View Single Post
  #2 (permalink)  
Old 01-14-2009, 02:33 PM
MrGamm's Avatar
MrGamm MrGamm is offline
WebProWorld Pro
 
Join Date: Dec 2007
Posts: 278
MrGamm RepRank 4MrGamm RepRank 4MrGamm RepRank 4
Default Re: do search engines follow robots rules?

Maybe you could try running your robots.txt file through a validator... Typically the allow statement is redundant.

I've never used this one before but maybe it will help...

Validators | Robots.txt Validator

Just because you add a robots does not mean google will immediately drop those pages. It could take a bit if it's anew addition.
__________________
James Weisbrod - programmer
Reply With Quote