Thread: robots.txt an open window?
View Single Post
  #9 (permalink)  
Old 10-01-2009, 03:33 AM
deepsand's Avatar
deepsand deepsand is offline
WebProWorld 1,000+ Club
WebProWorld MVP
  
Join Date: May 2004
Location: Philadelphia, PA
Posts: 4,249
deepsand RepRank 10deepsand RepRank 10deepsand RepRank 10deepsand RepRank 10deepsand RepRank 10deepsand RepRank 10deepsand RepRank 10deepsand RepRank 10deepsand RepRank 10deepsand RepRank 10deepsand RepRank 10
Default Re: robots.txt an open window?
Quote:
Originally Posted by simonm View Post
... I'll put the nofollow / noindex on page rather than robots.txt ...
If a bot is not so well behaved as to heed robots directives, why believe that it will heed meta tags?
Reply With Quote