View Single Post
  #10 (permalink)  
Old 02-04-2008, 06:38 PM
Penman Penman is offline
WebProWorld New Member
 
Join Date: Dec 2004
Posts: 13
Penman RepRank 0
Default Re: Excluding the "good" BOTS from /images folder

memaggiem this should do what your asking and not cause any problems with any of the bots.

User-agent: *
Disallow:/subdirectory/index.htm

or

User-agent: *
Disallow:/index.htm

But that might stop your home page from being indexed depending on site design.

As to the bigger question of allowing images to be indexed, we do not allow any of our images to be indexed. We do our own photography of products and want to protect them as much as possible so we completely disallow our images folder in our robots.txt file. This does not prevent the images from showing in the cache on the search engines as they use links to our images on our site instead of caching the images them selves.
Reply With Quote