Re: Excluding the "good" BOTS from /images folder
memaggiem this should do what your asking and not cause any problems with any of the bots.
User-agent: *
Disallow:/subdirectory/index.htm
or
User-agent: *
Disallow:/index.htm
But that might stop your home page from being indexed depending on site design.
As to the bigger question of allowing images to be indexed, we do not allow any of our images to be indexed. We do our own photography of products and want to protect them as much as possible so we completely disallow our images folder in our robots.txt file. This does not prevent the images from showing in the cache on the search engines as they use links to our images on our site instead of caching the images them selves.
|