View Single Post
  #1 (permalink)  
Old 02-03-2008, 08:58 PM
memaggiem memaggiem is offline
WebProWorld Member
 
Join Date: Apr 2004
Location: N.E.
Posts: 39
memaggiem RepRank 0
Default Excluding the "good" BOTS from /images folder

Hi again all! I had an issue way back with MSN trying to index my site from my /images/ folder due to the presence of an index.htm file

Since then I have excluded the MSN bot from the /images/ and VIOLA! my site is now being indexed properly!

Today I discovered that Google has my /images/index.htm page in a result for a keyword of mine. Mind you it's a pretty obscure keyword but nonetheless, it's been indexed. I see no evidence of Google trying to find my content within the /images/ folder as the MSN bot kept trying.

Question: Should I exclude the /images/ directory from all bots? If I do that then won't the cached pages not show the images? I tried to disable hotlinking once and the cached pages looked awful! Would you exclude the /images/ folder or is there another way that I can stop the contents of said folder from being displayed when some snarky user wants to see the images in the directory?

Many thanks!
Maggie
Reply With Quote