PDA

View Full Version : MSN not visiting but other are?



mcspec
01-24-2006, 09:46 PM
www.b-rlaw.com

We do great on google and fine on yahoo. We dont show up on MSN except for backlinks. Robots.txt does not exclude MSN bot, the bot is comming but not indiexing. Whats our problem?

Thanks for the help

cyanide
01-25-2006, 06:05 AM
The drop-downs at the top of your site, would be hurting you a little. You would do well to put a link to your site map at the bottom of every page.

greeneagle
01-25-2006, 11:03 AM
LOL - Not sure you really want them there - They voraciously suck down bandwidth.

In fact that is getting to be a problem for many, not just MSNBots!, but all.

Don't know where it's at, but WMW took a bold move and blocked all the major Bots a while back.

Ken

mcspec
01-25-2006, 05:16 PM
Thank you for your help. You think the navigation bar is preventing the BOT from getting through? That is unfortunate. Thanks very much for taking a look.

P

milonic
01-25-2006, 06:23 PM
With DHTML Menus it's always a good option to include the links that are in the menu inside a <NOSCRIPT> tag. The search engines won't penalize you for it because it's completely justified.

Take a look at our website: www.milonic.com - view the source and see the <NOSCRIPT> that we employ, it's automated on our site using PHP code but no reason why it can't be static.

Hope this helps

milonic
01-25-2006, 06:25 PM
Oh, and it also gives an alternative navigation technique for users that have Javascript turned off.

brian.mark
01-26-2006, 04:23 PM
LOL - Not sure you really want them there - They voraciously suck down bandwidth.

In fact that is getting to be a problem for many, not just MSNBots!, but all.

Don't know where it's at, but WMW took a bold move and blocked all the major Bots a while back.

Ken

Didn't that turn out to be a cloaked robots.txt?

http://72.14.203.104/search?q=cache:2O3CEgu774gJ:www.webmasterworld.com/robots.txt

Compared to
http://www.webmasterworld.com/robots.txt

Brian.