msn search robot

PhillipH · #1 (**permalink**) 10-17-2004, 09:03 PM

this lil dude was quick to come to my websites. i only registered a couple of days ago, maybe coincidence. It just begs the question whether paying for an index is wise.

Guest 4 18 Oct 2004 at 1:05am 18 Oct 2004 at 1:26am 21 minutes Search Robot MSN

bhartzer · #2 (**permalink**) 10-18-2004, 12:32 PM

The MSN bot is definitely being very aggressive. I've actually stopped it from indexing certain sites because of it being overly-aggressive, chewing up bandwidth on certain sites. I'm not going to let MSNBot spider certain sites until their search is available to the public.

sfowler · #3 (**permalink**) 10-27-2004, 03:31 AM

I agree. I do not yet have a problem with bandwidthe (more's the pity!), but MSN will be the first to go if it becomes an issue.

**jawn_tech** · #4 (**permalink**) 11-01-2004, 03:29 PM

I agree too. Go public, then we'll dance.

When it does become public, I'm glad their spider makes more frequent rounds though than Yahoo. I don't see Y as often for some reason.