Following a hunch -- MSN and robot tags

[jawn_tech]

Disclaimer -- this is totally unscientific. Just one site, just one instance, with just 3 keywords.

A particular site I manage does well in the SE's, and has been #1 on MSN for 3 important keywords.

Full knowing the major SE's will index a site even without a

<META NAME="ROBOTS" CONTENT="ALL">

or, a...

<META NAME="ROBOTS" CONTENT="INDEX,FOLLOW">

But hey, who knows if there's some search engine out there that it may help in, and a few extra visitors every now and then certainly won't hurt. After all, just because the majors don't use them (unless you tell them "noindex,nofollow", which even then is debatable sometimes), they certainly wouldn't knock you for using the "index,follow"...or would they?

So I go ahead and throw it in, just to see if it makes any difference in traffic/referrals.

In a couple days the 3 keywords getting first place in MSN suddenly go to page 2, in the 11-20th range. Hmm. So I remove the tags, and in a week they're back to number one. It's the only SE where I noticed a difference.

Just a coincidence? Probably, but the timing sure looked fishy, since no other changes were made, as far as code, content, or IBL's (that I was aware of). Would be interesting to run a test with a number of different sites.

[timmathews.com]

Wow. Nice hypothesis. I will run web pos report for MSN on one of my sites, then remove tags and run again in 14 days. I will let you know results.
Tim

[jawn_tech]

Thanks! MSN still seems like a new breed of animal, and it's interesting to test its temperment by throwing a stick at it once in a while.

[bblwise]

About the time that MSN introduced it's new search, my website dropped out completely from results. I've deleted my robot.txt code and will see what happens.

It was fine under the old sea.search

[webmasterjunkie]

I use the:
<META NAME="ROBOTS" CONTENT="INDEX,FOLLOW">
on websites. MSN places me well in the first page for a term I shoot for. One site has this code and no robots.txt file. Just thought I'd share.

[montytx]

I am suspicious that a change in code seems to move sites up in rankings. Almost as if MSN thinks that a fresher website is worth more than one with no changes. My site has yoyoed from no to 17 to 5 last week to 10 yesterday to 18 today. The only correlation I can find is that when I change the content of the page I notice a temporary boost in rankings. The problem is then 5 days later they seem to drop. Has anyone else noticed an similar occurance?

[webmasterjunkie]

All search engines will favor a site with fresher content than a website with stale content. But, they can also tell the difference from a website that needs fresh content from a website that has no need for fresh content.

IMO, it's a matter of what you are trying to get across to the user.

[garbageaccount]

I'll try it for a few weeks and see what happens. It really can't hurt at this point.

[wrmineo]

It's very possible you're on to something, but it's just as likely you're dealing with coincidence rather than conclusion.

I place these tags in all of my main pages and have only seen good results in MSN. Case in point, my main page at http://www.britney.freshdames.com has these metas and when I query MSN for "britney spears pics" my page is #4 out of nearly 3 Million results.

In all honesty, I'm not willing to wager that removing them will propel the page up a notch or two.

However, I'm happy to participate in a little curiousity and research. I'll take an unranked page or two and tweak them and let you know.

Good luck - keep the notes and information flowing.

Happy Surfing ~

[yeshua14]

Tim,

After the 14 days, did anything happen?