View Full Version : MSN Live Indexes Google Ads
SteveGerencser
12-19-2007, 11:18 AM
Microsoft Live Search crawls and indexes Google Ads.. Over 4 million Google adsense links now in MS Live.. Me thinks that Google needs to slap a nofollow / noindex on all their ads..
Microsoft Live Search Indexing & Listing Google AdWords Ads (http://searchengineland.com/071219-094304.php)
khurramali
12-19-2007, 11:41 AM
Microsoft Live Search crawls and indexes Google Ads.. Over 4 million Google adsense links now in MS Live.. Me thinks that Google needs to slap a nofollow / noindex on all their ads..
Microsoft Live Search Indexing & Listing Google AdWords Ads (http://searchengineland.com/071219-094304.php)
just heard that news on the daily search cast on webmasterradio.fm now.
SteveGerencser
12-19-2007, 11:55 AM
That's where I heard.. You listen to good radio :)
But it does add to the discussion.. Should Google be forced to retool their entire adwords/adsense program to accommodate the MS Live search algorithm??
Microsoft Live Search crawls and indexes Google Ads.. Over 4 million Google adsense links now in MS Live.. Me thinks that Google needs to slap a nofollow / noindex on all their ads..
Microsoft Live Search Indexing & Listing Google AdWords Ads (http://searchengineland.com/071219-094304.php)
Do they need to put nofollow on those links? Isn't it easy to identify these links?
Do you see any adwords links on the SERP's of this Norwegian meta SE
Twingine (http://www.twingine.com/) ?
Assumption: The links Google are talking about are more difficult to identify.
Most interesting indeed. Deserves a thread of its own. Give me a minute...
Dave
ETA... Copied and started another thread...
http://www.webproworld.com/google-discussion-forum/65309-msn-live-indexes-google-ads.html
No, it does not deserve a new thread in my view.
The traditional competition between Coke and Pepsi?
Sounds almost malicious to me, on the part of MSN. Especially the fact that Adwords ads are blocked by robots.txt and 302 redirects (fulfilling the requirements stated by Google regarding the marking of paid links, for everyone about to yell "hypocrite"). I think MSN crawls Iframe content in an unusual way, and that may be why the ads show up. Who knows... I doubt it will last though, now that its been brought to light.
crankydave
12-19-2007, 01:39 PM
Sounds almost malicious to me, on the part of MSN. Especially the fact that Adwords ads are blocked by robots.txt and 302 redirects (fulfilling the requirements stated by Google regarding the marking of paid links, for everyone about to yell "hypocrite"). I think MSN crawls Iframe content in an unusual way, and that may be why the ads show up. Who knows... I doubt it will last though, now that its been brought to light.
Hmmmm...
Google states that paid links must be coded in a way so that they cannot be followed by a machine. Google has paid ads that can be followed by a machine.
Doesn't make a difference how they do it or why they do it.
Sounds like in this instance you're suggesting the creator of the process needs to fix their problem and not the producers of the content.
Whereas in Googles instance, you're fine with the creator of the process not fixing their problem and "requesting" the creators of the content fix it for them.
Interesting.
Dave
This short post is inspired by an authoritative book "Microeconomic analysis" chapter 24, third edition by Hal Varian ISBN 0-393-96026-9
Economists have tools that dates back to Pigou (1920) to answer these questions. The subject is "externalties" in production and consumption.
When the actions of one agent directly (read sold or bought links) affect the environment of another agent (read Google's PageRank algorithme or competitors that do not buy links), we call this an externalty. In the case of an externalty in production, the production of one company (Google's PageRank) is affected by the actions or production by another company (e.g. sold links).
In general, market equilibria will be inefficient in the presence of externalties. This naturally leads to an examination of various measures to restore an efficient market equilibrium, like disallowing pagerank to be passed on sold links.
Source: http://www.webproworld.com/google-discussion-forum/65216-ability-pass-pagerank-can-lost.html#post352008
In my view, Google has doene a serious job to fight paid links that is not always easy to identify and restore market efficiency.
There would be no problem for Googel to put a "nofollow" attribute on adsense links. The only effect will be to put extra burdon on webmaster's that must change their code.
In my view this is similar to Google's pigeonrank algorithme (http://www.google.com/technology/pigeonrank.html).
May this post would deserve 5 stars in the break room.
crankydave
12-19-2007, 02:30 PM
There would be no problem for Googel to put a "nofollow" attribute on adsense links. The only effect will be to put extra burdon on webmaster's that must change their code.
Isn't that what they are doing now?
Dave
Hmmmm...
Google states that paid links must be coded in a way so that they cannot be followed by a machine. Google has paid ads that can be followed by a machine.
Doesn't make a difference how they do it or why they do it.
One of the ways Google recommends for this is using robots.txt to block spiders from crawling pages. The directory containing Adwords ads is blocked according to Google's requirements.
Line 1: User-agent: *
Line 15: Disallow: /pagead/
Sounds like in this instance you're suggesting the creator of the process needs to fix their problem and not the producers of the content.
Whereas in Googles instance, you're fine with the creator of the process not fixing their problem and "requesting" the creators of the content fix it for them.
Every major search engine has sworn up and down that their bots always respect robots.txt. That is almost a core principle of spider behavior on the Internet. MSN is spidering content that is blocked by robots.txt. That is absolutely unacceptable. That is a flaw in either the algorithm or the spiders' internal logic and needs to be addressed immediately.
Google is telling people to stop a pattern of behavior that they feel was designed to exploit their systems. MSN is doing something fundamentally counter to what a search engine is supposed to do.
You don't see a difference between these situations? Google is saying "mark your paid links". MSN is saying "we are going to crawl what we want, and disobey robots.txt."
crankydave
12-19-2007, 03:13 PM
You don't see a difference between these situations? Google is saying "mark your paid links". MSN is saying "we are going to crawl what we want, and disobey robots.txt."
But that is not what's happening wige.
Here's a quote from the article Feydakin linked to...
However, that doesn't prevent Live from having URL-based listings that come not from crawling the links but merely seeing them from other pages (Google does the same thing -- it may list a blocked URL)
MSN Live is disobeying nothing.
Should all the individual webmasters be forced to prevent Google Ads from being crawled? Should Google "practice what they preach" and prevent their paid ads from being followed? Is it MSN's problem so they should fix it themself?
Dave
Isn't that what they are doing now?
Yes.
A Norwegian meta SE has no problem filtering out links with Google Id on it and split the links in two disjoint sets.
If msn have problems with splitting the links in two disjoint set's that is definitely an argument for not using their SE.
Which links do you think Google talk about?
May be the problem is over stated, since it seems that Google seemingly have no problem with identifying these links (http://www.webproworld.com/google-discussion-forum/63904-ongoing-pagerank-massacre-hits-big-players.html#post34330).
May be a more relevant question: http://www.webproworld.com/google-discussion-forum/65216-ability-pass-pagerank-can-lost.html Why was that discussion stopped?
How does the MSN site display the page title if the spider does not actually retrieve the page?
Don't get me wrong, if MSN is not actually downloading the blocked content then I don't really see any problem with it. It's their index to do with as they please, although Google will need to make sure clicks on this content are handled correctly, as far as spam detection, etc. After all, it would suck if someone's Adwords account got marked as spam because it showed up in the MSN SERPs and got clicks.
I was unaware that search engines commonly listed pages in their indexes that were not crawled. How do the search engines know anything about these pages? How do you prevent visits to closed sections of your site if search engines index URLs without crawling the page or even checking the robots.txt rules for that page?
SteveGerencser
12-20-2007, 07:49 AM
MSN has already fixed this issue..
AdWords URLs were listed not only because of the way Live Search indexes blocked URLs, but also because of a change in how Google renders AdSense units. However, the blog post doesn't describe this change and the Google AdSense blog doesn't mention changes.
Bolding mine.. It just seems a little strange to have this happen at a time when Google is already forcing millions to change how they do business and build websites..
How does MSN qualify their SERP's?