Difference in rankings when searching Google datacenters

[wige]

I know that Google uses different datacenters, and as a result, I can expect some variations in the results depending on which datacenter responds to my query. However, I have noticed that when I query a datacenter directly, I get vastly different results. Let me give you an example. Run the following six queries in your web browser. The first queries google.com, the rest query specific, randomly selected datacenters, all with the same search terms and passing the same parameters.

seo web design - Google Search
seo web design - Google Search
seo web design - Google Search
seo web design - Google Search
seo web design - Google Search

You should notice that the first few results are the same, between the first and the other five, however from result 4 on they shuffle around quite a bit. In running these queries, I was not logged in to a Google account, I was in the US, and I was not using a proxy or anything else to hide my location. I tested this in Firefox and Internet Explorer, both with and without clearing my cookies.

Sorry about stealing the search terms from another thread, but there was a vigorous discussion there in part related to SERP positions, and I noticed that one side of the discussion was using the direct query method and the other was entering google.com, albeit through a proxy server.

My questions for the group are: Do you see this difference when visiting these sites? Why are these differences here? Do you take into account this issue when running reports for yourself or your clients? And which positions are more "important"? Do I want to be #1 on the direct access method, or #1 at Google.com (might more of the world see me at #1 if I am there in the direct access method, even though I am not at #1 when querying google.com from my present location)?

[SemAdvance]

I don't know.

More than likely datacenters work in some aspect for localization aspects.

More important question

Who is the target market?

Most of my family searches only google.com since none are webmasters Datacenters make no difference in their lives at least not that they are aware of and I doubt they would care.

So I want what would bring the most exposure!

Peace!

[Linknz]

From what I understand about the datacentres, they each hold the present information that's coming back from the googlebots visits and this will sit and remain in each datacentre until it's uploaded to the main server cluster, some some searches will show a variation in the results of the data held within each datacentre.
So I would consider this to be quite normal..

[amazonian]

I got 403 "forbidden" errors on all but the first link. For a few weeks now on most, but not all, of my searches, I've been required to type in a code to continue the search. Ostensibly my "query looks similar to automated requests from a computer virus or spyware application. To protect our users, we can't process your request right now."

I don't get it. How could my search for certain phrases hurt Google? It's not like I'm Adsense spamming! I suspect it has something to do with the way my McAfee virus protection software has been loading lately (or not), but what damn business is it of Google's?

[jeffposaka]

Originally Posted by amazonian

I got 403 "forbidden" errors on all but the first link. For a few weeks now on most, but not all, of my searches, I've been required to type in a code to continue the search. Ostensibly my "query looks similar to automated requests from a computer virus or spyware application. To protect our users, we can't process your request right now."

I don't get it. How could my search for certain phrases hurt Google? It's not like I'm Adsense spamming! I suspect it has something to do with the way my McAfee virus protection software has been loading lately (or not), but what damn business is it of Google's?

Sometimes I get this message when I am running a lot of queries with Seo for firefox.

[full house]

most data center doesn't give the same results . . some are just dammy or fake . . .

[dawilliams]

"Target Market" was mentioned above and I agree wholeheartedly. The majority of your clients (who use Google) will likely visit Google.com or use their Google Toolbar to find what they're seeking online. It is unlikely they are even cognizant of various datacenters and the means to query them.

Therefore - it seems to me that you would reach the majority of your target market and realize the best ROI for time and energy by striving to rank well on Google sans datacenter specific queries.

All the best in 2008.
Dave

[crankydave]

The thing to remember is that data sets are fetched and ordered at the time of the query. Different parameters, thresholds, etc. for the different DC's will cause this. Don't assume the exact same processes are at work across the board.

The place you want to be #1 is what is live at the time and place of the query. Afterall, this is what the searcher sees. It's also what makes SERP watching and reporting difficult.

Dave

[wige]

Actually, I came up with a new test for this as follows:

1. Use a tabbed browser, and clear your cookies before running the test.
2. Go to google.com, and enter in a search term. I used, once again, "seo web design" without quotes.
3. Click submit.
4. Open a new tab in your browser, and cut and paste the address from the SERPs page into the new tab. Verify you see the same results.
5. Open your command prompt. In WinXP, press windows+r and enter cmd. For Vista, type cmd in the Search bar at the bottom of your Start menu. If you have Linux, open your network control panel and find the ping utility.
6. Enter "ping google.com" at the command prompt.
7. You will see "Pinging google.com [x.x.x.x] with 32 bytes of data". x.x.x.x is the IP address of the Google datacenter you just ran your search on.
8. In the second tab's address bar, replace "www.google.com" with the IP address you got in step 7, and press enter.
9. The results will change, typically starting around position 5. Compare the two tabs.

I think this rules out localization factors and differences between data centers because both queries use the same data center. Simply put, it appears to me that Google serves different results or sorts results based on different parameters when you do a search directed at the IP address.

Because many tools that SEOs use to check rankings use the IP address of a specific datacenter rather than using the domain name, this can cause users of the tool to see different results from normal searchers.

[crankydave]

Consider though wige, that the "live" data center may have slightly different parameters, thresholds, etc. than a specific DC even though the same IP address is being queried. Same initial database of information being used but different criteria.

Dave

[wige]

But doesn't this make the tools unreliable at best? If a rank checking tool queries a specific datacenter, and different filters are applied when the request uses an IP address, this would make the results suspect because no searcher would see the same order of results.

[dann]

Originally Posted by Linknz

From what I understand about the datacentres, they each hold the present information that's coming back from the googlebots visits and this will sit and remain in each datacentre until it's uploaded to the main server cluster, some some searches will show a variation in the results of the data held within each datacentre.
So I would consider this to be quite normal..

To put it in simple terms this is very true.
A data center is not one huge server, but a bundle of 500 or more"relative cheap computers" to quote Google that matches your query based on "hundreds of factors".

Since Google has changed their algorithm slightly in the last year and a half or longer to represent freshness, the term everflux was invented.
I am not really sure of the total number of Googlebots out there, more than I can count though going through server logs.
So you have to imagine constant updates rolling through the DC's hundreds of computers that each provide only a few bytes per computer to each query. I hope that makes sense.
So it is completely understandable to get different results from different Data Centers all other factors already mention being excluded, all of this plays a role in the SERPS.

Don't spend too much time obsessing on Data Centers, it could be an indication of where you are or are going to be in the serps tomorrow or not. Most Google engineers probably wouldn't be able to answer that question.

When you are doing this research tough be sure you are not signed in to Google and use a browser set not to accept cookies, I use Opera for this specifically, but choose any other browser that you would not normally use.
Hope this helps, Dan

[crankydave]

Originally Posted by wige

But doesn't this make the tools unreliable at best? If a rank checking tool queries a specific datacenter, and different filters are applied when the request uses an IP address, this would make the results suspect because no searcher would see the same order of results.

I don't know about "unreliable" but it's not an exact science. Using the tools can give you an "idea". One that could be what was, what is, or what may or will be. Could also be that an actual searcher never see the exact same results. Hypothetically a "test" DC(s) where the tweaks and changes produce less than desirable results and never go "live".

Dave

[RichAtVNS]

ok, a little programming information, from someone who does systems design and development for many years on large scale systems (see VNSInc.com for my Profile of Richard Jay Silverman Cofounder/CEO resume lol)

Profile of Richard Jay Silverman Cofounder/CEO

1) Someone said that the local googlebots load ther spidering information to the local datacenters. This is a collection process done in almost all large distributed redundent multiprocessing systems.

2) In all collection systems, there is no real time analysis and storage going on which causes latency. This is called "offline processing" and "propagation lag"

3) all information collected is sent to a central location and then finally modified for use in the final system in this case google.com. This is what is called a simple "store and forward process"

4) I saw the results of your query and all but google they were identical
that meant that the information had been populated back to the other datacenters. Guess what this has a term also "bridgeback" which also has a processing lag.

5) Google had 1 site put ahead for the number one position. This was obvious a paid override or an algorithm to decide the expert in a highly competative field. And a core process on its main delivery system.

6) Google.com does resolve to different ip addresses all the time for me. I use multiple browsers and machines and routes to it. So Dynamic routers are used to decide which datacenters are delivering information when you go to google.com. This is to allow "redundent systems" and "distrbuted processing".

Now for a little hypothosis on my part.

This means if any datacenter is out of whack being worked on, it is possible for google to not have it show up in the dynamic routing,
So for more accurate results for your client run your queries off of google.com.

This way your assured to be accurate are not dealing with a datacenter which may be off line or a propogation lag.

---- one final term ... since we do not know the exact algorithms google or other search engines use to decide on their ranking systems we are using heuristic approaches to doing this. This is what Marvin Minsky and his chief rival Melvin Klerer (who I studied under) used to argue about in the artiifical intelligence debates of the late 60s, 70s and early 80s!

That's why I find this forum so enjoyable at times. It's the lack of understanding for simple programming paradigns....

Of course it still means I can't spell correctly or use correct grammer! LOL

[dann]

Originally Posted by RichAtVNS

This means if any datacenter is out of whack being worked on, it is possible for google to not have it show up in the dynamic routing,

Agreed, except for this. To say a DC is "out of whack" for returning different results is not accurate. Either it is ahead or being refreshed is more probable.
Out of whack means false or erroneous, I don't believe that's the case.

[RichAtVNS]

offline - definately could be out of whack.

Because they bring it down for software updates, testing software offline, removing worms and/or hacker attacks or have corrupted data!

It's not just about propagation errors...

[crankydave]

Originally Posted by RichAtVNS

offline - definately could be out of whack.

Because they bring it down for software updates, testing software offline, removing worms and/or hacker attacks or have corrupted data!

It's not just about propagation errors...

In these type instances would it not be able to be accessed at all? I'm not sure how this would apply if it cannot be accessed.

Dave