Fuzzy Search Engine Math: Who’s Really Bigger?

[jmiller]

Just when you thought someone was going to settle this once and for all, you’re thrown a curveball. Yahoo! says they’ve got a bigger index. Google says, “We don’t buy it.” And now a third party comes out on the side of Google, but has some suspect research methods. This is really, really confusing.

Last week, Yahoo!’s Tim Mayor came out twirling impressive numbers on Yahoo! Search Blog, claiming that with 20 billion indexed items, the #2 search engine had more than doubled Google’s retrievable information.

Google whirled around in their chairs and let everyone know that Google scientists were unable to verify those numbers.

This week, the National Center for Supercomputing Applications (NCSA) in conjunction with the University of Illinois at Urbana-Champaign, “conducted a brief study” of their own. The researchers, Matthew Cheney and Mike Perry, found that Google actually returned 166.9% more results than Yahoo!. Among the 10,012 test cases run, Yahoo! only returned more results than Google 3% of the time.

That’s a huge disparity in claims between Yahoo! and NCSA. One says it’s twice Google’s size, the other says Yahoo! isn’t even close to being in the same league as Google.

While the study would seem to lend support to Google’s contention that they have not been able to verify Yahoo!’s numbers, the study’s research methods give one pause.

“Unfortunately, both the Yahoo! and Google search engines truncate results returned to the user after 1,000 results. Thus, for the purposes of this study, we were forced to restrict our searches to those queries that returned less than 1,000 results on both Yahoo! and Google. Any search result found to have more than 1,000 returned results on either search engine was disregarded from our sample.”

The researchers who presented this study only used the most obscure of search terms and found that Google was able to plow the nether regions of the Internet better than Yahoo!.

But is that truly reflective of total size? Or is it only reflective of one search engine’s ability to find trivial items?

I’m no statistician, but these numbers don’t make a lot of sense to me. Would the real Search Giant please stand up?

[greeneagle]

IMHO
YAHOO has again revalidated this old cliche:
"Bigger Isn't Always Better!"

http://www.webproworld.com/viewtopic.php?t=50407

Ken

[kgun]

“Unfortunately, both the Yahoo! and Google search engines truncate results returned to the user after 1,000 results. Thus, for the purposes of this study, we were forced to restrict our searches to those queries that returned less than 1,000 results on both Yahoo! and Google. Any search result found to have more than 1,000 returned results on either search engine was disregarded from our sample.”

Google is better than Yahoo at restricted search. Better in the meaning of getting MORE hits.

In mathematics, a general mistake is to generalize from 1,2,3 dimensions two n dimensions. The "easiest" proof you can give in mathematics is (a contra proof):

You prove that something is valid for k=1. You assume that it is valid for k=s where s is a small number. You are able to prove that it is also valid for k=s+1. But your are not able to prove that your hyphotesis is still valid for k=z where z is very large. If a person shows that it is invalid for at lest one z, it is generally not valid. Then how many z's shall we require before we can draw the "fuzzy" conclusion:

SE A finds more unique hits than SE B for usual search terms?

"PageRank or PR(A) can be calculated using a simple iterative algorithm, and corresponds to the principal eigenvector of the normalized link matrix of the web. Also, a PageRank for 26 million web pages can be computed in a few hours on a medium size workstation. There are many other details which are beyond the scope of this paper."
http://www-db.stanford.edu/~backrub/google.html

Simple iterative alogrithm and link matrix of the web and the web is increasing exponentially. That requires computing power. Depending on the algorithm of other SE's, we may come to a point, where other algorithms is better (faster) in the meaning of indexing MORE sites in less time.

Linking outboud links is e.g. easier. So if SE A gives weight x (1-x) to IBL's (UBL's) and SE B weight y (1-y) to IBL's (UBL's) and x >> y, than SE B may be much more efficient at one time. But what is a good SERP?

It is a race between fast connections, computing power and algorithms.

Problem:
Are there infinite number of twin prime number pairs like (11,13), (17,19), (29,31) ....

There are s numbers, and s is large, but still unproved as far as I know, that there are an infinite or finite number of twin prime number pairs.

Digression:
It may require infinite precision. Computers have finite prcision, that is are finite state machines. You can not prove anything on a computer with finite precision that requires infinite precision. So it (may) require(s) an analytic proof.

Kjell Gunnar Bleivik
http://www.multifinanceit.com/
http://www.blognorway.com/

[kgun]

Problem:

Will it in n years be possible to train a computer (dataprogram) on learning analytic mathematics?

http://www.wolfram.com/


Kjell Gunnar Bleivik
http://www.multifinanceit.com/
http://www.blognorway.com/