Several threads recently have asked questions about directories like DMOZ/ODP -- one has a question about why Google's directory lags so far behind DMOZ. I came across an article (excerpted here) which makes reference to what I see as the fundamental problem with DMOZ or similar projects, including the "ESP" project mentioned here -- although well-intended, the exponentially increasing size of the internet makes it virtually impossible to keep up by reviewing each new site with "human hands".
Quote:
Associated Press
October 17, 2003
PITTSBURGH — Carnegie Mellon University researchers are using an Internet game to help improve artificial intelligence, in hopes of making Web searches more powerful. Graduate student Luis von Ahn and his mentor, professor Manuel Blum, believe search engines can one day adopt word labels generated by their ESP Game to help computers see images more as humans do.
(snip)
Search engines use algorithms — mathematical recipes designed to solve problems — to sort, rank and filter pages, text and images on the Internet. The ESP Game tries to improve upon that by asking two players who don't know each other to type in words that describe a series of images. Players win points when they match words — and those matches become labels Mr. von Ahn and Mr. Blum can affix to the image in question.It would take too long for researchers to label the hundreds of-millions of images that can be accessed by Google or other search engines. But Mr. von Ahn believes that task might be accomplished in a few months by getting a few thousand people to play the game each day.
Spokesmen for Google and Alta Vista were mum on that prospect, and some industry analysts were skeptical. Sophisticated algorithms can track which sites help the most users with specific questions — and that's generally faster and cheaper than using a phalanx of human editors, said Danny Sullivan, editor of the on-line newsletter SearchEngineWatch.com.
|