Any sitings of new search technology?

[Conficio]

Dear visitors,
has anyone seen some new search technology?

There is an open source project, called Nutch (http://www.nutch.org/), that implements an open source search engine.

They are backed by some well known people in the wider industry:

Peter Savich (Overture Research (http://research.overture.com/))
Mitch Kapor
Tim O'Reilly
Raymie Stata (UCSC)
Graham Spencer (Digital Consumer (http://www.digitalconsumer.org/))
Doug Cutting
Dan Fain (Overture Research)
Brewster Kahle (Internet Archive (http://www.archive.org/))
Michele Kimpton (Internet Archive (http://www.archive.org/))
Joe Kraus (Digital Consumer[/url)
Brett Bullington
Neil & Danny Rimer ([url=http://www.indexventures.com/]Index Ventures (http://www.digitalconsumer.org/))
R.J. Pittman (Groxis (http://www.groxis.com/))


Did Nutch come up in any presentation / discussion / forum / question? Have any of the supportes listed surfaced at the conference?

I courius about this.

K<o>

[redcircle]

That is one thing that I think should be kept private. All we need is some money hungry spammer getting the source and ruining it for everyone. I'm usually 100% pro open source but not on this one.

I'd rather have people keep guessing on what makes good search engine results than knowing the algorithem and tuning the website to do exactly that. You then have results that are just the best tuned sites and not the best content.

[Conficio]

I'd rather have people keep guessing on what makes good search engine results than knowing the algorithem and tuning the website to do exactly that. You then have results that are just the best tuned sites and not the best content.

That is a valid concern. But the argument can be made, that it only needs the right idea to make it spam proof. If you look at the cryptographic community, they only take for serious open source (or better open algorithm). If we attack the search issue from a similar angle, may be we get it fool proof.

The other benefit I see is to make search engine operation much cheaper. All I need is hardware and bandwith and a small amount of programming to define and tweak my ranking algorithm. That allows more offerings and should bolster competition (also makes life harder for the spammer, as more search engines need to be primed).

Lets play the scenario, where hundreds, may be thousands of search engines do their game and then the user uses a meta search engine (all the rage 4 - 5 years ago). This should give us an almost democratic process - voting with your own search engine.

Don't take my proposals too serious. I'm just trying to think out of the box.

K<o>