Very interesting discussion, even if somebody meant it ended in
SEO ethics (important enough), this
http://www.webmasterworld.com/forum30/29726-1-10.htm
discussion. It is a discussion between "Henk van Ess" and Google, that he has published copy right material at his site
http://www.searchbistro.com/
He claims to be a professional reporter and has not published anything (c) stuff.
Googlebot has an addtional option, archive, that other bots do not posess. As far as I know that is the default. That means that your complete site is copied and archived. In addition it has the index and follow option like other bots / crawlers.
This makes some interesting general questions:
1. What is copy righted material on the internet?
2. Who has the right to copy and archive that material?
3. What about password protected material?
4. Is there any international law on this subject?
The post also throws some light on how Google uses human beings to test / evaluate their rankings. It may be compared to wine tasters tasting wine. The input may be used to alter / maipulate the formulae (algorithm).
The discussion reminds me of a Dutch sailor that under the speculative tulip bulb craze that gripped seventeenth-century Holland was put in jail for eating a tulip bulb that he thought was an ordinary onion
http://www.litrix.com/madraven/madne004.htm
Conclusion: There is a great difference between an onion and tulip bulb. Be careful with what you eat, especially when you eat it.
Kjell Bleivik
http://www.multifinanceit.com/