I feel used.

[mtheory]

Let's examine the true driving force behind Google. Without it they would not exist.

It is the actual websites on the internet.

So far, the entire internet system has been lumped togethor as public domaim filled with intelectual properties and a vast gray area of legal and social liabilities and responsibilities.

We all pay to register, build, and maintain our real estate on the web.

Could some kind of class action be brought against Google for compensation for using our valuable "real estate" or should they be taxed to some extent for commercial use of this public domain.

[rlrouse]

No, IMO Google's spidering of our sites is fair use.

[mtheory]

Many sites have terms of use policies that strictly prohibit displaying there content within a frameset.

Google image search search does just that.

Google allows use of there search service on other sites but reserves the right to refuse usage and has an extensive "terms of use policy".

They also have a disclaimer that limits their liability for sites they display within their searches.

The whole Google concept is basically an extensive data mining operation that occurs with only internal scrutiny and no definitive compensation for the resources they(Google) exploit.

[Tubby]

I think of google (and other search engines) not so much as using our valuable real estate . . rather, more like the highways and footpaths, Take away google and the likes and I am once more stranded in the desert. Of course I would prefer google to build a bigger hyway to my little plot.

Google provides these roads tax free, they even pay me to show my users the road to other "valuable real estate".

Google might be losing some ground to Yahoo (using my own statistics here) But I love it dearly. No class action from me. . .

[Mel]

If you do not want your sites spidered and indexed (though why you would want a site no one can find does perplex me I admit) the solution is simple: Put up a robots.txt file that prohibits the whatever robots visiting what ever part of your site you do not want indexed.

[Tubby]

I rarely have a consistant view. my mind wanders around dizily pondering the altrnatives. . As a result my best arguments always evolve after the debate or discussion is over . . .

but Mtheory has a perspective I had not fully appreciated. What about a 'search fee'?

I shall now ponder whether we should charge by word count, or band width, - or ?. . I am happy not to kill this thought until I have we have given it a "good coat of looking at"

[buddhu]

Originally Posted by Mel

If you do not want your sites spidered and indexed (though why you would want a site no one can find does perplex me I admit) the solution is simple: Put up a robots.txt file that prohibits the whatever robots visiting what ever part of your site you do not want indexed.

Yep, same as Mel said.

[mtheory]

Originally Posted by Tubby

I rarely have a consistant view. my mind wanders around dizily pondering the altrnatives. . As a result my best arguments always evolve after the debate or discussion is over . . .

but Mtheory has a perspective I had not fully appreciated. What about a 'search fee'?

Tubby hit it right on the head. In addition to providing help to others, the forum environment is a great place to present new ideas an perspectives.

Originally Posted by Mel

If you do not want your sites spidered and indexed (though why you would want a site no one can find does perplex me I admit) the solution is simple: Put up a robots.txt file that prohibits the whatever robots visiting what ever part of your site you do not want indexed.

This point of view gives the search engine the assumtion that unless you "opt out" they are automatically entitled to exploit your content.

This is a dangerous precident and is why I think this issue need to be examined.

By this thinking, anyone may be able to assume that by placing your email address on your site, you have freely given them the right to spider your site, retreive the address and opt you into your service.

[greeneagle]

"Search Fees" already exist.

LOL
Ken

[jawn_tech]

lol. this is the first I've heard of anyone not wishing their 'real estate' was used more by Google. Usually it's people wishing their site wasn't 'ignored' by G, as they say.

[Tubby]

I don't think Mtheory in the original post said
he did not wanthis site spidered. so I think Mels point is a bit mute.

[mike]

It's an interesting argument in theory. However, it reminds me of some others that have been made in regard to things like cell phones and airline flight patterns. For example, if I own my property, how far up is it still mine? Should planes be able to fly in my 'yard'? Satellite pirates sometimes argue argue they signal the are receiving came to them, they didn't 'go get it'. But are people 'entitled' to signals that cross their private property- cellphones?

These arguments, while interesting, don't generally hold up all that well practically. The webpage crawling is a bit more compelling because the internet is still a relativly new thing and rights and entitlements are certainly unsure in many cases. The analogy I would use in your proposal though would be:
Imagine your website is a store or house. The internet is the highway by which people get to your store. Certainly you can choose who you do and don't let into your home or store but you can't necessarily keep people from looking at it from the highway. The search spider is essentially looking at your home/store and saying 'hey, that's over there'. You can tell it not to 'look' if you wish but I don't see a strong argument for saying they shouldn't look at it in the first place.

[mtheory]

Originally Posted by Mike

It's an interesting argument in theory. However, it reminds me of some others that have been made in regard to things like cell phones and airline flight patterns. For example, if I own my property, how far up is it still mine? Should planes be able to fly in my 'yard'? Satellite pirates sometimes argue argue they signal the are receiving came to them, they didn't 'go get it'. But are people 'entitled' to signals that cross their private property- cellphones?

These arguments, while interesting, don't generally hold up all that well practically. The webpage crawling is a bit more compelling because the internet is still a relativly new thing and rights and entitlements are certainly unsure in many cases. The analogy I would use in your proposal though would be:
Imagine your website is a store or house. The internet is the highway by which people get to your store. Certainly you can choose who you do and don't let into your home or store but you can't necessarily keep people from looking at it from the highway. The search spider is essentially looking at your home/store and saying 'hey, that's over there'. You can tell it not to 'look' if you wish but I don't see a strong argument for saying they shouldn't look at it in the first place.

How high up? Barbara Streisand sued a photographer for taking and ariel photograph of her home and publishing it, claiming invasion of privacy.

The "hey it's over there" anology isn't completely valid IMO because the spider does not stop at the "storefront" they also index content.
I also don't like the idea of a private enterprise telling me what roads I can or cannot see from, meaning, if the want to use these public throughways for profit the should not be able to selectively "ban" sites that don't meet their criteria or they should be obligated to make their "siteseeing" algorythms and practices available for public scrutiny.

[Fingers]

Very interesting point this.

I sort of see where you coming from with this..... but ultimately you have the choice to lock your door!

If that choice was not there then I think you have a really good argument.

A good shop keeper would not ban peeps from coming into his store and even taking a full invetory of his stock and showing all his friends for free...would he?

[Tubby]

I do not see the previous examples as a good analogy.

How about for example real estate. where one real estate agent is given exclusivity. Knowing that you are the only one able to sell a property gives an incentive to promote via newspapers etc.

What if an updated Google search engine offered to search your site daily if it was given exclusive rights to search your content, and would be willing to offer you payment. what if Google decided to buy the rights to search good content sites. . to restict the opposition.. what if you refused to give Yahoo permission?

Not likely to happen? . . maybe not, but this sort of agreement is everywhere in the business world.
'Locking your door' is a good business practise used widely, and wisely -

[MHenscheid]

Originally Posted by Mtheory

This point of view gives the search engine the assumtion that unless you "opt out" they are automatically entitled to exploit your content.

When you post a website, you are making it public. I would call this an "opt in" to any body who wants to see it, whether it be man or machine (google).

When displaying search results, one might argue that Google is committing plagerism, but I would say that they are properly siting the work.

It is very easy to make a site non searchable.

Originally Posted by Mtheory

By this thinking, anyone may be able to assume that by placing your email address on your site, you have freely given them the right to spider your site, retreive the address and opt you into your service.

I don't think this is a perfect analogy. Spidering for email addresses and then spamming has been deemed undersireable and has now been made illegal.

On the contrary spidering to provide information to the public is considered desireable (by 99% of people).

I don't believe we are setting bad precedents here by allowing google to search w/o explicit permission...we have already set a precedent regarding undesirable events (spamming emails) and could do so again in the future if something else undersireable occured (By undersirable, I mean as considered by a large amount of people).

[activeco]

Originally Posted by Fingers

..... but ultimately you have the choice to lock your door!

Otherwise, a visitor has the right to get what he wants from the house?

Yes, there is the robots.txt file, but I don't think the answer is so simple.

This is a long time dilemma and brings mostly copyright issue. Up to now no lawsuit is made, but it is a matter of time IMO.
There is a good article from last year which deals with the issue:
http://news.com.com/2100-1038_3-1024234.html

[MHenscheid]

Great article activeco! I think it covers a lot of what we have been talking about.

[Tubby]

Originally Posted by MHenscheid

Great article activeco! I think it covers a lot of what we have been talking about.

Not exactly;

what if the possibility existed for one search engine to offer higher rankings and guarenteed daily search, for an information site that it considered was of high value. what if it could exclude the opposition. Creat its own Portfolio . .
What if Google offered you a rating of 8 for all your information pages - in exchange for exclusive search rights.

Not that I can think of many sites that would get an offer like this, but we are talking concepts here. Do we realy need 150,000 results for a search of '1957 Chevrolet parts'? why would it not be a concept worth considering that a search engine could gain exclusive rights to a highly relevent site, why would it not be an advantage for search engines to seek out a better portfolio than the oppositions. Why would it not be an advantage to disadvantage new search engines from accessing site that users are looking for. Disable the opposition by fair means or foul. . This is not an uncommon business practise in industries throught the world, Why is it so unthinkable in this forum.

This may be a silly concept - or it may not be, but looking at what is, and pooh poohing a new thought is not good conversation.

Why isn't it silly that the owner of intellectual property should have limitations and restrictions placed. . . I can place an ad on the net and specify that I do not want American users to view the ad. . this is acceptable.

It is not a logical argument to presume that for all time - owners of intellectual property on the net will always want every bit of traffic from every source.

[TheWebDoctor(tm)]

Ok, here's the real problem and it needs to cease.

Graphics and image harvesting for the presentation on a search engine for commercial interests.

Where in the free domain rights does a search engine's rights begin? Why should a site owner have to opt out of having their graphics and images harvested, commercial profit made from it, and copyright infringements?

Anyone that says the search engines have the right to harvest your copyright protected art is insane. The search engines are harvesting your artwork in an attempt to gain more visitors to their services and thereby increasing their net value. I want my fair share of that billion dollar pie Google now has - not to mention the other billions of dollars Yahoo and the other search engines have from their advertising revenues generated from pages my graphics and art is on.

Talk about theft not to mention the resources used to provide the graphics to the search engines.

[Tubby]

I do recall getting a Birthday card made on microsoft publisher from a younger member of my family, on it was a photo of a Jaguar, I asked where the photo came from and I was told 'google'

I love google. but there are holes in the system. I do not suspect that google took the photograph. I do suspect whoever did take photo supplied it to my nephew for nothing. and I suspect there was an ad or two seen by my nephew while collecting the picture.

There is room for improvement in the system.