Tag: search

Disabling Google and Other Search Engines From Crawling a Site

Reader question: I have a online database of horror movies, and I have a good Google rank. In my traffic logs I noted the last month a really growing of the bandwidth: one of the most important browsers of the server logs is Googlebot, so this traffic was generated for the spidering engine of Google. I have the 20 Gb bandwidth limit and I don’t want to pay for excess, so I disable Google into my Web site. My question is:

The Problem with Site MatchTM : Creeping Commercialism Infiltrates Pure Search Listings

For over a year now, Yahoo has spent a lot of money acquiring assets that will allow them to grab market share from Google. They bought Inktomi, which is feeding search results to the Microsoft network, and they bought Alltheweb, and AltaVista, and Overture. Last month they unveiled plans that will integrate these assets. The most controversial of these is Site Match, which involves embedding paid links into the main index. There has been very little comment about Site Match from the noncommercial sector. Almost all of the grassroots discussion revolves around whether the typical ecommerce webmaster can benefit from the Site Match model, or whether he would be better advised to rely on free crawling and indexing. The commentary from the pundits is even more shallow, and discusses whether Yahoo has the right stuff to crush Google. This article, on the other hand, looks at Site Match from the perspective of a nonprofit webmaster.

Back To Top