Hi All,

I noticed that Google bots and Yahoo bots uses very much traffics on one of my site.

For example:

In the Month of May:

Googlebot uses 30 GB
Yahoo Lurp uses 11 GB

while in the month of May it was Googlebot 12.04GB Yahoo 6GB.

I don't want to waste my bandwidth unnecessary. Can anybody tell me how I can control on it or what can be done to reduce it?

I think by using robot.txt file we can restrict any search engine on the site and do control increase & decrease crawling rate but apart from this is there any other possible way to control on Google bots & Yahoo.

Please do assist me in this regard.