Originally Posted by josephx

Hello everybody,

I have submitted my site to a search engine a few months ago, but lately, unknown robots eats up my 2.5 GB Bandwidth in just about 10-20 days. I only have about 2 short videos and about 2-30 images on my site, so there's no reason why it would eat up that amount of bandwidth.

I have checked my access log, and found out:
38.99.13.123 - - [23/Jun/2007:20:44:45 +0900] "GET /t/imagery/ HTTP/1.0" 404 17507 "-" "Mozilla/5.0 (Twiceler-0.9 http://www.cuill.com/twiceler/robot.html)"

61.135.162.52 - - [23/Jun/2007:20:44:53 +0900] "HEAD /folder/folder/folder/content.html HTTP/1.1" 200 0 "-" "Baiduspider+(+http://www.baidu.com/search/spider.htm)"

They were crawling on my site every minute!

May I know how to block these robots using the .htacces? Please let me know if this is not the right forum to discuss about this issue.

Thank you.