I don't mind being crawled repeatedly, even by bots that produce little result--provided that the spider is associated with some legitimate engine.
These are the bots I allow:
Google, Yahoo/Inktomi, MSN Search, AltaVista, AOL Search, AllTheWeb, Lycos, Compass Communications Inc, Excite, Fast Search Inc, IBM Almaden Research Center, iWon, LookSmart, Naver, Overture, SurfWax, WiseNut, InfoMinder, Walhello, Alexa.
All others are blocked.
A bot database and .htaccess deny list is here:
http://hometown.aol.com/botlist22/botlist.txt
There are many others but I maintain that myself.
Andi