Prevent hijacking:
http://www.loriswebs.com/hijacking_web_pages.html
You may take a look at my
http://multifinanceit.com/robots.txt
file.
I also have a htaccess.txt file,
http://multifinanceit.com/htaccess.txt
that is not implemented as .htaccess.
I also have another version
http://multifinanceit.com/htaccess1.txt
Recommended links:
http://techpatterns.com/downloads/spider_blocking.php
http://www.hostpronto.com/article/9/6
"Setting a Spider-trap
The best method of identifying bad bots is to create what is known as a Spider-trap. Create a directory, block that directory to all agents using robots.txt and link to the directory from a page (usually as a small 1x1 pixel link).
Only bad bots will access that directory (ie they've ignored our robots.txt exclusion). These bots can then be directed to a script that will immediately grab their IP address, User Agent or Referrer and add it to an .htaccess file - so that they're banned from the site".
Found this excellent
http://www.garykeith.com/browsers/downloads.asp
link where you can download files.
Why reinvent the wheel?
KBleivik
http://multifinanceit.com/
http://www.multifinansit.no/