Quote:
|
Originally Posted by mawells
If I were to put a robot.txt "no index" tag on the form page - do you think it would cut down on these spammers since the page would not be included in the search engines?
|
Ethical, law-abiding organizations choose to follow the suggestions laid out in your robots.txt file. The ones you're dealing with clearly choose not to -- and never will.
Said differently: robots.txt is not a replacement for security. Think of it as a sign in your front window that is read and observed only by good neighbors. Criminals, no-gooders and others will ignore it or even use its information against you.
Use it as intended and for nothing else.