Re: robots.txt question / coldfusion
Extensions are not required. Exclusions in robots.txt are processed as text strings from left-to-right, so you only need to enter the string as far as needed to be unique - so if you wanted to block every page starting with name-of-, you can do that by just leaving off the last word.
__________________
The best way to learn anything, is to question everything.
|