Re: How to stop Google from crawling secure content/directory?
Hi
All good advice has gone before me.
In case you are taking the robots.txt route, best to use the meta tag as well. If a link somehow exists to a page, or is created by accident, that could still be crawled regardless of robots.txt.
|