View Single Post
  #5 (permalink)  
Old 01-02-2008, 04:42 PM
Palindrome Palindrome is offline
WebProWorld Member
 
Join Date: Oct 2007
Posts: 51
Palindrome RepRank 0
Default Re: How to stop Google from crawling secure content/directory?

Hi

All good advice has gone before me.

In case you are taking the robots.txt route, best to use the meta tag as well. If a link somehow exists to a page, or is created by accident, that could still be crawled regardless of robots.txt.
Reply With Quote