View Single Post
  #17 (permalink)  
Old 08-11-2006, 08:54 PM
jrb@w3f.com jrb@w3f.com is offline
WebProWorld Pro
 
Join Date: May 2006
Location: Arizona, USA
Posts: 194
jrb@w3f.com RepRank 0
Default Re: Robots.txt

Quote:
Originally Posted by computergenius
Assuming that you want all robots to access all pages, just create a blank file, called robots.txt, and put it in your root directory.

Complicated, huh? <G>

Alternatively, put the following two lines in the file, for the same effect.
User-agent: *
Disallow:
Yes, then go to google, and look up and read how to handle your robots.txt. I has a significant effect. It is also lengthly and complex. Too much for a reply here. In the mean-time, use the example above.
__________________
http://MADEinUSA.org and others.
Reply With Quote