Here I go again, another question.

[LiviLou]

What is robot text and how does it work? I went to the meta tags analizer page and had the free evaluation done. It came back that my site was not robot friendly. I have no idea what that means, how to change that or even if I should. Everything else seemed to check out fine.

Comments - Suggestions?

[carbonize]

A robots.txt file is a file that all search engine spiders have agreed to read and obey. The file is based in your domains root directory and tells the robots where they can and cannot go. You can even specify different rules for different bots.

http://www.searchengineworld.com/robots/robots_tutorial.htm explains it in more detail and has some examples you can use.

[le_gber]

The robots.txt is a text file in which you tell the search engine spiders/robots what files, folder/directories they should ignore. It is the first file that robots/spiders should 'hit' when arriving on your website.

This text file goes on the root level of your website tat is on the same level as your homepage.

ex:

User-agent: googlebot
Disallow: /cgi-bin/

in this example I tell googlebot to ignore the cgi-bin directory.


User-agent: *
Disallow: /

In this example I tell all robots (the wildcard) to completely ignore my site (/ stands for all files/directories that are included in the root folder).

If you want to disallow all robots to go in one of your folders use

User-agent: *
Disallow: /somefoldername/


For more information about robots.txt visit: http://www.robotstxt.org/

Hope this helps

Leo

[LiviLou]

Yes, actually it did. Thanks for answering.