robot.txt file

[midiman52]

Hello
Just found out my webhosting company has blocked my robot.txt file.
When i type my website address /robot.txt into search it redirects to their company.
Having checked around the internet i came up with some information that said you have to manually create this file to your root folder.
Bit of a novice can anyone tell me where i find the root folder.
It also mentioned that without a robot.txt file spiders can never scrawl your website. Is that true?

Thanks Roger

[incrediblehelp]

Search engines will always look for a robots.txt sand if they don't find one it will generate a 404 error in your logs. No fear they will still crawl your website as long as other websites point links to yours.

It is just good practice to have one, but not absolutely necessary to rank.
Just upload a basic robots.txt file to the root of your server. Your hosting company should give you support on how to do this. If they don't consider switching hosting companies.

Here is the robots website with more help:

The Web Robots FAQ

[idansh]

How can his host prevent the robots.txt file it seems like not real, every host in the world knows the importance of robots.txt file and it usage

[amar]

In your post your mentioned that you are checking for robot.txt.

Are you checking for robot.txt or robots.txt .

The correct file name for robots file is is robots.txt.

[edhan]

Originally Posted by amar

In your post your mentioned that you are checking for robot.txt.

Are you checking for robot.txt or robots.txt .

The correct file name for robots file is is robots.txt.

Yes. Amar is right about robots.txt. Anyway, if you are thinking of creating it, you can simply upload the robots.txt (after creating) to your root directory. And it should work as you can check your log file. Without it, your logfile will only show as 404 (as mentioned by incrediblehelp).