robots.txt file Question

[watto]

I recently made some changes to my robots.txt file and I would like some feedback as to whether this is correct:

http://www.business-trader.com.au/robots.txt

All help is greatly appreciated.

watto

[caravan]

Hi Watto,

Have you tried running it through the Google Webmaster Analyse robots.txt Tool. You simply put in the contents of your robots file and a list of your websites url's and Google will show you how the file affects your website pages. It is an excellent way of debugging the file.

[watto]

Thanks for the tip!

Regards

watto

[incrediblehelp]

Watto their is no right or wrong for robots.txt. Just whatever works for your website.

[watto]

Thanks incrediblehelp. I did realise that robots.txt files are different and a robots.txt file needs to be customized to your site, I really just wanted some feedback to make sure I had used correct coding.

Cheers

watto

[incrediblehelp]

I would also add:

Disallow: /cgi-bin
Disallow: /_private
Disallow: /_vti_bin
Disallow: /_vti_cnf
Disallow: /_vti_log
Disallow: /_vti_pvt
Disallow: /_vti_txt

[watto]

Thanks for the tip!

Kind Regards

watto