Google and Reading Of Server Stats

[citypublife]

Sometimes I feel confused with my weekly server stats. Could somebody please explain in plain English what the following means in relation to Google and how should I utilise this information?

Listing the top 20 organisations by the number of requests

• 616: 0.80%: googlebot.com

Listing the top 50 hosts by the number of requests

• 169: 0.13%: crawler14.googlebot.com
  102: 0.17%: crawler11.googlebot.com

Listing the top 30 referring sites by the number of requests

• 1095: http://www.google.co.uk/
  636: http://www.google.com/
  39: http://www.googleadservices.com/

Also, this one has confused me for a while:

Listing the top 30 files by the number of failed requests

• 124: /robots.txt

Would appreciate if anyone can decipher for me. How can I tell how many hits my site gets from Adwords?

www.citypublife.co.uk

[minstrel]

Originally Posted by citypublife

Listing the top 20 organisations by the number of requests

• 616: 0.80%: googlebot.com

This means googlebot has visited your site - number of requests means how many files it requested, but that includes html pages, images, scripts, everything...

Listing the top 50 hosts by the number of requests

• 169: 0.13%: crawler14.googlebot.com
  102: 0.17%: crawler11.googlebot.com

googlebot has more than one crawler - two fo them have visited you...

Listing the top 30 referring sites by the number of requests

• 1095: http://www.google.co.uk/
  636: http://www.google.com/
  39: http://www.googleadservices.com/

These aren't spiders - these are hits from people who found your website from Google, Google UK, and Google Ads...

Also, this one has confused me for a while:
Listing the top 30 files by the number of failed requests

• 124: /robots.txt

It means you don't have a robots.txt file and Google always looks for one to see whether you have any restrictions on which files or directories you don't want indexed. The fact that you don't have one means Google will crawl the whole site.

You can get rid of this entry by creating a robots.txt file and uploading it to your root directory.

In notepad, create a file with the following lines:

User-agent: *
Disallow:

This says "all robots should index my entire site".

Save it as robots.txt and upload it to the root directory on your server.