Search Engine Bots

[wclew]

Let me apologize in advance, this is probably a stupid question for most of you. I read a post on here somewhere where a person said that the Google Bot had not been back to there site in ?? days. How can you tell when the Bot was last on a site? And, does that meta tag work that tells the bot to revisit in x days?
Thanks in advance,
Dennis

[jackson992]

I personall look at my latest visitors to see when the googlebot has been by:)

[minstrel]

How can you tell when the Bot was last on a site? And, does that meta tag work that tells the bot to revisit in x days?

As jackson992 said, look at your server logs to see how often googlebot is visiting. As for the "revisit after" meta tag, it's usually better to omit that - let Google determine when it comes back, which will be according to how often the content on your pages changes and reportedly the PR of your site.

[wclew]

As jackson992 said, look at your server logs to see how often googlebot is visiting.

OK, another stupid question. How do I look in my server logs?

[minstrel]

Hi, wclew:

Most hosting services will provide you with a URL where you can view your server logs. You might try contacting your host and asking them how to do that - you can also try simply FTPing to your site and look for a subdirectory on your site called "stats" or something similar. In that directory, you will probably find some html pages, which are assembled by whatever log-parsing software your host uses. If you don't see anything like that, call your host and ask them what they do provide and how you access it.

[wclew]

Thanks Minstrel. I checked using FTP and saw nothing relating to stats, I will email them to find out.
Thanks again,
Dennis

[vilesilencer]

wclew,

there is another method if your already in your ftp client. I only know how to do it via PHP though, and it does involve some familiarity with your FTP client and CHMOD commands.

PM me if your interested.

The output I get from my personal site is like this, based on a PHP file i have pulling variables off the visitors. It's a pretty common way of grabbing useful info from visitors:


Date: January 25th 2004, 10:09PM
IP: 64.68.82.170
Agent: Googlebot/2.1 (+http://www.googlebot.com/bot.html)
URL: None
Referrer: None
Username: None

I've never used the "revisit after" tag and googlebot visits at least once a day, sometimes up to 20+ times a day from different IP addresses, so I don't think that meta tag has anything to do with the bots visiting.

[wclew]

wclew,

there is another method if your already in your ftp client. I only know how to do it via PHP though, and it does involve some familiarity with your FTP client and CHMOD commands.

PM me if your interested.

The output I get from my personal site is like this, based on a PHP file i have pulling variables off the visitors. It's a pretty common way of grabbing useful info from visitors:


Date: January 25th 2004, 10:09PM
IP: 64.68.82.170
Agent: Googlebot/2.1 (+http://www.googlebot.com/bot.html)
URL: None
Referrer: None
Username: None

I've never used the "revisit after" tag and googlebot visits at least once a day, sometimes up to 20+ times a day from different IP addresses, so I don't think that meta tag has anything to do with the bots visiting.

Could you tell me more about this? What does it require to install this? I use WS_FTP for ftp software.
Thanks,
Dennis