Crawler Alert: Instant Notification When Google Visits

Garrett · #1 (**permalink**) 05-21-2004, 04:55 PM

If you want to know the instant Google's bot visits your site then you're a severe algoholic. And Crawler alert will help you get your fix. Crawler alert will send you an email when any of the major search bots visit the page you specify.

You will have to place a file on your site along with a link to this file on your front page. When crawlers roll through your site they call up the content from the file and the Crawler Alert server checks the crawler signature and compares it to the crawlers for which you wanted notification.

The crawlers it checks for includes the biggies like GoogleBot and Slurp on to the lesser known bots like KIT_Fireball by Fireball, and NetScoop by NetScoop.

The creators of the service run a SEO firm in Israel and received regular requests from their clients with new content who wanted to know if they had been crawled yet. In their words, "we decided that it could really be a useful tool for every web site owner or a web design/search engine optimization (SEO) firm that works with customers, so we opened it for the general public."

If you'd prefer not to dig around in your log files, or if you're too busy to monitor them, the Crawler Alert service might be for you.

I didn't investigate the file they have you put on your server though, or any possible link benefit they may receive from having the file pointing to their server.

Their privacy policy states, "the information you provide Crawler Alert will be used only to provide you the Crawler Alert service." I haven't seen any of these alerts - there may be some advertising for the Crawler Alert creators.

If you'd like to try it out visit Crawler Alert.

danbroughton · #2 (**permalink**) 05-24-2004, 12:08 PM

Just signed up and added to my site - waiting for the emails to flow in!
Just a quick tip - add a text link back to your homepage in the php page you download from their site - stops PR drain .. it might as well go somewhere.

chris_g · #3 (**permalink**) 05-26-2004, 12:26 AM

We usually monitor search engine activity through the log files and a robot manager but this service makes it nice when getting access to log files is difficult. What I do like about this service is it’s something the website owner can monitor on his own if they are so inclined.

Is there a way to see what page the particular spider visits?

I only can hope that people don't get obsessed with spider visits like they do about page rank.

kokopoko · #4 (**permalink**) 05-26-2004, 02:49 PM

Wow. That's pretty cool. I think I'll try it.

Hurga · #5 (**permalink**) 05-27-2004, 02:14 PM

How often does slurp usually comb a site? I haven't really seen it in the past month. MSN's been going crazy over the last week tho.

bhartzer · #6 (**permalink**) 05-27-2004, 02:34 PM

This service sounds suspicious to me, especially since the website is currently banned in Google and has a PR0.

Just because a site is banned in Google and has a PR0 doesn't mean that it's not a good service, though. That's for you to decide. But Trust me, I'll be staying away. I'd rather check my web stats program.