Googlebot quit visiting. Pages missing!

Ed_S_1 · #1 (**permalink**) 03-08-2005, 03:03 PM

We noticed that a few of our key pages (including our home page) have dropped off of Google over the weekend.

I searched our access log files (going back to 2/7/05) and Googlebot is not in any of them.

However, Yahoo, MSN, Ask Jeeves are spidering away...

We didn't do any configuration changes recently.

At first, we thought that pages were being deleted because of some duplicate content ("why you should visit us" type stuff at the bottom) appearing on those pages, so we fixed that. But, then we noticed that Googlebot wasn't even visiting. :(

Are we worrying for nothing?

bhartzer · #2 (**permalink**) 03-08-2005, 03:26 PM

Is your site still in the Google index?

Without knowing specifics (put your website in your profile, please) I'm not able to comment any further.

Ed_S_1 · #3 (**permalink**) 03-08-2005, 04:06 PM

Our website is: http://www.babysupermall.com

We still have 20K+ pages in google. Used to be up around 30K. And, its the most popular pages that are disappearing...

It's almost as if googlebot can't get to our site (network issue), so it is deleting the pages as it tries to fetch them - Naturally, deleting the most popular ones first.

Ed_S_1 · #4 (**permalink**) 03-09-2005, 03:22 PM

It turns out to be a connectivity issue.

When I do a tracer (or ping) to a googlebot, it stops short (right at the bot). But, when I do a tracer from another server, it goes through.

Now I have to figure out how to get Google to fix the problem...

Wish me luck!

Ed_S_1 · #5 (**permalink**) 03-09-2005, 05:55 PM

Problem was with our web hosting company: Valueweb (Affinity)

They had blocked some traffic, thinking it would not affect their dedicated servers. But, it did!

Oops...

scubajared · #6 (**permalink**) 03-10-2005, 05:10 PM

I also use Valueweb for my hosting company for www.hansenhomesaventura.com. I just noticed the same problem. When I do a cache for pages that have been indexed and placed very high for keywords do not show up at all. Did you talk to Valueweb about this problem and if so what did they say?

Thank you

Jared Emin

Ed_S_1 · #7 (**permalink**) 03-10-2005, 05:24 PM

They were totally unaware of the problem when I told them. They fixed it around 3:30PM CST on 3/9.

They said they set a router to block some spider traffic to part of their network, but inadvertently blocked it to their dedicated servers as well.

We think the problem started on 2/28.

It may take Google a while to show your pages again as they re-spider your site. :(

It's a shame we didn't see it earlier. I spent a good part of the last few days beating my head.

You may want to check your access logs to make sure Googlebot is showing up.

You could also complain. The agreement doesn't have much in the way of recourse, but I think they can credit you for 3 months of service.