Greetings to all, since this is my first post in this forum.
I noticed a strange behavior of the googlebots some days after I redesigned a website (maybe an unrelated coincidence, though). The bots don't spider most of our pages anymore. They spider
- plain html
- plain shtml which include php scripts
- plain php scripts
They *used* to spider .html pages, which we generate by php and turn them via mod_rewrite into .html
Unfortunately, rewritten .html pages don't get spidered anymore ...
What did we change?
- We switched from non-validated HTML 4.01 to validated XHTML 1 transitional, replaced TABLES with DIVs, made the whole site better accessible.
- We added mime-type-related caching via mod_expires.
- No URL change, no content change, no change in link structure.
The site is http://www.wiesentbote.de (it's a local online newspaper in german language), the PR is the same as ever.
The google bots used to spider all pages linked from the home page, which was fine, since we had about five to six different google spiders visiting the site several times a day. Since our page gets updated several times a day, we managed to get almost all articles of a given day into google - with a two-day delay. Now we do not get *any* article into google, which ist quite a setback.
Any ideas anyone?
Any help is appreciated.
Greetings from Germany,