Stats show excessive hits to error page (but not 404)

[cronic]

Hi all, niggling problem ,...

I have awstats on all of my client sites so am no stranger to how it works or what to expect. My main client site is www.naturalwayhealth.co.uk which does ok, I spend a lot of time looking through the stats of all my sites and this problem is one site only.

The worrying stat is the most hit page on the site is our error page, however there are no reported 404's or broken links. So this makes me think people are physically typing in the wrong addresses but that can't be right looking at the figures:

We get aproximately 5000 hits a month.
Most hit page is error.php veiwed 23301 times!!
HTTP Error Codes show no problems at all.
2nd most hit page is the index with 1229 hits.

I wish we did get 23301 hits to the site!

So what could be causing this anomoly? I suspect it is a robot or something as I'm sure the visitor stats are accurate; Google passed 2861 visitors over and thats our biggest income of hits. I do use htaccess a lot for moving pages around so thats a possible culpret but still 23301 visitors is not realistic.

Any ideas would be appreciated as this bugs me!
Cheers,
Chris

[incrediblehelp]

Well do you have page names changing all of the time?

Did you recently go through a re-design?

Do you have products coming off of the website all of the time?

[modrewrite]

It sounds like your error.php is not sending back the 404 (http://www.askapache.com/2007/htaccess/apache-status-code-headers-errordocument.html#status-404) response code.. if it sends a 200 (http://www.askapache.com/2007/htaccess/apache-status-code-headers-errordocument.html#status-200) OK response than google has real issues and it can really damage your ranking.

If that is your problem than add this to the very top of your error.php file so it looks like this


<?php
ob_start();
header('HTTP/1.1 404 Not Found');
header('Status: 404 Not Found');

[modrewrite]

You might really want to start using a simple Google Analytics trick (http://www.askapache.com/2007/seo/tip-google-analytics-404-error-page.html) that tells you the page that couldn't be found and where the link was referred from.

So instead of
50000 /error.php
you get
1 /error.php?notfound=therequest&camefrom=referrer



So your code becomes this (but only on the error.php file)

<script src="http://www.google-analytics.com/urchin.js" type="text/javascript"></script>
<script type="text/javascript">
_uacct = "UA-732133-x";
urchinTracker("/error.php?notfound=" + _udl.pathname + _udl.search + "&camefrom=" + _ubd.referrer);
</script>

[cronic]

Thankyou for your replies :)

Yes I've got a lot of htaccess 'permanant redirects' from old shtml pages to new php pages etc. These all appear to be working fine; Google picked up on them ages ago but I've left them for slower search engines like yahoo/msn etc.

A friend has pointed out that my htaccess had a full path for the error page like this:
ErrorDocument 404 http://www.mysite.co.uk/error.php

That gave a 302 response, I've now changed it to:
ErrorDocument 404 /error.php
And now it gives a proper 404 response.

Hopefully the stats will reflect this change and problem solved. But it still doesn't explain why the stats claim the error page has been hit 23000 times when the amount of traffic through the entire site is about 5000.

Progress .. Thanks again!