Old Pages

[cppgenius]

We all know any active website changes a lot over time. You remove pages, rename pages and add pages. However adding pages is never a problem for visitors or search engines, but I am not quite sure what is a good guideline to follow when it comes to removing and renaming pages.

Last year we changed the extension of our pages from .htm to .php. We implemented the proper redirects and links from the old pages to the new ones. Once the search engines picked up the new pages and indexed them, we removed the old ones. However I recently studied our error log and found that we still receive several queries a day for our old pages, including pages that had been removed on previous occasions (like products, articles and information we no longer publish on our site).

My question is, how long should you keep the old page on your site before removing it. What I actually mean by saying 'old page' I do not imply an exact copy of a page, located under a new name elsewhere on our site, because that will raise duplicate content concerns. These old pages only include a 301 redirect, or where 301 redirect is not possible a meta refresh tag and in all cases a notification on the page containing links about the new location.

We will be looking at upgrading our hosting package to make use of custom error documents, but in the meantime we want to upload some of the pages that we removed in the past, each consisting only of a notification that the page had been removed and supplying a link to our homepage. Some of these pages are no longer relevant to our website, so is there any sense in uploading a notification about these pages also, because that may lead to un-targeted traffic and we do not want the search engines to see it as spam. On the other hand, who says these visitors will be un-targeted, they might see something interesting on our homepage and convert into a customer.

[wige]

301 redirects are the recommended method for moving pages, because it tells the search engine to stop looking at the old location. A 302 redirect tells the search engine that the page has moved, but may return to the original location. A 410 message tells the search engine that the requested page has been removed and will never come back, but since most webmasters have never heard of 410, much less know how to implement it, search engines may not handle this properly. A meta redirect does not tell the search engine anything, and may or may not be followed properly.

301 redirects, as long as the page has no links pointing to the original location, should probably be removed six months after the last link pointing to the old location has been deleted. Search engines should only check a few times before removing the URL from their database. However, this is how things "should" work, not how they do. I have had a few pages redirected with 301s keep getting spidered because a certain search engine never updates their index, and others where I was unable to get a link to the old URL changed.

In theory, search engines don't like sites that have a lot of broken links (the spider could see a lot of error 404s as inefficient, and reduce the crawl rate as a result). This is mostly an unconfirmed rumor though. Customers, on the other hand, hate landing on an error message, and if there is a chance that an old forgotten link somewhere on the web could take the user to a non-existent page, they should see a friendly error message that gives a link to the correct place or even better an invisible redirect.

[incrediblehelp]

Originally Posted by cppgenius

However I recently studied our error log and found that we still receive several queries a day for our old pages, including pages that had been removed on previous occasions (like products, articles and information we no longer publish on our site).

Even if the pages were removed the redirects should still handle the requests without any issues.

Did you remove the 301 redirects along with the old pages? If so put them back.

[pablowerk]

Yea I would agree with incrediblehelp leave the 301's up. Even after the SE have properly crawled them you more than likely will still have old visitors that may have bookmarked those old pages.

I updated my site a couple of years ago and have left all the 301's in the code. I still receive hits to these old pages.

[MarcThai]

A simple way to ensure everyone finds the pages you renamed from HTM to PHP is to enter the following in your .htaccess file:

ErrorDocument 400 /400.php
ErrorDocument 400 /400.php
ErrorDocument 400 /400.php
ErrorDocument 404 /404.php
ErrorDocument 500 /500.php

Just define the document and then define where the equivalent can be found.

[GunRunner]

Can you put an example of the 301 code that you used? Thanks

[Peter (IMC)]

Leave the redirects up. But do the redirects using the .htaccess file in stead of some code in the pages. It's most likely that those requests come from links pointing at the old pages and it would be a waste to lose the "link juice" from those links.

There are more sources of visitors than just the search engines,.

[netabilitysue]

YES, keep those 301 redirects there forever! Why not? Getting rid of them costs traffic and loyalty.

As someone who owns/edits a directory of Websites (with over 9,000 unique links to job search resources), I get very annoyed with people who change their sites without doing permanent redirects.

Please notice that you have other sources of traffic and visitors in addition to search engines:

What about all those bookmarks/favorites set by previous visitors?

What about those e-mailed URL's to check out this great site?

What about my PR 7 link to your site? Plus, all the other links to your site?

People keep trying to buy my "high value links" (NOT for sale). Yet, EVERY month, several hundred of the sites that I GIVE them to, throw them away by changing the URL of the page I link to. ARRGH!

Yes, I run a linkrot checker (LinkAlarm - highly recommended!) every week, and, yes, LinkAlarm indicates which are permanent redirects. And we update those links, but they are less urgent maintenance issues than the newly dead-ended links, and that is greatly appreciated.

BTW, the sites that can figure out how to map from the old pages specifically to the appropriate new pages are greatly appreciated because they are nicer to their visitors AND have made my research into the new URLs much easier.

Dumping everyone onto the new home page is better than a 404, but not by much.

End of rant.

[Orion]

Here's the .htaccess code for 301 redirects...

### redirects ###

redirect 301 /page_name.ext http://www.domainname.tld/new_page_name.ext

When I'm going to replace or remove a page...

I create the new page
upload new page
check site for any links to old page update to new page
check sitemaps and update (google yahoo etc.)
search web for IBL's to the old page... contact them to update links to the new page
then add 301 to site
remove old page..

usually when you make a change to a page name or remove a page you lose a bit on the rankings. It does come back, but I've found the loss and time to recovery tends to be a bit less if I leave the old page up until all the other things are done.

Hope that helps!

[Aquahobby.com]

How bad is it if there are still links pointing to the old page on your site, together with links pointing to the new page? I've recently done some 301 redirects and updated the wrapper links, but there are still hundreds of links in the actual content of my pages that need to be updated individually...

[brandrocker]

What's the harm in keeping 301 redirects? It will be particularly helpful for those visitors who trust and follow their bookmarks.

[wige]

Originally Posted by Aquahobby.com

How bad is it if there are still links pointing to the old page on your site, together with links pointing to the new page? I've recently done some 301 redirects and updated the wrapper links, but there are still hundreds of links in the actual content of my pages that need to be updated individually...

I think you lose some of the rank value of the incoming links. One posting I saw indicated that Google didn't pass pr through the 301 or 302 redirects, but they do follow them to ensure the pages are indexed. I don't think that is the case, because there are a few Google hacks that are known and officially acknowledged by Google that allow you to manipulate the displayed URLs in the SERPs by creative use of redirects. But, it is possible links to 301/302 redirects do not carry the same weight as a direct link. Part of the equation is apparent trustworthyness, and how trustworthy will the SE consider a moved page with the links not updated (or since you are dealing with internal pages, pages that link to redirects).

[cppgenius]

Originally Posted by netabilitysue

Please notice that you have other sources of traffic and visitors in addition to search engines:

What about all those bookmarks/favorites set by previous visitors?

What about those e-mailed URL's to check out this great site?

Originally Posted by wige

Customers, on the other hand, hate landing on an error message, and if there is a chance that an old forgotten link somewhere on the web could take the user to a non-existent page, they should see a friendly error message that gives a link to the correct place or even better an invisible redirect.

My point exactly. The links from articles you wrote, that got syndicated with the old link, directory listings or link exchanges with websites who do not respond to requests to update the link, discussions on forums or blogs containing one of your old links, all made me realise that I need to keep these pages up and running, not for the sake of search engines, but for the sake of visitors. I am a bit concerned about loosing page rank, but hell, you can always try to improve your page rank later on, but you cannot regain lost traffic due to an unfriendly "Page Not Found" error, a page that could have cost you a client.

Originally Posted by wige

I have had a few pages redirected with 301s keep getting spidered because a certain search engine never updates their index, and others where I was unable to get a link to the old URL changed.

Like MSN (and a couple of our local search engines here in South Africa)

Originally Posted by incrediblehelp

Did you remove the 301 redirects along with the old pages? If so put them back.

Originally Posted by Peter (IMC)

But do the redirects using the .htaccess file in stead of some code in the pages.

Here is the code I'm currently using:
header("HTTP/1.1 301 Moved Permanently");
header("Location: http://www.newurl.com/newpage.php");

It seems like a lot of people prefer redirects via .htaccess instead of using a script within the page. I will keep that in mind, thanks for the tip.

So it seems pretty clear, I should keep those redirects up, thanks for the replies.

[MK]

I guess I'm a slob. I have a cute design for 400 and 404 errors - it has links to my site homepages. Otherwise, I don't care if people get errors. They know how to skim back the url in the bar to the homepage and start all over again. The problems are from ancient links on sites that don't bother to update their info, anyway.

[wige]

Originally Posted by MK

They know how to skim back the url in the bar to the homepage and start all over again.

No, they don't. Usability testing, and several years working in technical support has shown me that the average Internet user does not hack URLs; even if the user knows what the text in the address bar means, they will prefer to use the back button to go to a page they know than use trial and error to try to find what they are looking for. The typical consumer on the web wants to find what they want as quickly and easily as possible, without having to hunt around or edit URLs to find it. There are some exceptions to this... more technical users may be more comfortable hacking a URL, for example.
In addition, error messages can erode confidence in the web site. Users do not have much trust on the web. The faith they have in a site depends largely on how well they know the company that the web site represents. If you are a small business that the customer has never heard of before, the user is already coming to the site with a low level of trust, and each error that the user encounters reduces the chance that that user will place an order. A simple redirect or a "smart" error page can either prevent the loss of trust or recover the lost trust and keep the visitor much better than a generic error 404 message.