Discontinued products / broken links

[amxfan]

I have two questions here that I would really appreciate some info on.

First Question

On an e-commerce site, when a product becomes discontinued what is the best way to handle this to prevent broken links?
I run a few different e-commerce sites and about once a month different products become discontinued. I quickly remove them from the site and update my site map, but Google, and I guess other search engines, still crawl these pages and Google reports them as broken links in webmasters tools. What is the best way to handle this? I have tried several to help me out with this, including setting a redirect back to the main page on a 404 error. This worked rather well, but Google did not like it as it reported a status 200 and not a status 404. This was quickly removed. I have also tried to leave the pages, but removing all links to them. This also worked until customers who either followed a cached link or a bookmark, and placed an order. Going into the cart and putting out of stock or no longer available for every product can be overwhelming, so removing the pages is the easiest, but I am getting Google reporting broken links.

Question two

With the popularity of wide screen monitors what is the best resolution to design a web site around? I have seen sites that auto-size and look fine on a standard monitor but when viewed on a wide screen monitor look bad. Pictures become short and fat, text becomes spaced out, and white space becomes overbearing. What is the best size to create a site to so it displays properly on wide and standard monitors and also TV's? As of now I create with a fixed size of 1000px x 700px. What is the best size? The sites that looked bad when viewed from one monitor to the other had scrollbars disabled. I also noticed a major difference when viewed on a TV as some TV's try to stretch the size to fit the TV. This is mostly seen when viewed through a video out port to a TV.

[wige]

As far as the first question, you could add a 410 Gone message, or use a 301 redirect to send the user back to the search function of the site. The 301 method would be better from an SEO perspective, because it keeps any link weight from the product active in your site. A 410 Gone message should cause the page to be deindexed quickly, but you lose the benefit of any links to the page.

Based on your description, it seems that you have created a page for each product, which you delete when the product is discontinued. To redirect traffic to pages when removed, the easiest method is to create a PHP file in the root of whatever directory contains your product pages. If there are subdirectories of products as well, that is ok, this will still work. The PHP file will contain the following:


<?php
header('Location: http://www.yourdomain.tld/folder/page.php',301);
?>


In the same folder, create an .htaccess file with the following line:

ErrorDocument 404 nameoffile.php

where nameoffile.php is the name of the file you created above.

[amxfan]

Yes you are correct in that each item has it's own page. These pages are not being reached by onsite links or a search function within the site, but rather they are being reached by indexed links from Google, bookmarks, etc.
If I understand what you are stating right this would replace any custom 404 error page that I have already and the visitors would be redirected to a chosen page "more than likely the home page". What I'm unsure of is would this also not return a 200 status ok ?

When a visitor hits a page that is not longer there they would get redirected "301" to another page that does exist, like the home page which would return a status 200. This I have done but I did it on the server side, not with a script. Google did not like it because it returned a status 200 rather than a status 404. Is there a difference of using a script or doing it on the server side, or am I really misunderstanding what it is you are doing? My main issue isn't really visitors reaching these pages, but search engines reaching these pages. It seems although I submit a new sitemap, it takes time for Google to remove the old or drop links that are no longer existing. Also having links from other sites that happen to pick up on a feed, that the search engines follow to get to a page that no longer exist.

I do feel what you are telling me to do will work, and will work well. I'm unsure if Google would see it as a status 200. That is where I am confused.

[cw1865]

As to your first question, with respect to products that I am no longer offering I have done one of three things:

1. If it made sense to simply offer a similar product, I would simply change the page and keep it.
2. If that wasn't an option, on my old system, I would simply sever the link and I would keep the page.
3. On the new system which is database driven, I simply 'hide' the product but keep it, which is the same as severing the link to it.

You could of course do a redirect.

Like yourself I saw in the logs that google was trying to crawl pages that didn't exist. On one site, which I picked up off of the expired list, I saw that google was trying to crawl pages from the previous owner and I went out of my way to build content for those urls it was trying to crawl, figuring that if google was going there I may as well put something there for it to crawl and cache!

[Jean-Luc]

Regarding you first question, my recommendation is to go for pure 301 redirects. This is the best thing to do from a marketing prospective: when a customer looks for a non-existing product, you suggest him another product that hopefully meets his needs.

The best way to do the 301 redirect differs from system to system. Sometimes you can automatically handle this within your e-commerce application, sometimes you can manually do it in the .htaccess file and there may be many other possibilities depending on the details of your environment.

wige: in my logic, your combination of a 404 with a built-in 301 as you suggested should not give the expected result. Is there a flaw in my logic ?:confused:

Jean-Luc

[PaganIL]

I would put something in your code that disables the order buttons and puts a large red lettered sold out or discontinued text on the page. I would assume you are using a dynamic language of sorts, so this can be checked agaisnt he record for the item. If an item is marked as discontituned or sold out, this can automatically happen and you are not having to update an individual page...it would all be handled by whatever admin tool you are using for your inventory. You can also dynamically insert the redirects mentioned above so that the page is still there and just insert them if the item is no longer in your inventory...again, this makes the process of doing this dynamic so all you have to do is mark something discontinued as you normally would and then you are done.

Eric

[PaganIL]

For question 2...

You really have to program for the lowest common denominator. 800x600 was the industry standard, but i think that has been slowly getting bumped up to 1024x768. I wouldn't go higher than that because most people are going to be viewing the site on a monitor, not a widescreen TV. To keep the pictures from getting stretched, I would set the image size int he tag (and also keep all images the same size or if they have to be different, specify the size in the database so the size can be set appropriately for each image that needs to be different (product pics would be the onloy ones i can think offhand that really would use this. most of your images will probably be of a static size.)

Eric

[heavener]

Regarding your second question on screen size/resolution, I still believe that 800x600 is the de facto standard. That means almost anyone can use it -- and if your tables dynamically reallocate, it'll fill any sized screen.

I am forced to use 800x600 on my computer that powers the video projector -- and, yes ... I do look at web pages on the big screen for a variety of reasons.

I have two new laptops that will not go higher than 1024x768, so that's my "rock-bottom outside limit" for designing bigger sizes.

And I work with a designer who insists that sites should be designed at 464x280 !!! Why? Do the initials PSP mean anything? Nintendo DS? Blackberry? Yup, people want their browsers wherever they are, and the more portable and multi-functional, the better.

Really, it's a "do your own thing" world -- your design always comes back to your demigraphics. Where are all of your potential customers? What screens and devices are they likely to aggregate toward? How many can you risk alienating by picking a screen size that forces them to scroll horizontally?

Michael - heavener@heavenr.com

[Steelchord]

I'd think that the 301 redirects coming from the page instead of the .htaccess file will do the trick; but it depends on your server and php version. Wiser heads than mine will have to expound on that.

The liquid design issue, though, I can pitch my two cents into. The fact is, there's a multitude of ways you can attempt to resolve this--css, or detection scripts, or flash-driven templates; but the fact is that people with large screens are aware of the fact, and frequently will have things set the way they like them to be. So program to what makes you happy, but there's no real need to be overzealous unless you're specifically targeting that market. Make sure your site follows best practices where accessibility is concerned (Web Accessibility Initiative (WAI) - home page (http://www.w3.org/WAI/)) and you'll be fine.

[amxfan]

I would like to thank everyone for replying, for every answer given gives me alot to think about. cw1865's suggestion of leaving it on the server is just one of the possibilities. I happen to like this as several of my product pages have similar or featured items listed on that page. Combining his suggestion with PaganIL's suggestion of disabling the add button I think is the way I am going to go, and leave those pages until Google drops them since they will no longer be in the site map and they will have no links going to them. My thinking is about after 3 months, then I should be able to get rid of those pages. I am sure the 301 redirects would also work, and work very well, but from a sales standpoint I am thinking if I redirect to a product that is similar and not the actual product, then my visitor did not find what they are looking for and leave. These similar items and featured items may work better as far as a higher conversion rate but I still may get a lot of visitors leaving to find a site that does not have it labeled discontinued. I am going to go this way because if I cannot provide the product to my customers, I am going to tell them so and let other sites get a bad rep for offering a product that has been discontinued.

As far as the second question goes, again thank you to those who responded. As far as the screen size goes, for psp, Blackberry and such, I totally understand why he would want his site designed on such a small size and scale. What I am unsure of is why wouldn't he have a .mobi and have his script display that site if being viewed on such a device. As of now I think my best bet is to stick with the 1024 X 768 standard as that is becoming the standard across the board and see where the industry goes with monitors, web TV's, etc. before I jump the gun and start redesigning. As far as mobiles go, I am tracking that I am getting sales and inquiries from these devices, but as of yet, the numbers do not warrant me to make a redesign and from what I have personally seen my sites display ok, they just have scroll bars.

Again thank you for all the right answers.

[maxsun]

We have the same problem--on a daily basis.

We removed hundreds of links of items that weren't selling right after the Holidays. Our logs did go down and so did sales--but we really blamed that on the recession not the removal of "dead weight".

First off--just logically, a "clean internet" is a "good internet". Delete the crap!

Our second thought was Google MUST have learned to deal w/this situation long ago. I doubt any of us is getting dinked too heavily for it. In fact, Google is probably happy when dead pages are removed--less CPU for them to index.

You might try setting the meta tags to "nofollow, noindex", setting your sitemap to Archive and then after a month or two, delete the pages if you're really worried.

Regards,

[Nick Spence]

So far as page size goes, we build to 800 wide, not because most people don't have a larger monitor, but because they didn't buy a larger monitor so that we could fill it up with a larger page, but so they could fill it up with multiple pages.

~Nick

[MarkGatESS]

I've run into the problem of Google continuing to index pages and subfolders that no longer exist. They're not in my SiteMap.xml nor do they exist on the web server, yet Google keeps trying to access (and give 404 Not Found errors) to these same pages over and over. Some of those pages were removed from the server over a year ago.

:confused: Should I take the URLs of those listed in the 404 errors on Google's webmaster tools and add them to my robots.txt file as "Disallow"?

The webserver/hosting provider our company uses doesn't allow me to access and/or create an .htaccess or any other server admin functions (I can't even get the FrontPage admin functions to work, even though the server is supposed to have FrontPage server extensions installed.)

[wige]

wige: in my logic, your combination of a 404 with a built-in 301 as you suggested should not give the expected result. Is there a flaw in my logic ?:confused:

Its a server hack. ErrorDocument simply tells the server what to do when a file is not found, letting you substitute the missing document with a custom document explaining the error, which exists at the requested URL, along with a header containing the 404 error code. The custom error document created in PHP overrides the 404 response code with a redirect response code.

User requests the missing page.
Server sees that the page does not exist, so responds to the request with the error document.
The PHP parser tells the server to override the "404 - Not Found" header with a "301 - Permanent Redirect" header
The user receives a redirect to the intended destination.

This allows you to create redirects as files are deleted without creating custom rewrite rules.

[beesknees]

MarkG - That's a bummer that you can't access or edit .htaccess. I've had the same issue with Google trying to find pages that don't exist anymore, and the 301 redirect is the SEO friendliest way to handle this issue. I don't want to sound snobby, but if it is possible for you to go with another host, I think that would be your best option. It's not right, IMHO, for a host to restrict you from accessing the files on your server that you are paying for, especially the ones that allow you to increase the security and functionality of your site.

If you need a recommendation for a new host, PM me, and I'll give you the name of my hosting company. They are small, inexpensive and they ROCK!

[Peter (IMC)]

I have two questions here that I would really appreciate some info on.

First Question

On an e-commerce site, when a product becomes discontinued what is the best way to handle this to prevent broken links?
I run a few different e-commerce sites and ....

Question two

With the popularity of wide screen monitors what is the best resolution to design a web site around? I have seen sites that auto-size and ...


1) Leave the product up there but make it no longer possible to add to the cart. Or use a custom 404 page so that the visitor will find a real page with the site menu and everything still available.

2) Check your visitor stats and base your decision on that info. If you use Google Analytics you can even set goals and compare resolutions with conversions.

[amxfan]

User requests the missing page.
Server sees that the page does not exist, so responds to the request with the error document.
The PHP parser tells the server to override the "404 - Not Found" header with a "301 - Permanent Redirect" header
The user receives a redirect to the intended destination.


Ok I think I'm a little slow tonight... This sounds and looks like a cool way to handle my first issue looking at it from a visitors point of view. But from a SE point of view will it see a status 404, 301 or a status 200? I'm asking due to if Google does not see a 404 when it's expects to it makes your site "Not verified" in web masters tools.

If I'm reading it right "it's been a long night already" I think the SE will see a 301 since it's over rides the 404 and with this any link weight that the missing page might have had is now passed onto the page where the 301 directs you to.

Do I have it or I'm I still in left field.....