Best way to update a site while minimizing downtime

[nametaginc]

Hello everyone,

The web team at my company has been and is continuing to work on revising our website. The revision is going to be a major one and as such, we aren't really able to update/revise the site in bits and pieces.

We are creating the new site using CSS: we plan on completing all the changes and then will upload all the pages and revisions at once. Due to the infrastructure changes of the new site, some of the pages that are on the current/old site will need to remain on the new site but will be 301 redirected until the SEs are aware and have indexed the changes.

From what I've read a change of this scale will draw the attention of the SEs. My question is...is the method we are going about to make the changes the best way to do so?

- I've read that others have had their site disappear from the SE results until their site has been reindexed...will an instantaneous switch from the old site to the new one leave us at the mercy of the SEs to reindex us with the changes?

- Is setting up 301 redirects the best way to pass the optimization of the old pages that have been indexed and rank well, to the new pages that will replace them?

- Lastly, for anyone that has gone through something similar to this...approximately how long did it take for the SEs to see and reindex your revisions?


Thanks in advance!

[kevsta]

how big a site and how much is it changing? are many (ranking) pagenames & urls changing? ..is the main question.

if they are you must 301 the old pages to the new ones otherwise you'll drop like a stone. & yes 301 is the absolute best (only) way to hold onto serps.

we just did it to a 20k page site with minimal upheaval to serps but there weren't too many url / filename changes, so Google just recached the whole thing over about 3 weeks.

[Dubbya]

You will take a hit, there's no doubt there, but you're wise to implement as many 301's as possible to minimize the impact of the overall change.

Don't forget to create a new sitemap and robots.txt file. Be sure to submit them when you update.

Use Xenu Link Sleuth to check for broken links and missing pages.

After you've been indexed, check your Webmaster tools pages for 404's and spidering errors. Ensure that any outdated IBL's are covered with 301's and fix any errors ASAP.

Don't worry too much though. While one does tend to panic, it's normal to see your rankings drop for a few weeks until things even out. They'll bounce back.

Looks like you've done your homework though, so do what you can to minimize the flux and try to relax.

.02

[joer80]

I took all of my pages, and copied them, adding the letter x to the end of the file name.

I wrote code that grabbed the users ip address. If it was mine, show the new include file. (Add an x to the filename to import)

[Peter (IMC)]

Just build the whole site on another IP address so you can test straight online. The 301's are a good idea too, to not lose the visitors that find the old pages in the SERP's + they help the search engines find the new pages quickly.

If the whole thing is optimized, then what you will see is an increase in visitors from the day that you put the new site online. (We do this a couple of times a week and it goes like this in 99% of the cases.) Unlikely that you'll see a reduction in the number of visitors.

However, if you start monitoring for a couple of specific keywords, then you might get dissapointed. You have to look at over all traffic, not a a couple of specific keywords.

[bj]

One of the reasons I've gone to using dynamic scripting for sites is that it's so much easier to implement a change of this sort, since all the data is in the database, all the html structure is in the template, and you simply drop in the new skin and turn it on. No worries about broken links, and no url changes, so no canonical headaches or 404 worries or other stuff of that sort. Though internal html structure can change, and there may be some shifting in SERPS due to that, the hit is very minimal and very short lived.

I know you're not going to do that this round, but you may consider it for next time. And next time seems to come in 3-5 year integrals. Do you really want to go through this page by page headache next time?

[vijaysaki]

If i were you if the site is not too big I would actually name the files the same.

You are thinking about pushing new site all at once is a good idea. But 301 redirect i dont know. If you used 404 and databased the old page contents and on the head if you leave it as 404 (or depending on the page name you cleverly redirect to the newer appropriate pages) SEs will soon forget those pages and index the new pages. That way people will visit you page but soon it will be gone and new indexes will appear.

Another solution good be the 404 can carry your newer look with old content...

I dont know if it helped

[Orion]

I know of no other way to do it

upload new pages overwriting old ones and adding new ones

Definitely 301 the pages...

immediately update your google site map and yahoo URL list and resubmit them in your webmaster control panel.. that gives you quicker reindexing...

find all your IBLs and get them updated to link to the new pages (if they are different)...

[B0B]

I assume we are talking about structural changes, adding, deleting and changing entire sections of the site. I assume it’s a big site. I also assume that it's up on a temporary IP, fully tested, optimized and everyone is happy with it. This includes the CSS, site map and everything. I would run some search bot tests on it to make sure everything is visible and can be easily indexed. Set the robots.text file so the bots don't accidently find anything. You don't want to take a hit for duplicate content while testing.

Keep the page URLs the same for similar keyword optimized content, if you can or for as many pages as you can. New pages with new URLs whose content (closely) maps to a new page use server side redirect to serve up the new (similar) page. If your navigation structure is blatantly obvious to the bots and you have site maps and submitted them to the SEs, they will drop the old URLs.

Bj is right; love those databases, especially if it’s multilingual. A little lookup of what maps where and what template to use and away you go. But you still have to test it and get all the approvals before a major change.

Enjoy Bob

[GonnaLuvIt]

I just did this exact thing 2 weeks ago. I left my old site intact and 301'ed all my old pages to a subdomain. That way I would not lose my current rankings. Then I went live with the new site, now I am slowly getting indexed for the new pages and gaining new SERPs. Eventually, when the traffic has shifted, I will phrase out the old site.

[Peter (IMC)]

Originally Posted by GonnaLuvIt

I just did this exact thing 2 weeks ago. I left my old site intact and 301'ed all my old pages to a subdomain. That way I would not lose my current rankings. Then I went live with the new site, now I am slowly getting indexed for the new pages and gaining new SERPs. Eventually, when the traffic has shifted, I will phrase out the old site.

Why not 301 the old pages to the new pages???

[GonnaLuvIt]

Originally Posted by Peter (IMC)

Why not 301 the old pages to the new pages???

Mainly, because I have over 30,000 pages and there was no easy way to match page for page with a 301. So, I'm basically leaving my old pages online until my new pages get indexed then I will remove them.

Honestly, this is my first time doing this so I cannot claim I know what I'm doing. All I do know is that its working for me and I haven't lost any traffic keeping the old site on a subdomain.

[nametaginc]

Hi everyone,

Thanks for all the responses/input.

Just to help clarify some things.

The current/old site has around 300 pages or so...each page is uniquely created and coded. Some of the site uses CSS, some parts of it don't. The design team is somewhat restricted because we are essentially building on top of the existing site...using the existing infrastructure, CSS, navigation system, etc. We have tried working within these boundaries and have made changes to various pages...but not nearly as quickly as we would like. In addition, the new design of the site will be organized slightly different...right now each product individual process has its own landing page...while this presents a lot of information, in our opinion it confuses the customer and breaks the continuity of the ordering process.

The new site, which will be built from ground up, will be hosted on our test site until it is completely finished and ready to be uploaded over the old site. Due to the new structural/navigation system, the total number of pages will be greatly reduced and the new site will have its own new CSS, used by every page...as opposed to whenever convenient on the old site. Since we will be making a shift from organizing our products from process to a "logo/no logo approach", the individual process landing pages (such as laser engraved, rotary engraved, full color printed) will no longer be available...instead one page (tags with a logo) with all the processes listed will be used. While not as effective as dynamic site like BJ suggests, it will offer us a template that we can use on all of the product lines.

So technically while we should be able to get parts and pieces of the old site updated, it would seem that the new approach of building a new site would be easier, more effective and would help to unify our efforts.

The major areas of concern though are that we will no longer have the process-specific landing pages which have been optimized and well ranked...which is where the 301 redirects come in; since we are able to create and rename pages, some of the more popular pages can still be retained but structured slightly differently. We would leave the old pages on the new site with 301 redirects until the SEs reindex the site completely and thoroughly.

The other primary concern is that a simultaneous upload of the new site to replace the old would "shock" the SEs to the point where we would disappear from the results. While we are hoping to not be out of the index for long, because we are primarily an e-commerce site, any downtime...especially prolonged downtime, can be quite concerning.

At this time, the new site and the test domain are still a work-in-progress and should still take a few more months to be ready to be launched. In the meantime, I wanted to learn as much about the process and from others' experiences who have done something similar to this.

[Dubbya]

I've found that implementing a few changes at a time will significantly reduce the "shock" you referred to. Here's what I did on a couple of sites, one of which is an ecommerce site, the other, a busy payroll services site.

Given that you've got months before it's all complete, there's no better time than the present to get started. The longer it takes to complete, the less significant the impact will be.

Here's how you can complete your site redesign without getting dropped out of the SERPs:

*These recommendations are based on my personal experiences, so take that for what you feel it's worth. I'm not saying this is the only way or the best way to implement a site redesign, this is simply a basic guide outlining what I did and what worked for me.*

Changing the header and footer first, allows you to implement some major changes in terms of document layout without incurring a significant hit to your current positions in the SERPs.

Don't make any changes to the meta tags or page titles at this stage. Try to focus your efforts on implementing the basic "look and feel" update including your CSS stylesheet. You'll likely need to create a "crossover" version that incorporates elements of the old and new sites, but it won't be much of an issue as you move through to completing the overhaul.

With these changes have been implemented, submit a new sitemap and wait for the spiders to hit your site.

When that's had a week or so to propagate through the SE's, focus on changing the navigation system. Don't add or delete any pages and don't change the link text if you don't have to. Again, submit a new sitemap and wait a week or so for the spiders to visit and for these changes to propagate.

At this point, you'll have significantly changed your code structure without making any changes to the content, link text, page titles or headings. In my experience, I actually found it improved my rankings because the code was better structured, which helped improve "spiderability".

So, with the basic site design CSS stylesheet and new navigation system in place, you can start modifying some content. First, update the page content, page titles, page headings and links that will be least likely to impact your results in the SERPs.

Your "About Us", "Contact Us", "Privacy Policy", "Shipping Policy", "Warranty" and "Terms of Use" pages would be good candidates. If you've got a link directory page, you might want to go ahead and update that as well. Just try to restrict major redesigns to pages that receive little traffic and that don't carry much Page Rank. When you've completed these changes, submit a new sitemap and give the changes time to propagate.

At this point you might want to focus solely on completing your home page update, going as far as you can in terms of navigation changes, without modifying URLS to your existing content. At this point, you should probably update your Meta tags, description, keywords and page title. This is where you're most likely to see the most significant changes in the SERPs, so be mindful of that if you're planning to change the home page title.

There's no need at this point to resubmit a sitemap at this point because you it's a pretty safe bet that your homepage is going to get indexed. In addition, you won't have made any additional changes to any other pages during this particular phase.

Again, give that a week or two to propagate and keep an eye on your rankings in the SERPs and check your site stats or Webmaster tools for spidering errors, 404's and the like. After you're sure the site has been spidered and the pages have been indexed, go ahead and start modifying your product pages.

As best you can, try to leave the categories and subcategories hierarchy intact for now. Try to visualize the categories as parent folders, the sub categories as subfolders containing the products and sub products inside them. Make changes to the smallest pieces of the puzzle first and work your way up the chain.

At this point, complete the layouts to the final product pages, add/remove products and implement any required 301's as required. Create a new sitemap, submit it and wait a week or two for the changes to propagate.

With that done, go ahead and change your categories as required, add/remove URLS and implement any required 301's to your new pages. Create and submit a new sitemap and again, give the SE's some time for the changes to propagate.

At this point you'll have updated your site layout, meta tags, CSS file, javascripts, contact and policy pages, links page, home page, your product pages and categories. You'll have added and removed products, manufacturers, categories, subcategories, page titles, page headings and link text and Google will love the fact that you're constantly updating and improving the site.

With the 301's in place, you will retain page rank and uninterrupted traffic, plus your users won't have suffered much inconvenience at all.

As I said, this worked for me and helped minimize the impact that an overwhelming site overhaul can have on your rankings in the SERPs.

Good Luck.

[crankydave]

Excellent post Dubbya!

I've recently gone through a similar situation with an ecommerce site. Some things to add which will probably overlap what Dubbya pointed to. This also is my own person experiences...

Try and keep the "most important" text elements the same initially. Titles, meta info, internal anchors, main text. They can be slowly and methodically adjusted once live.

Start working and continue to work on putting your page to page redirects together on a notepad file for example. Check, double check, and triple check it for any errors.

Create a custom 404 page that matches the new layout. Be clear and consise about explaining why they landed there. Include an on site search function. Include your most important nav links. Provide a "contact" so visitors can alert you to what they were looking for when they landed there if they want.

Have a site map(s) ready to upload.

Don't be surprised if the old pages remain in the SERP's for 30 days or more. Especially the deeper ones. Check to make sure the 301's are functioning properly for these pages on a regular basis. For example, I still have old pages in the SERP's today. The 301's were put in place and site map(s) were uploaded on the 26th of February.

Don't panic. You'll likely see some shifts and changes. Not neccessarily intially if you have all your ducks in a row. Don't overreact to them if/when they happen.

Keeping the "feel" or "flavor" of the look of the site similar will help to reassure and instill confidence in returning/repeat visitors.

A press release announcing the "improvements" visitors see can help in a variety of ways.

Dave

[Dubbya]

Thankya Dave!

Try and keep the "most important" text elements the same initially. Titles, meta info, internal anchors, main text. They can be slowly and methodically adjusted once live.

And that's really the key point. "Slow, methodical changes" implemented and adjusted over time.

I'd say three months would be a great time frame to work within. Get moving ASAP to make the most of the time you've got.

nametaginc, the recommendations you've received are by no means cast in stone. Use our suggestions as a framework for your own implementation plan.

Draft a logical plan that will have the least impact on your current rankings while implementing your site redesign.

[nametaginc]

Dubbya and Crankydave -- much thanks for your advice and suggestions.

I will re-evaluate the method we go about updating our site with the web team again...though it may ultimately be easier for us to redesign the site by making a completely new one to replace the old, like you said, the key to not triggering the wrath of the SEs is to make slow, methodical changes.

I'm sure if we take the time we can integrate the new with the old and mix the CSS, coding and navigation.