Site Restructure Impact on Search Referrals

[office7]

We are moving our site from a table-based layout to css as well as doing some other significant enhancements to the site. When we first created the site back in 2003 using a web design program, we unwittingly accepted the default directory where the html files are kept >> www. businessperform.com/html/

Later on, we did create other directories to store specific html files and did do 301 redirects for older files. We are now considering restructuring the directories at the same time as we upgrade the site. I am worried about the impact on our search ranking and search referrals by doing this.

My main worries are that a restructure will upset our inbound links and doing a 301 redirect on all /html/ pages will lead to a huge htaccess file. Currently, all 400 of our pages are indexed in Google, etc.

The current structure with number of pages and number of pages with at least one inbound link is:

/ (18 files -pdf, doc- with inbound links)
/html 135 pages (80 pages with inbound links)
/articles 194 pages (39 pages with inbound links)
/links
/newsletters

The idea is to redistribute the 135 pages in /html/ to five or six other directories, all under the root. Inbound links are as reported by Google Webmaster Tools. Currently, our htaccess file lists 178 301 redirects. 161 of those are redirects to the /articles directory. What if we simply redirect in htaccess only those pages with inbound links (some 119 pages)? If we add all old pages to htaccess, it will have 329 redirects. Won't that be too many for fast access?

This site produces our main business income, so we can't afford to upset Google for too long. Who has done such a restructure and lived to tell the story? How did you do it successfully? Any advice will be greatly appreciated.

Les Allan

[wige]

Actually, redirecting the entire folder would be rather simple in .htaccess. The following line is all you need:

RedirectMatch 301 /html/(.*) http://yourdomain.com/$1

This assumes you are moving all of the pages from the folder to the root. You can do the redirect to any directory you want. As long as the files in the /html/ folder have some sort of directory structure or other file name pattern that can be used as a "key" for redirection, you should be able to do this with minimal overhead.

That being said, 178k redirects is A LOT and might start slowing your server down. I would review those redirects and see if they can be reduced using either RedirectMatch or mod_rewrite. These two technologies exist just for the purposes of simplifying redirection, although they also provide lots of other cool tricks as well. Note: you might need some familiarity with regular expressions to really make the change successful.

Also, I would mention that Google does not always list all of the pages that actually have inbound links. If you only redirect those pages, you might miss pages that have IBLs but simply aren't listed by Google.

[mjamrock]

Dear Les-

Definitely proceed with all the 301s. We just renamed most of our URLs and moved them into a folder structure that was more Google SEO friendly. (Like you several hundred pages). We were unable to set up 301 redirects because of our host provider and as such- I am facing heavy Google penalties for duplicate content in their index (the old URL and the new URL - even after removing the old from their index by request).

It has been going on for months.

If you can set up the 301s it is well worth the time. I wouldn't worry about loosing credit for your internal linking structure- Google will catch up quickly although your reports may not show it. Once Google sees you have added hundreds of pages of "new" content they will be visiting much more frequently. Just be sure to submit a new xml site map as I am sure you already know.

Good Luck!

Britt

[SEObyCanz]

Hi Les,

Fancy meeting you in a place like this... :)

Anyway to your concern, I would be looking closely at how much your page rank for various internal pages influences your SERP's I don't know what sort of key phrases you are doing well for, but if you are chasing highly sort after phrases, losing pagerank could really effect your SERP's,

I know people will jump down my throat and say PR doesn't matter, and of course other onpage factors do count, but still study out your highest ranking keyphrases, and most visited pages in analytics, especially and see if rank has anything to do with your SERPs. You know the elements it takes for ranking highly in SERP's, we have talked about it a lot.

I know you have spent a lot of time on your PR over the years, and possibly losing your PR for your internal pages because of their name changes, could and I repeat could have a profound effect.

Personally I would find out if the above code for multiple re-directs would work if you put in redirects for specific pages first that that after them to pick up the rest, I guess though there would be no way of knowing which redirect worked unless you had a page in another folder, and it was above the multiple redirect...

of course you might be changing your onpage SEO at the same time, etc, so what I just said doesn't count :) I'd just hate to see you lose your ranking for the most important keywords.

Lynny

[crankydave]

I wouldn't call 400 pages huge. I've 301'd quite few more than that without issue. Time consuming, yes. But not problematic IMO.

IMO... focus on the pages that rank well and ones with external links. Getting potential visitors to land on the correct page rather than having to to click further to find it is a good idea while the 301's get sorted out. That can take Google for example, months. Yahoo still has monster issues with recognizing 301's.

The more page to page 301's you can do the better.

Dave

[office7]

Thank you everyone for taking the time to respond. Your comments give me some confidence that doing a 301 redirect in htaccess for all 329 old pages will not adversely impact our site.

To respond in particular to a couple of posters:

Wige, I wish I could use RedirectMatch 301 /html/(.*) http://yourdomain.com/$1, as you suggest. Unfortunately, the files in the /html dir have widely varying names. Pattern matching won't help here. Sorry, when I said that our htaccess file lists 178 301 redirects, I didn't mean it had 178,301 redirects. I meant that it has 178 redirects of the 301 (permanent) type. You're right about GWT possibly not listing all inbound links. And that's the risk we'd be taking in not listing all 329 old pages in htaccess. I tried to run an inbound links report with my two favorite programs, WebCEO and BacklinksMaster. Both started giving errors for Google part way through. I suspect I've upset Google with the large number of requests to their servers.

Britt, I'm surprised you can't set up 301 redirects with your provider. If they don't provide a control panel facility to do that, can't you just create and upload a htacess file to your root folder? Re duplicate content, you can add a ROBOTS metatag to each of your page headers with the variable "noindex".

Hello Lynny. Great to see you are hanging around. Our main pages have a PR of 3/4. Minor pages are mostly PR1 or 0. With a 400 page site and five key themes, we rank for a lot of different keywords. Yes, some of our content will be changing as we reorganize the site, so A/B testing may be difficult. I share your concern with maintaining our PR. That's why I am so concerned about changing our directory structure.

Dave, thanks for your confidence. So, I could easily do a 301 redirect on 329 pages in htaccess. Have you done a big increase in 301 redirects, in one go? And you didn't notice any drop in search referrals? Am I understanding you right? It's not time consuming at all for me to set up that many redirects. I have all of our pages listed in Excel. I have set up a formula that writes the line that needs to go in htaccess for each page I want to redirect. Then I just copy and paste into Notepad. I am worried when you say that Google can take months to rerank our pages after a redirect. The redirected pages are our most important. I guess you'd say to expect a drop in income while this happens?

Thanks again for taking the time to comment.

Les Allan

[crankydave]

A site a moved completely a year and a half ago has that many redirects still in place. Just make sure you double and triple check your formatting and URL's to make sure everything is accurate so you don't busticate something.

While Google sorts out the 301's, your old URL's could remain in the serps for 2-3 months. That's why the 301's are so important. I would expect a bit of a dip but by making the move properly, you can minimize it.

Don't hold your breath when it come to Yahoo.

Dave