Site almost disappears from Google

[suzstephens]

Late last week, the domain for JoeJanus.com was redirected from his older Advanced Access site to a new site hosted on Point2. Last week, his site was coming up #4 on Google for "phoenix homes for sale". Yesterday, a Google site map for his new site was submitted and today the site has virtually disappeared from Google. The only way I've been able to get results for it is by searching "joe janus" or searching phrases from his blog hosted at the same domain. I can't think of anything significantly different that was done to his site that might cause this. I don't know what to do. Is there any way to communicate directly with Google about this?

[mjtaylor]

How was the redirect done?

[swcreations]

Google has stated in the past that their results are fully automated and they have also said updates are made after the next spider crawl.

IMO it appears the criteria for G results are changing drastically by the day. In the past I have seen plenty of fluxuations but never to the degree lately. I have also noticed newer sites 1-2 years old have gained top 10 rank all of a sudden.

For years the same sites stayed in the top 10 positions and I would gather than people may have stopped looking if they saw the same thing every time they put in a specific keyword. Any thoughts about this?

[suzstephens]

mjtaylor, as far as I know, Joe simply submitted the IP address for his Point2 site to his registrar (GoDaddy, I think). There have not been any 502 or 503 (or whatever those numbers are -- I get really dyslectic about remembering them) redirects done. Since this happened right after submission of the site map, my ill-informed hunch is that there's some relationship between the site map submission and the site falling off Google.

Stephanie, I believe that the last time the site was crawled was 10/17. How often does Google typically crawl sites?

[swcreations]

I think that depends on the website. When I was newer (8 years ago) it was months before they would crawl my website.

I noticed in the past several years they crawl my website every few days. I usually see my new product pages in the search engines within a few days.

Does anyone know why they crawl some websites more often than others?

[suzstephens]

Oh gosh, it seems like there may be a simple solution. The client had gone through and clicked a check box to add "no index" to his site while it was under construction. When he went back through and unchecked the "no index" box, apparently he missed the home page.

[mjtaylor]

Go back and set up 301 redirects; otherwise you are essentially starting over. But if the old pages are still in Google and it's likely they are if this is within the last 30 days), setting up 301 redirects will help you recover your position.

Originally Posted by swcreations

Does anyone know why they crawl some websites more often than others?

Usually has to do with how often content changes.

[activeco]

Check this old Matt's article:
Moving to a new web host

Assumung your new IP address(es) are in good order (e.g. check it here), the probable failure is point #5, altough the sitemap could also be the culprit.

Originally Posted by suzstephens

There have not been any 502 or 503 (or whatever those numbers are -- I get really dyslectic about remembering them) redirects done.

This is a serious change. You really have to know what you do and why. All the questions should have been asked in advance.

[suzstephens]

It is impossible to set up 301 redirects on sites hosted on either Advanced Access or Point2, so my dyslexia regarding the numbers is irrelevant.

I have set up new sites on Point2 for former Advanced Access clients several times and have never had any problem before. Usually they retain their SE position and toolbar PR, and both often improve within a short while. (For example, one site that I recently moved had no toolbar PR in spite of being on Advanced Access for a couple of years; within 2 months of moving it to Point2, the toolbar PR is now PR1.)

I feel pretty sure now that the inadvertent failure to delete the "no index" meta tag on the home page is the root cause of the problem. I hope that when Google indexes the site again, its Google SERPs will be restored.

[crankydave]

Be sure and do a header check on the old domain and make sure that it is a 301 redirect and not a 302. Typically, when I've checked domains that have been forwarded through godaddy for example, they are 302 redirects.

Dave

[wige]

Another thing that can happen, especially if you simply redirected the domain is that Google may have "lost" the site - it takes Google a while to find the new IP address of a domain, sometimes up to a week, because they cache known DNS entries. As a result, if your site was removed from the old host before Google got the new IP, Google might have knocked the site down because it thinks the site was inactive.

I have also seen a few comments in Google Groups about a possible sitemaps bug, that may cause a site to be dropped in the rankings when a sitemap is submitted for the first time. This is unconfirmed, but another possibility.

[scot184]

Regarding new sites being in the top 10 for search phrases, Google is pushing to be a sort of timely search engine. So news stories that happened say today would already be in the SERPs the same day.

It's pretty cool, but makes it more difficult to hold onto a top ten ranking if your phrase becomes newsworthy. Though your position will usually stick around after all the buzz dies down.

[Clint1]

Originally Posted by suzstephens

Late last week, the domain for JoeJanus.com was redirected from his older Advanced Access site to a new site hosted on Point2. Last week, his site was coming up #4 on Google for "phoenix homes for sale". Yesterday, a Google site map for his new site was submitted and today the site has virtually disappeared from Google. The only way I've been able to get results for it is by searching "joe janus" or searching phrases from his blog hosted at the same domain. I can't think of anything significantly different that was done to his site that might cause this. I don't know what to do. Is there any way to communicate directly with Google about this?

For over a year now, the G forums have been full of posts regarding site owners submitting site maps, and then having their pages and even entire sites deleted. Those type posts have died down since G (as always) never had any comments on it. That's why I've never used one, obviously too risky. Although seeing this happen the next day it a bit quick, I'd remove the sitemap and see what happens. Most saw their sites reappear after removing their sitemap file.

[seiretto]

I think this has little to do with sitemaps (other than in this case alerting googlebot to go and crawl the site, thus causing the coincidental drop in ranking).

BEFORE you move hosts you MUST setup your existing web pages and folders on your new server, read more here:
WHY choose the UK for web hosting?: Changing host - will it affect my Google search ranking or pr?

So many fail to do this! Fail here and you can lose months or even years worth of SEO.

Do it right and your ranking will be safe, and if you are moving to a faster more reliable host then your ranking can and will improve.

[suzstephens]

Seiretto, this step, mentioned in the article that you referenced, "Correctly copy all pages and folders from your old host to the new web space, then test and test again until your sure all pages are present and correct" is simply not possible with the large real estate web site providers" because users can't FTP to the sites.

Once again, I'm fairly certain that the problem resulted from my client's mistakenly failing to properly delete the "no index" meta tag from his home page.

[seiretto]

Originally Posted by suzstephens

Seiretto, this step, mentioned in the article that you referenced, "Correctly copy all pages and folders from your old host to the new web space, then test and test again until your sure all pages are present and correct" is simply not possible with the large real estate web site providers because users can't FTP to the sites."

No FTP - I see, you are providing the CMS. And its not really a large site.

Even if it was a large site there is NO excuse, you, or the site owner, can check ALL links are present. Check ALL links (even on a real large site) automatically for broken hyperlinks and broken image links using:
Find broken links on your site with Xenu's Link Sleuth (TM)
Its free.

So long as your new host provides you with a sub-domain as we do (i.e. [Username].s4-web-hosting.co.uk) this enables you to setup your web site prior to transferring in your existing domain. This ensures any down time of your web site is kept to a minimum. And your ranking will be spared.

If you are considering transferring to a new host always ensure the new web hosting account has a method to test your website prior to moving. If they can not provide this go elsewhere. And for best SEO results host in the country where your target market is based (all our servers are UK based).

[suzstephens]

No, I don't provide the CMS and I have little if any control over either the current host's CMS or the previous host's CMS. Both the old and new sites were hosted by two different third parties, each of which automatically generates page files and page names, and each of which has its own, different, page naming conventions. So I can't simply download a site from one host and upload it to the other. The process involves basically building a completely new site on one host, then redirecting the domain from the old host and killing off the old hosting account. Of course we check for broken links on the new site, but there's absolutely no way to simply move the site files from one host to the other.

Point2's system is incredibly powerful in its own right, and it offers an amazing list of features. For example, if a realtor posts a listing of home for sale, the listing can be automatically syndicated to over two dozen other web sites, including CraigsList and Point2Homes.com. But working with it is nothing like working with a standard web host. Building a custom site on top of a Point2 site involves workarounds piled upon workarounds.

I'm not quite as stupid as you imply: I've been building web sites since 1995, however working on top of the Point2 system involves its own very complex set of challenges. I've spent over a year and a half learning to customize these sites and I figure out a new workaround almost every day. I'm also proud to mention that JoeJanus.com, the site in question, was named "Site of the Month" out of over 150,000 Point2 sites this month. Too bad the huge volume of traffic visiting it this week is mostly other realtors! I sure hope Google will re-index it soon!

[mjtaylor]

Originally Posted by suzstephens

<snip>process involves basically building a completely new site on one host, then redirecting the domain from the old host and killing off the old hosting account. Of course we check for broken links on the new site, but there's absolutely no way to simply move the site files from one host to the other.

What's done is done; I know you can't address the problem you now have this way ... but out of curiousity, why couldn't you download the site as is (there are other ways to do that without FTP; FrontPage can do it, for example, and there are software programs that do it) and then upload it to the new host, test it and then start making your changes?

Just curious, MJ

[isharli]

Originally Posted by mjtaylor

Go back and set up 301 redirects; otherwise you are essentially starting over. But if the old pages are still in Google and it's likely they are if this is within the last 30 days), setting up 301 redirects will help you recover your position.



Usually has to do with how often content changes.

How often does content need to change and how extensive do the changes need to be? Is this especially true for the index page?

[subhzash]

Redirection may be the reason for your site down. What kind of redirection you made?? If you made automatic redirection that may had created the problem for your site

[incrediblehelp]

Since the redirection was done you need to be patient while the SE's catch up with the change.