Yahoo Canonical Issue

[crankydave]

Perhaps someone could shed some light on this. I've yet to be able grasp what the problem is.

I've generally had good rankings on Yahoo. For a targeted phrase "religious jewelry", one of my sites had been on page one for quite some time. For around 8 months now, the site (chainzonline) is on page 5 or completely disappears...

religious jewelry - Yahoo! Search Results

The only thing I noted is that the www is omitted from the URL. There has been a canonical 301 in place for over a year now. I should point out, that a little better than a year ago, the site was moved from one host to another and uses a different platform. I wouldn't think that should matter.

Granted, I understand that things change over time. However, years of being on page one, then suddenly falling to page 5, and sometimes completely vanishing for extended periods, has me a bit perplexed.

The most "visible" change I see is where they have dropped the www and I don't know why. If anyone has any ideas or see's what I'm missing, please let me know.

Dave

[kushty]

We had the same problem, we were number one for the term Orlando Villas for rent, then we disappeared and started to return without the www and slowly came back then we added our website to the yahoo directory and disappeared altogether. We have fallen back to around page 5.

I don't know if there is anything you can do about it, it is there crap indexing system.

You can of course use the new "rel link" in the meta tags now and supposedly the main3 search engines will pick it up.

[gerardkelley]

I clicked your link and you are #3 and without the www

[crankydave]

Thanx kushsty.

Fortunately, I don't rely on any single source for traffic.

If Yahoo can't handle a 301 that's been in place for quite a long time, I've little confidence that a meta tag would cure that. Besides, the meta tag isn't a "directive" so any "value" passed by links would be lost.

For some more "fun", here's some searches using only the title elements...

gold religious jewelry
gold religious jewelry - Yahoo! Search Results
Top of page 2 with the www

14K religious jewelry
14K religious jewelry - Yahoo! Search Results
Bottom of page 1 without the www

chainzonline religious jewelry
chainzonline religious jewelry - Yahoo! Search Results
lol... Top spot without www and indented internal page with www

Dave

[crankydave]

Originally Posted by gerardkelley

I clicked your link and you are #3 and without the www

Thanx Gerard. The link is to page 5 in the results.

Dave

[jflesher]

I would suggest dropping the www for non (@ or root, no www)

This will resolve this issue in time.

Copy after RewriteEngine On in .htaccess file

[jflesher]

sorry about the incomplete post; seems it will not allow me to post links till I've posted 10 times; I was trying to add the code to show you how to do this in the .htaccess file.

[crankydave]

Originally Posted by jflesher

I would suggest dropping the www for non (@ or root, no www)

This will resolve this issue in time.

Copy after RewriteEngine On in .htaccess file

Why?

I have the 301 in place to the www and has been for a long time. Site is on page 1 of all the other SE's.

Dave

[jflesher]

I see; right now your doing the non to www; no reason to change it for 1 key word; so how it got indexed this way is a mystery; but I have the same issue with a lot of key words; not sure if it makes a difference which way you go, www to non just seems to index better, but changing it now may make your ranks worse.

Do you have a Sitemap.xml?

Have you verified in Yahoo?

[jflesher]

I did notice you have some validation problems when I ran your site at validator w3 org. Some people don't put much stock into this; but I do; if the search engines have problems parsing the page; it could effect your ranking.

I don't think it has anything to do with this problem, seeing how it does not do in the other SE, I just think overall a validated site will parse better and may index better as a result.

Then again; you may not be able to fix these problems if its your CMS that is producing them.

[Webnauts]

Some questions:

1. You only have changed your host or you made any other on-pages changes, like title tag, etc.

2. Your unordered lists are incorrectly marked up, which breaks the semantics of their content, and since Yahoo cares more that ever about the appropriate implementation of semantical elements, could be that has affect you.

I am sure we will figure out what is going on.

[crankydave]

Originally Posted by jflesher

I did notice you have some validation problems when I ran your site at validator w3 org. Some people don't put much stock into this; but I do; if the search engines have problems parsing the page; it could effect your ranking.

I don't think it has anything to do with this problem, seeing how it does not do in the other SE, I just think overall a validated site will parse better and may index better as a result.

Then again; you may not be able to fix these problems if its your CMS that is producing them.

Thank you

Yes, I am aware of the validation problems. I don't see it as a parsing problem. Could be mistaken but a parsing problem should having nothing to do with the homepage canonical problem with Yahoo.

Dave

[Webnauts]

Me again:

1. 4.55% of the images are missing alt attributes. Maybe you do not take advantage of them, or they are just candy images?

2. You have on two or more pages the same keywords meta tag. That can trigger filters.

3. I found on your site questionable links to sites (possible seen as bad neighborhoods from Yahoo who is most strict about that so far I experienced so far. Are those links to the below web sites nofollowed?

Found on: Copyright Website
URL: The Naked Gun
Anchor text: Naked Gun

Found on: Copyright Website
URL: The Naked Gun
Anchor text: Naked Gun

Found on: revellian.com/2008/06/15/bobbys-batch-17-zen-and-enlightenment/
URL: The Reality of Casino Bartending- Sex, Drugs and Videotape | Revellian Dot Com
Anchor text: The Reality of Casino Bartending - Sex, Lies and Videotape

Found on: revellian.com/2008/06/15/bobbys-batch-17-zen-and-enlightenment/
URL: More Bizarre Casino Bartender Stories | Revellian Dot Com
Anchor text: More Bizarre Casino Bartending Stories

Found on: revellian.com/2008/06/15/bobbys-batch-17-zen-and-enlightenment/
URL: Freakish Jobs: Copa Casino - Worst Employee Ever | Revellian Dot Com
Anchor text: Freakish Jobs: Copa Casino

Found on: revellian.com/2008/06/15/bobbys-batch-17-zen-and-enlightenment/
URL: Freebase Nicotine - The Cigarette Drug | Revellian Dot Com
Anchor text: Freebase Nicotine - The Cigarette Drug

Found on: revellian.com/2008/06/15/bobbys-batch-17-zen-and-enlightenment/
URL: Extremely Disturbing Facts | Revellian Dot Com
Anchor text: Extremely Disturbing Facts - Drug Testing

Found on: revellian.com/2008/06/15/bobbys-batch-17-zen-and-enlightenment/
URL: The Hypocrisy of Alcohol Versus Drugs | Revellian Dot Com
Anchor text: The Hypocrisy of Alcohol Versus Drugs

Found on: revellian.com/2008/06/15/bobbys-batch-17-zen-and-enlightenment/
URL: Drugs | Revellian Dot Com
Anchor text: drugs

Found on: revellian.com/2008/06/15/bobbys-batch-17-zen-and-enlightenment/
URL: Psychosexual Disorders | Revellian Dot Com
Anchor text: psychosexual disorders

[crankydave]

Originally Posted by Webnauts

Some questions:

1. You only have changed your host or you made any other on-pages changes, like title tag, etc.

2. Your unordered lists are incorrectly marked up, which breaks the semantics of their content, and since Yahoo cares more that ever about the appropriate implementation of semantical elements, could be that has affect you.

I am sure we will figure out what is going on.

Only changes are host, and platform. Some minor content additions. Removed 1 word from title... 'fine'.

Aware of the unordered lists John, but thank you.

Not sure how this pertains to the canonical issue though.

With the www the homepage was on page 1 for years. Now, Yahoo has decided that the www in the URL for the homepage is not only query specific, it kills the rankings and uses the non www URL depending upon what you search for.

You can check the server response for the different canonical homepage URL's if you like. All 301's have been in place for a long time.

Dave

[Webnauts]

It depends how you have setup the redirects.

Did you set a redirect for the homepage alone or for the whole site?

For example if you are on Apache and you want to solve the homepage canonical issues, you do something like this:

### Require the www to avoid cannonicalization issues ###
RewriteCond %{HTTP_HOST} !^www\.seoworkers\.com [NC]
RewriteRule ^(.*)$ http://www.seoworkers.com/$1 [L,R=301]

### Require to add trailing slash if not present to avoid cannonicalization issues ###
RewriteCond %{HTTP_HOST}   !^www\.seoworkers\.com [NC]
RewriteCond %{HTTP_HOST}   !^$
RewriteRule ^/(.*)         http://www.seoworkers.com/$1 [L,R]

### Redirect index.php to / ######
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.*index\.html?\ HTTP/
RewriteRule ^(.*)index\.html?$ http://www.seoworkers.com/$1 [R=301,L]

If you want to redirect the entire site you do this:

RedirectMatch 301 ^.*$ http://www.seoworkers.net

What happen in that case?

[crankydave]

John.. all the images have alt attributes. Which ones specifically are you referring to?

None of the meta tags are identical. Some very similar but none are identical.

I'm aware of the links to revellian.com. Bobby is an interesting guy and I read his blog. There are links from blog.chainzonline.com to his blog but not from the TLD.

Besides, I wouldn't remove any of them anyway. They're blog comments on my blog and I don't have a problem with them. No, none of the links on the blog are nofollowed

Dave

[crankydave]

John... The 301's are done properly.

For the hompage canonical URL's and all the others.

Dave

[Webnauts]

And I found only 3 pages indexed in Yahoo without WWW:
Site Explorer - Suchergebnisse

[crankydave]

Originally Posted by Webnauts

And I found only 3 pages indexed in Yahoo without WWW:
Site Explorer - Suchergebnisse

Yeah, that's part of what I'm trying to get after.

There's been a 301 in place for a long time. Why is it indexed?

Here's with the www..

http://siteexplorer.search.yahoo.com...&fr=sfp&bwmf=d

Dave

[jflesher]

Originally Posted by crankydave

Thank you

Yes, I am aware of the validation problems. I don't see it as a parsing problem. Could be mistaken but a parsing problem should having nothing to do with the homepage canonical problem with Yahoo.

Dave

I was referring to the SE parsing the page, the SE will parse the page using some programming code, looking for keywords, if there are errors in the page it might not parse it correctly; and this could effect how it sees key words on a page; and yes it could effect ranking but not the canonical problem; this problem was due to linkage, maybe a back link, I've heard all kinks of reasons why this happens; like getting listed for https when you don't have an SSL site.

If you have a Sitemap.xml you will want to make sure its updated and only contains links with the www

Also I don't see a verify for Yahoo; that's why I asked; I'd do this.

[Webnauts]

Originally Posted by crankydave

Yeah, that's part of what I'm trying to get after.

There's been a 301 in place for a long time. Why is it indexed?

Why? I recently found in Yahoo dead or redirected pages from 2005!

Originally Posted by crankydave

Here's with the www..

Site Explorer - Search Results

Dave

Did you try to login in your Yahoo Site Explorer account and delete those 3 links?

[Webnauts]

Originally Posted by jflesher

I was referring to the SE parsing the page, the SE will parse the page using some programming code, looking for keywords, if there are errors in the page it might not parse it correctly; and this could effect how it sees key words on a page; and yes it could effect ranking but not the canonical problem; this problem was due to linkage, maybe a back link, I've heard all kinks of reasons why this happens; like getting listed for https when you don't have an SSL site.

... Also I don't see a verify for Yahoo; that's why I asked; I'd do this.

I fully agree!

[crankydave]

Well, been doing some more digging and getting some PM's and emails, and this appears to be a Yahoo problem that's not getting fixed/resolved.

They're not handling 301's properly. In a nutshell, when they find a URL and subsequently crawl the page, they're attributing it to the URL they find and not what the URL resolves to. There are some instances when they say they do this but the www and non-www isn't supposed to be one of them.

And John, if you use site explorer you'll see the same thing for seoworkers...

http://siteexplorer.search.yahoo.com...o-rd-se&bwmf=d

Dave

[jflesher]

I did a search again today; your still page 5 in Yahoo for Religious Jewelry and it points to the non-www; now at your site you do the 301 from non to www; such that ~ it should always resolve to the www and not the none; this part doesn't surprise me, I'd have to look at your Sitemap.xml; but I don't see one; I'd sujest adding a Sitemap to resolve this issue; otherwise with Yahoo, I do not see how to resolve this; with Google you can set your perferance in Webmaster Tools; which I'd sujest doing if you haven't; but this is what I don't understand; how is it hurting your page ranking?

It seems you have concluded that this is the only factor; what I'd like to know is what information you are basing it on.

Jeff

[crankydave]

Jeff...

Yahoo has a single current page (the homepage) indexed without the www

They have well over 1000 internal pages as well as the homepage indexed with the www

They have over 7000 links to the www and 27 to the non-www

For years the homepage has been listed with the www

Suddenly, they list both the www and non-www versions of the homepage URL, suddenly they decide to return the non-www version for some queries but the www version for others.

They, nor you, nor anyone else can get to the non-www version of the homepage because a 301 is and has been in place.

I appreciate your replies and thoughts. I simply fail to see how a sitemap changes or supercedes the above. Yahoo does not have a problem finding, crawling, or indexing any of the pages. They have used the www version of the homepage for years and suddendly stopped.

The very same thing has been discussed, albeit very briefly, on WMW and SEORoundtable. I've discussed this matter privately with collegues for months. Since having started this thread I've received PM's and emails detailing the very same thing for other sites which I've verified.

I know how dilligent, particular, and precise John is about his site and both the www and non-www version of his site indexed in Yahoo. At a glance, and fortunately, he does not appear to be suffereing from a ranking issue because of it at all like I and others are.

That, in part, is what my basis is for my opinion is.

Again, I appreciate you taking time to share your thoughts and opinions as well as John's and the others I've received.

Dave

[jflesher]

I must be missing something about Sitemap; I though most SE's including Yahoo said they will use it to resolve the issue of www vs non-www; as well as to find links that may be hard to find from the home page; I can't find where I read that; but it was one of the reasons I decided to do it in the first place; not to mention the hard to find pages.

I might be wrong but if I am I don't see any reason to use a Sitemap, I just can't image this is the case.

[jflesher]

I did a search on "sitemap seo www vs non www" and found a few other people saying the same thing; unless you have knowlege of a reason to not use it, besides its too much work; I still think its the answer you are looking for.

This quote is from mattcutts.com/blog/sitemaps-with-www-vs-non-www
"The only thing I’d add to that answer is one recommendation: check your internal links and to make sure that they’re consistent (either all to www or all to non-www). That will also help search engines pick the root page that you prefer."

[Feydakin]

The only reason to use a sitemap is when your site navigation sucks so bad that a search engine can't crawl it properly.. And even then you should solve the problem by fixing your navigation rather than dumping an xml sitemap in there..

Also, Matt Cutts works for Google, not Yahoo! so any answer he may have would be vague for Google at best..

Some issues that Dave and I have discussed :

14K Gold Religious Jewelry from CHAINZonline

That is the cache that Yahoo has for that page.. It is the "correct cache" but they have it for the non www webpage that you simply can not get to without violating the 301 redirect command.. It even says it's for the www version, but the listed link is non www..

Try it.. Try going to chainzonline without the http://www.webproworld.com/yahoo-discussion-forum/www.. Can't get there..

A header check also shows the correct 301 redirect..

I think it is simply a busted Yahoo! and is a great example of why they are #3 when they had #1 all locked up years ago..

[jflesher]

Are you saying Yahoo doesn't honour Sitemap?

I'm not trying to harp on this issue; only to understand your reasoning for not wanting to use the only tool I know of that was designed to prevent this from happening.

Google, Yahoo, MSN and other search engines encourage Webmaster's to use Sitemap.xml files.

I understand your issue; but I still only see one solution; giving up or saying its a Yahoo bug is fine with me, I use to have issues like this and have solved them on all 30 of my web sites just by adding a sitemap; so I know for a fact it works. I wrote a program that creates the sitemap from my menu system; it was a lot of work and it paid off as far as not having this issue; although I still get backlink and https for non-ssl site issues; but that's a differnt problem altogether.

Good luck.

[Webnauts]

Originally Posted by crankydave

Well, been doing some more digging and getting some PM's and emails, and this appears to be a Yahoo problem that's not getting fixed/resolved.

They're not handling 301's properly. In a nutshell, when they find a URL and subsequently crawl the page, they're attributing it to the URL they find and not what the URL resolves to. There are some instances when they say they do this but the www and non-www isn't supposed to be one of them.

And John, if you use site explorer you'll see the same thing for seoworkers...

Site Explorer - Search Results

Dave

I know. There are 13 links pointing to my site without www. Therefore you are right that Yahoo does not resolve 301 properly.

[Webnauts]

Originally Posted by jflesher

I understand your issue; but I still only see one solution; giving up or saying its a Yahoo bug is fine with me, I use to have issues like this and have solved them on all 30 of my web sites just by adding a sitemap; so I know for a fact it works. I wrote a program that creates the sitemap from my menu system; it was a lot of work and it paid off as far as not having this issue; although I still get backlink and https for non-ssl site issues; but that's a differnt problem altogether.

I understand if you say that the problem is because Dave has no XML sitemap. But I use XML sitemaps since they exist. Why doesn't that solve my problem?

[jflesher]

So much for my fix; I looked at my Yahoo links in Jan, where they looked good (besides a few https which I deleted); but you just peeked my curiosity, so I looked again; low and behold Yahoo is now indexing me on www; when in fact I have the 301 setup for non-www; you are right; it appears now that Yahoo is indeed ignoring my Sitemap.

I have been looking at this as if it was an old problem; but I think its a new one and yes you are right, it looks like a bug; I should have looked at my indexs prior to all this, they haven't changed alot since Sep, didn't think they would now; boy was I wrong.

Although I must admit I have been focusing on Google, since my Yahoo rankings are not all that bad; which blinded me to this problem till now.

[crankydave]

I'm not sure why this didn't occur to me before. Here's part of an email I received from Yahoo...

I understand that you would like your site listed with the leading
"www".

I have checked on your site but it appears to be already listed with the
"www" subdomain:

chainzonline.com - Yahoo! Search Results

Please let me know if I am missing anything.

The www in the URL is technically a subdomain we just don't normally refer to it as such. The 301 should certainly be enough to resolve that problem though.

I can't help but wonder if this might be related to how Yahoo is handling redirects to a subdomain.

Dave

[jflesher]

I noticed that you went with www, it seems a lot of people do, yet your blog is non-www, I use WP, it only uses non-www, this was one of the reasons I went with non, if I add in my sitemap for the blog I want all sub-domains to be the same.

I heard a lot of people say that most people will type in www; but the 301 fixes that; so what else can be a good reason for going with the www over non?

[crankydave]

The blog is relatively new so it wasn't a consideration years ago when the site went up.

At that time, canonical issues were not handled well by the SE's and could cause a problem. I simply decided on the www Handling redirects properly were more of problem then as well.

It really doesn't (shouldn't) matter which route you go as long as you're consistant with whatever you choose as your preferred URL.

I feel rather confident that the times when the HP disappears completely is because they have both versions in the index.

On an additional note...

http://suggestions.yahoo.com/detail/...rer&fid=121713

Dave

[crankydave]

Well, after hearing little to nothing via email with Yahoo, I decided to pose the issue here...

www and non-www problems | Yahoo! Site Explorer Suggestion Board

Let's see what (if anything) Yahoo has to say.

Dave

[crankydave]

I guess I should have expected this. Here's part of the reply from a Yahoo employee...

We see and recognise the redirect and as a result you won't be penalised for the www and non-www versions....

...I'm not sure of the reasons for the 2 forms of the site being returned for different queries, but I don't believe this is significant.

lol... Won't "penalize" because of the 301 but the 2 forms of the URL are not significant.

So, in honor of this reply, I'm placing the following link here...

enilnozniahc

It's the site name spelled backwards and linked to the non-www URL. Any bets on what's going to happen?

Anyway, as much as I don't want to, I'm going to add the canonical <link> tag to the page. Don't expect anything different but what the heck.

Dave

[Webnauts]

Originally Posted by crankydave

Anyway, as much as I don't want to, I'm going to add the canonical <link> tag to the page. Don't expect anything different but what the heck.

Dave

It will be an interesting test. Please keep us updated if you don't mind.

[crankydave]

Absolutely John.

I'm going to wait a day or two for the link to get found and followed then go live with the canonical <link> tag.

Another thing I'm considering, don't like the idea, but considering just the same. Is to start building additional links to the non-www URL using the specific queries as anchor text.

The other SE's shouldn't have a problem transferring "value" because of the 301 and I'm simply growing old(er) and losing traffic and conversions waiting for Yahoo to fix what's busticated.

Dave

[Webnauts]

Originally Posted by crankydave

Absolutely John.

I'm going to wait a day or two for the link to get found and followed then go live with the canonical <link> tag.

Another thing I'm considering, don't like the idea, but considering just the same. Is to start building additional links to the non-www URL using the specific queries as anchor text.

The other SE's shouldn't have a problem transferring "value" because of the 301 and I'm simply growing old(er) and losing traffic and conversions waiting for Yahoo to fix what's busticated.

Dave

I wouldn't build links to the non-www version Dave. Try the canonical tag for that case. If it is true what Matt Cutts claimed here Google’s Advice On Using The New Canonical Tag and is handled the same way from Yahoo, your problem should be solved within some days.

[crankydave]

Originally Posted by Webnauts

I wouldn't build links to the non-www version Dave. Try the canonical tag for that case. If it is true what Matt Cutts claimed here Google’s Advice On Using The New Canonical Tag and is handled the same way from Yahoo, your problem should be solved within some days.

Thanx John

The canonical tag is live, so we'll see what happens. Interesting though, as soon as Yahoo found the couple of links I posted using the domain name backwards and the non-www URL, the rankings moved up from page 5 several spots and is now on page 4. The page 5 rankings had been steady, within 3 places, for months. Google has already attributed the anchor text to the www homepage, at least on some DC's.

Dave

[crankydave]

FYI... Slurp has visited and crawled the site multiple times since the canonical tag went live.

Dave