Supplemental club

[incrediblehelp]

Rob at TextLinkBrokers wrote on an article related to this topic via Site-Reference.com

Here is why I think this article is related to these recent supplemental indexing issues. Since this new spider has been released according to Rob, Google has been able to index a far lager amount of pages and content for everyone's website. This new bot can possibly crawl JavaScript and other previously impossible methods of linking. As Rob has mentioned:

"This is my concern - if the duplicate content gets indexed by Google what will they do? There's a good chance that the site would be penalized or even banned for violation of the webmaster quality guidelines set forth by Google."

So what Is Google doing about it now? Seems like not much for those of you out there that are stuck in the supplemental index.

Here are my questions, for those of you out there that are having issues:

Do you use any JavaScript based navigation along with regular text/image links?

Do you have a website version for US AND UK or Canada that contains basically the same content?

Do you have dynamic websites that could lead to different URL's contain the same content through out your website?

Since you go dumped into the supplemental index, did the total URLs that Google indexed go up?

I think answering yes to one or more of these questions could help define why you are in the supplemental index now.

[Pascal77]

Originally Posted by incrediblehelp

Rob at TextLinkBrokers wrote on an article related to this topic via Site-Reference.com

Here is why I think this article is related to these recent supplemental indexing issues. Since this new spider has been released according to Rob, Google has been able to index a far lager amount of pages and content for everyone's website. This new bot can possibly crawl JavaScript and other previously impossible methods of linking. As Rob has mentioned:

So what Is Google doing about it now? Seems like not much for those of you out there that are stuck in the supplemental index.

Here are my questions, for those of you out there that are having issues:

Do you use any JavaScript based navigation along with regular text/image links?

no

Do you have a website version for US AND UK or Canada that contains basically the same content?

no

Do you have dynamic websites that could lead to different URL's contain the same content through out your website?

my forum perhaps? but then all forums should have this and the site without forum is in the same condition.

Since you go dumped into the supplemental index, did the total URLs that Google indexed go up?

a bit but nothing unusual

I think answering yes to one or more of these questions could help define why you are in the supplemental index now.

[crankydave]

Originally Posted by incrediblehelp

Rob at TextLinkBrokers wrote on an article related to this topic via Site-Reference.com

Here is why I think this article is related to these recent supplemental indexing issues. Since this new spider has been released according to Rob, Google has been able to index a far lager amount of pages and content for everyone's website. This new bot can possibly crawl JavaScript and other previously impossible methods of linking. As Rob has mentioned:

So what Is Google doing about it now? Seems like not much for those of you out there that are stuck in the supplemental index.

Here are my questions, for those of you out there that are having issues:

Do you use any JavaScript based navigation along with regular text/image links?

Do you have a website version for US AND UK or Canada that contains basically the same content?

Do you have dynamic websites that could lead to different URL's contain the same content through out your website?

Since you go dumped into the supplemental index, did the total URLs that Google indexed go up?

I think answering yes to one or more of these questions could help define why you are in the supplemental index now.

First, as an aswer to your questions. No on all counts.

Second, I don't think was or is a matter of ending up in the supplemental index. I think it was and is a matter of the short barrel data being lost or corrupted and could not be used, and google had to use the supplemental index in order to provide results.

This would tend to explain why some sites were not showing supplemental results but URL only partially indexed results like mine. Pages weren't being dumped there. It was the pages that were there all along being displayed.

Also, the fix appears to be happening as sites are reindexed and the data used in the short barrels is being repopulated. This would also lend credence to pages not being moved to the supplemantal index but those pages being lost or unusable leaving only the supplemental results to fill queries.

Dave

[incrediblehelp]

I agree Dave I am sure a fix is being implemeneted now and it should handle most of the website issues currently. I was just trying to figure out why it happened in the first place, but like I said most will eventually be fixed anyways.

[janeth]

Has anyone seen the new bot crawl JavaScript?

[incrediblehelp]

I havent personally, but according to Rob it can emulate itself as a Mozilla browser and thus execute JavaScript.

[Ellio]

lets hope tomorrow brings a few more sites back from supplemental hell.

[Ellio]

For those that might not have read it this is in the Google Webmaster help pages;

3. Why is my site labeled "Supplemental"?
Supplemental sites are part of Google's auxiliary index. We're able to place fewer restraints on sites that we crawl for this supplemental index than we do on sites that are crawled for our main index. For example, the number of parameters in a URL might exclude a site from being crawled for inclusion in our main index; however, it could still be crawled and added to our supplemental index.

The index in which a site is included is completely automated; there's no way for you to select or change the index in which your site appears. Please be assured that the index in which a site is included does not affect its PageRank.

"there's no way for you to select or change the index in which your site appears" Interesting sentance for club members.

[Ellio]

I know its a little off subject but looking for indexing answers I found this and read it for the first time.

http://www-db.stanford.edu/pub/papers/google.pdf

Facinating stuff.

[greeneagle]

http://www.seo4fun.com/notes/supplementals.html

Ken

[Pascal77]

Originally Posted by greeneagle

http://www.seo4fun.com/notes/supplementals.html

Ken

I don't think it's a problem that comes from the sites. The problem is at Google and they are not very open about it.($prices stocks)
If it is then i aspect that Google should communicate more over the things you have to do to do it right. But they don't

I saw Matt Cutts started another topic on his blog, he better concentrated at the problems they are dealing with.(just a thought, he does what he wants)
I don't think that this is a small problem and that it is going to be fixed very soon. I hope so but the problem is to big for fixing it in a minute.

[williamc]

Ken, that does not apply to the current situation as this is an admitted google ****-up, not normal issues that placed all these sites in supplemental.

[williamc]

Originally Posted by Pascal77

I don't think that this is a small problem and that it is going to be fixed very soon. I hope so but the problem is to big for fixing it in a minute.

With the size of the indexes, I would bet on a week or 2 before we see all the sites that should not be in supplemental back in the regular index.

[Ellio]

http://www.seo4fun.com/notes/supplementals.html

Ken

I agree not necessarily relevant to the current Google mistake but very interesting as it includes issues we should all take care off.

After all the "Google Mistake" could be something like:

GoogleBot Mozilla now being able to index JavaScript and then Google applying duplicate content filters because of indexed JS Navigation making pages look duplicate and taking up half of every pages code.

Not saying this is the Google Mistake by any means just that the Google error may have been caused by the new and possibly far reaching Googlebot Mozilla's abilities.

[Pascal77]

Originally Posted by williamc

With the size of the indexes, I would bet on a week or 2 before we see all the sites that should not be in supplemental back in the regular index.

To do that they have to now what the problem is. And seeing GoogleGuy's responses in WMW they don't even have a clue.
I am just a rookie but i have eyes in my head.

This thing isn't going to be fixed very soon.
(Bye dreams of starting an own business this year for me)

[williamc]

Originally Posted by Pascal77

To do that they have to now what the problem is. And seeing GoogleGuy's responses in WMW they don't even have a clue.
I am just a rookie but i have eyes in my head.

This thing isn't going to be fixed very soon.
(Bye dreams of starting an own business this year for me)

GoogleGuy stated that they did know what the problem was and that they were already fixing it, as can be seen from all the sites that have already come out of the supplemental index. What remains to be seen now, is just how long it will take for ALL the sites to be out that should be.

[Ellio]

GoogleGuy latest:

Okay, quite a few people should see their pages coming back. If you haven't seen any change (that is, if your pages are still supplemental), I'd like to look into that too so that I can see if there's any common factor remaining.
So: if your pages are still supplemental, feel free to write to sesnyc06 [at] gmail.com with the subject line of "stillsupplemental" (all one word), and I'll ask someone to check the emails out.

Hope that helps, and I'm glad that lots of people are seeing a full recovery,
GoogleGuy

[williamc]

Thanks for that ellio.

[Pascal77]

Originally Posted by Ellio

GoogleGuy latest:

read the next pages after that

Okay, quite a few people should see their pages coming back. If you haven't seen any change (that is, if your pages are still supplemental), I'd like to look into that too so that I can see if there's any common factor remaining.

So: if your pages are still supplemental, feel free to write to sesnyc06 [at] gmail.com with the subject line of "stillsupplemental" (all one word), and I'll ask someone to check the emails out.

Hope that helps, and I'm glad that lots of people are seeing a full recovery,
GoogleGuy

so i can see if there's any common factor ...

He doesn't now what is going on means this.

They did find problems, but they didn't find every problem and i don't think they are going to fix this very soon. First it was a week, now 2 weeks more.

I always liked Google and i even defended them always, but this time i don't even think about doing that. This is something that may not happen by a big company like Google.

[Ellio]

pascall77,

Your GG quote is identical to mine.

Whats pages are we supposed to read "after that"

?

[janeth]

We have learned that there are more then one issue that caused the problem.

So we are learning more about the problem. (-;

[Pascal77]

Originally Posted by janeth

We have learned that there are more then one issue that caused the problem.

So we are learning more about the problem. (-;

yep and they are also, by believing they solved it they are not going to find more.

But that they have a big problem is quit clear i think


to Elio; read next comments and see that they have not fixed it for many sites. Sites reported by SEO-people, there are a lot more out there who lost all traffic and don't even have a clue what they are doing wrong.

[Pascal77]

They did recover and fixed the crash of the barrels but in my eyes they didn't fixed the cause.

I am also convinced that there were already problems in februari. I saw a lot of my pages going up and down. Not like with the jagger update, there they only getted better rankings and not such drastically changes. Maybe i see this better in dutch language results because there are a lot less results than for English language.
And BigDaddy wasn't an update or am i wrong with this?

[Pascal77]

Originally Posted by Pascal77

They did recover and fixed the crash of the barrels but in my eyes they didn't fixed the cause.

I am also convinced that there were already problems in februari. I saw a lot of my pages going up and down. Not like with the jagger update, there they only getted better rankings and not such drastically changes. Maybe i see this better in dutch language results because there are a lot less results than for English language.
And BigDaddy wasn't an update or am i wrong with this?

I also don't understand why people are speaking about a supplemental problem.
The supplemental pages only showed up because they lost the barrels.

Correct me if i'm wrong or if you think something else

[janeth]

Does anyone have any ideas on what % was fixed and what % was not?

[westie]

Oh-Oh!!

Having told you that we were clear yesterday - we have now gone backwards to half in half out.
Yesterday pm we had all pages out with recent cache for all pages.
Today about half and half - new pages (2 days old) are out of SR but some of cache is showing pages that are over a year old!! Weird!

Somebody must have pressed the wrong button in the goggleplex. Interesting to see that Matt Cutts does not even mention the problem! Maybe he pushed the wrong button!!

Regards all.
http://www.ukcoastguide.co.uk

[Ellio]

to Elio; read next comments and see that they have not fixed it for many sites. Sites reported by SEO-people, there are a lot more out there who lost all traffic and don't even have a clue what they are doing wrong.

Quite agree and we are one of them!

I also don't understand why people are speaking about a supplemental problem.
The supplemental pages only showed up because they lost the barrels.

Well put - perhaps it should be the "Lost Barrells Club" !

[williamc]

Originally Posted by Ellio

Well put - perhaps it should be the "Lost Barrells Club" !

Incorrect. Yes, they lost the word barrels also IMO, but that is NOT why so many sites had all their pages dumped into the supplemental index except their homepage.

It is why so many supplemental pages were showing in the serps however, IMO.

But this thread was about why so many pages all of a sudden got supplemental'd

[SPC2]

I disagree with the short and long barrels idea - the idea that the affected sites see supplementals because Google lost the normal indexes. It it were true, it would mean that all the supplementals that those sites see are supplemental regardless of what's in the main indexes, and that's clearly not true. In my case, pages that were normaly ranked highly (non-supplemental pages) went into supplemental, and then they came out again. My site was a perfect example of the problem, and it shows that they were not forced to revert to the supplemental index after losing the data in the main indexes.

I also disagree that Google doesn't know what's wrong. From the GG quotes, they found what was wrong and fixed it. He asked for examples that weren't fixed in case there is still a problem that they don't know about - some, or all, of which are bound to be genuinely penalised sites that they intend to be in supplemental, and have nothing to do with the current problem. That's one way of interpreting what he wrote, but it's not the only way, and there may be another problem that they hadn't spotted when he wrote it. But what he wrote doesn't mean that there *is* another problem.

[janeth]

Originally Posted by SPC2

But what he wrote doesn't mean that there *is* another problem.

A lot of sites not getting fixed would?

[Ellio]

SPC" Wrote:I also disagree that Google doesn't know what's wrong. From the GG quotes, they found what was wrong and fixed it. He asked for examples that weren't fixed in case there is still a problem that they don't know about - some, or all, of which are bound to be genuinely penalised sites that they intend to be in supplemental, and have nothing to do with the current problem. That's one way of interpreting what he wrote, but it's not the only way, and there may be another problem that they hadn't spotted when he wrote it. But what he wrote doesn't mean that there *is* another problem.

I can only disagree with your conclusion.

Our site has not been fixed, was effected at the same time as everybody else, is PR6 as has been for over 5 years, is white hat and a fairly major brand in our sector. We certainly do not think it has suddenly been penalised by removing all pages but the homepage and dispaying a randon bunch of supplementals. A penalty would remove the homepage if anything.

NO we were effected by the "Mistake" but have yet to be fixed. I wait to see if my email to GG is dealt with.

I will report back if any action is taken.

[crankydave]

Originally Posted by SPC2

I disagree with the short and long barrels idea - the idea that the affected sites see supplementals because Google lost the normal indexes. It it were true, it would mean that all the supplementals that those sites see are supplemental regardless of what's in the main indexes, and that's clearly not true. In my case, pages that were normaly ranked highly (non-supplemental pages) went into supplemental, and then they came out again. My site was a perfect example of the problem, and it shows that they were not forced to revert to the supplemental index after losing the data in the main indexes.

I tend to believe that it was because of the normal indexes being lost or somehow unusable. There were many webmasters pointing out that their sites were suddenly being listed as URL only, just the same as those that were supplemental. Perhaps even a worse fate because there was no content at all, just a URL. Mine included. A URL only result in the supplemental index would not have a supplmental tag attached to it because their is nothing on the page indexed or cached.

Take a look at this...

http://216.239.59.99/search?q=site:w...=&start=0&sa=N

Starting on page 9 there are URL only results for all of the pages. These are not pages that that got "dumped" into the supplemental index only to return later on the repaired DC's. These are most likely pages that have been URL only partially indexed pages that have been in the supplemental index all along.

<edit to add>
Since I'm unaware of any webmasters pointing out they've gone from supplemental to URL only, this does not have any appearance of being an interim step in the recovery process, a further indication of these pages being in the supplemental index all along.
</edit>

Dave

[SPC2]

Your site doesn't sound to be a penalty, Ellio, but I'm sure than some people who are claiming to be affected by the problem, aren't affected by it, and that their pages went supplemental due to a penalty, but they hadn't noticed. It happened to one of my sites not long ago, so I know that going supplemental is a form of penalty. (Matt Cutts sorted it out for me, and it's back to normal now).

It may be more than one problem. All I was saying is that GG's statement doesn't necessarily imply that there is more than one. They may have had problems with the main indexes, but that can't have been the only problem, because pages were moved into supplemental successfully. If it was that the main indexes were currupted or lost, they couldn't have moved the pages into supplemental.

[Pascal77]

Originally Posted by SPC2

Your site doesn't sound to be a penalty, Ellio, but I'm sure than some people who are claiming to be affected by the problem, aren't affected by it, and that their pages went supplemental due to a penalty, but they hadn't noticed. It happened to one of my sites not long ago, so I know that going supplemental is a form of penalty. (Matt Cutts sorted it out for me, and it's back to normal now).

It may be more than one problem. All I was saying is that GG's statement doesn't necessarily imply that there is more than one. They may have had problems with the main indexes, but that can't have been the only problem, because pages were moved into supplemental successfully. If it was that the main indexes were currupted or lost, they couldn't have moved the pages into supplemental.

No, BigDaddy wasn't an update so sites can't be penalized if it isn't an update.
My site has nothing to do, so far as i now, with any sort of SPAM or something that could get me penalized.

If there's the possibility that there is 1, 2, 3, ... problems then this means that they don't now what the problem is.
I think they figured out the problem why they lost the barrels but not the cause off it.

And i repeat, i saw very strange behavour in februari in SERP's overhere. Especially sites which are not so old getted less traffic and i saw very, very much SPAM in SERP's.

[incrediblehelp]

Originally Posted by crankydave

Take a look at this...

http://216.239.59.99/search?q=site:w...=&start=0&sa=N

Starting on page 9 there are URL only results for all of the pages. These are not pages that that got "dumped" into the supplemental index only to return later on the repaired DC's. These are most likely pages that have been URL only partially indexed pages that have been in the supplemental index all along.

Why do you feel that if Google shows URL (as title) only when doing a site operator that it is in the supplemental index? I only assume that URLs are in the supplemental index if it is actually labeled as such in the Google SERPs. In your example that is not the case.

[crankydave]

Originally Posted by incrediblehelp

Why do you feel that if Google shows URL (as title) only when doing a site operator that it is in the supplemental index? I only assume that URLs are in the supplemental index if it is actually labeled as such in the Google SERPs. In your example that is not the case.

Why they're not labeled as supplemental is because there's nothing to label. No content. Nothing to search. It's only a URL and will not show up in the SERP's because there's no content associated with it.

Dave

[janeth]

The link you posted shows pages, but I'm with incrediblehelp here, I thought they had to be labeled as supplemental.

[Ellio]

We have had URL listings in the past, I thought these were pages google was aware of but had not fully indexed yet.

I do not think this necessarily means they are "in" the supplemental index which i thought is mainly for redundant pages.

[incrediblehelp]

Originally Posted by crankydave

Why they're not labeled as supplemental is because there's nothing to label. No content. Nothing to search. It's only a URL and will not show up in the SERP's because there's no content associated with it.

Right but that doesnt mean they are in the supplemental index. That just means they are have trouble even getting indexed in the first place or are brand new to the Googlebot and Google just hasnt indexed the page fully yet. These are two seperate issues.

[incrediblehelp]

Ellio beat me by a couple of seconds, LOL.

[crankydave]

Originally Posted by janeth

The link you posted shows pages, but I'm with incrediblehelp here, I thought they had to be labeled as supplemental.

Janeth,

There's nothing to label. No content. Just a URL. Where else could they be storing this info? "The Really Long URL Only Barrel"?

Looking closer you'll see that the category pages that are actually cached and displayed on the first 9 pages are also listed as URL only.

They are pulling this info from 2 different places.

Dave

[crankydave]

Originally Posted by Ellio

We have had URL listings in the past, I thought these were pages google was aware of but had not fully indexed yet.

I do not think this necessarily means they are "in" the supplemental index which i thought is mainly for redundant pages.

Exactly. In the past. This is how new pages/sites are first indexed. After they go back and fully index the page, where do you suppose they put the partially indexed page?

Dave

[crankydave]

Originally Posted by incrediblehelp

Right but that doesnt mean they are in the supplemental index. That just means they are have trouble even getting indexed in the first place or are brand new to the Googlebot and Google just hasnt indexed the page fully yet. These are two seperate issues.

These are pages that have been fully indexed for more than 2 years in some cases. They are not brand new.

Same question... Where do you suppose they put the partially indexed page once it becomes fully indexed?

Dave

[SPC2]

Originally Posted by crankydave

There's nothing to label. No content. Just a URL. Where else could they be storing this info? "The Really Long URL Only Barrel"?

That's an interesting question, Dave, but I don't agree with your conclusion - if you are saying that they are stored in the supplemental index.

Google fetches a page, and then the page is parsed. URLs that are found on the page are stored for the spider to fetch at a later time. In storing a URL, they also store the words that are associated with it - link/alt text, but where do they store that data? I don't think there's any reason to suppose that they store it in the supplemental index, and I do think that there's good reason to suppose that they store it in the normal indexes.

[crankydave]

Originally Posted by SPC2

That's an interesting question, Dave, but I don't agree with your conclusion - if you are saying that they are stored in the supplemental index.

Google fetches a page, and then the page is parsed. URLs that are found on the page are stored for the spider to fetch at a later time. In storing a URL, they also store the words that are associated with it - link/alt text, but where do they store that data? I don't think there's any reason to suppose that they store it in the supplemental index, and I do think that there's good reason to suppose that they store it in the normal indexes.

If what you are saying were correct, then they would be storing the information they fetch from the fully indexed page and the partially indexed page in the same index. In another words, two sets of information, two exactly identical URL's, for the same page stored in the same barrel. Makes no sense.

Dave

[Ellio]

Did a mod modify the thread?

Last time I looked there were 6 pages and 130+ replies.

Now 4 pages and 90+ replies.

What happened to the missing 40+ replies.

Can a Mod please advise?

[greeneagle]

Ellio,

This is a great and timely topic, for many Website Owners, Web Developers and Designers right now.

It appears that the Admins may have stepped in doing an expert job removing non-core comments, rendering the thread more valuable yet. Someone spent a great deal of time assuring the thread owner (yourself), the participants and readers here maximum thread value.

I believe, after reading through it several times that we should all take the "cue", myself included.

We all get a little excited when revenue suddenly drops, dreams seemed "dashed", livlihoods are challenged and the transition isn't as light as what seems to have been indicated by authoritive sources.

Let's carry on and see where this goes. Thanks for bringing this topic on.

Ken

[Ellio]

Ken Wrote: We all get a little excited when revenue suddenly drops, dreams seemed "dashed", livlihoods are challenged and the transition isn't as light as what seems to have been indicated by authoritive sources.

Ken,

I do not remember anybody getting excited, a little humour perhaps but that was based on your rather long winded and less than accurate posts.

Not sure who's dreams have been dashed either, certainly not mine as we do not rely on Google natural serps for our profitibilty. Why do you always assume that somebody posting about a problem is doing so in desperation or because of financial reasons?

What would help this thread, and others, is if posters, including yourself, used plain English that all can be understood by all including those outside of Texas.

The thread is possibly better off without your original posts and the related replies but lets not forget who and what caused that burst of replies....

[greeneagle]

Is anyone still seeing recovery changes, or is it a done deal like GG indicated?

A while back I posted on our turning down a client in a popular industry. The reason being that they owned many ticket sales Sites purchased from a "site mill" that also held a leading dominant database control position.

The game "at large" in that "mill" was to resale many Sites based on thier database. The sites were extremely similar while lacking fundamental SEO, however the tight grip excluded outside SEO to a great extent, leaving only the possibility of SEM.

IMO - Potential Site owners should be extremely wary of vendors that don't extend proper access and controls to the Sites and hosting they purchase.

Walking down that road a little further it seems to be easy to get caught in the "ruts" provided in and by the commonality of the "databases for sale" game.

Ken

[Pascal77]

Originally Posted by greeneagle

Is anyone still seeing recovery changes, or is it a done deal like GG indicated?

Ken

Recovery? I have the impression that it's getting worser.

Some people say they are out of supplemental. Then my question is : and how about rankings and visitors?