Is Google Removing the Supplemental Index Tag?

[incrediblehelp]

A recent comment at SEOmoz.org from Matt Cutts says they might remove the SI tag sometime soon.

I guess Matt and Google were feeling that people are getting to "fixated" on their pages being in the SI. Well yes they are because these pages simply don't rank. They have absolutely no chance against pages outside of the SI. If they felt "fixation" was an issue why not take away PR number from the toolbar to that is so inaccurate. I know that will help me spend less time explaining what "truly" means.

Matt goes on to say:

"Over time, the supplemental results are less and less supplemental and more and more likely to show up for any given query. As I mentioned at SMX Seattle, my personal preference would be to drop the "Supplemental Result" tag altogether because those results are 1) getting fresher and fresher, and 2) starting to show up more and more often for regular web searches. Especially as the supplemental results get more fresh,"

More fresh? What the heck is that?

Micheal Martinez goes on to say this about "freshness":

Matt Cutts confirmed at SMX Advanced 2007 that Supplemental Results pages are not parsed the same way as pages in the Main Web Index. So if Google only "does away" with the "Supplemental Results" label, they will be playing a game of smoke and mirrors with both users and Webmasters.


Either Google should start fully parsing Supplemental Results pages so that they have a fair chance of ranking in search results, or they should keep the label so that Webmasters can see they still need to get more inbound links to move those pages into the Main Web Index.


Google should not under any circumstances pretend to be combining the two indexes. Either Supplemental Results go away completely or they stay. This is not a "freshness" issue. This is a "Google will not fully parse and index Supplemental Pages" issue.

So what does everyone think about this?

Here are some more source postings on the subject:

Update: Google is removing the supplemental query » Small Business SEM

Is Google Gearing Up To Drop The Supplemental Result Label?

[crankydave]

Well I can't say that I'm surprised but I don't like the idea of simply dropping the label.

Eary this year (i believe) Matt intimated that he thought the trend for 2007 would be for Google to start looking into the SI when they didn't find enough "good" matches in the RI for a query. Take a 3 word query for example. If Google was not able to find enough pages that contained all 3 words they'd then look into the SI for for pages that did, something that was not being done in the past. This is where the more and more SI pages showing up in regular search results comment is coming from IMO.

Additionally, as I have been watching, pages in the SI appear to be having less and less "value". I've noted that pulling a page out of the SI has had a dramatic effect on the page it is intended to target.

I said I'm not surprised because Google likely sees it at a means for webmasters to try and "manipulate" the pages that they want to have in the RI currently. This is adverse to what's been said about their thinking of what they want their "main index" to be. More comprehensive. More sites represented. Given that I'm of the firm belief that Google has decided to keep their forward index at a finite size, the supplemental label gives webmasters an opportunity to try and force more of "their" pages into the RI at the expense of other pages from other sites.

While I agree with MM that Google should fully parse and index pages in the SI so that they have a "better" chance of ranking, I don't see it as a "fair" chance since they look in the RI first and then the SI if they don't find enough "good" matches. A page in the SI may never even be considered for no other reason than it being in the SI to begin with. If Google were to take the best 20K matches from the RI and then the best 20K matches from the SI for their data set, then the pages in the SI would have a "fair" chance at ranking if they were fully parsed and indexed. To my knowledge, that's not being done.

Personally I see this move as a means for Google to take more control over the pages that are contained in the RI by not allowing webmasters and SEO's to see and target the ones that are not.

Dave

[Steven1976a]

What is the reason the SI appear anyway? Is it because of duplicate content? Or other factors involved?

Our new website is database driven and for a catagory like our necklaces we have 25 pages of necklaces which are exactly the same apart from the product pictures, stock number, prices.

If they all appeared in the serps then this would be bad for the serps right? They must filter out pages which are duplicate right?

[incrediblehelp]

If they all appeared in the serps then this would be bad for the serps right? They must filter out pages which are duplicate right?

true. I am not saying it is right or wrong to show people what pages are in the SI, but the issue is if you know your main pages are in the SI at least you know your doing something wrong.

Sure maybe me, you or Dave may not run into this, but for regular webmasters, knowing your pages are in the SI is a god reason to figure out where you went wrong.

[crankydave]

true. I am not saying it is right or wrong to show people what pages are in the SI, but the issue is if you know your main pages are in the SI at least you know your doing something wrong.

Sure maybe me, you or Dave may not run into this, but for regular webmasters, knowing your pages are in the SI is a god reason to figure out where you went wrong.

Just to add to this, even if "duplicate" pages (folders) do make it into the RI index from the same site, take your product pages for example, they get filtered when the results sets are ordered. You might get an indented result for 2 pages but you won't get a whole list of them. They don't need the SI for that purpose.

You can do a search for something and use filter=0 and see what I mean.

Dave

[dvduval]

I think maybe they don't want to show webmasters which pages they don't like?

[tacimala]

From what I understood, they aren't removing the "tag" that says Supplemental Index next to the URL, they were simply removing the option to query just for supplement results using "site:domain.com **** -alskd". This command would show you all of your site's pages in the supplemental index and now it just gives you the same data as the normal site: command.

You can still run a site: command and go towards the end to find the pages tagged as a Supplement Result. The old command still works on some foreign Google TLD's though...

[laurynsmith]

When our rankings dropped dramatically, it was the supplemental index that lead us to find the answer why. We simply updated our (5 year old) code with do not follow and do not index tags to let Google know we were not spamming them or the user with useless information (it was photo galleries and former versions of updated pages that were supplemental). This was a valuable tool for us to make sure our code stays in compliance with Google. It will be a shame if they take it away from honest webmasters, just trying to keep content and code as up to date as possible.

[mpilatow]

Yeah Tacimala, they have already taken that functionality away. They are now talking about removing the Supplemental tag away completely. The reasoning according to Matt is that the supplemental index is being updated more often so it is more like the regular index. He also intimated that it would force SEO's to focus on traditional SEO techniques and not try to get their supplemental pages in the regular index since there is no supplemental index anymore.

I could see this going two ways. The first would be those supplemental pages get pulled into the regular index. The second would be those pages get dropped out of any index. In a way it would be preferable to have the pages be removed because at least you know there is a problem with them, which would help you figure out which pages need work. If they are in the index then you would need to do a bit more work to figure out what needs help. Either way still requires more investigation than the current method where you see exactly what pages are supplemental and need work.

[laurynsmith]

Sorry about that old signature tag everyone. Lauryn