Any ideas why this page is supplemental?

[thindenim]

I'm desparing a bit with google at the moment. Certain pages of my site, which I believe have quality unique content and are well linked are supplemental, while others with very little content are in the main index for no apparent reason.

An example is this page http://www.girlznight.co.uk/shop/hai...air-dryer.html. Here's why I don't believe it should be in the supplemental index.

- 742 words of unique content
- A link from the home page
- Unique Meta Tags
- Code to text ratio of 31%, which I don't think is bad

Anyone have any ideas as to why this may be supplemental?


Thanks,
David

[wige]

Only thing I can think of is that there are no external inbound links to the page.

[minstrel]

Yes. That would do it.

[thindenim]

But not every page has to have an external link to be non supplemental? Links from internal pages pass pagerank to pages they are linked to, particularly links from the home page, which generally is the page with highest pagerank in the site?

[wige]

It is not the only factor, but increasing relevant inbound links from outside your site is one of the best ways to get a page out of the SI.

[minstrel]

Internal navigation links will help somewhat, but remember they are diluted from the home page PR by however many pages are linked on your site.

[coolguy27]

Originally Posted by thindenim

I'm desparing a bit with google at the moment. Certain pages of my site, which I believe have quality unique content and are well linked are supplemental, while others with very little content are in the main index for no apparent reason.

An example is this page Besoke Labs T3 Tourmaline Ionic Evolution Hair Dryer | New T3 Evolution Hair Dryer. Here's why I don't believe it should be in the supplemental index.

- 742 words of unique content
- A link from the home page
- Unique Meta Tags
- Code to text ratio of 31%, which I don't think is bad

Anyone have any ideas as to why this may be supplemental?


Thanks,
David

Maybe because it has a little backllinks....

[incrediblehelp]

From John (Webnauts):

Supplemental Pages

Common Causes of Supplemental Pages:

• Duplicated content;
• Too much content similarity;
• Pages with low or no content;
• Orphaned web pages. Pages that no one links to, including yourself;
• Error pages, if a site does not use If-Modified-Since, Last Modified and/or Expires rules;
• Poor website navigation;
• Pages due to canonicalization problems;
• Too low PageRank;
• Long URLs, especially with long parameters, starting with a question mark (?) and being separated with an ampersand (&) and are not rewriten;
• Suspicious pages for spamindexing, like non-unique and irrelevant to page content meta tags, or linking to bad neighborhoods, etc.

In this case I think it is back links as mentioned above. Sure not every page needs a back link to be out of the SI, but your main website better have plenty of quality back links so that when you link out to these sub-pages they don't fall into the SI.

[minstrel]

Matt Cutts: Supplemental vs. Duplicate Content

I'd agree with most of your video and your bullet points, but lean more toward Michael's viewpoint on bullet point #2; duplicate content doesn't make you more likely to have pages in the supplemental index in my experience. It could be a symptom but not a cause, e.g. lots of duplicate content implies lots of pages, and potentially less PageRank for each of those pages. So trying to surface an entire large catalog of pages would mean less PageRank for each page, which could lead to those pages being less likely to be included in our main web index. I'm not aware of an explicit mechanism whereby duplicate content is more likely to be in our supplemental results, but I'm also happy to admit that as supplemental results are different from webspam, I'm not the expert at Google on every aspect of supplemental results.

One fine point to make: when people see the "show duplicate results" link at the end of search results and click it to add "&filter=0" as an extra parameter, the new results you see are not all from the supplemental results. Lemme see if I can find a query to demonstrate that. Ah, here we are, the first one I tried. [site:mattcutts.com foxmarks] returns one result. If I click to see more results, that post has also been indexed at other urls, but at least two of the extra urls are in our main web index, not the supplemental index.

Infrastructure status, January 2007
Matt Cutts
January 10, 2007

As a reminder, supplemental results aren’t something to be afraid of; I’ve got pages from my site in the supplemental results, for example. A complete software rewrite of the infrastructure for supplemental results launched in Summer o’ 2005, and the supplemental results continue to get fresher. Having urls in the supplemental results doesn’t mean that you have some sort of penalty at all; the main determinant of whether a url is in our main web index or in the supplemental index is PageRank. If you used to have pages in our main web index and now they’re in the supplemental results, a good hypothesis is that we might not be counting links to your pages with the same weight as we have in the past. The approach I’d recommend in that case is to use solid white-hat SEO to get high-quality links (e.g. editorially given by other sites on the basis of merit).

I think going forward, you’ll continue to see the supplemental results get even fresher, and website owners may see more traffic from their supplemental results pages. To check out the current freshness of the supplemental results, I grabbed 20 supplemental pages from my site and checked out their crawl date using the “cache:” command and looking in the cached page header. The oldest supplemental results page that I saw was from September 7th, 2006 (and I only saw 2-3 pages from September; most were from December or November). The most recent of the 20 pages was from January 7, 2007, which shows that supplemental results can be quite fresh at this point.

[mjtaylor]

Very interesting quotes from MC ... taken with a grain of salt, of course ... we had a big discussion about Are Supplemental Pages Evil? not very long ago ...
MJ

[jeffposaka]

IMO, an external link to that page is the most important point. Try to get a link to that page and see what happens.

[thindenim]

I've just added a link on a blog I maintain and I guess there is one on this thread too, so I'll keep an eye on the results.

In terms of the supplemental issue. For a few months there, Google was going through a phase where it seemed to run in a cycle: -

For 2 weeks it would add just about every page in our site to the main index (c 1,500)
For the next 2 weeks it would filter them all out until it got down to around 100 - 150 in the main idex and the rest supplemental.

This happened for about 4 months in a row, until last month when it just seemed to stick at the lower end.

Needless to say, traffic was almost double when it was in the "up" phase.

[SemAdvance]

There are many reasons for pages going supplemental, duplicate content, no links, etc.

The important thing to remember is only the "supplemental results" tag was removed.

The supplemental index is still there and most sites have pages within it.

There has been and always will be two indexes.

Speed of returned results dictates this need.

Documents are sorted into the two indexes for various reasons within Google's algorithm

The Anatomy of a Search Engine See Sec 4.1 and accompanying illustration to better understand.

At one time these indexes were known as

'forward' index

'inverted index'

The forward index holds just the documents most likely to be returned for the average persons search query. (See Google Trends - Google Zeitgeist)

The inverted / supplemental index holds all pages and will have results pulled from this index when docs within the forward index do not return the results needed for obscure search queries.

Lastly relying on an search engine to build your business is a recipe for disaster and a precursor to filing for bankruptcy.

You should set up a myspace - facebook - ivillage-bebo profile page. Join groups in fashion, design, beauty, modeling, anywhere large groups of women are to be found. Gain friends and then advertise special "community friends" discount.....sort of the employee discount of the Internet.

Soon you will never worry of a search engines method of operation.

Peace!

[tamecrow]

I actually have an article on this printed out on my desk, funnily enough. You can view it here:-

How to escape Google's Supplemental Index

[Clarrie]

Originally Posted by minstrel

January 10, 2007

I'd take anything Matt says about Supplemental Results with a pinch of salt - he's had to do quite a lot of spinning over the last year to sell the idea that the SI isn't evil.

I'll accept that things have improved somewhat, but its still way less than ideal. (As you can tell I'm still in the "SI is evil" camp!)

The problem of e-commerce product pages dropping into the SI has been a real problem for over a year now, and primarily comes from the perceived "lack of popularity" of such pages.

The glib answer is "that the page needs inbound links" - yeah, right - as if every page on a site is going to have its own personal inbound links! Most pages of any site of any reasonable size have to rely on the transfer of page rank from the home page (or other key popular pages), and the further away the product page is away from the home page in terms of links the more the PageRank is diluted and down into the SI it goes. (Note this different from the URL parameters issue that Incredible has already flagged up).

I've found the two best ways of attacking the problem are to 1) redesign the internal link structure of the site to shorten the link path to the individual product pages from the home page; and 2) work on building IBLs to link nodes within the site (e.g. individual categories). I've found it works, but it takes time....

If you have one really important product you need to have out of the SI, then consider linking to it direct from the home page (create a list of featured products or something like that).

[thindenim]

Originally Posted by Clarrie

If you have one really important product you need to have out of the SI, then consider linking to it direct from the home page (create a list of featured products or something like that).

Actually, that's one of the reasons I can't figure this one out as the page is linked directly from the home page. As you say, it's pretty difficult to achieve quality external links to all your product pages

[Clarrie]

Originally Posted by thindenim

Actually, that's one of the reasons I can't figure this one out as the page is linked directly from the home page. As you say, it's pretty difficult to achieve quality external links to all your product pages

OK, well that's a b****r then! Guess you need to look at Incredible's list of possible problems then and see if there's anything you can do there.

[minstrel]

Originally Posted by Clarrie

The problem of e-commerce product pages dropping into the SI has been a real problem for over a year now, and primarily comes from the perceived "lack of popularity" of such pages.

The glib answer is "that the page needs inbound links" - yeah, right - as if every page on a site is going to have its own personal inbound links! Most pages of any site of any reasonable size have to rely on the transfer of page rank from the home page (or other key popular pages), and the further away the product page is away from the home page in terms of links the more the PageRank is diluted and down into the SI it goes. (Note this different from the URL parameters issue that Incredible has already flagged up).

Exactly. But that's as it should be. If you have a shopping site with a range of, say, cell phones and accessories, or a clothing site with a range of products for males and females of varying ages and sizes, why would you expect those individual pages to be seen as "important enough" for a primary listing, even by your repeat customers?

And as a searcher looking for such products, would you want every product page for every online merchant coming up and cluttering the search results? I wouldn't. What I'd want to see is the listing for the main index page(s), well optimized for on-page content and with a well-written meta description tag as the snippet - something that would entice me to explore the site further to see what specific products it has to offer.

The fact that your individual product pages are in supplemental is actually a bonus in such cases. It means that if someone searches using a very specific serach phrase that matches your product page, the page will come up in the search results - and it will do so without cluttering up more generic searches.

And of course you get your site to show up well in the more generic searches by targeting the appropriate search terms and aiming anchor text at the home page or main index page.

Originally Posted by Clarrie

If you have one really important product you need to have out of the SI, then consider linking to it direct from the home page (create a list of featured products or something like that).

I totally agree. But as with search engine results, you wouldn't do this for every product you offer, would you? That would make your home page look cluttered, cheesy, and spammy. That's exactly why Google delegates certain pages to Supplemental (one of the reasons).

[ctabuk]

A friend of mine Michael Martinez recently wrote this. Michael is by contract unable to join many forums - but many of the SEO guys here know where he posts (almost alone sometimes)

PageRank is now used as a quality filter on the other end — Google divides the Web into pages with sufficient PageRank (the elite pages admitted into the full Main Web Index) and pages with too little PageRank (the pages — probably the majority of Web documents — that are stored in the Supplemental Results Index).

But Google also says it uses PageRank to help determine crawling priorities. In a natural crawling system PageRank would only be an indicator of the probability of a crawl, but Matt Cutts and other Google representatives have made it sound (to me) like Google actually favors pages which meet some internal PageRank threshold requirement.

PageRank is also used as a reward system. Google allows pages to pass PageRank if they comply with its Webmaster Guidelines, and they allow pages to accumulate PageRank if they comply with its Webmaster Guidelines. Hence, Google has perverted the original PageRank concept from a probabilty measurement to an credibility valuation.

That transition from probability to credibility has forced Google to become intransigent about paid links and other link manipulation schemes. Most people who have manipulated links professionally were doing so primarily for the anchor text (which directly influenced the relevance scoring). But now Google has given Webmasters compelling reasons to pursue PageRank solely for the sake of PageRank.

PageRank was never very important to RANKINGS, but now it has become extremely critical to INCLUSION

------

He is of course relating to internal pagerank

[minstrel]

Originally Posted by ctabuk

PageRank is now used as a quality filter on the other end — Google divides the Web into pages with sufficient PageRank (the elite pages admitted into the full Main Web Index) and pages with too little PageRank (the pages — probably the majority of Web documents — that are stored in the Supplemental Results Index).

But Google also says it uses PageRank to help determine crawling priorities. In a natural crawling system PageRank would only be an indicator of the probability of a crawl, but Matt Cutts and other Google representatives have made it sound (to me) like Google actually favors pages which meet some internal PageRank threshold requirement.

PageRank is also used as a reward system. Google allows pages to pass PageRank if they comply with its Webmaster Guidelines, and they allow pages to accumulate PageRank if they comply with its Webmaster Guidelines. Hence, Google has perverted the original PageRank concept from a probabilty measurement to an credibility valuation.

That transition from probability to credibility has forced Google to become intransigent about paid links and other link manipulation schemes. Most people who have manipulated links professionally were doing so primarily for the anchor text (which directly influenced the relevance scoring). But now Google has given Webmasters compelling reasons to pursue PageRank solely for the sake of PageRank.

PageRank was never very important to RANKINGS, but now it has become extremely critical to INCLUSION

Excellent post - is this article published anywhere?

What I'd add to that last line, though, is "it has become extremely critical to INCLUSION in Google's primary index. That said, being in the Supplemental Index is not a bad thing at all and pages in the SI can and do show up in listings for specific search phrases."

Try the site:domain.com query for a website of your choice. Then try the same thing with site:domain.com/*. This is rumored to tell you how many pages for that site are in the main index vs. the supplemental index.

Results 1 - 10 of about 129,000 from webproworld.com

Results 1 - 10 of about 3,200 from webproworld.com/*

So, assuming the latter query does in fact weed out supplemental pages (I cannot verify this conclusively, although it looks about right), that would tell us that WPW has about 3200 pages in the primary index and about 125800 in the supplemental index.

Do you think that's hurting WPW?

More examples:

Results 1 - 10 of about 657,000 from webmasterworld.com

Results 1 - 10 of about 172,000 from webmasterworld.com/*



Results 1 - 10 of about 495,000 from digitalpoint.com

Results 1 - 10 of about 81,500 from digitalpoint.com/*

[privatefleet]

I have run in to a similar problem with my site.

This might be a simplistic answer but maybe you have too many pages in relation to incoming links! Google has indexed 1790 pages

If PR is spread amongst so many pages it might dilute so much that some pages don't have anough to make the main index.

So get more links, delete some pages, adjust navigation or use nofollow tages to funnel PR

BTW your site appears to be down at the moment...

[thindenim]

Originally Posted by privatefleet

I have run in to a similar problem with my site.

This might be a simplistic answer but maybe you have too many pages in relation to incoming links! Google has indexed 1790 pages

If PR is spread amongst so many pages it might dilute so much that some pages don't have anough to make the main index.

So get more links, delete some pages, adjust navigation or use nofollow tages to funnel PR

BTW your site appears to be down at the moment...

Oops... looks like we forgot to pay the bill.... :-p Back up now!

Thanks for that. Certainly I'm working on building more inbound links all the time, but it can be hard for a commercial site.

I agree with you that certain of the pages further down may be not getting enough pagerank filtering through, which was the reason for linking key products directly from the home page. I have also used nofollow on various pages, such as the privacy policy and contact page, yet they are appearing in the main index. The mind boggles!

[Webnauts]

Originally Posted by ctabuk

PageRank is now used as a quality filter on the other end — Google divides the Web into pages with sufficient PageRank (the elite pages admitted into the full Main Web Index) and pages with too little PageRank (the pages — probably the majority of Web documents — that are stored in the Supplemental Results Index).

I fully agree! Doesn't that mean that it is better to have more pages in the main index instead of being in the supplementals? Why should I waste my PageRank with pages like order forms, tell a friend, or so ever? And by the way I had this discussion with Jaan (Incrediblehelp) a while ago.

Originally Posted by ctabuk

But Google also says it uses PageRank to help determine crawling priorities. In a natural crawling system PageRank would only be an indicator of the probability of a crawl, but Matt Cutts and other Google representatives have made it sound (to me) like Google actually favors pages which meet some internal PageRank threshold requirement.

I never heard that from someone, but I have noticed that on my sites and on my clients sites.

Originally Posted by ctabuk

PageRank is also used as a reward system. Google allows pages to pass PageRank if they comply with its Webmaster Guidelines, and they allow pages to accumulate PageRank if they comply with its Webmaster Guidelines. Hence, Google has perverted the original PageRank concept from a probabilty measurement to an credibility valuation.

I noticed that already, as I comply to every single guideline including Valid Markup.

Originally Posted by ctabuk

That transition from probability to credibility has forced Google to become intransigent about paid links and other link manipulation schemes. Most people who have manipulated links professionally were doing so primarily for the anchor text (which directly influenced the relevance scoring). But now Google has given Webmasters compelling reasons to pursue PageRank solely for the sake of PageRank.

That is why I love Google.

Originally Posted by ctabuk

PageRank was never very important to RANKINGS, but now it has become extremely critical to INCLUSION

I am preaching that since a long time now, if you all remember our hot debates. But who cares. At least some one confirms that I was and am right.

Originally Posted by ctabuk

He is of course relating to internal pagerank

Obviously. And of the one we will soon in the green bar, but only for that very day.

[minstrel]

Originally Posted by Webnauts

I never heard that from someone, but I have noticed that on my sites and on my clients sites.

Yes, although it's a bit more complicated than that - you don't need all of your pages to have high PR or to be in the main index to attract Googlebot - and of course, once Googlebot is there it does what all good spiders do - it follows links, some of which will lead to low or even zero PR pages.

[kgun]

Originally Posted by Webnauts

Obviously. And of the one we will soon in the green bar, but only for that very day.

I am not so sure, even if you ctabuk has declared it dead, since the toolbar rank is a delayed metric, and as such hopefully more reliable.

[Webnauts]

Originally Posted by minstrel

Yes, although it's a bit more complicated than that - you don't need all of your pages to have high PR or to be in the main index to attract Googlebot - and of course, once Googlebot is there it does what all good spiders do - it follows links, some of which will lead to low or even zero PR pages.

Just a question: Don't the better optimized web site pages with higher PR rank higher in the search results?

[Webnauts]

Originally Posted by kgun

I am not so sure, even if you ctabuk has declared it dead, since the toolbar rank is a delayed metric, and as such hopefully more reliable.

Very strange, because: How to contact matt cutts? - Page 3

[kgun]

From that thread:

"As far as the toolbar PageRank, I definitely wouldn't expect to see it in the next few days. Probably not even in the next couple weeks, if I had to guess".

My bolding.

If Google uses Bayesian (like the Kalman filter) filtering, the new (updated) toolbar rank should be a weighted average of the last position (old toolbar rank and) the new measurement (the new internal pagerank), where the weights are the Kalman gain matrix. (Rounded to x/10, 0<= x <= 10).

Only guessing. Somebody that has exact information? Does Matt Cutts have or is he one of Google's PR persons?

There are more general nonormal filters like the Masrieliez (unsure of the spelling since it is years since I studied it) filter with the Kalman filter as a special case.

[incrediblehelp]

Originally Posted by Webnauts

I fully agree! Doesn't that mean that it is better to have more pages in the main index instead of being in the supplementals? Why should I waste my PageRank with pages like order forms, tell a friend, or so ever? And by the way I had this discussion with Jaan (Incrediblehelp) a while ago.

Ah yes that was a long night of debate. I still dont think you need to nofollow out bound shopping cart URLs or order forms. As you know I am not a big fan of the idea of PR leak or PR hoarding, page by page.

[Webnauts]

Originally Posted by incrediblehelp

Ah yes that was a long night of debate. I still dont think you need to nofollow out bound shopping cart URLs or order forms. As you know I am not a big fan of the idea of PR leak or PR hoarding, page by page.

1. For shopping carts you rather use robots.txt.
2. About PageRank you seem to be totally out of date. David posted some interesting information here. PR is more important than ever.

[wige]

Originally Posted by kgun

From that thread:

"As far as the toolbar PageRank, I definitely wouldn't expect to see it in the next few days. Probably not even in the next couple weeks, if I had to guess".

My bolding.

If Google uses Bayesian (like the Kalman filter) filtering, the new (updated) toolbar rank should be a weighted average of the last position (old toolbar rank and) the new measurement (the new internal pagerank), where the weights are the Kalman gain matrix. (Rounded to x/10, 0<= x <= 10).

Only guessing. Somebody that has exact information? Does Matt Cutts have or is he one of Google's PR persons?

There are more general nonormal filters like the Masrieliez (unsure of the spelling since it is years since I studied it) filter with the Kalman filter as a special case.

I am not sure what you are commenting on... I think this quote was in response to a question about when Google will update the toolbar? Not sure how the filtering applies to that.

From what I have heard, the toolbar data is not live, nor does it "touch" the live data. It is exported from their datacenter and loaded on it's own server (something like tbpagerank.google.com). Because I see so many comments about when it will be done, and what schedule, etc, I think this is likely a manual process that someone at Google initiates, and thus it will always be out of date. It is a snapshot of the base pagerank for all known pages at one particular moment in time after a refactoring (probably the wrong term). This toolbar data seems to be intentionally less than useful - it contains only the base PR, and that base PR is probably recalculated immediately after the export or the data is aged before they move it to the tbpagerank server.

[incrediblehelp]

John once again you dont understand what I mean.

Feel free to nofollow these links to your hearts content as I will not and still rank just fine.

[Webnauts]

Originally Posted by incrediblehelp

John once again you dont understand what I mean.

Feel free to nofollow these links to your hearts content as I will not and still rank just fine.

Hey man, if I do not understand what you mean, why don't you explain? Do you probably mean that if you PageRank will be reduced, you will still rank the same? That is the point what I met, that you are most probably out-of-date. Or did I misunderstand something?

[incrediblehelp]

What I am commenting on John is this statement from you:

Originally Posted by Webnauts

Why should I waste my PageRank with pages like order forms, tell a friend, or so ever?

I don't believe you are wasting anything here. If you want to stop the bots from crawling these pages fine. If you don't do it, that is fine too!

I don't want visitors to this thread to think it would be a huge problem if you don't. Millions of websites rank fine and very well without adding nofollow to links or using robots.txt to block order forms and shopping cart pages.

Now will that change in the future? Maybe, but so could 1000 other variables for why pages rank.

[kgun]

Originally Posted by wige

I am not sure what you are commenting on... I think this quote was in response to a question about when Google will update the toolbar? Not sure how the filtering applies to that.

One method to compute the TbRank. Only guessing.

Originally Posted by wige

From what I have heard, the toolbar data is not live, nor does it "touch" the live data. It is exported from their datacenter and loaded on it's own server (something like tbpagerank.google.com).

That is what I have heard too. May be a more stable / reliable measure than a measure that is constantly fluctuating. Since it is internal, I do not know how much it fluctuates. But to use an analogy from my profession, finance and economics, a fund rating that fluctuated from day to day or even week to week, will be a little confusing.

Originally Posted by wige

Because I see so many comments about when it will be done, and what schedule, etc, I think this is likely a manual process that someone at Google initiates, and thus it will always be out of date. It is a snapshot of the base pagerank for all known pages at one particular moment in time after a refactoring (probably the wrong term).

Manual. I doubt that. Probably a delayed metric updated after a time, eg. like I described above. It is known that SE's use Bayesian filtering.

KW search: Bayesian filtering

Originally Posted by wige

This toolbar data seems to be intentionally less than useful - it contains only the base PR, and that base PR is probably recalculated immediately after the export or the data is aged before they move it to the tbpagerank server.

Useful in what sense is the question.

[Webnauts]

Originally Posted by incrediblehelp

John once again you dont understand what I mean.

Feel free to nofollow these links to your hearts content as I will not and still rank just fine.

Well me and Dan Thies have a totally different opinion and experience: Dynamic Linking & Nofollow - Practical Examples, Diagrams, + FAQs

If you read his article I found today, and my comment on it, I am pretty sure that you will change your mind.

[wige]

Originally Posted by kgun

Manual. I doubt that. Probably a delayed metric updated after a time, eg. like I described above. It is known that SE's use Bayesian filtering.

KW search: Bayesian filtering

I actually think the export of data is not on automated because it is not on a set schedule. From my understanding of the patent, in order for the refactoring of pagerank to be valid, a certain number of pages (described as a percentage of the total index) needs to have been recrawled first, then the pagerank is recalculated to generate a new base value. If Google automatically released a toolbar update it could be possible to calculate the size of the index or the processing speed of the engine. Also, based on the fact that Matt said he didn't know when the toolbar values would be updated again, but it wouldn't be soon (note, in the listed quote he is only saying when, not how) tells me this is based on a schedule of "when they feel like it" rather than any specific algorithm-based timeframe.

[kgun]

"Tja" (Norwegian for yes / no), the number of sites / pages on the web was the main reason for my answer. Only guessing again.

Exponential growth of sites / pages. That (and speed) are (is) the main advantage of an (electronic) SE index. There would be an increasing burdon on the Google staff that I think has a much slower (relative) growth.

That does not imply that Google does not use polls to fight spam? In some regards, polls may be fairly effecient.

[activeco]

Google divides the Web into pages with sufficient PageRank (the elite pages admitted into the full Main Web Index) and pages with too little PageRank (the pages — probably the majority of Web documents — that are stored in the Supplemental Results Index).

I wouldn't fully agree with this.
Too little PR on pages in supplemental index is a consequence and not the reason.
G have enough tools to measure traffic. Bring more traffic to those pages and you're out. That, in turn, brings more links and of course higher PR.
Chicken & egg? Maybe.

[Webnauts]

Originally Posted by activeco

I wouldn't fully agree with this.
Too little PR on pages in supplemental index is a consequence and not the reason.
G have enough tools to measure traffic. Bring more traffic to those pages and you're out. That, in turn, brings more links and of course higher PR.
Chicken & egg? Maybe.

Can you please be more specific? Do you mean that if a site gets traffic, but doesn't get backlinks, can that still increase PR?

[activeco]

Originally Posted by Webnauts

Can you please be more specific? Do you mean that if a site gets traffic, but doesn't get backlinks, can that still increase PR?

No. It's about going out of the supplemental index. Traffic means main index.
And of course, more traffic tends to attract natural backlinks, hence PR increase.

[Webnauts]

Originally Posted by activeco

No. It's about going out of the supplemental index. Traffic means main index.
And of course, more traffic tends to attract natural backlinks, hence PR increase.

Well I found easier ways to come out of the supplementals, but your idea even if it is time consuming and therefore expensive, it could work.

[minstrel]

Originally Posted by Webnauts

Can you please be more specific? Do you mean that if a site gets traffic, but doesn't get backlinks, can that still increase PR?

Originally Posted by activeco

No. It's about going out of the supplemental index. Traffic means main index. And of course, more traffic tends to attract natural backlinks, hence PR increase.

Originally Posted by Webnauts

Well I found easier ways to come out of the supplementals, but your idea even if it is time consuming and therefore expensive, it could work.

I highly doubt your premise, activeco. If you have any evidence at all to support your claim, I'd be surprised. Post it if you have it.

[activeco]

I don't want to push too much on this, however anyone can do its own test.
If you have supplemental pages, take your stats and extract the NON-Google-reffered traffic pages. In most cases majority of those pages should be in the main index while the rest is supplemental.

[Webnauts]

Originally Posted by activeco

I don't want to push too much on this, however anyone can do its own test.

Do our own test? I am impressed!

Did your site have pages in the supplemental results and you got rid of them that way?

Well if you think you can do that it could be challenging to help the senator you promote with a link in your signature seems like he needs some help:

Supplemental Ratio for obama.senate.gov: 9.62%

* Google has a total of 1040 pages indexed from obama.senate.gov
* 940 are in the main index
* 100 are in the supplemental index

So facts, or let it be. I hate theories, and excuses when they cannot be backed up.

[activeco]

Originally Posted by Webnauts

Did your site have pages in the supplemental results and you got rid of them that way?

Yes.

Do our own test? I am impressed!

I don't care about that. My goal was to help the OP and not to get credits from anyone.
Especially not from people who use the help forums as the main platform for self promotion.

[Webnauts]

Originally Posted by activeco

Yes.

URL?

Originally Posted by activeco

Especially not from people who use the help forums as the main platform for self promotion.

Which people do you mean?

[Webnauts]

David if you really want to reduce your pages in the supplemental results do the following:

If a ? indicates a session ID, you may want to exclude all URLs that contain them to ensure Googlebot doesn't crawl duplicate pages. But URLs that end with a ? may be the version of the page that you do want included. For this situation, you can set your robots.txt file as follows:

User-agent: *
Allow: /*?$
Disallow: /*?

That will eliminate the cart pages. You don't need them. And you are waisting valuable PR and Rankings. You can take my word for that!!!

When you do that, drop a note here when uncle Google visits you, so I can see what can be done next.

But its up to you.

[Webnauts]

Oh David, I forgot to mention:

Supplemental Ratio for girlznight.co.uk: 94.15%

• Google has a total of 1830 pages indexed from girlznight.co.uk
• 107 are in the main index
• 1723 are in the supplemental index

OUCH!

[thindenim]

Yes, as you can see supplemental pages are a real problem. It seems that none of the actual product pages are getting indexed, this has also really just happened in the last couple of months.

I have added the lines to the robots.txt file that you suggested. This will filter out all my "browse by brand" pages, i.e. - http://www.girlznight.co.uk/?init=br...did=&b=55&c=73, but I guess they could have been causing duplicate content anyway. Probably better to cut down the number of indexed pages as you suggest and hopefully pass more pagerank to the more important product pages.

Thanks for your help, I'll let you know if there is any change