The Ethics of rel="nofollow"

[webpresence]

Hi All,

There has been a lot said recently about the effectiveness of the rel="nofollow" attribute with regards to internal PR.

Many SEO professionals believe that the rel="nofollow" is the golden nugget for funnelling PR from non-themed pages (contact, terms, clients, login pages etc.) to their most important on-topic pages. The Effect? The PR that was allocated to the non-themed pages will then be passed on to the on-topic pages, thus helping rankings of the latter.

I can see how this would work short term, but I can also see how in the long term the SE's will treat this as a black hat technique as the site owner is effectively manipulating PR.

An alternative to 'nofollow' would be to use a robots.txt file and exclude spiders from indexing the non-themed pages - the big question however; Would this technique accomplish the same goal and funnel PR from the non-themed to the on-topic pages?

IMO the robots.txt method will accomplish PR funnelling and is the truly ethical way to go, you may disagree though. Let's discuss!

Thanks for reading! Lee

[wige]

I should start by saying I have not read the entire patent involving pagerank thoroughly, but I have been wondering how pagerank flows in special circumstances. The patent application does not seem to mention robots.txt or the nofollow tag, but these two developments I believe came into use after that patent application. I know the original concept of the flow of pagerank was that the value of each link was based on the page's rank divided by the number of links. I have been unable to find any reference from Google or any other search engine about what happens to the page rank when a link has nofollow or points to a page that is restricted by robots.txt.

In theory, nofollow would have a more immediate effect on page rank because at the time that the spider is looking at the links, it can see that these links should not be valued. Because robots.txt is an external file, it may not be immediately obvious to the spider, so this may not be taken into account when the PR is divided.

But, there are two things that happen when a page is spidered. The first is "url discovery". This is the process of finding the links that are on the page, and adding them to the list of known URLs to be fetched on a later cycling. The second step is to calculate the value of the links on the page. The original intent of nofollow was to tell the spider that the link should not be added to the future crawls database, to restrict an administrative section of the site, for example. This is the same reason robots.txt was created.

Today, nofollow has been recommended because according to Google pagerank is no longer transferred by links so marked. Nowhere have I seen it definitively stated that these links are removed from the page rank equation.

To sum up in an example, imagine a page with ten links, and a page rank of 10. Originally, each link would be worth 1 point. Now if one link is marked with a nofollow tag, the theory seems to be that the other nine links get 1.111- points and the marked link gets 0. I have seen absolutely no statement that backs this up, and so I wonder if the nine unmarked links keep their value of 1 point and the marked link gets 0. Basically the 1 point is reserved for that link, but because of the nofollow tag it never reaches its destination.

If anyone has any documentation for or against this theory, please post it.

[incrediblehelp]

To me I don't understand nofollowing a link. It you don't want to pass the juice then why are you linking anways? Also relevant outbound surely help your document rank better.

[genetrader]

I had heard Leslie Rohde once state that Matt Cutts himself had said that the act of using NOFOLLOW to 'shift' Pagerank was not against the Google TOS and was allowable. If someonse can find that on Matt's blog and verify than the question becomes why would they start doing that?

[ArthurKay]

I use the "nofollow" attributes for links posted on my site's forum. Not that my site gets enough traffic to worry about it, but I don't like the idea of people posting just to add a link to their site. I've run a few message boards where spammers post constantly with links to all kinds of junk.

I do have a link to my website in my footer (hoping to get some traffic), but I'm not spamming the message boards just trying to build incoming links. I think the two are entirely different.

[wige]

Matt Cutts posted some comments as to the reason for using nofollow:

Originally Posted by mattcutts.com

In an ideal world, nofollow would only be for untrusted links. Let’s take the example of a forum that wants to avoid linking to spam, but the same advice applies to wikis or any other web software. If an off-domain link is made by an anonymous or unauthenticated user, I’d use nofollow on that link. Once a user has done a certain number of posts/edits, or has been around for long enough to build up trust, then those nofollows could be removed and the links could be trusted. Anytime you have a user that you’d trust, there’s no need to use nofollow links.

This quote indicates that nofollow makes the link no longer an editorial vote for the page, but doesn't discuss much about what is done with the pr that is omitted.

Originally Posted by http://www.mattcutts.com/blog/text-links-and-pagerank/

The nofollow tag allows a site to add a link that abstains from being an editorial vote. Using nofollow is a safe way to buy links, because it’s a machine-readable way to specify that a link doesn’t have to be counted as a vote by a search engine.

[nipplecharms1]

Originally Posted by ArthurKay

I do have a link to my website in my footer (hoping to get some traffic), but I'm not spamming the message boards just trying to build incoming links. I think the two are entirely different.

How are they different? Seems to me this is a case where you benefit twice; one with the "juice" of this forum and second with readers here that you hope visit Chi-town or are from there.

Not sure you can have it both ways. Well you can but that just makes one greedy! . And Greed is good to quote G. Gecko.

But seriously, I have used the no follow because although my customers might get a bennie from visiting a site we link to (and no, before you ask, they are not affiliates for us and we dont get a spif) I do not want the bad "ju-ju" from that site coming back to us. We link to some soft porn-type sites yet I do not want to get smeared by that brush. We have links for our customers, period.

Does that help?

My 2c and a grape soda.

Michael

[jrb@w3f.com]

I prefer the nofollow too, as I don't like putting stuff in the robots file
or the htaccess file either, because it gives away paths to places I don't
really want spyders going. Now google, and yahoo, and a few others
may follow "the rules" but it is the other spyders that I worry about.
And, they are the ones that do all the damage after looking in to every
possible nook and cranny to get info.

Yeah, nofollow is also good if you have a few affiliate links on your pages.
Since these links all go to the same looking page for all the affiliates, they
will rank poorly with google due to their duplicate content derating.

Also, Google is now starting to penalize websites with a few affiliate links
because they "might" just be redirector pages, and have no valuable content.

So, I put noIndex,noFollow on all my affiliate links, and any other links
that may have a bad taste to googlebot.

Ethical? Use it for the purpose intended, and you should be ok. (IMHO)

[RegDCP]

Originally Posted by incrediblehelp

To me I don't understand nofollowing a link. It you don't want to pass the juice then why are you linking anways? Also relevant outbound surely help your document rank better.

I use them in affiliate products links.
As I am being "paid" to place a link on the page, I do not want to pass page rank to the affiliate site.

I tried an experiment a while back. I had a page promoting an affiliate product that ranked well in Google. I removed the nofollow and the page disappeared.. I replaced it and the pages was once again listed.

Reg

[ArthurKay]

Originally Posted by nipplecharms1

Not sure you can have it both ways. Well you can but that just makes one greedy!

I don't argue that WebProWorld may be passing the "juice" onto my site, but they could very easily add the "nofollow" tag to links in this forum. I would still have my link in the footer hoping you would visit the site.

I think the same goes with blogs and other social sites. I simply don't want my site associated with questionable links that could hurt my ranking, so my forum has "nofollow" on every outbound link.

Plus, I doubt my site is going to gain any SEO ground as a result of being on WebProWorld. Do a link population check on my site, and you'll see it's close to zero

[SemAdvance]

PageRank???

People actually care about it???

Makes no difference in rankings so what good is beating it to death?

Google is not going to tell you the factual number so at best your speculating about an arbitrary digit.....

Also the nofollow tag is not for paid links.....certainly not for links to affiliate programs, as your not trying to boost PageRank..

It is to be used when any link, paid or not, could be found to artificially boost another sites PageRank

If you notice Google has a page full of paid links and none of their pages have nofollow tags. When in Rome do as the Romans do.....

Hey, I wonder when Googles going to kick Yahoo and MSN out of the the Google index for having paid links...

Try not to be Lemmings....

[SemAdvance]

Originally Posted by RegDCP

I use them in affiliate products links.
As I am being "paid" to place a link on the page, I do not want to pass page rank to the affiliate site.

I tried an experiment a while back. I had a page promoting an affiliate product that ranked well in Google. I removed the nofollow and the page disappeared.. I replaced it and the pages was once again listed.

Reg

Unless this experiment lasted over three months between pulling the tag and replacing it I would say it is in no way related to the use of a nofollow tag.

Google would need a full update to see the changes to the page, score and in drop it's ranking and it its a page other than your index page it may need more than one update which would make it 6 months to score it and then after you replaced it another 3 months to pick up the changes.

Lastly there is no reason to place nofollow tags to your affiliate programs, The affiliate is not buying a link from you to inflate PageRank, so that is a misconception you have, and another reason your experiment is not valid.

[ccnj]

Originally Posted by wige

Matt Cutts posted some comments as to the reason for using nofollow:


This quote indicates that nofollow makes the link no longer an editorial vote for the page, but doesn't discuss much about what is done with the pr that is omitted.

OK, so what about using a javascript on the links? For example, look at my linkback page - I have a number of links that are using a javascript to help the PR issue on my side... CCNJ Link Exchanges

Will that do the same thing, or do I waste my time on the javascript?

[spronger]

Originally Posted by webpresence

There has been a lot said recently about the effectiveness of the rel="nofollow" attribute with regards to internal PR.

I think everyone's missing the point here, apart from wige. Lee asks about the effectiveness of using nofollow on internal links to boost PR on pages designed to generate income, by not parsing PR to non-important pages such as contact, privacy etc. Nofollowing your external and affiliate links is a different issue. Although, most of my affiliate links are redirect pages. So they are still internal pages and nofollowing them my well be a wise thing to do. Whether or not Google sees this as appropriate I've yet to learn.

I don't think using robots.txt would be a proper/effective substitute. This would remove them from the index altogther as thus any internal linking benefit they confer to other pages. The idea is to have your unimportant pages give up their internally assigned PR to the benefit of important pages, not be removed from the index altogether, yes?

I would like to hear from anyone who has applied this strategy i.e. nofollowed all their unimportant pages. What results were achieved? And have you nofollowed your redirect affiliate pages? You don't expect these pages to rank. They only exist to cloak an affiliate link.

Originally Posted by SemAdvance

PageRank???

People actually care about it???

Makes no difference in rankings so what good is beating it to death?

If it made no difference in ranking nofollow wouldn't exist. It wouldn't need to because your outbound links would give no value for ranking. Google actually gave you a way to withhold PR from sites you deem to be unworthy. They care. They created it for US to use. Agreed, obsessing about PR is not advisable, but dismiss it as having "no difference"? Can't see the logic.

Originally Posted by SemAdvance

Also the nofollow tag is not for paid links.....

Says who? I believe that is one of the reasons it was specifically created for, so a paid link would not confer PR. This would allow you to buy/sell links for traffic (legitimate advertising in Google's view) and not for the sole purpose of boosting PR.

Steve

[SemAdvance]

Originally Posted by ccnj

OK, so what about using a javascript on the links? For example, look at my linkback page - I have a number of links that are using a javascript to help the PR issue on my side... CCNJ Link Exchanges

Will that do the same thing, or do I waste my time on the javascript?

To answer your question Google does not parse links within javascript.

Originally Posted by spronger

If it made no difference in ranking nofollow wouldn't exist. It wouldn't need to because your outbound links would give no value for ranking. Google actually gave you a way to withhold PR from sites you deem to be unworthy. They care. They created it for US to use. Agreed, obsessing about PR is not advisable, but dismiss it as having "no difference"? Can't see the logic.

If it did make a difference then the sites with the highest PR would make up most of the # 1 listings for any keyword term searched.

Since Yahoo & Google are the sites with the highest PR they would dominate the SERPs upper positions.

They don't, so PR does not matter to obtain top results. It is but 1 factor in an algorithm of 100+ parts.

Originally Posted by SemAdvance

Originally Posted by SemAdvance View Post

Also the nofollow tag is not for paid links.....

Originally Posted by spronger

Says who? I believe that is one of the reasons it was specifically created for, so a paid link would not confer PR. This would allow you to buy/sell links for traffic (legitimate advertising in Google's view) and not for the sole purpose of boosting PR.

Steve

Originally Posted by mattcutts.com
In an ideal world, nofollow would only be for untrusted links. Let’s take the example of a forum that wants to avoid linking to spam, but the same advice applies to wikis or any other web software. If an off-domain link is made by an anonymous or unauthenticated user, I’d use nofollow on that link. Once a user has done a certain number of posts/edits, or has been around for long enough to build up trust, then those nofollows could be removed and the links could be trusted. Anytime you have a user that you’d trust, there’s no need to use nofollow.

I'll take two sentences out of the paragraph to validate my statement.

In an ideal world, nofollow would only be for untrusted links.

Anytime you have a user that you’d trust, there’s no need to use nofollow.

If you can show me where it says anything about the links being paid, for I will gladly say you are correct.

Till such time, payment has no bearing in the intended usage of the links.

As I've said and written for the last 4 years

There is no link, like a natural link......

everything else is artificial and devalued most often.

[webpresence]

Think we're moving away from the subject slightly here! Has anyway done any research to determine the effectiveness of using nofollow on non-themed internal links (contact, privacy etc.) to boost PR on themed pages (optimised pages).

Can a robot exclusion be used as an alterantive to nofollow?

[SemAdvance]

Originally Posted by webpresence

Think we're moving away from the subject slightly here! Has anyway done any research to determine the effectiveness of using nofollow on non-themed internal links (contact, privacy etc.) to boost PR on themed pages (optimised pages).

Can a robot exclusion be used as an alterantive to nofollow?

Contact us, privacy and the like pages won't rank well and the search engines expect most sites to have these so it's unlikely an nofollow is needed as these page have no value to the search engines or your visitors as far as distributing 'relevant' information.

I doubt the exercise would raise PageRank as the value of an internal link has to be so minimal, it likely could not raise or lower the value.

Another aspect you would need to look at this from is the competition level of the keyword term....in some cases where there is little or no competition it may have an effect.

In the amount of time it would take to conduct such an experiment (3+ months between PR updates) one could build a few one way links to the sites main pages and add much more value.

Robots txt is used to stop robots from crawling folders and files, so it could work in the same way but again probably futile.

From Matt Cutts: Anytime you have a user that you’d trust, there’s no need to use nofollow.

That also should answer why this test would be futile.

[kgun]

Originally Posted by RegDCP

I use them in affiliate products links.
As I am being "paid" to place a link on the page, I do not want to pass page rank to the affiliate site.

I tried an experiment a while back. I had a page promoting an affiliate product that ranked well in Google. I removed the nofollow and the page disappeared.. I replaced it and the pages was once again listed.

Reg

Interesting. So there are simple tagging aspects in addition to affiliate link hijacking.

[wige]

Originally Posted by SemAdvance

To answer your question Google does not parse links within javascript.

Actually, on my site I have had AJAX content that was only accessible through a javascript indexed by Google. Google does claim they do not crawl what is in a javascript function, but this content was indexed three days after I added AJAX to the server, and I have found no other way Google could have discovered the URL.

Originally Posted by SemAdvance

If you can show me where it says anything about the links being paid, for I will gladly say you are correct.

The quote you cited was pretty much the only paragraph in that entire blog posting that didn't talk about using nofollow on paid links, which I think was the topic of the posting.

[crankydave]

As far as the "ethics" go, PR gets "manipulated" every day by every single site through the use of their navigation or lack thereof. Directing "value" where they want it directed to.

Often, links to pages like "View Cart" or "Contact" are placed site wide for the convenience of vistors. Because Google uses page rank to determine which "allotted" pages from a site get placed into the RI, with the remaining going into the SI, directing PR to the "content" pages becomes very important. Keeping in mind that recent developements regarding the SI hints towards Google's desire for all queries to reach the SI makes this less of a factor.

Personally, I don't see an issue in using internal <nofollow> tags when done so as a means to tell SE's which of your pages are "important" to the index. I know which of my pages are most important. And again, we do this all the time through our links and their on page placement.

I think where the trepidation comes in is more because of the definition of the tag rather than its use. Things like "untrusted" or "can't vouch for" for external links.

If internal use of the tag was defined as "not important page" then I would suspect that using it would be looked at differently. Let's not forget that the <nofollow> tag is not for your visitors. It is to help SE's determine which pages are "important".

Dave

[RegDCP]

Originally Posted by SemAdvance

Unless this experiment lasted over three months between pulling the tag and replacing it I would say it is in no way related to the use of a nofollow tag.

Google would need a full update to see the changes to the page, score and in drop it's ranking and it its a page other than your index page it may need more than one update which would make it 6 months to score it and then after you replaced it another 3 months to pick up the changes.

Lastly there is no reason to place nofollow tags to your affiliate programs, The affiliate is not buying a link from you to inflate PageRank, so that is a misconception you have, and another reason your experiment is not valid.

Sorry SemAdvance, I cannot agree with you.
As only one change was made on the page, I would have to conclude that the nofollow tag was responsible.
Why do you think that it would take 6 months?
I see changes to pages resulting in ranking changes showing in MUCH less time.
The page disappeared following the removal of the nofollow tag. The page reappeared after it was put back in. The experiment worked.

While the affiliate is not paying me to put the link, it is in effect, a paid link as I stand to make a profit from it.
If I were to leave the nofollow off the link, I would be telling Google that I endorse the page rank organically and that is not the case. It has nothing to do with what the affiliate site is buying, but pertains to the "honesty" in my page's linking.

Reg

[wige]

Originally Posted by RegDCP

As only one change was made on the page, I would have to conclude that the nofollow tag was responsible.
Why do you think that it would take 6 months?
I see changes to pages resulting in ranking changes showing in MUCH less time.

Ranking and PageRank are only loosely connected. The nofollow tag probably did not have an effect on the page rank in a short time, because Google calculates PageRank on a semi-annual basis, not live as it spiders sites. However, removing the tag may have caused the spider to be suspicious of those links for some other reason that was covered by the fact that those tags were in place.

[SemAdvance]

Originally Posted by RegDCP

Sorry SemAdvance, I cannot agree with you.
As only one change was made on the page, I would have to conclude that the nofollow tag was responsible.
Why do you think that it would take 6 months?
I see changes to pages resulting in ranking changes showing in MUCH less time.
The page disappeared following the removal of the nofollow tag. The page reappeared after it was put back in. The experiment worked.

While the affiliate is not paying me to put the link, it is in effect, a paid link as I stand to make a profit from it.
If I were to leave the nofollow off the link, I would be telling Google that I endorse the page rank organically and that is not the case. It has nothing to do with what the affiliate site is buying, but pertains to the "honesty" in my page's linking.

Reg

You're free to disagree and no need to be sorry. I never say sorry.

That's why the USA rocks and other countries like Iran suck, freedom of speech!

Why do I think Google would take that long? Because they do not re-index pages that often. Google only updates links every quarter and therefore would only update no follow tags that often as well since these are included in the link itself.

More than likely you are seeing effects of links and other issues happening in the background or in the past month or two.

Peace!

[spronger]

Originally Posted by SemAdvance

If it did make a difference then the sites with the highest PR would make up most of the # 1 listings for any keyword term searched.

Since Yahoo & Google are the sites with the highest PR they would dominate the SERPs upper positions.

They don't, so PR does not matter to obtain top results. It is but 1 factor in an algorithm of 100+ parts.

You're missing the point. Google PageRank, as displayed in the Google toolbar as a number from 0 - 10, is NOT the same PageRank which the Google algorithm uses to rank pages. They are totally different. One is a tool which G provides as an indicator of the "importance" of a particular page. It has limited value other than showing how well linked a particular page is. The other forms the basis of the Google algo. Just read the patent if you don't believe me. It's no secret.

You're right when you say that a high PR page - as in high TOOLBAR PageRank - doesn't always equate to a high ranking. A site can have a gazillion IBLs and thus a high PR, but that doean't mean it will rank well for that particular query. The toolbar PR matters very little in this case, and yes, it's not worth obsessing about.

But PageRank, as in the ACTUAL PageRank you distribute via links, matters quite a lot in how pages are ranked. Hence, the reason for the nofollow tag. Google don't want you confering PR in various circumstances, which INCLUDES paid links.

Why do I think Google would take that long? Because they do not re-index pages that often. Google only updates links every quarter and therefore would only update no follow tags that often as well since these are included in the link itself.

This is incorrect. Again you're confusing toolbar PR with actual PR. The toolbar is only updated periodically as you say. New links are found and indexed daily and the effects on ranking can happen just as quickly.

[SemAdvance]

Originally Posted by spronger

You're missing the point. Google PageRank, as displayed in the Google toolbar as a number from 0 - 10, is NOT the same PageRank which the Google algorithm uses to rank pages. They are totally different. One is a tool which G provides as an indicator of the "importance" of a particular page. It has limited value other than showing how well linked a particular page is. The other forms the basis of the Google algo. Just read the patent if you don't believe me. It's no secret.

You're right when you say that a high PR page - as in high TOOLBAR PageRank - doesn't always equate to a high ranking. A site can have a gazillion IBLs and thus a high PR, but that doean't mean it will rank well for that particular query. The toolbar PR matters very little in this case, and yes, it's not worth obsessing about.

But PageRank, as in the ACTUAL PageRank you distribute via links, matters quite a lot in how pages are ranked. Hence, the reason for the nofollow tag. Google don't want you confering PR in various circumstances, which INCLUDES paid links.



This is incorrect. Again you're confusing toolbar PR with actual PR. The toolbar is only updated periodically as you say. New links are found and indexed daily and the effects on ranking can happen just as quickly.

I actually understand the algorithm quite well. Toolbar PR is the PageRank of your site 3 months ago. I also understand how PageRank is used in the algorithm to determine rankings.

New links do effect the rankings indeed, this however is typically what you see with new sites that pop up for a bit and then fall back down the rankings to where they belong.

Google cannot assign trust immediately and being as you said new links do effect rankings and overzealous webmaster could pay to have a mutlitude of posting done on social bookmarking sites for example.

Once the algorithm is tripped for acquiring too many links, too quickly the brakes are applied.

Also link changes do effect the SERPs however on page changes are not quickly scored or re-indexed.

Also if any of this were true, could you please tell me which of the ten sites listed for the following term has seen it's PR devalued and it's listing removed??

Google Search:

buy links

Let me know what you find....

Also while you are there, please take a lookat the paid ads.

How do you propose Google should cancel all the Adwords Advertisers accounts that promote the service with the intent of improving Google search engine rankings??

How will they absorb the lost revenues? How will they inform the shareholders of the devalued stock prices once they do??

And thats just one keyword term...I'm sure there are 100s others filled with sites with PR, rank well, and sell, or buy links, and do not employ nofollow tags.

Google says one thing

does another......

[crankydave]

I have had on page changes reflected in the SERP's in as little as 1 hour. I have had new pages listed and ranked in as little as 1 hour and remain there without a sitemap.

On site changes can be and are scored quickly.

Nothing wrong with buying or selling links. Depends upon the purpose. It's not about having their PR devalued it's about having their ability to pass PR devalued and that has and can be done internally. The page still has PR but is treated much like a dangling link and removed from the actual calculation and then added back in after the fact so it has no effect.

Dave

[RegDCP]

Originally Posted by SemAdvance

You're free to disagree and no need to be sorry. I never say sorry.

That's why the USA rocks and other countries like Iran suck, freedom of speech!

Why do I think Google would take that long? Because they do not re-index pages that often. Google only updates links every quarter and therefore would only update no follow tags that often as well since these are included in the link itself.

More than likely you are seeing effects of links and other issues happening in the background or in the past month or two.

Peace!

SemAdvance, I was being polite when I said "sorry".

If you think that the SERPs are calculated or recalculated on a quarterly bases, then you cannot understand how databases work.
It would be quite strange if this were to happen as this would mean storing a great number of site algo results, then importing them into the various databases all at once.

This would also mean that a newly spidered site would not show up until the quarter, which is not the case.

As my test showed the page disappearing shortly after the removal of the nofollow tag and reappearing shortly after its reinstatement, I would suspect that updates are made to the SERPs on a continual, real time basis.

Reg

[SemAdvance]

Originally Posted by RegDCP

SemAdvance, I was being polite when I said "sorry".

If you think that the SERPs are calculated or recalculated on a quarterly bases, then you cannot understand how databases work.
It would be quite strange if this were to happen as this would mean storing a great number of site algo results, then importing them into the various databases all at once.

This would also mean that a newly spidered site would not show up until the quarter, which is not the case.

As my test showed the page disappearing shortly after the removal of the nofollow tag and reappearing shortly after its reinstatement, I would suspect that updates are made to the SERPs on a continual, real time basis.

Reg

I'd rather we don't have a match of wits or try to determine what one or another knows about things. I have worked with the first set of algorithms developed over 20 years ago within the retail & pharmaceutical sectors

There is but one algorithm and it is run over the server. Others call it a search spider. To each his own.

Others can learn more here

The Anatomy of a Search Engine

As for the speed of which you and Dave see changes, I have no clue what the level of competition is for the keywords you are using.

I will grant you that changes for 'khazakistan woolen area rugs' will occur much sooner than say 'Manhattan condo'.

I would agree I see changes quicker for some terms than others. Most of the ones I do see in the more competitive arena, take quite a while to rank, and the walk up from position # 8 or 9 to # 1, 2 or 3 can take a year or more.

In the various categories we work in, mileage may vary

Originally Posted by crankydave

I have had on page changes reflected in the SERP's in as little as 1 hour. I have had new pages listed and ranked in as little as 1 hour and remain there without a sitemap.

Dave

Sitemap usage has no effect on rankings whatsoever.

It simply enables the spider to index pages it otherwise would have difficulty parsing.

Peace

[RegDCP]

Originally Posted by SemAdvance

As for the speed of which you and Dave see changes, I have no clue what the level of competition is for the keywords you are using.

2,290,000 competing pages for the search term so it is fairly competative.
My page is #1.

Reg

[crankydave]

Originally Posted by SemAdvance

Sitemap usage has no effect on rankings whatsoever.

I was simply pointing out the the changes were found via the web and not via a submission. I said nothing at all about it affecting ranking.

As far as the level competiton. Not relevant to the statement you made which was...

Originally Posted by SemAdvance

Also link changes do effect the SERPs however on page changes are not quickly scored or re-indexed.

This is wrong. Simply because you cannot see any changes in some instances doesn't mean they are not scored.

Originally Posted by SemAdvance

Why do I think Google would take that long? Because they do not re-index pages that often. Google only updates links every quarter and therefore would only update no follow tags that often as well since these are included in the link itself.

This is also wrong.

Links and their value are updated internally constantly. Just like PR.

Dave

[deepsand]

Originally Posted by SemAdvance

<snip>

There is but one algorithm and it is run over the server. Others call it a search spider. To each his own.

<snip>

Actually, the servers which handles the spiders, and the database server(s) which store and analyze the data retrieved by the spiders are quite independent & different.

The database servers constinuously both accept new data from the spiders, and analyze it, along with previously accumulated data, in real time.

[kgun]

Originally Posted by SemAdvance

I actually understand the algorithm quite well. Toolbar PR is the PageRank of your site 3 months ago. I also understand how PageRank is used in the algorithm to determine rankings.

New links do effect the rankings indeed, this however is typically what you see with new sites that pop up for a bit and then fall back down the rankings to where they belong.

Google cannot assign trust immediately and being as you said new links do effect rankings and overzealous webmaster could pay to have a mutlitude of posting done on social bookmarking sites for example.

Once the algorithm is tripped for acquiring too many links, too quickly the brakes are applied.

Also link changes do effect the SERPs however on page changes are not quickly scored or re-indexed.

I noted this as interesting. Can other members confirm that?

My own minor conclusion after rereading this whole thread:

Nofollowing links on a document / page makes the document more foccused in my view. Foccused and minimalistic documents are better then too complex and unstructured documents IMO.

[wige]

I can confirm, especially for external links, that links are processed almost immediately. I have posted links to an existing page in a few related blogs and, with no changes to the actual page, watched the page jump as much as ten positions within 48hours.

As far as pagerank, there are I think two different pageranks. One is immediate, and is a factor of the pagerank that has been calculated by previous pagerank calculations combined with the value of incoming links to the page. This is the pagerank you can directly affect in the short term by building links. However, the value of those incoming links gets recalculated every few months, kind of like a market correction, where pr is redistributed. Unfortunately I am horrible at math and don't really know the best way to describe this process.

[kgun]

I think that was a good explanation.

Let PR(t) = PageRank in periode t.

Then

PR(t)=Delta[PR(t)]+K1*PR(t-1)+K2*PR(t-2) + ... + Kn*PR(t-n), where

Delta[PR(t)] is the new flow of PR from Period t-1 to t and the K1, K2, ..., Kn are coefficients computed in one way or an other. A simplified version is:

PR(t)=Delta[PR(t)]+K1*PR(t-1) and K1=1, so

PR(t)=Delta[PR(t)]+PR(t-1).

In words if t=2007 and t-1=2006.

The pagerank in 2007 equals the flow of pagerank from 2006 to 2007 + the pagerank in 2006.

Then back to the etichs of nofollow:
My own view: Nothing unetical in using nofollow on links that are irrelevant to the message of the site, like affilliated links even if they are related contextually.

[crankydave]

Internal PR is calculated constantly. Previous values cannot be used. Each new calculation would need to use the initial value assigned to a page in order for it to work.

How often this is done would depend solely on how long it takes to the computer to run the desired number of iterations using the known data, export it, and start again.

Dave

[deepsand]

Originally Posted by crankydave

Internal PR is calculated constantly. Previous values cannot be used. Each new calculation would need to use the initial value assigned to a page in order for it to work.

How often this is done would depend solely on how long it takes to the computer to run the desired number of iterations using the known data, export it, and start again.

Dave

1) With a properly constructed mathematical expression, one need not store all of the previously collected values of the data elements involved. Rather, one need only store the most recently calculated sub-results for each element, and then compute the new sub-results & total result using the weighting coefficients assigned to each element.

2) PR cannot be "constantly" calculated in real time; dynamic real time results require dynamic real time input for the various data elements. As the input is provided by the spiders, and is therefore static data derived from the last time that a site was crawled, a new PR can only result when the spiders present new data.

[crankydave]

Originally Posted by deepsand

1) With a properly constructed mathematical expression, one need not store all of the previously collected values of the data elements involved. Rather, one need only store the most recently calculated sub-results for each element, and then compute the new sub-results & total result using the weighting coefficients assigned to each element.

2) PR cannot be "constantly" calculated in real time; dynamic real time results require dynamic real time input for the various data elements. As the input is provided by the spiders, and is therefore static data derived from the last time that a site was crawled, a new PR can only result when the spiders present new data.

I wasn't saying or trying to suggest that PR was caculated in "real time". Just that internal PR calculations are constant, continuous, or a rolling calculation if you will, limited only by the time it takes to make the calculation and export the results.

The data used is static. Spiders crawl and present new data on a constant, continuous, basis. A single link *can* change the PR of every single web page depending upon the the decimal point you want to take it to.

I have no idea how long it takes Google to run the desired number of iterations and export the results with a known data set. My point was that doing so was only limited to the time it takes the computer(s) to run the caculations, export the results, then move on to the current known data set and start over again, that each subsequent calculation started from the base value and did not use previous calculation results.

Dave

[SemAdvance]

Originally Posted by wige

I can confirm, especially for external links, that links are processed almost immediately. I have posted links to an existing page in a few related blogs and, with no changes to the actual page, watched the page jump as much as ten positions within 48hours.

As far as pagerank, there are I think two different pageranks. One is immediate, and is a factor of the pagerank that has been calculated by previous pagerank calculations combined with the value of incoming links to the page. This is the pagerank you can directly affect in the short term by building links. However, the value of those incoming links gets recalculated every few months, kind of like a market correction, where pr is redistributed. Unfortunately I am horrible at math and don't really know the best way to describe this process.

I could have sworn I said this very thing.

Links can have an effect on low to mid level competition terms, but their value is degraded when it is determined they are indeed artificial as most links are today.

They may be processed immediately however they are not scored immediately.

PR is constant and updated live daily to Google employees.

Toolbar PR is an export to keep webmaster warm & fuzzy....

I will say Googles propaganda is having its effect... as I see many large sites going to nofollow for all external links..

Gotta love it ...makes my knocking competitors off even easier.

[deepsand]

Originally Posted by crankydave

I wasn't saying or trying to suggest that PR was calculated in "real time". Just that internal PR calculations are constant, continuous, or a rolling calculation if you will, limited only by the time it takes to make the calculation and export the results.

The data used is static. Spiders crawl and present new data on a constant, continuous, basis. A single link *can* change the PR of every single web page depending upon the the decimal point you want to take it to.

I have no idea how long it takes Google to run the desired number of iterations and export the results with a known data set. My point was that doing so was only limited to the time it takes the computer(s) to run the calculations, export the results, then move on to the current known data set and start over again, that each subsequent calculation started from the base value and did not use previous calculation results.

Dave

Based on my nearly 50 yrs. experience with DP, as it was once known, I would offer the following re. the inner working & hidden mechanisms here involved.

1) Results will be stored, to provide input for future iterations, so as to avoid as much re-computation as possible. Storage space is dirt cheap compared to CPU cycles, both in terms of the physical devices & operating costs.

2) Once the PR for a particular page has been determined, such will remain static until such time as new relevant input is received.

3) Relevancy will be deterministic. Other entities in the database, which serve to provide data values for those elements used re. a particular page's PR will contain pointers to said page; most likely these will be bi-directional pointers.

4) When spiders return with new data re. a given site, its elements will be searched for pointers re. other potentially affected sites.

5) Then, and only then, will the PR of the affected site be readdressed by the database application which determines PR.

6) In addition to redetermining the affected site's page PRs, the link(s) between the referring and referred to sites will be reconstructed and/or deconstructed as need be.

Hope this makes sense.

[mjtaylor]

Originally Posted by SemAdvance

PageRank???

People actually care about it???

They do. I do. It *is* part of the algorithm. A patented part of the algorithm. When was the last time you applied for a patent that meant nothing to you?

Yes, PR gets more attention/discussion than h1 tags or titles, but that's probably because it is (apparently) quantifiable and that makes it feel more accessible.

Originally Posted by SemAdvance

Makes no difference in rankings so what good is beating it to death?

Of course, it makes a difference in rankings. I rarely, if ever, see sites with considerably less PR above any of mine in competitive SERPs ... yes, a PR5 site can outperform a PR8 site on a given term ... but you will not see PR0-3 sites at the top in very many instances ... of course, there are exceptions ... but *generally* the higher PR sites are in the top 10 and the lower ones are not ... so, yeah, it affects SERPs.

Take 'web design' for example - among the top ten sites are 2 PR4s and the rest of PR6 or 7 ... and I would not be at all surprised if the PR4 sites show up with higher PR on the next update ...

So, let's see, if I have a PR3 site, but I know it's possible to move my into PR6 you don't think it makes sense to put that effort into it?

Originally Posted by SemAdvance

If you notice Google has a page full of paid links and none of their pages have nofollow tags. When in Rome do as the Romans do.....

How specious! But really, where does Google have a page of paid links? On SERPs? There is no PR to pass from SERPs as they are not indexed, ranked pages.

Cheers, MJ

[mjtaylor]

Originally Posted by SemAdvance

Unless this experiment lasted over three months between pulling the tag and replacing it I would say it is in no way related to the use of a nofollow tag.

Google would need a full update to see the changes to the page, score and in drop it's ranking and it its a page other than your index page it may need more than one update which would make it 6 months to score it and then after you replaced it another 3 months to pick up the changes.

Lastly there is no reason to place nofollow tags to your affiliate programs, The affiliate is not buying a link from you to inflate PageRank, so that is a misconception you have, and another reason your experiment is not valid.

Nofollow is no vote - nothing to do with paid or not paid. It was an attribute created to try and stop people from spamming blogs for PR ... it is only recently that it was suggested as a way to show no vote for paid links ...

More importan6ly, I am not at all sure on what you base your suggestion that it would take 3 months to test a change on a page. I make frequent changes and the position in the SERPs changes the same day as the page is spidered.

Perhaps some years ago Google was slow to assimilate change, but that doesn't seem to be the case anymore.

I am confused ... in an earlier post you said PR has not effect on ranking and now you seem to be saying it does ...

MJ

[kgun]

Originally Posted by SemAdvance

PR is constant and updated live daily to Google employees.

Toolbar PR is an export to keep webmaster warm & fuzzy....

I will say Googles propaganda is having its effect... as I see many large sites going to nofollow for all external links..

Gotta love it ...makes my knocking competitors off even easier.

Like temperature outside my house, that is updated daily by a very high dimensional system.

[kgun]

Originally Posted by mjtaylor

Of course, it makes a difference in rankings. I rarely, if ever, see sites with considerably less PR above any of mine in competitive SERPs ... yes, a PR5 site can outperform a PR8 site on a given term ... but you will not see PR0-3 sites at the top in very many instances ... of course, there are exceptions ... but *generally* the higher PR sites are in the top 10 and the lower ones are not ... so, yeah, it affects SERPs.

Take 'web design' for example - among the top ten sites are 2 PR4s and the rest of PR6 or 7 ... and I would not be at all surprised if the PR4 sites show up with higher PR on the next update ...

So, let's see, if I have a PR3 site, but I know it's possible to move my into PR6 you don't think it makes sense to put that effort into it?

How specious! But really, where does Google have a page of paid links? On SERPs? There is no PR to pass from SERPs as they are not indexed, ranked pages.

Agree, and in addition it is not always enough to look at the PageRank of the actual page.

If the Google alogorithme is a hiearchical system, that it most probably is, the PageRank of the home page and the sum of related internal pages may also be important. To put it in mathematical / economic terms, it is input in the production function that produces SERP's as output.

The relative importance of PageRank in one form or an other need not be a constant and the importance will most probably be changed programmatically in future versions of the algorithme.

[crankydave]

Originally Posted by deepsand

Based on my nearly 50 yrs. experience with DP, as it was once known, I would offer the following re. the inner working & hidden mechanisms here involved.

1) Results will be stored, to provide input for future iterations, so as to avoid as much re-computation as possible. Storage space is dirt cheap compared to CPU cycles, both in terms of the physical devices & operating costs.

2) Once the PR for a particular page has been determined, such will remain static until such time as new relevant input is received.

3) Relevancy will be deterministic. Other entities in the database, which serve to provide data values for those elements used re. a particular page's PR will contain pointers to said page; most likely these will be bi-directional pointers.

4) When spiders return with new data re. a given site, its elements will be searched for pointers re. other potentially affected sites.

5) Then, and only then, will the PR of the affected site be readdressed by the database application which determines PR.

6) In addition to redetermining the affected site's page PRs, the link(s) between the referring and referred to sites will be reconstructed and/or deconstructed as need be.

Hope this makes sense.

Of course data is stored.

My point is and remains that to calculate the PR of a page any existing PR is thrown out and recalculated. PR is calculated and recalculated as it relates to the entire web as a whole and not part of it.

As I said, I don't know how often that happens but they are only limited by the time it takes and nothing more. At some point they need to recalculate. At some point the entire web is affected. Their initial model was able to index (reindex) more than 8.6 million pages a day and it's safe to assume IMO that it's a whole lot more today. Also at that time, the PR for 26 million pages could be calculated in a few hours and again IMO, they can do it considerable faster today.

The amount of new data and reindexed data that they can fetch during the period of time it takes to run the caculation neccessitates them recalculating PR on an ongoing basis using the subsequent updated data. Not only because PR is based upon the web as a whole but at some point the entire web is affected.

Dave

[kgun]

Originally Posted by crankydave

Of course data is stored.

My point is and remains that to calculate the PR of a page any existing PR is thrown out and recalculated. PR is calculated and recalculated as it relates to the entire web as a whole and not part of it.

Do you think of this cite:


"PageRank or PR(A) can be calculated using a simple iterative algorithm, and corresponds to the principal eigenvector of the normalized link matrix of the web. Also, a PageRank for 26 million web pages can be computed in a few hours on a medium size workstation. There are many other details which are beyond the scope of this paper."

[SemAdvance]

Originally Posted by mjtaylor

They do. I do. It *is* part of the algorithm. A patented part of the algorithm. When was the last time you applied for a patent that meant nothing to you?

Yes, PR gets more attention/discussion than h1 tags or titles, but that's probably because it is (apparently) quantifiable and that makes it feel more accessible.


Of course, it makes a difference in rankings. I rarely, if ever, see sites with considerably less PR above any of mine in competitive SERPs ... yes, a PR5 site can outperform a PR8 site on a given term ... but you will not see PR0-3 sites at the top in very many instances ... of course, there are exceptions ... but *generally* the higher PR sites are in the top 10 and the lower ones are not ... so, yeah, it affects SERPs.

Take 'web design' for example - among the top ten sites are 2 PR4s and the rest of PR6 or 7 ... and I would not be at all surprised if the PR4 sites show up with higher PR on the next update ...

So, let's see, if I have a PR3 site, but I know it's possible to move my into PR6 you don't think it makes sense to put that effort into it?




How specious! But really, where does Google have a page of paid links? On SERPs? There is no PR to pass from SERPs as they are not indexed, ranked pages.

Cheers, MJ

Lots of companies apply for patents win them and do nothing with them. We have seen this from Google and others.

You cannot say for sure what the PR of any site is given you do not work for Google so are not privy to reality.

What you can and are doing is speculating on an arbitrary number the search giant downloaded to a desktop toolbar.

Your whole paragraph going back and forth on the sites and PRs shows there is no consistency so therefore it impossible to benchmark

Every google SERP page with a paid advertisement on it is a paid ad without a nofollow tag.

You're trying to argue validity on something that cannot be validated...

If you really think Goolge enforces a nofollow tag explain to me why all of these keywords return websites that would seem to violate googles stance?

buy text links

sell text links

buy links to increase rankings

buy links to increase PageRank

I could list many more....1000s of sites still listed in Googles index and I would venture that in 6 months or even 1 year or more they will still be listed.

Google speaks with forked tongue and the sooner people learn that the better off their business will be.

Peace

[SemAdvance]

Originally Posted by crankydave

Of course data is stored.

heir initial model was able to index (reindex) more than 8.6 million pages a day and it's safe to assume IMO that it's a whole lot more today.

Dave

Where did you come up with this?

Could you please post a Google URL where it states it indexs say 1 million pages daily.

[wige]

Originally Posted by SemAdvance

Every google SERP page with a paid advertisement on it is a paid ad without a nofollow tag.

Why would the ads be marked with nofollow?

1 - They can not be indexed by any search engine, because they can only be viewed by entering a value into a form and visiting the resulting page, and no search engine parses HTML forms with text fields.

2 - They can not be indexed because they use a redirect system (javascript based last time I checked) that search engines are supposed to ignore.

3 - Because Google uses a special bot to analyze and "vet" the landing page of the ad along with other heuristic tools, the links can be considered "trusted" to some extent and may not require the nofollow tag, which was originally created for links that were posted to a site through an untrusted user content system such as a blog comment board.

4 - Because Google is the only site that uses pagerank, Google is the only search engine that would be affected by the nofollow directive, and Google does not spider it's own SERPs.

5 - Because in accordance with Google's terms of service, all adwords links are listed as Sponsored Links, and there is no apparent or obvious attempt to mislead browsers or bots.

[SemAdvance]

Wige

Excellent valid points.

Except one.

It's okay for Google to sell links... but not you or I??

B.S.

;->

[crankydave]

Originally Posted by SemAdvance

Where did you come up with this?

Could you please post a Google URL where it states it indexs say 1 million pages daily.

First suggestion would be to read my posts more carefully. Take careful note of where I said "was able".

My second suggestion would be for you to actually take the time to read the information you yourself post links to. Had you done so, you'd know where the figure comes from.

Kgun has posted a link in this thread so I see no need in reposting it.

Dave