The Ethics of rel="nofollow" [Archive]

webpresence

08-02-2007, 10:56 AM

wige

08-02-2007, 12:48 PM

I should start by saying I have not read the entire patent involving pagerank thoroughly, but I have been wondering how pagerank flows in special circumstances. The patent application does not seem to mention robots.txt or the nofollow tag, but these two developments I believe came into use after that patent application. I know the original concept of the flow of pagerank was that the value of each link was based on the page's rank divided by the number of links. I have been unable to find any reference from Google or any other search engine about what happens to the page rank when a link has nofollow or points to a page that is restricted by robots.txt.

In theory, nofollow would have a more immediate effect on page rank because at the time that the spider is looking at the links, it can see that these links should not be valued. Because robots.txt is an external file, it may not be immediately obvious to the spider, so this may not be taken into account when the PR is divided.

But, there are two things that happen when a page is spidered. The first is "url discovery". This is the process of finding the links that are on the page, and adding them to the list of known URLs to be fetched on a later cycling. The second step is to calculate the value of the links on the page. The original intent of nofollow was to tell the spider that the link should not be added to the future crawls database, to restrict an administrative section of the site, for example. This is the same reason robots.txt was created.

Today, nofollow has been recommended because according to Google pagerank is no longer transferred by links so marked. Nowhere have I seen it definitively stated that these links are removed from the page rank equation.

To sum up in an example, imagine a page with ten links, and a page rank of 10. Originally, each link would be worth 1 point. Now if one link is marked with a nofollow tag, the theory seems to be that the other nine links get 1.111- points and the marked link gets 0. I have seen absolutely no statement that backs this up, and so I wonder if the nine unmarked links keep their value of 1 point and the marked link gets 0. Basically the 1 point is reserved for that link, but because of the nofollow tag it never reaches its destination.

If anyone has any documentation for or against this theory, please post it.

incrediblehelp

08-02-2007, 02:53 PM

To me I don't understand nofollowing a link. It you don't want to pass the juice then why are you linking anways? Also relevant outbound surely help your document rank better.

genetrader

08-02-2007, 04:14 PM

I had heard Leslie Rohde once state that Matt Cutts himself had said that the act of using NOFOLLOW to 'shift' Pagerank was not against the Google TOS and was allowable. If someonse can find that on Matt's blog and verify than the question becomes why would they start doing that?

ArthurKay

08-02-2007, 04:30 PM

I use the "nofollow" attributes for links posted on my site's forum. Not that my site gets enough traffic to worry about it, but I don't like the idea of people posting just to add a link to their site. I've run a few message boards where spammers post constantly with links to all kinds of junk.

I do have a link to my website in my footer (hoping to get some traffic), but I'm not spamming the message boards just trying to build incoming links. I think the two are entirely different.

wige

08-02-2007, 04:37 PM

Matt Cutts posted some comments as to the reason for using nofollow:

In an ideal world, nofollow would only be for untrusted links. Let’s take the example of a forum that wants to avoid linking to spam, but the same advice applies to wikis or any other web software. If an off-domain link is made by an anonymous or unauthenticated user, I’d use nofollow on that link. Once a user has done a certain number of posts/edits, or has been around for long enough to build up trust, then those nofollows could be removed and the links could be trusted. Anytime you have a user that you’d trust, there’s no need to use nofollow links.

This quote indicates that nofollow makes the link no longer an editorial vote for the page, but doesn't discuss much about what is done with the pr that is omitted.

The nofollow tag allows a site to add a link that abstains from being an editorial vote. Using nofollow is a safe way to buy links, because it’s a machine-readable way to specify that a link doesn’t have to be counted as a vote by a search engine.

nipplecharms1

08-02-2007, 04:46 PM

I do have a link to my website in my footer (hoping to get some traffic), but I'm not spamming the message boards just trying to build incoming links. I think the two are entirely different.

How are they different? Seems to me this is a case where you benefit twice; one with the "juice" of this forum and second with readers here that you hope visit Chi-town or are from there.

Not sure you can have it both ways. Well you can but that just makes one greedy! :) :) :). And Greed is good to quote G. Gecko.

But seriously, I have used the no follow because although my customers might get a bennie from visiting a site we link to (and no, before you ask, they are not affiliates for us and we dont get a spif) I do not want the bad "ju-ju" from that site coming back to us. We link to some soft porn-type sites yet I do not want to get smeared by that brush. We have links for our customers, period.

Does that help?

My 2c and a grape soda.

Michael

jrb@w3f.com

08-02-2007, 04:50 PM

I prefer the nofollow too, as I don't like putting stuff in the robots file
or the htaccess file either, because it gives away paths to places I don't
really want spyders going. Now google, and yahoo, and a few others
may follow "the rules" but it is the other spyders that I worry about.
And, they are the ones that do all the damage after looking in to every
possible nook and cranny to get info.

Yeah, nofollow is also good if you have a few affiliate links on your pages.
Since these links all go to the same looking page for all the affiliates, they
will rank poorly with google due to their duplicate content derating.

Also, Google is now starting to penalize websites with a few affiliate links
because they "might" just be redirector pages, and have no valuable content.

So, I put noIndex,noFollow on all my affiliate links, and any other links
that may have a bad taste to googlebot.

Ethical? Use it for the purpose intended, and you should be ok. (IMHO)

RegDCP

08-02-2007, 05:02 PM

To me I don't understand nofollowing a link. It you don't want to pass the juice then why are you linking anways? Also relevant outbound surely help your document rank better.

I use them in affiliate products links.
As I am being "paid" to place a link on the page, I do not want to pass page rank to the affiliate site.

I tried an experiment a while back. I had a page promoting an affiliate product that ranked well in Google. I removed the nofollow and the page disappeared.. I replaced it and the pages was once again listed.

Reg

ArthurKay

08-02-2007, 05:24 PM

Not sure you can have it both ways. Well you can but that just makes one greedy!

I don't argue that WebProWorld may be passing the "juice" onto my site, but they could very easily add the "nofollow" tag to links in this forum. I would still have my link in the footer hoping you would visit the site.

I think the same goes with blogs and other social sites. I simply don't want my site associated with questionable links that could hurt my ranking, so my forum has "nofollow" on every outbound link.

Plus, I doubt my site is going to gain any SEO ground as a result of being on WebProWorld. Do a link population check on my site, and you'll see it's close to zero :-)

SemAdvance

08-02-2007, 05:45 PM

PageRank???

People actually care about it???

Makes no difference in rankings so what good is beating it to death?

Google is not going to tell you the factual number so at best your speculating about an arbitrary digit.....

Also the nofollow tag is not for paid links.....certainly not for links to affiliate programs, as your not trying to boost PageRank..

It is to be used when any link, paid or not, could be found to artificially boost another sites PageRank

If you notice Google has a page full of paid links and none of their pages have nofollow tags. When in Rome do as the Romans do.....

Hey, I wonder when Googles going to kick Yahoo and MSN out of the the Google index for having paid links...

Try not to be Lemmings....

SemAdvance

08-02-2007, 05:52 PM

I use them in affiliate products links.
As I am being "paid" to place a link on the page, I do not want to pass page rank to the affiliate site.

I tried an experiment a while back. I had a page promoting an affiliate product that ranked well in Google. I removed the nofollow and the page disappeared.. I replaced it and the pages was once again listed.

Reg

Unless this experiment lasted over three months between pulling the tag and replacing it I would say it is in no way related to the use of a nofollow tag.

Google would need a full update to see the changes to the page, score and in drop it's ranking and it its a page other than your index page it may need more than one update which would make it 6 months to score it and then after you replaced it another 3 months to pick up the changes.

Lastly there is no reason to place nofollow tags to your affiliate programs, The affiliate is not buying a link from you to inflate PageRank, so that is a misconception you have, and another reason your experiment is not valid.

ccnj

08-02-2007, 08:33 PM

Matt Cutts posted some comments as to the reason for using nofollow:

This quote indicates that nofollow makes the link no longer an editorial vote for the page, but doesn't discuss much about what is done with the pr that is omitted.

OK, so what about using a javascript on the links? For example, look at my linkback page - I have a number of links that are using a javascript to help the PR issue on my side... CCNJ Link Exchanges (http://www.creditcardsnj.com/CCNJlink.php)

Will that do the same thing, or do I waste my time on the javascript?

spronger

08-02-2007, 08:54 PM

There has been a lot said recently about the effectiveness of the rel="nofollow" attribute with regards to internal PR.

I think everyone's missing the point here, apart from wige. Lee asks about the effectiveness of using nofollow on internal links to boost PR on pages designed to generate income, by not parsing PR to non-important pages such as contact, privacy etc. Nofollowing your external and affiliate links is a different issue. Although, most of my affiliate links are redirect pages. So they are still internal pages and nofollowing them my well be a wise thing to do. Whether or not Google sees this as appropriate I've yet to learn.

I don't think using robots.txt would be a proper/effective substitute. This would remove them from the index altogther as thus any internal linking benefit they confer to other pages. The idea is to have your unimportant pages give up their internally assigned PR to the benefit of important pages, not be removed from the index altogether, yes?

I would like to hear from anyone who has applied this strategy i.e. nofollowed all their unimportant pages. What results were achieved? And have you nofollowed your redirect affiliate pages? You don't expect these pages to rank. They only exist to cloak an affiliate link.

PageRank???

People actually care about it???

Makes no difference in rankings so what good is beating it to death?

If it made no difference in ranking nofollow wouldn't exist. It wouldn't need to because your outbound links would give no value for ranking. Google actually gave you a way to withhold PR from sites you deem to be unworthy. They care. They created it for US to use. Agreed, obsessing about PR is not advisable, but dismiss it as having "no difference"? Can't see the logic.

Also the nofollow tag is not for paid links.....

Says who? I believe that is one of the reasons it was specifically created for, so a paid link would not confer PR. This would allow you to buy/sell links for traffic (legitimate advertising in Google's view) and not for the sole purpose of boosting PR.

Steve

SemAdvance

08-03-2007, 07:20 AM

OK, so what about using a javascript on the links? For example, look at my linkback page - I have a number of links that are using a javascript to help the PR issue on my side... CCNJ Link Exchanges (http://www.creditcardsnj.com/CCNJlink.php)

Will that do the same thing, or do I waste my time on the javascript?

To answer your question Google does not parse links within javascript.

If it made no difference in ranking nofollow wouldn't exist. It wouldn't need to because your outbound links would give no value for ranking. Google actually gave you a way to withhold PR from sites you deem to be unworthy. They care. They created it for US to use. Agreed, obsessing about PR is not advisable, but dismiss it as having "no difference"? Can't see the logic.

If it did make a difference then the sites with the highest PR would make up most of the # 1 listings for any keyword term searched.

Since Yahoo & Google are the sites with the highest PR they would dominate the SERPs upper positions.

They don't, so PR does not matter to obtain top results. It is but 1 factor in an algorithm of 100+ parts.

Originally Posted by SemAdvance View Post

Also the nofollow tag is not for paid links.....

Says who? I believe that is one of the reasons it was specifically created for, so a paid link would not confer PR. This would allow you to buy/sell links for traffic (legitimate advertising in Google's view) and not for the sole purpose of boosting PR.

Steve

Originally Posted by mattcutts.com
In an ideal world, nofollow would only be for untrusted links. Let’s take the example of a forum that wants to avoid linking to spam, but the same advice applies to wikis or any other web software. If an off-domain link is made by an anonymous or unauthenticated user, I’d use nofollow on that link. Once a user has done a certain number of posts/edits, or has been around for long enough to build up trust, then those nofollows could be removed and the links could be trusted. Anytime you have a user that you’d trust, there’s no need to use nofollow.

I'll take two sentences out of the paragraph to validate my statement.

In an ideal world, nofollow would only be for untrusted links.

Anytime you have a user that you’d trust, there’s no need to use nofollow.

If you can show me where it says anything about the links being paid, for I will gladly say you are correct.

Till such time, payment has no bearing in the intended usage of the links.

As I've said and written for the last 4 years

There is no link, like a natural link......

everything else is artificial and devalued most often.

webpresence

08-03-2007, 08:12 AM

SemAdvance

08-03-2007, 08:36 AM

Think we're moving away from the subject slightly here! Has anyway done any research to determine the effectiveness of using nofollow on non-themed internal links (contact, privacy etc.) to boost PR on themed pages (optimised pages).

Can a robot exclusion be used as an alterantive to nofollow?

Contact us, privacy and the like pages won't rank well and the search engines expect most sites to have these so it's unlikely an nofollow is needed as these page have no value to the search engines or your visitors as far as distributing 'relevant' information.

I doubt the exercise would raise PageRank as the value of an internal link has to be so minimal, it likely could not raise or lower the value.

Another aspect you would need to look at this from is the competition level of the keyword term....in some cases where there is little or no competition it may have an effect.

In the amount of time it would take to conduct such an experiment (3+ months between PR updates) one could build a few one way links to the sites main pages and add much more value.

Robots txt is used to stop robots from crawling folders and files, so it could work in the same way but again probably futile.

From Matt Cutts: Anytime you have a user that you’d trust, there’s no need to use nofollow.

That also should answer why this test would be futile.

kgun

08-03-2007, 08:50 AM

wige

08-03-2007, 09:32 AM

To answer your question Google does not parse links within javascript.
Actually, on my site I have had AJAX content that was only accessible through a javascript indexed by Google. Google does claim they do not crawl what is in a javascript function, but this content was indexed three days after I added AJAX to the server, and I have found no other way Google could have discovered the URL.

If you can show me where it says anything about the links being paid, for I will gladly say you are correct.
The quote you cited was pretty much the only paragraph in that entire blog posting that didn't talk about using nofollow on paid links, which I think was the topic of the posting.

crankydave

08-03-2007, 11:09 AM

As far as the "ethics" go, PR gets "manipulated" every day by every single site through the use of their navigation or lack thereof. Directing "value" where they want it directed to.

Often, links to pages like "View Cart" or "Contact" are placed site wide for the convenience of vistors. Because Google uses page rank to determine which "allotted" pages from a site get placed into the RI, with the remaining going into the SI, directing PR to the "content" pages becomes very important. Keeping in mind that recent developements regarding the SI hints towards Google's desire for all queries to reach the SI makes this less of a factor.

Personally, I don't see an issue in using internal <nofollow> tags when done so as a means to tell SE's which of your pages are "important" to the index. I know which of my pages are most important. And again, we do this all the time through our links and their on page placement.

I think where the trepidation comes in is more because of the definition of the tag rather than its use. Things like "untrusted" or "can't vouch for" for external links.

If internal use of the tag was defined as "not important page" then I would suspect that using it would be looked at differently. Let's not forget that the <nofollow> tag is not for your visitors. It is to help SE's determine which pages are "important".

Dave

RegDCP

08-03-2007, 03:37 PM

Unless this experiment lasted over three months between pulling the tag and replacing it I would say it is in no way related to the use of a nofollow tag.

Google would need a full update to see the changes to the page, score and in drop it's ranking and it its a page other than your index page it may need more than one update which would make it 6 months to score it and then after you replaced it another 3 months to pick up the changes.

Lastly there is no reason to place nofollow tags to your affiliate programs, The affiliate is not buying a link from you to inflate PageRank, so that is a misconception you have, and another reason your experiment is not valid.

Sorry SemAdvance, I cannot agree with you.
As only one change was made on the page, I would have to conclude that the nofollow tag was responsible.
Why do you think that it would take 6 months?
I see changes to pages resulting in ranking changes showing in MUCH less time.
The page disappeared following the removal of the nofollow tag. The page reappeared after it was put back in. The experiment worked.

While the affiliate is not paying me to put the link, it is in effect, a paid link as I stand to make a profit from it.
If I were to leave the nofollow off the link, I would be telling Google that I endorse the page rank organically and that is not the case. It has nothing to do with what the affiliate site is buying, but pertains to the "honesty" in my page's linking.

Reg

wige

08-03-2007, 03:48 PM

As only one change was made on the page, I would have to conclude that the nofollow tag was responsible.
Why do you think that it would take 6 months?
I see changes to pages resulting in ranking changes showing in MUCH less time.
Ranking and PageRank are only loosely connected. The nofollow tag probably did not have an effect on the page rank in a short time, because Google calculates PageRank on a semi-annual basis, not live as it spiders sites. However, removing the tag may have caused the spider to be suspicious of those links for some other reason that was covered by the fact that those tags were in place.

SemAdvance

08-03-2007, 09:46 PM

Sorry SemAdvance, I cannot agree with you.
As only one change was made on the page, I would have to conclude that the nofollow tag was responsible.
Why do you think that it would take 6 months?
I see changes to pages resulting in ranking changes showing in MUCH less time.
The page disappeared following the removal of the nofollow tag. The page reappeared after it was put back in. The experiment worked.

While the affiliate is not paying me to put the link, it is in effect, a paid link as I stand to make a profit from it.
If I were to leave the nofollow off the link, I would be telling Google that I endorse the page rank organically and that is not the case. It has nothing to do with what the affiliate site is buying, but pertains to the "honesty" in my page's linking.

Reg

You're free to disagree and no need to be sorry. I never say sorry.

That's why the USA rocks and other countries like Iran suck, freedom of speech!

Why do I think Google would take that long? Because they do not re-index pages that often. Google only updates links every quarter and therefore would only update no follow tags that often as well since these are included in the link itself.

More than likely you are seeing effects of links and other issues happening in the background or in the past month or two.

Peace!

spronger

08-04-2007, 09:53 AM

If it did make a difference then the sites with the highest PR would make up most of the # 1 listings for any keyword term searched.

Since Yahoo & Google are the sites with the highest PR they would dominate the SERPs upper positions.

They don't, so PR does not matter to obtain top results. It is but 1 factor in an algorithm of 100+ parts.

You're missing the point. Google PageRank, as displayed in the Google toolbar as a number from 0 - 10, is NOT the same PageRank which the Google algorithm uses to rank pages. They are totally different. One is a tool which G provides as an indicator of the "importance" of a particular page. It has limited value other than showing how well linked a particular page is. The other forms the basis of the Google algo. Just read the patent if you don't believe me. It's no secret.

You're right when you say that a high PR page - as in high TOOLBAR PageRank - doesn't always equate to a high ranking. A site can have a gazillion IBLs and thus a high PR, but that doean't mean it will rank well for that particular query. The toolbar PR matters very little in this case, and yes, it's not worth obsessing about.

But PageRank, as in the ACTUAL PageRank you distribute via links, matters quite a lot in how pages are ranked. Hence, the reason for the nofollow tag. Google don't want you confering PR in various circumstances, which INCLUDES paid links.

Why do I think Google would take that long? Because they do not re-index pages that often. Google only updates links every quarter and therefore would only update no follow tags that often as well since these are included in the link itself.

This is incorrect. Again you're confusing toolbar PR with actual PR. The toolbar is only updated periodically as you say. New links are found and indexed daily and the effects on ranking can happen just as quickly.

SemAdvance

08-04-2007, 11:36 AM

You're missing the point. Google PageRank, as displayed in the Google toolbar as a number from 0 - 10, is NOT the same PageRank which the Google algorithm uses to rank pages. They are totally different. One is a tool which G provides as an indicator of the "importance" of a particular page. It has limited value other than showing how well linked a particular page is. The other forms the basis of the Google algo. Just read the patent if you don't believe me. It's no secret.

You're right when you say that a high PR page - as in high TOOLBAR PageRank - doesn't always equate to a high ranking. A site can have a gazillion IBLs and thus a high PR, but that doean't mean it will rank well for that particular query. The toolbar PR matters very little in this case, and yes, it's not worth obsessing about.

But PageRank, as in the ACTUAL PageRank you distribute via links, matters quite a lot in how pages are ranked. Hence, the reason for the nofollow tag. Google don't want you confering PR in various circumstances, which INCLUDES paid links.

This is incorrect. Again you're confusing toolbar PR with actual PR. The toolbar is only updated periodically as you say. New links are found and indexed daily and the effects on ranking can happen just as quickly.

I actually understand the algorithm quite well. Toolbar PR is the PageRank of your site 3 months ago. I also understand how PageRank is used in the algorithm to determine rankings.

New links do effect the rankings indeed, this however is typically what you see with new sites that pop up for a bit and then fall back down the rankings to where they belong.

Google cannot assign trust immediately and being as you said new links do effect rankings and overzealous webmaster could pay to have a mutlitude of posting done on social bookmarking sites for example.

Once the algorithm is tripped for acquiring too many links, too quickly the brakes are applied.

Also link changes do effect the SERPs however on page changes are not quickly scored or re-indexed.

Also if any of this were true, could you please tell me which of the ten sites listed for the following term has seen it's PR devalued and it's listing removed??

Google Search:

buy links

Let me know what you find....

Also while you are there, please take a lookat the paid ads.

How do you propose Google should cancel all the Adwords Advertisers accounts that promote the service with the intent of improving Google search engine rankings??

How will they absorb the lost revenues? How will they inform the shareholders of the devalued stock prices once they do??

And thats just one keyword term...I'm sure there are 100s others filled with sites with PR, rank well, and sell, or buy links, and do not employ nofollow tags.

Google says one thing

does another......

crankydave

08-04-2007, 12:11 PM

I have had on page changes reflected in the SERP's in as little as 1 hour. I have had new pages listed and ranked in as little as 1 hour and remain there without a sitemap.

On site changes can be and are scored quickly.

Nothing wrong with buying or selling links. Depends upon the purpose. It's not about having their PR devalued it's about having their ability to pass PR devalued and that has and can be done internally. The page still has PR but is treated much like a dangling link and removed from the actual calculation and then added back in after the fact so it has no effect.

Dave

RegDCP

08-04-2007, 12:12 PM

You're free to disagree and no need to be sorry. I never say sorry.

That's why the USA rocks and other countries like Iran suck, freedom of speech!

Why do I think Google would take that long? Because they do not re-index pages that often. Google only updates links every quarter and therefore would only update no follow tags that often as well since these are included in the link itself.

More than likely you are seeing effects of links and other issues happening in the background or in the past month or two.

Peace!

SemAdvance, I was being polite when I said "sorry".

If you think that the SERPs are calculated or recalculated on a quarterly bases, then you cannot understand how databases work.
It would be quite strange if this were to happen as this would mean storing a great number of site algo results, then importing them into the various databases all at once.

This would also mean that a newly spidered site would not show up until the quarter, which is not the case.

As my test showed the page disappearing shortly after the removal of the nofollow tag and reappearing shortly after its reinstatement, I would suspect that updates are made to the SERPs on a continual, real time basis.

Reg

SemAdvance

08-04-2007, 12:47 PM

SemAdvance, I was being polite when I said "sorry".

If you think that the SERPs are calculated or recalculated on a quarterly bases, then you cannot understand how databases work.
It would be quite strange if this were to happen as this would mean storing a great number of site algo results, then importing them into the various databases all at once.

This would also mean that a newly spidered site would not show up until the quarter, which is not the case.

As my test showed the page disappearing shortly after the removal of the nofollow tag and reappearing shortly after its reinstatement, I would suspect that updates are made to the SERPs on a continual, real time basis.

Reg

I'd rather we don't have a match of wits or try to determine what one or another knows about things. I have worked with the first set of algorithms developed over 20 years ago within the retail & pharmaceutical sectors

There is but one algorithm and it is run over the server. Others call it a search spider. To each his own.

Others can learn more here

The Anatomy of a Search Engine (http://infolab.stanford.edu/~backrub/google.html)

As for the speed of which you and Dave see changes, I have no clue what the level of competition is for the keywords you are using.

I will grant you that changes for 'khazakistan woolen area rugs' will occur much sooner than say 'Manhattan condo'.

I would agree I see changes quicker for some terms than others. Most of the ones I do see in the more competitive arena, take quite a while to rank, and the walk up from position # 8 or 9 to # 1, 2 or 3 can take a year or more.

In the various categories we work in, mileage may vary

I have had on page changes reflected in the SERP's in as little as 1 hour. I have had new pages listed and ranked in as little as 1 hour and remain there without a sitemap.

Dave

Sitemap usage has no effect on rankings whatsoever.

It simply enables the spider to index pages it otherwise would have difficulty parsing.

Peace

RegDCP

08-04-2007, 01:05 PM

As for the speed of which you and Dave see changes, I have no clue what the level of competition is for the keywords you are using.

2,290,000 competing pages for the search term so it is fairly competative.
My page is #1.

Reg

crankydave

08-04-2007, 01:11 PM

Sitemap usage has no effect on rankings whatsoever.

I was simply pointing out the the changes were found via the web and not via a submission. I said nothing at all about it affecting ranking.

As far as the level competiton. Not relevant to the statement you made which was...

Also link changes do effect the SERPs however on page changes are not quickly scored or re-indexed.

This is wrong. Simply because you cannot see any changes in some instances doesn't mean they are not scored.

Why do I think Google would take that long? Because they do not re-index pages that often. Google only updates links every quarter and therefore would only update no follow tags that often as well since these are included in the link itself.

This is also wrong.

Links and their value are updated internally constantly. Just like PR.

Dave

deepsand

08-04-2007, 02:21 PM

<snip>

There is but one algorithm and it is run over the server. Others call it a search spider. To each his own.

<snip>

Actually, the servers which handles the spiders, and the database server(s) which store and analyze the data retrieved by the spiders are quite independent & different.

The database servers constinuously both accept new data from the spiders, and analyze it, along with previously accumulated data, in real time.

kgun

08-06-2007, 10:57 AM

I actually understand the algorithm quite well. Toolbar PR is the PageRank of your site 3 months ago. I also understand how PageRank is used in the algorithm to determine rankings.

New links do effect the rankings indeed, this however is typically what you see with new sites that pop up for a bit and then fall back down the rankings to where they belong.

Google cannot assign trust immediately and being as you said new links do effect rankings and overzealous webmaster could pay to have a mutlitude of posting done on social bookmarking sites for example.

Once the algorithm is tripped for acquiring too many links, too quickly the brakes are applied.

Also link changes do effect the SERPs however on page changes are not quickly scored or re-indexed.

I noted this as interesting. Can other members confirm that?

My own minor conclusion after rereading this whole thread:

Nofollowing links on a document / page makes the document more foccused in my view. Foccused and minimalistic documents are better then too complex and unstructured documents IMO.

wige

08-06-2007, 11:50 AM

kgun

08-06-2007, 04:17 PM

I think that was a good explanation.

Let PR(t) = PageRank in periode t.

Then

PR(t)=Delta[PR(t)]+K1*PR(t-1)+K2*PR(t-2) + ... + Kn*PR(t-n), where

Delta[PR(t)] is the new flow of PR from Period t-1 to t and the K1, K2, ..., Kn are coefficients computed in one way or an other. A simplified version is:

PR(t)=Delta[PR(t)]+K1*PR(t-1) and K1=1, so

PR(t)=Delta[PR(t)]+PR(t-1).

In words if t=2007 and t-1=2006.

The pagerank in 2007 equals the flow of pagerank from 2006 to 2007 + the pagerank in 2006.

Then back to the etichs of nofollow:
My own view: Nothing unetical in using nofollow on links that are irrelevant to the message of the site, like affilliated links even if they are related contextually.

crankydave

08-06-2007, 05:41 PM

deepsand

08-06-2007, 06:33 PM

Internal PR is calculated constantly. Previous values cannot be used. Each new calculation would need to use the initial value assigned to a page in order for it to work.

How often this is done would depend solely on how long it takes to the computer to run the desired number of iterations using the known data, export it, and start again.

Dave

1) With a properly constructed mathematical expression, one need not store all of the previously collected values of the data elements involved. Rather, one need only store the most recently calculated sub-results for each element, and then compute the new sub-results & total result using the weighting coefficients assigned to each element.

2) PR cannot be "constantly" calculated in real time; dynamic real time results require dynamic real time input for the various data elements. As the input is provided by the spiders, and is therefore static data derived from the last time that a site was crawled, a new PR can only result when the spiders present new data.

crankydave

08-06-2007, 07:29 PM

1) With a properly constructed mathematical expression, one need not store all of the previously collected values of the data elements involved. Rather, one need only store the most recently calculated sub-results for each element, and then compute the new sub-results & total result using the weighting coefficients assigned to each element.

2) PR cannot be "constantly" calculated in real time; dynamic real time results require dynamic real time input for the various data elements. As the input is provided by the spiders, and is therefore static data derived from the last time that a site was crawled, a new PR can only result when the spiders present new data.

I wasn't saying or trying to suggest that PR was caculated in "real time". Just that internal PR calculations are constant, continuous, or a rolling calculation if you will, limited only by the time it takes to make the calculation and export the results.

The data used is static. Spiders crawl and present new data on a constant, continuous, basis. A single link *can* change the PR of every single web page depending upon the the decimal point you want to take it to.

I have no idea how long it takes Google to run the desired number of iterations and export the results with a known data set. My point was that doing so was only limited to the time it takes the computer(s) to run the caculations, export the results, then move on to the current known data set and start over again, that each subsequent calculation started from the base value and did not use previous calculation results.

Dave

SemAdvance

08-06-2007, 07:41 PM

I can confirm, especially for external links, that links are processed almost immediately. I have posted links to an existing page in a few related blogs and, with no changes to the actual page, watched the page jump as much as ten positions within 48hours.

As far as pagerank, there are I think two different pageranks. One is immediate, and is a factor of the pagerank that has been calculated by previous pagerank calculations combined with the value of incoming links to the page. This is the pagerank you can directly affect in the short term by building links. However, the value of those incoming links gets recalculated every few months, kind of like a market correction, where pr is redistributed. Unfortunately I am horrible at math and don't really know the best way to describe this process.

I could have sworn I said this very thing.

Links can have an effect on low to mid level competition terms, but their value is degraded when it is determined they are indeed artificial as most links are today.

They may be processed immediately however they are not scored immediately.

PR is constant and updated live daily to Google employees.

Toolbar PR is an export to keep webmaster warm & fuzzy....

I will say Googles propaganda is having its effect... as I see many large sites going to nofollow for all external links..

Gotta love it ...makes my knocking competitors off even easier.

deepsand

08-06-2007, 10:26 PM

I wasn't saying or trying to suggest that PR was calculated in "real time". Just that internal PR calculations are constant, continuous, or a rolling calculation if you will, limited only by the time it takes to make the calculation and export the results.

The data used is static. Spiders crawl and present new data on a constant, continuous, basis. A single link *can* change the PR of every single web page depending upon the the decimal point you want to take it to.

I have no idea how long it takes Google to run the desired number of iterations and export the results with a known data set. My point was that doing so was only limited to the time it takes the computer(s) to run the calculations, export the results, then move on to the current known data set and start over again, that each subsequent calculation started from the base value and did not use previous calculation results.

Dave

Based on my nearly 50 yrs. experience with DP, as it was once known, I would offer the following re. the inner working & hidden mechanisms here involved.

1) Results will be stored, to provide input for future iterations, so as to avoid as much re-computation as possible. Storage space is dirt cheap compared to CPU cycles, both in terms of the physical devices & operating costs.

2) Once the PR for a particular page has been determined, such will remain static until such time as new relevant input is received.

3) Relevancy will be deterministic. Other entities in the database, which serve to provide data values for those elements used re. a particular page's PR will contain pointers to said page; most likely these will be bi-directional pointers.

4) When spiders return with new data re. a given site, its elements will be searched for pointers re. other potentially affected sites.

5) Then, and only then, will the PR of the affected site be readdressed by the database application which determines PR.

6) In addition to redetermining the affected site's page PRs, the link(s) between the referring and referred to sites will be reconstructed and/or deconstructed as need be.

Hope this makes sense.

mjtaylor

08-07-2007, 08:40 AM

PageRank???

People actually care about it???

They do. I do. It *is* part of the algorithm. A patented part of the algorithm. When was the last time you applied for a patent that meant nothing to you?

Yes, PR gets more attention/discussion than h1 tags or titles, but that's probably because it is (apparently) quantifiable and that makes it feel more accessible.

Makes no difference in rankings so what good is beating it to death?

Of course, it makes a difference in rankings. I rarely, if ever, see sites with considerably less PR above any of mine in competitive SERPs ... yes, a PR5 site can outperform a PR8 site on a given term ... but you will not see PR0-3 sites at the top in very many instances ... of course, there are exceptions ... but *generally* the higher PR sites are in the top 10 and the lower ones are not ... so, yeah, it affects SERPs.

Take 'web design' for example - among the top ten sites are 2 PR4s and the rest of PR6 or 7 ... and I would not be at all surprised if the PR4 sites show up with higher PR on the next update ...

So, let's see, if I have a PR3 site, but I know it's possible to move my into PR6 you don't think it makes sense to put that effort into it?

If you notice Google has a page full of paid links and none of their pages have nofollow tags. When in Rome do as the Romans do.....

How specious! But really, where does Google have a page of paid links? On SERPs? There is no PR to pass from SERPs as they are not indexed, ranked pages.

Cheers, MJ

mjtaylor

08-07-2007, 08:47 AM

Unless this experiment lasted over three months between pulling the tag and replacing it I would say it is in no way related to the use of a nofollow tag.

Google would need a full update to see the changes to the page, score and in drop it's ranking and it its a page other than your index page it may need more than one update which would make it 6 months to score it and then after you replaced it another 3 months to pick up the changes.

Lastly there is no reason to place nofollow tags to your affiliate programs, The affiliate is not buying a link from you to inflate PageRank, so that is a misconception you have, and another reason your experiment is not valid.

Nofollow is no vote - nothing to do with paid or not paid. It was an attribute created to try and stop people from spamming blogs for PR ... it is only recently that it was suggested as a way to show no vote for paid links ...

More importan6ly, I am not at all sure on what you base your suggestion that it would take 3 months to test a change on a page. I make frequent changes and the position in the SERPs changes the same day as the page is spidered.

Perhaps some years ago Google was slow to assimilate change, but that doesn't seem to be the case anymore.

I am confused ... in an earlier post you said PR has not effect on ranking and now you seem to be saying it does ...

MJ

kgun

08-07-2007, 08:50 AM

PR is constant and updated live daily to Google employees.

Toolbar PR is an export to keep webmaster warm & fuzzy....

I will say Googles propaganda is having its effect... as I see many large sites going to nofollow for all external links..

Gotta love it ...makes my knocking competitors off even easier.

Like temperature outside my house, that is updated daily by a very high dimensional system.

kgun

08-07-2007, 09:04 AM

Of course, it makes a difference in rankings. I rarely, if ever, see sites with considerably less PR above any of mine in competitive SERPs ... yes, a PR5 site can outperform a PR8 site on a given term ... but you will not see PR0-3 sites at the top in very many instances ... of course, there are exceptions ... but *generally* the higher PR sites are in the top 10 and the lower ones are not ... so, yeah, it affects SERPs.

Take 'web design' for example - among the top ten sites are 2 PR4s and the rest of PR6 or 7 ... and I would not be at all surprised if the PR4 sites show up with higher PR on the next update ...

So, let's see, if I have a PR3 site, but I know it's possible to move my into PR6 you don't think it makes sense to put that effort into it?

How specious! But really, where does Google have a page of paid links? On SERPs? There is no PR to pass from SERPs as they are not indexed, ranked pages.

Agree, and in addition it is not always enough to look at the PageRank of the actual page.

If the Google alogorithme is a hiearchical system, that it most probably is, the PageRank of the home page and the sum of related internal pages may also be important. To put it in mathematical / economic terms, it is input in the production function that produces SERP's as output.

The relative importance of PageRank in one form or an other need not be a constant and the importance will most probably be changed programmatically in future versions of the algorithme.

crankydave

08-07-2007, 10:46 AM

Based on my nearly 50 yrs. experience with DP, as it was once known, I would offer the following re. the inner working & hidden mechanisms here involved.

1) Results will be stored, to provide input for future iterations, so as to avoid as much re-computation as possible. Storage space is dirt cheap compared to CPU cycles, both in terms of the physical devices & operating costs.

2) Once the PR for a particular page has been determined, such will remain static until such time as new relevant input is received.

3) Relevancy will be deterministic. Other entities in the database, which serve to provide data values for those elements used re. a particular page's PR will contain pointers to said page; most likely these will be bi-directional pointers.

4) When spiders return with new data re. a given site, its elements will be searched for pointers re. other potentially affected sites.

5) Then, and only then, will the PR of the affected site be readdressed by the database application which determines PR.

6) In addition to redetermining the affected site's page PRs, the link(s) between the referring and referred to sites will be reconstructed and/or deconstructed as need be.

Hope this makes sense.

Of course data is stored.

My point is and remains that to calculate the PR of a page any existing PR is thrown out and recalculated. PR is calculated and recalculated as it relates to the entire web as a whole and not part of it.

As I said, I don't know how often that happens but they are only limited by the time it takes and nothing more. At some point they need to recalculate. At some point the entire web is affected. Their initial model was able to index (reindex) more than 8.6 million pages a day and it's safe to assume IMO that it's a whole lot more today. Also at that time, the PR for 26 million pages could be calculated in a few hours and again IMO, they can do it considerable faster today.

The amount of new data and reindexed data that they can fetch during the period of time it takes to run the caculation neccessitates them recalculating PR on an ongoing basis using the subsequent updated data. Not only because PR is based upon the web as a whole but at some point the entire web is affected.

Dave

kgun

08-07-2007, 11:16 AM

Of course data is stored.

My point is and remains that to calculate the PR of a page any existing PR is thrown out and recalculated. PR is calculated and recalculated as it relates to the entire web as a whole and not part of it.

Do you think of this cite:

"PageRank (http://infolab.stanford.edu/~backrub/google.html) or PR(A) can be calculated using a simple iterative algorithm, and corresponds to the principal eigenvector of the normalized link matrix of the web. Also, a PageRank for 26 million web pages can be computed in a few hours on a medium size workstation. There are many other details which are beyond the scope of this paper."

SemAdvance

08-07-2007, 02:52 PM

They do. I do. It *is* part of the algorithm. A patented part of the algorithm. When was the last time you applied for a patent that meant nothing to you?

Yes, PR gets more attention/discussion than h1 tags or titles, but that's probably because it is (apparently) quantifiable and that makes it feel more accessible.

Of course, it makes a difference in rankings. I rarely, if ever, see sites with considerably less PR above any of mine in competitive SERPs ... yes, a PR5 site can outperform a PR8 site on a given term ... but you will not see PR0-3 sites at the top in very many instances ... of course, there are exceptions ... but *generally* the higher PR sites are in the top 10 and the lower ones are not ... so, yeah, it affects SERPs.

Take 'web design' for example - among the top ten sites are 2 PR4s and the rest of PR6 or 7 ... and I would not be at all surprised if the PR4 sites show up with higher PR on the next update ...

So, let's see, if I have a PR3 site, but I know it's possible to move my into PR6 you don't think it makes sense to put that effort into it?

How specious! But really, where does Google have a page of paid links? On SERPs? There is no PR to pass from SERPs as they are not indexed, ranked pages.

Cheers, MJ

Lots of companies apply for patents win them and do nothing with them. We have seen this from Google and others.

You cannot say for sure what the PR of any site is given you do not work for Google so are not privy to reality.

What you can and are doing is speculating on an arbitrary number the search giant downloaded to a desktop toolbar.

Your whole paragraph going back and forth on the sites and PRs shows there is no consistency so therefore it impossible to benchmark

Every google SERP page with a paid advertisement on it is a paid ad without a nofollow tag.

You're trying to argue validity on something that cannot be validated...

If you really think Goolge enforces a nofollow tag explain to me why all of these keywords return websites that would seem to violate googles stance?

buy text links

sell text links

buy links to increase rankings

buy links to increase PageRank

I could list many more....1000s of sites still listed in Googles index and I would venture that in 6 months or even 1 year or more they will still be listed.

Google speaks with forked tongue and the sooner people learn that the better off their business will be.

Peace

SemAdvance

08-07-2007, 02:58 PM

Of course data is stored.

heir initial model was able to index (reindex) more than 8.6 million pages a day and it's safe to assume IMO that it's a whole lot more today.

Dave

Where did you come up with this?

Could you please post a Google URL where it states it indexs say 1 million pages daily.

wige

08-07-2007, 04:10 PM

Every google SERP page with a paid advertisement on it is a paid ad without a nofollow tag.
Why would the ads be marked with nofollow?

1 - They can not be indexed by any search engine, because they can only be viewed by entering a value into a form and visiting the resulting page, and no search engine parses HTML forms with text fields.

2 - They can not be indexed because they use a redirect system (javascript based last time I checked) that search engines are supposed to ignore.

3 - Because Google uses a special bot to analyze and "vet" the landing page of the ad along with other heuristic tools, the links can be considered "trusted" to some extent and may not require the nofollow tag, which was originally created for links that were posted to a site through an untrusted user content system such as a blog comment board.

4 - Because Google is the only site that uses pagerank, Google is the only search engine that would be affected by the nofollow directive, and Google does not spider it's own SERPs.

5 - Because in accordance with Google's terms of service, all adwords links are listed as Sponsored Links, and there is no apparent or obvious attempt to mislead browsers or bots.

SemAdvance

08-07-2007, 04:30 PM

Wige

Excellent valid points.

Except one.

It's okay for Google to sell links... but not you or I??

B.S.

;->

crankydave

08-07-2007, 04:32 PM

Where did you come up with this?

Could you please post a Google URL where it states it indexs say 1 million pages daily.

First suggestion would be to read my posts more carefully. Take careful note of where I said "was able".

My second suggestion would be for you to actually take the time to read the information you yourself post links to. Had you done so, you'd know where the figure comes from.

Kgun has posted a link in this thread so I see no need in reposting it.

Dave

wige

08-07-2007, 04:48 PM

Where does it say you are not allowed to buy or sell sponsored links? All the references I have been able to find have been in reference to buying and selling links purely to gain pagerank from Google, which is considered a deceptive practice. Selling links similar to Google's Adwords program has been around much longer than even Google itself. However, back in the early days of the Internet, these purchased links were always shown as being "sponsored" or "paid" or "advertisement."

All I see Google saying is "hey, we see people buying contextual links, trying to hide them as part of the text or disguise them as editorial links, and we are going to make our spiders hunt for these types of deceptive links and punish the sites where we find them. If you sell links, and they are not clearly marked or obvious, and don't want to take the risk of losing ranking, add the nofollow tag."

If Google applies the same rule to their own SERPs, I can't see how their spider or any other, would miss the clearly labeled paid links as being editorial. Matt Cutts even made similar comments in his blog that part of this arose from e-mails he had seen with instructions about hiding paid links in the text, and how they were going to get more aggressive in trying to find deceptive links and suggesting nofollow as a way to mitigate the risk from whatever heuristic analysis they end up implementing.

SemAdvance

08-07-2007, 04:56 PM

Wige

Read this

If you sell links, and they are not clearly marked or obvious, and don't want to take the risk of being punished, add the nofollow tag."

Who in the hell is Google to punish anyone??????

I guarantee businesses can be built and successful without annoying little threats from Google.

Google can kick everyone out of it's index but then they won't have the inventory to sell their ads I mean links...

Dave

Just wanted to be sure its okay to post things without citing sources ;->

wige

08-07-2007, 05:10 PM

Ok, I rephrased the paraphrased quote. As far as who is Google to punish anyone, they are the ones with 60% or better market share. If they don't trust you, they have the right not to link to you, and if you want to buy a link they have the right to mark the link as paid. It seems like that is exactly what they are asking from webmasters.

As far as citing sources, that is my point. I can't find any that say there is any policy against paid links, the only thing I can find is a policy against deceptive content as mentioned at Google Terms of Service (http://www.google.com/accounts/TOS) and Matt Cutts' various comments such as Quick comment on nofollow (http://www.mattcutts.com/blog/quick-comment-on-nofollow/)

SemAdvance

08-07-2007, 05:33 PM

wige

08-07-2007, 05:52 PM

Right on!! The puppet has spoken and all must fear the mighty Google.....
Heres a quick mathematical equation on how to regard issues from Google and Matt Cutts
Google + /- MattCutts = Propaganda

What's your point?

Google isn't the only company that can set the rules through nothing more than the sheer power of their massive market share. As soon as Microsoft even mentions a new version of IE, don't you prepare your site to make sure it will be viewable to the (arguably) 70% of Internet users that use IE? Making sure your site conforms to MS's latest interpretation of the standards? The whole point of having a web site for your business is to make your site available to the largest audience possible, isn't it? Are you suggesting that a web designer should not make every effort to ensure their content is available and friendly to the sources that have the largest market shares?

As far as Matt Cutts being Google propaganda, until I get a job at Google, or find someone else who talks about the inner workings of the search engine, I will listen to the advice and insights he posts as to how Google looks at pages, if that means my site can do better.

mjtaylor

08-07-2007, 05:52 PM

Wige

Excellent valid points.

Except one.

It's okay for Google to sell links... but not you or I??

B.S.

;->

Glad you can see wige's points.

Since Google SERPs do not pass PR, it would seem you are not comparing apples to apples ...

Google has not stated that it's not okay to sell links. It said (through Matt Cutts) that it would prefer to prevent PR from passing via paid links and will attempt via the algorithm to do so. And Cutts did suggest that rel-nofollow would be a great way to help Google do that.

Google has not, as far as I can see, done anything yet to devalue the PR of paid link directories; but that doesn't mean it won't or that there it has not already done so. I recently bought a slew of links for a few sites from excellent high PR paid link directories and I saw no change in SERPs ... whereas a few non-paid links from other sources seemed to have a most positive effect.

I say *seem* a lot, because as you noted earlier, SemAd, this is all speculating. Indeed, it is. Precious little is known absolutely about SEO, but with the information on forums like these and other places where SEO experience is shared, we can come to some logical conclusions that are highly effective in many cases.

MJ

mjtaylor

08-07-2007, 06:09 PM

Hi All,

There has been a lot said recently about the effectiveness of the rel="nofollow" attribute with regards to internal PR.

Many SEO professionals believe that the rel="nofollow" is the golden nugget for funnelling PR from non-themed pages (contact, terms, clients, login pages etc.) to their most important on-topic pages. The Effect? The PR that was allocated to the non-themed pages will then be passed on to the on-topic pages, thus helping rankings of the latter.

I can see how this would work short term, but I can also see how in the long term the SE's will treat this as a black hat technique as the site owner is effectively manipulating PR.

An alternative to 'nofollow' would be to use a robots.txt file and exclude spiders from indexing the non-themed pages - the big question however; Would this technique accomplish the same goal and funnel PR from the non-themed to the on-topic pages?

IMO the robots.txt method will accomplish PR funnelling and is the truly ethical way to go, you may disagree though. Let's discuss!

Thanks for reading! Lee

First of all, Lee, my apologies to you for participating in the hijack of your thread.

To your point and question: I don't believe that Google will take issue with the use of rel=nofollow internally but I do think the use of robots.txt file would be the way to go ... there was a recent thread Internal Linking Strategies and Structure (http://www.webproworld.com/search-engine-optimization-forum/61487-link-me-scotty-internal-linking-strategies-structure.html)that touched upon how one passes or doesn't pass PR within a site that may interest you.

Now back to the hijacked portion of this thread.

MJ

SemAdvance

08-07-2007, 06:13 PM

Wige

Yes that's what I am saying. There is more than one way to build a business than with a free handout from a search engine.

live by google, die by google

Did you ever stop to consider what Google says to do and whats reality are two different things?

What good would it serve them to tell people how to manipulate the organic SERPs?

Do you think Google sees the SERPs as an asset or expense? (Its a needed expense to carry their ads)

MJ

if you look at the post title it states

Re: The Ethics of rel="nofollow"

What of the ethics of a company whose main purpose is to sell links, telling webmasters who "NEED" to earn income, they could be punished if they sell links???

See a hypocrisy in that?

Why should anyone need worry of their ethics? when Googles own ethics are grossly questionable........

<snip Please refrain from derogatory comments. CD>

mjtaylor

08-07-2007, 06:17 PM

I prefer the nofollow too, as I don't like putting stuff in the robots file
or the htaccess file either, because it gives away paths to places I don't
really want spyders going.

There is not a file or folder on your server that Google or any other spider cannot find whether you give it a link or a path in a robots.txt or htaccess file or not. There are programs that will download all the folders and files below a certain path whether there are links to them or not ....

Further, the rel=nofollow doesn't mean a spider will not follow the link, it only tells the spider that you are not 'voting' for the link and in Google's case, I believe we know that Google will honor that by not passing PR to that link. I don't necessarily believe they won't follow or index the link ... just that they won't pass your PR to it.

Cheers, MJ

mjtaylor

08-07-2007, 06:28 PM

MJ

if you look at the post title it states

Re: The Ethics of rel="nofollow"

Yes, I see that, and I don't think we've really been discussing ethics at all.

What of the ethics of a company whose main purpose is to sell links, telling webmasters who "NEED" to earn income, they could be punished if they sell links???

See a hypocrisy in that?

No, honestly I don't. I see a free market.

And you can take the hijack slight and shove it......

Not sure why you took the hijack apology to Lee personally. It had nothing to do with you.

Who do you think you are??? Some princess???

Bless you, I know who I am and princess I am not. I may be high maintenance, but I maintain myself.

Really, Sem, that comment was not about you or directed toward you.

I laid no blame on anyone but myself for the hijacking of this thread ...You should know me well enough by now to know that I am direct and open anytime I disagree with you or anyone else. I hope I am always kind in my approach and I would appreciate it, in the spirit of a friendly forum, if you would be, too.

Peace, MJ

wige

08-08-2007, 10:12 AM

I also think there might be a more severe issue to using nofollow tags in pages than simply manipulating pagerank. In theory, this tag is telling the spider that the url should not be crawled, not just that the url should not receive a vote. Should there be a concern that if you mark all the links to, lets say, your privacy policy with no follow, that the Search Engines will drop that page from the index since it no longer has any links, or in the case of a new site, has never legitimately been crawled?

As far as the ethics of using this for your own internal PR, basically telling Google which pages you think are the most important, there are other recommended methods to do so, so it would seem the SEs want that type of perspective. If you read the "priority" tag description listed at https://www.google.com/webmasters/tools/docs/en/protocol.html, Google doesn't seem to mind being told what pages you think are important. What it does with this information, though, is anyone's guess.

mjtaylor

08-08-2007, 10:36 AM

I also think there might be a more severe issue to using nofollow tags in pages than simply manipulating pagerank. In theory, this tag is telling the spider that the url should not be crawled, not just that the url should not receive a vote.

Wige, thanks ... I didn't understand that before. I thought the tag was simply "no vote" but in reviewing Google's original post about the No Follow Tag (http://googleblog.blogspot.com/2005/01/preventing-comment-spam.html) it does suggest that adding the no follow attribute would prevent both crawling and passing PR.

Matt and Jason wrote on Google's Official Blog (1-18-2005): "From now on, when Google sees the attribute (rel="nofollow") on hyperlinks, those links won't get any credit when we rank websites in our search results."
And in a Q/A further down that post, it reads:
"Q: Should I put rel="nofollow" on the link to my comments page?
A: Probably not, because lots of interesting discussion can happen there. Also, if other people link to your comments page, a spider can follow that link and find any spam that's lurking on the comments page."That would, indeed, suggest Google will not crawl and index such a page unless it finds a link elsewhere without the nofollow attribute.

It's always interesting to go to the source ... and I often amaze myself when I discuss a topic at some length without doing so ...

This SEW blog (http://blog.searchenginewatch.com/blog/050118-204728)post post also has some light to shed on this topic.

Cheers, MJ

crankydave

08-08-2007, 10:53 AM

I also think there might be a more severe issue to using nofollow tags in pages than simply manipulating pagerank. In theory, this tag is telling the spider that the url should not be crawled, not just that the url should not receive a vote. Should there be a concern that if you mark all the links to, lets say, your privacy policy with no follow, that the Search Engines will drop that page from the index since it no longer has any links, or in the case of a new site, has never legitimately been crawled?

I don't think there's cause for concern.

As long as your privacy policy is navigatable by site visitors, is it neccessary for it to be in the index at all? Perhaps if you subscribe to the notion that it adds to trust SE's place in the site. Other than that, how important is it for a searcher to be able to reach that page via a SE?

If one subscribes to the "trust" idea, then leaving one crawlable link would keep the page indexed.

Dave