The Power of the WebProWorld SEO Community

[Feydakin]

I am reading the thread.. What I am getting from the thread is that people are linking to a page you would rather they didn't.. So you are looking for a way to take that link juice and funnel it in a direction of your choosing to take advantage of those links..

[williamc]

yep, that's the takeaway I get from it as well.

[Webnauts]

Originally Posted by Feydakin

I am reading the thread.. What I am getting from the thread is that people are linking to a page you would rather they didn't.. So you are looking for a way to take that link juice and funnel it in a direction of your choosing to take advantage of those links..

Exactly Steve. But one think I should ad here:

Do NOT use the robots.txt "disallow" directive for a page you already use the meta robots "noindex" directive.

More about read this discussion with MC: Sphinn - Graywolf, Matt Cutts, g1smd and Halfdeck discuss robots.txt disallow & meta noindex exclusion!

[Webnauts]

Oh, there we go. I found something about the robots.txt "noindex" directive you might would like to read check too: A Deeper Look At Robots.txt

[williamc]

Great, someone else spreading an experimental directive which google admits may disapear. I actually feel sorry for newbies nowadays, so much of this flying around.

[Webnauts]

William,

I am using the "noindex" robots.txt directive with full successful results since 2007. I would have the same success if I would use the "noindex" meta robots directive too. I know and repeat that the "noindex" directive is unofficially supported by Google. And I always said that!

Should I be ashame for looking for a place to discuss my experiments and thoughts with other professionals?

If that is fact, then all SEOs who advise their customers that all ages they want to block from the SE should be done via "disallow" robots.txt directive, should also be ashame. Also how much garbage and hurt sites we saw already with the "nofollow" thingy. Even if it was unofficial.

Just to make things clear here.

I decided to share my experience of almost 2 years testing with the public on May 12th, 2009 publishing at article at Search Engine Journal. If you would like to read what I wrote, search with Google for this: Bot Herding: The Ultimate Tool for PageRank Sculpting. (I do not want to add the link here, because a member in this thread reported me for spam or selfpromotion.)

For the last time: I did not try to convince anyone to follow this technique. Since I stated that it is unofficially supported by Google, I do not feel responsible. Isn't that disclaimer clear enough?

Now for the newbies, I think we came up with an officially supported by Google solution:

<meta name="robots" content="noindex">

Are we good now? Or should we keep this thread going for some years repeating again and again the same stuff?

About myself and my customers? I will continue using the "noindex" directive as long it works for me and my customers who are aware of this unofficial Google support and wish the implementation. If something changes I will reverse my work.

I hope now we can come to an end and start a new constructive discussion.

Would you like to choose a topic?

P.S. To make sure that no newbies will get hurt, I will ask the admin to delete this thread.

[williamc]

I was not actually slanting you John, but that article which does NOT tell people the whole truth in that it can disapear at anytime and google admits it might "break" their robots.txt later.

You admitted that prior so I have no gripe there

[Webnauts]

Originally Posted by williamc

I was not actually slanting you John, but that article which does NOT tell people the whole truth in that it can disapear at anytime and google admits it might "break" their robots.txt later.

You admitted that prior so I have no gripe there

Excellent.

OFF-TOPIC:
So what should be discuss now? I started yesterday a cool thread that is already sticky:
Wordpress SEO & Security Friendly .htaccess & Pugins

[Terry Van Horne]

I think Feydakin read and he doesn't see the point in herding users when if they linked to that page then they think that is the best page to link to for their users. You may have other desires but it is quite likely they know better than you why they sent people to that page. In fact if it was me and you redirected the link my solution is easy... I remove the link. Been there done that, IMO, that's disrepectful to the webmaster who linked to you.

In 15 years I've never used Robots.txt or a goofy meta tag directive that is wonkier than...

Robots.txt isn't even a real protocal with a real body behind it which is why Google can add things to it that may or may not work or be changed on some whim. Robot.txt is useless for the prupose to which it was conceived. If you want sensitive content kept out of the index then... use authentication... otherwise it's like locking the door and leaving the key in the lock.

[Webnauts]

Terry I am just wondering how you deal with supplemental pages. For example and to be specific with online shops.

Upon the chance, if you don't mind, I could add here a real live example so you can see yourself what the whole thing is about? If you want, please crawl this site gameshop.gr and we can go from there.

[Webnauts]

I checked that Google can index 661 pages, the site: operator less (never mind) and their Google Webmaster Tool shows 4,372 internal links. They launched the optimized new site on the 27th of February 2009, since their old site did not have more than 30 visitors per month, and here you can see their traffic the last 30 days without any PPC or other ads/promotion involved: http://getclicky.com/stats/home?site...e=last-30-days

And maybe you would like to look at the robots.txt http://www.gameshop.gr/robots.txt

I am sure William will start telling how dangerous the "noindex" in the robots.txt could be, but the customer is aware of that and told me that he will ask me to reverse that if problems occur. If the customer would have paid more, I told them we can add the meta tag for all those pages that we do not want to have indexed, but he did not want that.

So you you want to say that the site would perform better without the work done?

P.S. I forgot to mention that their green bar was 2 in February and now they have 4 without any additional efforts. If that makes sense at some point.

[Terry Van Horne]

I think people that obsess over supplemental results have too much time on their hands

John are you saying that the robots.txt is the reason the traffic went up? A factor maybe but not the only reason. I have a hard time grasping how that could be unless the old Robots.txt was blocking engines from indexing.

I build sites for users I could care less about these sorts of issues unless it is an issue of content not getting indexed. Supplemental results are not a negative it is often just a case of the site having so much content Google doesn't see the need to have a page in the index for each color an item comes in or regularly indexing pages that are very similar to others. In my experience these pages still do get into results it's just a lot less frequently.

SE's don't do anything to help me do my work so... I leave this sort of stuff up to them to sort out. If it had a negative impact then... I'd adjust things. I do investigate to determine the reason but if it's just a case of Google not wanting it in the index because it's already there in another form then... are they going to reward activity that is manipulative? IMO, not for very long.

[Feydakin]

The latest update on sculpting page rank, siloing, whatever you want to call it..

Question:

It seems like you supported PageRank sculpting a year ago and now it seems like you don’t support it anymore. Why is that and will it become a negative indicator?

Answer:

Matt Cutts: No, it won’t hurt your site. You can do your links however you want. You can use it to eliminate links to sign in forms and whatnot, but its a better use of your time to fix your site architecture and fix the problem from the core. Suppose you have 10 links and 5 of them are nofollowed. There’s this assumption that that the other 5 links get ALL that PageRank and that may not be as true anymore (your leftover PageRank will now “evaporate”, says Matt.). You can’t shunt your PageRank where you want it to go. It’s not a penalty. It’s not going to get you in trouble. However, it’s not as effective. It’s a better use of your time to go make new content and do all the other things. If you’re using nofollow to change how PageRank flows, it’s like a band-aid. It’s better to build your site how you want PageRank to flow from the beginning.

Emphasis mine..

Source: You&A With Matt Cutts

[Terry Van Horne]

TSuppose you have 10 links and 5 of them are nofollowed. There’s this assumption that that the other 5 links get ALL that PageRank and that may not be as true anymore (your leftover PageRank will now “evaporate”, says Matt.).

Thanks Feydakin! It seems like all the assumptions that were made about sculpting/nofollow either were never true or Google realized they provided another way to manipulate PR. One has to wonder if it's true for sculpting... when you use nofollow the link still may be included in the PageRank calculation so... in a sense Google is penalizing people who do the right thing and follow their rules. I have to wonder... how do they differentiate sculpting from "proper" implementation?
Proper being the way Google wants to see it implemented.

[Feydakin]

Agreed.. It became yet another example of Google 'suggesting' one thing, realizing that they handed SEOs a powerful tool, and then SEOs abusing the hell out of it forcing Google to try to put the Genie back in the bottle..

This is why I stopped caring about Google long ago.. As far as I'm concerned they are just another traffic source, and not a very good one at that.. I take their suggestions, use the ones that suit me and my needs, and forget the rest.. Let Google worry about Google.. It's just far too much effort to try to find and manipulate minor loopholes for the minor benefit that you might get from finding them..

[crankydave]

Nice post Fey!

Let's take a look at this in particular...

...Suppose you have 10 links and 5 of them are nofollowed. There’s this assumption that that the other 5 links get ALL that PageRank and that may not be as true anymore...

Assumption? Not exactly. MC suggested this was the case so it's a bit more that just an assumption.

May not be as true anymore... Translation... Was true, now...

I have certainly not been alone, but I've been openly critical of Googles position when it comes to tagging links they don't want to pass value. Especially paid links. It's something they have to figure out how to do themselves if they want it to work.

Dave

[Terry Van Horne]

What the newbies don't know and those who sell training and information don't tell is that manipulation and chasing algos is a mugs game because these "loopholes" in the algo have diminishing value the instant they are disclosed or detected by SEs. Anyone who thinks that any trick they see on the web is still working a year later just isn't paying attention, possibly, because they're too busy implementing the latest fad exploit. That was just as true in 96 as it is now! That's why most of us who have been around a while stopped sharing or using exploits because we realized they are easy for SEs to fix and a headache to undo especially if you used them on a lot of client sites.

Many don't agree but I truly believe there is wanted and unwanted manipulation. Optimizing titles, descriptions, navigation, site architecture, markup and copy is wanted manipulation because these aid in indexing and determining "true" relevance. Manipulation by over aggressive link building, buying links, spammming blogs and guest books is unwanted manipulation. I believe Google doesn't care that you promote content or buy links but... they do care if you over promote the home page. It has never been normal to see 10,000 links to the home page and and a few hundred to the deeper pages. That doesn't indicate quality that reaks of manipulation via promotion. Keep in mind that Google's founders invent PageRank as a way to determine page quality/popularity not to create a link building industry or provide monetization for publishers or a way for SEOs to get lousy content to the top of the results. SEO has gone from SE facilitator to unwanted manipulator.

[Tubby]

I am I think far less complicated a person than webnauts. So much less complicated that when I come across a link to one of my pages. . I simply feel pleased. (it doesn't happen every day)

I do like the concept of 'herding link juice' but It does remind me of my army days when as young soldiers we occasionally washed (volunteered) the glasses in The officers mess on special event days like the Queens Birthday. The juice we herded back into selected bottles by smell, taste, and sometimes by direct description. "Johny walker in them 4 glasses lads"

These days I am so uncomplicated That I simply Buy my bottles, full, pure and uncomplicated simple. I probably miss out on a good few nips. . . One thing I did learn in the army. everything that runs breaks down. the simpler it is the less it breaks down.

But . . I do like the concept of herding all the little bits of left overs, spare bits, scraps . . . I would hate the thought of anyone seeing me doing it and asking. Is he that hard up? (for traffic) - Or getting used to that extra nip every evening, simply to find one day that google had taken to pouring it into a 'leftover google bucket' . . . and left me dry.

I have enjoyed the thread so far....

[Feydakin]

Originally Posted by Tubby

I am I think far less complicated a person than webnauts. So much less complicated that when I come across a link to one of my pages. . I simply feel pleased. (it doesn't happen every day)

As do I..

I would hate the thought of anyone seeing me doing it and asking. Is he that hard up? (for traffic) -

OMG.. Thanks for the awesome laugh this morning.. You are on fire man..

I have enjoyed the thread so far....

Same here, but much more lately..

[williamc]

Originally Posted by CHUMPSFROMOXFORD@YAHOO.CO

I have learned that loopholes and shortcuts can spell disaster for your site.

You didn't seem to learn that spamming forums is not smart tho.

[inertia]

Suppose you have 10 links and 5 of them are nofollowed. There’s this assumption that that the other 5 links get ALL that PageRank and that may not be as true anymore (your leftover PageRank will now “evaporate”, says Matt.).

What are we supposed to think about this? How about all those people with nofollowed outbound links?

[williamc]

Originally Posted by inertia

What are we supposed to think about this? How about all those people with nofollowed outbound links?

Then they should have gotten on the DoFollow bandwagon faster

But, this is a good example of why people should not jump on every little thing a search engine tells you you should do for them at first. Build your site the way you want, and for users, not search engines.

[Terry Van Horne]

I should have qualified buying links by posting "buying links for SEO". Buying links was around way before Google's algo made it an SEO technique. Back then we called it web promotion not SEO.

[Terry Van Horne]

Originally Posted by inertia

What are we supposed to think about this? How about all those people with nofollowed outbound links?

Agreed, but I wouldn't be surprised if there's one rule for sculpting and another for those who are "playing ball". Matt has this way of giving subtle hints it's almost like reading the Feds minutes for monetary policy. This is important "that may not be as true anymore". Which could be his way of saying it applies to sculpting only the assumptions may be still true in other cases.

[williamc]

Originally Posted by Terry Van Horne

Which could be his way of saying it applies to sculpting only the assumptions may be still true in other cases.

They can't even find hidden text or paid links programmaticly yet. Do you really think they can differentiate between intents in any meaningful manner?

Inertia: Travelling SEO's is a neat idea. Kudos.

[Terry Van Horne]

Originally Posted by williamc

They can't even find hidden text or paid links programmaticly yet. Do you really think they can differentiate between intents in any meaningful manner?

I didn't post they could do it programmatically did I?

All search engines can find hidden text, one was doing it long before Google came along. There are just too many false positives due to CSS and cell back ground colors to rely entirely on programmatic discovery.

williamc, Do you really believe SE's only catch those that are ratted on?

You can pretty much assume most scuplting is in site wide navigation, would have text like "contact us" etc. In other words there's a huge footprint that says heay you might want to look at this. I think there's a lot of stuff that people think is totally programmatic that is done/reviewed by humans as well. That includes the top five results for big money querries. Don't have proof and don't care if they say otherwise, they have to, or they'd need more lawyers than engineers.

[Feydakin]

Originally Posted by inertia

What are we supposed to think about this? How about all those people with nofollowed outbound links?

As I understand the best guessing flying around the webbernet today, some from smart folks, internal and external nofollows are being treated differently these days.. Which would make a lot of sense..

IMO, the last 2 years of nofollow and PR sculpting was simply an exercise by Google to get people to modify their websites in a manner that would allow Google to further refine their algorithms and apply those new findings to sites not using PR sculpting..

This could be a good thing in that they 'might' be taking what they learned and applying filters to pages like login and contact us across the internet regardless of whether you are sculpting or not..

[Terry Van Horne]

Originally Posted by Feydakin

As I understand the best guessing flying around the webbernet today, some from smart folks, internal and external nofollows are being treated differently these days.. Which would make a lot of sense..

Thanks that's entirely programmatic and it makes sense since when you sculpt you're basically saying you don't trust your own pages.