Google and Cloaking

[activeco]

Originally Posted by Webnauts

I added this as you have requested. Should I still unhide the other text in blue?

It doesn't matter, you can remove it completely, it wasn't indexed anyway. You assumed that adding/revealing the menu caused some problems, so good time to test it.

P.S. Don't reveal the blue text, it will have influence, so you wouldn't know what caused eventual SERP changes.

[Webnauts]

Originally Posted by activeco

It doesn't matter, you can remove it completely, it wasn't indexed anyway. You assumed that adding/revealing the menu caused some problems, so good time to test it.

P.S. Don't reveal the blue text, it will have influence, so you wouldn't know what caused eventual SERP changes.

OK. Then I will leave the blue text hidden too.

The probelm I met was that the fist visible text in the body were the skip navigation links and then the logo and the h1.

This test is not good enough for that issue since it is just a test page and not using my site template. Anyway...

[spiderbait]

I think this is an interesting topic.

In my opinion, it raises the legitimate question of whether all cloaking is evil (in the arbitrary eyes of Google), or whether there are certain circumstances where cloaking can be justified.

Unfortunately, I also believe it's a purely academic question, given that our findings and opinions will ultimately matter very little - if at all. Even the most compelling rationale for cloaking will probably have little effect in changing Google's policies.

However, for the sake of the discussion I think it's worth pursuing the question: Is all cloaking evil?

So, I would suggest that there's probably a simple litmus test to apply. There may be others, but this is the one that comes first to my mind. And like so many things in life, it comes down to motivation.

Is the cloaking intended to produce an advantageous result in the SERPs?

• If the answer is yes, then I think the example fails the test and should be considered evil.
• If the answer is no, then we might consider it as an example of legitimate cloaking.

So, what is the purpose for wanting to hide these accessibility links from the search engines? Does it benefit the users in some way?

[Webnauts]

Originally Posted by spiderbait

So, what is the purpose for wanting to hide these accessibility links from the search engines? Does it benefit the users in some way?

If I hide those links to non-visually impaired users, why shouldn't I hide them from the SE?

[spiderbait]

Originally Posted by spiderbait

So, what is the purpose for wanting to hide these accessibility links from the search engines? Does it benefit the users in some way?

What's the benefit to the user?

[activeco]

Originally Posted by spiderbait

In my opinion, it raises the legitimate question of whether all cloaking is evil (in the arbitrary eyes of Google), or whether there are certain circumstances where cloaking can be justified.

Sure, even Google accept 'legitimate' cloaking(s). E.g. geo-cloaking is acceptable under condition that the page shows the same content to the bot coming from the targeted area.

So, what is the purpose for wanting to hide these accessibility links from the search engines? Does it benefit the users in some way?

The thread was triggered with the problem Webnauts experienced when the links became visible to SE.
It presumably caused ranking drop for some keywords. See #13 of this thread.
So, it's not only academic, but rather practical issue.

[Webnauts]

Originally Posted by spiderbait

What's the benefit to the user?

It benefits blind users only.

[spiderbait]

Originally Posted by activeco

Sure, even Google accept 'legitimate' cloaking(s). E.g. geo-cloaking is acceptable under condition that the page shows the same content to the bot coming from the targeted area.

Good example. The argument could me made though, that geo-cloaking is not really cloaking, but rather is simply "conditional content" - and the principle is whether the same conditional content is shown to all visitors, human or bot.

Originally Posted by activeco

The thread was triggered with the problem Webnauts experienced when the links became visible to SE.
It presumably caused ranking drop for some keywords. See #13 of this thread.

So the purpose has nothing to do with benefitting users and is solely designed to achieve an advantageous result in the SERPs?

Originally Posted by activeco

So, it's not only academic, but rather practical issue.

I understand that the issue has real-world practical import to many webmasters. But without a method to bring Google to the table to discuss it and possibly even issue a waiver or exception for certain unique circumstances, I'd respectfully suggest that it is still an academic issue.

[spiderbait]

Originally Posted by Webnauts

It benefits blind users only.

Okay, then to answer your question of whether it's cloaking or not, I'd say it definitely is cloaking.

So, I guess what you're really looking for is a method to achieve the effect of cloaking, but which would not be perceived or penalized as cloaking by Google.

Is that right?

[spiderbait]

Do you have the ability to detect whether a user's browser is equipped with a screen reader of some sort?

If so, you could do a conditional include based on that detection.

[Webnauts]

Originally Posted by spiderbait

Do you have the ability to detect whether a user's browser is equipped with a screen reader of some sort?

If so, you could do a conditional include based on that detection.

Oh my God. Good point though.

Well screenreaders are desktop software like Jaws and WindowEyes. Maybe someone can help me out at this case?

So far I know, they use the browser (user-agent) of the user.

[activeco]

Originally Posted by spiderbait

Do you have the ability to detect whether a user's browser is equipped with a screen reader of some sort?

Good thinking spiderbait. Unfortunately the detection is very difficult and unreliable.
Some try to use flash: Screen Reader Detection Example , but there are warnings about it: Developer Beware: Using Flash to Detect Screen Readers - The Paciello Group Blog.

[spiderbait]

Originally Posted by activeco

Good thinking spiderbait. Unfortunately the detection is very difficult and unreliable.
Some try to use flash: Screen Reader Detection Example , but there are warnings about it: Developer Beware: Using Flash to Detect Screen Readers - The Paciello Group Blog.

That's good to know. I had no idea, which is why I asked. I've never built a truly accessible site (hangs head) so didn't know whether it was possible to reliably detect.

It would be nice if the browser headers of users with screen readers could contain a reference, much the way the PDF and other applications are reported. Optional for the user of course, if they chose not to reveal that through their browser.

Maybe there's a standards agency working on that very problem? It might not help webnauts in the present, but it seems a useful idea for development.

[SemAdvance]

Actually Google does not frown upon all cloaking activities.

It does frown upon cloaking done to manipulate search engine rankings.

It does not frown upon cloaking to give users a better experience when visiting a website.

:->

[Webnauts]

Originally Posted by SemAdvance

Actually Google does not frown upon all cloaking activities.

It does frown upon cloaking done to manipulate search engine rankings.

It does not frown upon cloaking to give users a better experience when visiting a website.

:->

That was always my opinion. But when I hear the stories of Matt Cutts, I feel like it is more risky than ever to implement techniques to improve users experience. Do I suck, or Google? Or am I suffering from paranoia?

[activeco]

Originally Posted by Webnauts

Do I suck, or Google? Or am I suffering from paranoia?

Or all of that?

[malice95]

I think google sucks in this case..

You do have a valid use for cloaking in this case.. Instead of checking to see if it is a
search engine bot you could probably get away with checking for Lynx, or other screen reader
programs and provide the extra content to them.

Mike

[seopo]

I can think of four ideas to do what you want, none of which are confirmed

1. Do screen readers and lynx browser read process javascript? If they do you could add this in via DOM.

2. There was talk of noindex on sections of content such as robots-exclusion-brainstorming - Microformats or <div rel="noindex", doubt anything is in place yet (anyone tested?) but may be in the next year?

3. Can the jumpto links be added the the <link> tag? HTML link tag

4. Lastly, ok so its cloaking but it's hardly keyword/link stuffing. People hide links all the time with drop downs, not cloaking but it still presents the user with a different view to the spider.... I think Google would take a view on this and and wouldn't think the differences warrant blacklisting. You could even show what you were hiding by wrapping it in comment tags for a search engine not sure if that'd be worse of better though...

[Webnauts]

Originally Posted by malice95

I think google sucks in this case..

You do have a valid use for cloaking in this case.. Instead of checking to see if it is a
search engine bot you could probably get away with checking for Lynx, or other screen reader
programs and provide the extra content to them.

Mike

We will suffering for years with the web sites "print versions". Since we can do that with CSS, we save a hell of time and we do not need to setup those alternative pages.

Now because we want to make our site as friendly as possible to users with disabilities, and in this case for the blind, and in this case the blind, we must pay with minus rankings. On the other hand Google have setup the Google Accessible Search

So my conclusion so far is, that if the practice we are discussing here is violating Google's guidelines, then it will be the first time in my career than I will tell that GOOGLE SUCKS, and that BIG TIME!!!

[crankydave]

Just some general comments...

From what I read, technically, you are not showing the bots "different" content when you block part of a page. You are showing them the same content, just not all of it. I don't see where this would be considered cloaking at all. As long as what the bots see, the user can see also, I don't see an issue. If it were, then if you blocked an entire page from a bot it would be considered cloaking.

Now, if you were to show bots content that your visitors don't see or redirect them to content the visitors don't see, you *might* be creating a problem *if* detection methods and an subsequent reactions are automated. If the detection methods are automated (ie red flag) and the subsequent reactions are manual then "intent" would likely come into play.

Dave

[activeco]

Originally Posted by crankydave

From what I read, technically, you are not showing the bots "different" content when you block part of a page. You are showing them the same content, just not all of it. I don't see where this would be considered cloaking at all. As long as what the bots see, the user can see also, I don't see an issue. If it were, then if you blocked an entire page from a bot it would be considered cloaking.

Now, if you were to show bots content that your visitors don't see or redirect them to content the visitors don't see, you *might* be creating a problem

Very questionable.
Going extreme, let's say you create a regular content rich page and a bot sees only <h1> header and a single link.
Cloaking?

[crankydave]

Originally Posted by activeco

Very questionable.
Going extreme, let's say you create a regular content rich page and a bot sees only <h1> header and a single link.
Cloaking?

As long as the visitor sees the same <h1> and link that the bot does I don't see how it would be cloaking.

Now that being said, IF that <h1> and link were "Blue Widgets" and the rest of regular content was "pr0n" you'll likely be creating a problem no matter how you deliver the content, but just the same, you're still not cloaking IMO.

Dave

[Webnauts]

Originally Posted by seopo

1. Do screen readers and lynx browser read process javascript? If they do you could add this in via DOM.

Can you please specify?

Originally Posted by seopo

2. There was talk of noindex on sections of content such as robots-exclusion-brainstorming - Microformats or <div rel="noindex", doubt anything is in place yet (anyone tested?) but may be in the next year?

I know microformat by heart and in some cases I already use them. The attributes you mentioned are not supported by the major SE yet.

Originally Posted by seopo

3. Can the jumpto links be added the the <link> tag? HTML link tag

If you could explain how?

Originally Posted by seopo

4. Lastly, ok so its cloaking but it's hardly keyword/link stuffing. People hide links all the time with drop downs, not cloaking but it still presents the user with a different view to the spider.... I think Google would take a view on this and and wouldn't think the differences warrant blacklisting. You could even show what you were hiding by wrapping it in comment tags for a search engine not sure if that'd be worse of better though...

I must agree with you ate this point. I am not afraid that Google can detect that issue automatically using MD5sum, since that content part is dynamically generated. I am just concerned about manual diffing.

[Webnauts]

Originally Posted by crankydave

Just some general comments...

From what I read, technically, you are not showing the bots "different" content when you block part of a page. You are showing them the same content, just not all of it. I don't see where this would be considered cloaking at all. As long as what the bots see, the user can see also, I don't see an issue. If it were, then if you blocked an entire page from a bot it would be considered cloaking.

Now, if you were to show bots content that your visitors don't see or redirect them to content the visitors don't see, you *might* be creating a problem *if* detection methods and an subsequent reactions are automated. If the detection methods are automated (ie red flag) and the subsequent reactions are manual then "intent" would likely come into play.

Dave

Just a reminder: YOU RULE MAN!

[Webnauts]

Originally Posted by activeco

Very questionable.
Going extreme, let's say you create a regular content rich page and a bot sees only <h1> header and a single link.
Cloaking?

I thought I am Mr. paranoia? Are you too?

Cool thread man. Great idea telling me to get this started man!

[crankydave]

One additional point and correct me if I'm mistaken. When it comes to cloaking aren't we looking at the source and not neccessarily the rendering of the source?

In other words, if you are delivering something via the source, in the way of what could be considered "content" that is to the bots only = cloaking

Dave

[activeco]

Originally Posted by crankydave

As long as the visitor sees the same <h1> and link that the bot does I don't see how it would be cloaking.

Cloaking, sneaky Javascript redirects, and doorway pages

Originally Posted by crankydave

When it comes to cloaking aren't we looking at the source and not neccessarily the rendering of the source?

True. That's why I suggested frames in the first place.

[activeco]

Originally Posted by Webnauts

I thought I am Mr. paranoia? Are you too?

Hah, not really.
But, that would be an easy way to channel popular pages to desired focused keywords.

[seopo]

Sure thing.... If screen readers process javascript then you could use something like jQuery to easily prepend some of your content to the font of your H1 or Content i.e. Manipulation/prepend - jQuery JavaScript Library

The link tag isn't just for CSS etc, this link explains better HTML link tag

Something like this might do what you are after

<head>
<title>Some Title</title>
<link rel="Skip to main content" href="#main-content" />
<link rel="Skip to sub-navigation" href="#sub-nav" />
</head>

I've never got round to testing this but sure it would be wide spread knowledge by now but has anyone reading this given <div rel="noindex"> a go?

[craigmn3]

I hate to interupt this watch repair clinic....but sometimes the inelegant hammer is the best approach.

If there is content you want the viewerr to see and not the SE's put it in a graphic. It won't be scanned, it won't be indexed, it won't be lookat all by the SE's except as one more graphic....where as your viewer will see exactly what you want them to see without knowing it's a graphic

To be the bot....you have to beat the bot.


I guess I am missing the point

[crankydave]

Originally Posted by activeco

Cloaking, sneaky Javascript redirects, and doorway pages

From this link the first thing I note is the use of the word "different" and not using the word "identical" when it comes to cloaking. The impression I get when reading the quote John cited using the word "identical" is when it comes to the source.

Originally Posted by activeco

True. That's why I suggested frames in the first place.

And I agreed with both John and kgun (i believe it was kgun) that suggested iFrames to Dave Barnes. Feydakin also suggested delivering the dropdowns via java right off the bat. Many java navs are not being indexed.

Dave

[Webnauts]

Originally Posted by seopo

The link tag isn't just for CSS etc, this link explains better HTML link tag

Just to make things clear here. I have a very strong experience in design, and I do know about the use of the link tag.
The link you are adding here for the second time doesn't tell me anything new. I use the link tags and not only for CSS. Look at the source code of my site.

Originally Posted by seopo

Something like this might do what you are after

<head>
<title>Some Title</title>
<link rel="Skip to main content" href="#main-content" />
<link rel="Skip to sub-navigation" href="#sub-nav" />
</head>

Can you setup a sample page to demonstrate how that can work?

Originally Posted by seopo

I've never got round to testing this but sure it would be wide spread knowledge by now but has anyone reading this given <div rel="noindex"> a go?

I did. It doesn't work.

I will add you wish though on the test page we are discussing about right now.

[activeco]

Originally Posted by crankydave

And I agreed with both John and kgun (i believe it was kgun) that suggested iFrames to Dave Barnes.

I referred to this thread. Regarding the other one, I already gave you the credit: Block spiders from PART of a page?

[Webnauts]

Originally Posted by seopo

I've never got round to testing this but sure it would be wide spread knowledge by now but has anyone reading this given <div rel="noindex"> a go?

Your wish is added to the test page here Hidding content from search engines

[crankydave]

John... I understand your intent. It's certainly not to deceive. However, if you are delivering a different "source" to bots than you are to visitors, then IMO, as Google puts it you are "high risk".

Now, I seriously doubt that anyone physically looking at the page would ever take issue however if it is something that is "automated" as I said earlier, it's likely not going to be able to discern "intent".

Dave

[crankydave]

Originally Posted by activeco

I referred to this thread. Regarding the other one, I already gave you the credit: Block spiders from PART of a page?

lol... sorry. In which case I agree with you here.

It's a "sensible" way to do it without having to worry at all.

Dave

[Webnauts]

Originally Posted by craigmn3

I hate to interupt this watch repair clinic....but sometimes the inelegant hammer is the best approach.

If there is content you want the viewerr to see and not the SE's put it in a graphic. It won't be scanned, it won't be indexed, it won't be lookat all by the SE's except as one more graphic....where as your viewer will see exactly what you want them to see without knowing it's a graphic

To be the bot....you have to beat the bot.


I guess I am missing the point

Oh my God Craig! I think you have a point there:

<ul id="skip-link" title="Keyboard Tabbing and Screen Reader users functions">
<li><a href="#main-content" accesskey="2"><img src="/images/skip-main.gif" alt="Skip to main navigation" title="Skip to local navigation" width="100" height="80" /></a></li>
<li><a href="#sub-nav" accesskey="3"><img src="/images/skip-main.gif" alt="Skip to local navigation" title="Skip to local navigation" width="150" height="80" /></a></li>
</ul>

Well this sounds like a good idea! If everything else will not work, it will be a way to go. But still would prefer to have the text links when the user is tabbing and not alt tags. Exactly as I have now on my life site.

[Webnauts]

Originally Posted by crankydave

From this link the first thing I note is the use of the word "different" and not using the word "identical" when it comes to cloaking. The impression I get when reading the quote John cited using the word "identical" is when it comes to the source.



And I agreed with both John and kgun (i believe it was kgun) that suggested iFrames to Dave Barnes. Feydakin also suggested delivering the dropdowns via java right off the bat. Many java navs are not being indexed.

Dave

But Dave I cannot use iframes, because my site is coded in XHTML 1.0 Strict and is not supported by that document type, and second there are several workarounds necessary to make them really accessible. If I go for 100% valid, I will have to implement "noframes" and add the links there, which will be visible by the bots.

[Webnauts]

Originally Posted by craigmn3

If there is content you want the viewerr to see and not the SE's put it in a graphic. It won't be scanned, it won't be indexed, it won't be lookat all by the SE's except as one more graphic....where as your viewer will see exactly what you want them to see without knowing it's a graphic

Are you sure SE cannot read text on graphics? (WO/2008/003095) RECOGNIZING TEXT IN IMAGES

[crankydave]

Originally Posted by Webnauts

But Dave I cannot use iframes, because my site is coded in XHTML 1.0 Strict and is not supported by that document type, and second there are several workarounds necessary to make them really accessible. If I go for 100% valid, I will have to implement "noframes" and add the links there, which will be visible by the bots.

I understand your dilemma John.

What is best for your visitors?

Also (because I can't help myself ) what if you weren't going backwards John. What if you were to actually be going forward and didn't know it...

Dave <---ducks all the flying vegetables

[seopo]

I'm not a HTML guru and out of office hours so I'm can't check if this is 100% correct, but I think screen readers may pick up the link tags? If they do then this may work Some Title

I used to have issues with netscape 4 following these types of tags like meta refreshes....

[deepsand]

Hiding, or attemping to hide, content from an SE is distinctly different from presenting content to an SE that mis-represents that presented to other agents, and is not cloaking in the sense used by SEs.

Google's definition of cloaked content is more properly called a chimera.

[Webnauts]

Originally Posted by deepsand

... called a chimera.

Do you mean that animal of the Greek Mythology?

[deepsand]

Originally Posted by Webnauts

Do you mean that animal of the Greek Mythology?

That would be Chimera.

With a lower-case "c" it's any entity consisting of multiple diverse elements, such that its appearance changes depending on how it's observed.

[Webnauts]

Originally Posted by deepsand

That would be Chimera.

With a lower-case "c" it's any entity consisting of multiple diverse elements, such that its appearance changes depending on how it's observed.

Something like 3D graphics?

[Webnauts]

So after all the conclusion is that Google would tolerate hiding content parts with iframes or javascript, but not with the php method I developed?

And I only want to hide internal page links. They are only jump links within the page.

If that is the result of the thread, Google sucks more than I ever could imagine before!

[Terry Van Horne]

Originally Posted by Webnauts

The content of the linked page is important. That is what I would hide with an IFRAME, disallowing bots to follow the target though a noindex meta tag or robots.txt.


What is the difference if I would disallow the bots accessing a page with robots.txt or with a 403?

Sorry a lot of work with Robots.txt that the minute you stick that in the Iframe it's as good as dead or in much the same state as if you Robots.txt it. Google does little or nothing with I/Frames other than to look at the pointers src. I often put search features with long dropdowns in an IFrame. That way it doesn't dilute the text I optimize. I will also put trademark and other "required" disclaimers doing the same thing. I wouldn't call that cloaking... I'd call it having a F'in clue how things actually work. Give the user what they need... maximize relevance by removing delutive elements with legit techniques. That's definitely not cloaking but some would call it that to get a headline.

Trying to hide it is what makes a beningn action more risky.

[ami_iss]

You may or may not get away with it. Search engines are not omnipotent and omniscient, and what is "crime" from their POV, probably does pay.
Black Hat SEO never works is a superstition - the weight of the evidence is against it. Crime pays.

My guess is that more sophisticated cloakers have a better chance of getting away with it than innocents who can easily fall into various automated traps that they weren't even thinking about.

A lot of awesome technical knowledge has been displayed in this discussion, but the fancy techniques do not answer your question. The guidelines intend that you should not be showing visitors something different from what is seen by the search engine spider. So the simple answer to your question is that you are violating the guidelines. The intent is to prevent people from making a page that, for example, looks like it is about the Beijing Olympics and may actually have some user visible text about that, but is really, for example, about Texas holdem or nude photos of 14 year old kids doing you know what or redirects to an affiliate site.

From Google's POV, if you are doing "organic SEO" there is no reason to hide the navigational part of your site, which should be packed with appropriately titled relevant links. Their problem is to ensure that the best content content gets the highest listing. They just want a fair way to compare your page with other pages, which presumably have the same limitations and problems in presenting navigational links. They are going to figure that if the spider sees something different from the visitor, it is "fishy" - and chances are they are right. If you get banned, you will have to do a lot of arguing to get listed again.

Your real question is can you get away with it? As far as I can see, technically it is a no brainer to make a second spider that ignores your directives and compares results. Spammer spiders don't follow robots.txt or other directives, so why can't Google do that? There is no reason for Google to "play fair" if you don't. That spider would be used only for checking and not for indexing. Whether they use that technique or any other particular automated technique to catch cloakers is anyone's guess. They might respond to a complaint by a visitor - or a competitor. If your use of navigational links is truly legitimate and you are losing positioning because of it, then you are a victim of the limitations of the Google algorithm, but cloaking may get you in worse trouble - or not.

It seems to me that they aren't catching all the cloaked sites, whatever techniques they are using. There are a lot of real black hat sites and pages listed in Google and you can find them all the time when you do searches. So someone "gets away with it."

[deepsand]

Originally Posted by Webnauts

Something like 3D graphics?

Perhaps I should have explicitly included "& by whom" in my definition, rather than taking such to be implied.

A 3D model, viewed from any given perspective, will look the same to all observers; thus it is not a chimera.

On the other hand, a colored model, regardless of the number of dimensions, will look different to persons with color blindness, and thus might rightly be called chimeras.

An example of real chimeras would be those rare individuals who carry multiple & distinctly different genes. To the casual observer, only those genes which express themselves in those parts of the body which are externally visible are evidenced. However, to a trained practitioner, those "hidden" genes become visible.

[deepsand]

Originally Posted by ami_iss

YThe guidelines intend that you should not be showing visitors something different from what is seen by the search engine spider. So the simple answer to your question is that you are violating the guidelines.

Actually, said guidelines address the issue of holding out a resource as being of materially different content, with intent to deceive, depending on the viewing agent, not that of whether or not all agents view all possible content.