Duplicate Content NOT to be indexed

[ArthurNYC]

I own a large network of review sites and for the past 3 years we have done a dance with Google where all of a sudden they bump everything up to a $10 Min Bid and we are done. Over three years it has happened 3 times and every time we slightly change the URL (s, -, ...) and go back to work. The issue we have is that we never get natural listings.

Three months ago the site was redesigned by a top level SEO guy for us and we are getting natural listings all over the place. (If anyone needs a recommendation, PM me as he did a great job). But now we got whacked by Google again with a $10 min bid (please do not try to analyze the reason for this, even our reps only say it has something to do with the landing page, we have been on every forum, spoken with every rep at any partner company and we get no real clear answer).

So now the question is, how do I leave the OLD site live (the one that is getting natural listings) and do PPC on a new URL. There are hundreds, if not thousands of pages of Unique content but it is now duplicated and that will eventually catch up to us and hurt natural rankings.

How do I completely stop Google (and others) from seeing the PPC site and listing it in their search engines? If it never gets spidered, never gets listed, then I assume/hope the natural listing site will not be penalized for duplicate content.

Thanks in advance.

BigFish

[Webnauts]

Watch this interview with Adam Lasnik from Google, and if you still have question, post again:
Video: Interview with Adam Lasnik of Google

[thindenim]

That's a great interview. Some interesting comments on sculpting internal pagerank, food for thought...

[Webnauts]

Originally Posted by thindenim

That's a great interview. Some interesting comments on sculpting internal pagerank, food for thought...

I agree. Adam have confirmed my position about not implementing the nofollow attributes for internal pages like contact, etc.

[incrediblehelp]

Originally Posted by Webnauts

Watch this interview with Adam Lasnik from Google, and if you still have question, post again:
Video: Interview with Adam Lasnik of Google

What does this have to do with his PPC dilemma?

Either way Arthur you could set up another domain and simply block the robots from accessing this domain and only focus on PPC here. I do not recommend this because you lose out a lot from a branding perspective trying to market two different domains. It is also very confusing to the end user. You could help build the brand of your "main" domain and just set up a sub domain for PPC reasons.

[Webnauts]

The question was: Duplicate Content NOT to be indexed?

[DoneInStyle]

Great video. Thanks for posting the link, Webnauts!

[incrediblehelp]

Originally Posted by Webnauts

The question was: Duplicate Content NOT to be indexed?

Just saying a nofollow doesnt seem to fit right here.

[davidweb]

Originally Posted by ArthurNYC

I own a large network of review sites and for the past 3 years we have done a dance with Google where all of a sudden they bump everything up to a $10 Min Bid and we are done. Over three years it has happened 3 times and every time we slightly change the URL (s, -, ...) and go back to work. The issue we have is that we never get natural listings.

Three months ago the site was redesigned by a top level SEO guy for us and we are getting natural listings all over the place. (If anyone needs a recommendation, PM me as he did a great job). But now we got whacked by Google again with a $10 min bid (please do not try to analyze the reason for this, even our reps only say it has something to do with the landing page, we have been on every forum, spoken with every rep at any partner company and we get no real clear answer).

So now the question is, how do I leave the OLD site live (the one that is getting natural listings) and do PPC on a new URL. There are hundreds, if not thousands of pages of Unique content but it is now duplicated and that will eventually catch up to us and hurt natural rankings.

How do I completely stop Google (and others) from seeing the PPC site and listing it in their search engines? If it never gets spidered, never gets listed, then I assume/hope the natural listing site will not be penalized for duplicate content.

Thanks in advance.

BigFish

Hi, Sorry, but it was bit difficult to perceive your exact problem. Please clarify whether you are facing issues with duplicate content or PPC bidding ??

As far as duplicate content is concerned, it is a serious issue. People on several blogs argue that duplicate content is just a filter and not penalty. Moreover they will justify that duplicate content only gets added into the supplement index.

However the real question is what will happen to all those pages linking to a particular page ( which has been perceived as duplicate by google). The fact is that all those links connected to duplicate page will loose their rankings, PR and will eventually die down.

It is best to avoid duplicate content filter in the first place. If google detects any then remove every instance of duplicate content from your website.

It will help you score higher rankings on both Google and Yahoo.

[JezC]

Duplicate Content in the same site is not a huge deal (ref, Matt Cutt's blog, earlier this year).

Duplicating content between sites *is* a big deal (same article by Matt) - because Google may prefer the "wrong" site, as much as any other reason. See Matts replies to comments to his article, about half way down the current list of responses.

However, I think SEO is a red herring for this. Looks like the crucial issue is the PPC problem.

Landing Page Quality Scores for AdWords are *usually* easy to resolve - the MinCPC primarily affects the initial value for the bid, and is only slightly related to the AvCPC (the paid CPC). So long as the advert gets a good CTR, the MinCPC will usually decline with time. This is how major brands end up with $0.02 MinCPC (No one *starts* with $0.02 MinCPC - you start with $0.05 or $0.10 or whatever, and work it down). Very often the cause of a high MinCPC is the way that the AdGroup was built, and the precise way that the landing page URL was determined by Google. Under some circumstances, altering the order of adding adverts and keywords can affect the MinCPC. Adverts with landing pages determined using DKI can be a particularly nasty problem to resolve. The nature of Google's auction *usually* means that a high MinCPC is not well correlated with the AvCPC - IOW, a $10.00 MinCPC may result in a $0.20 AvCPC because of the other participants in the auction.

From the brief outline of the problem, I'd:

1/ avoid duplicating content on another site (that site should be poorly ranked, eventually, even if you don't use robot exclusions or other techniques to prevent spidering) - mostly for reasons of brand experience and the internal costs of running/managing both sites.

2/ not worry too much about duplicating content within the site (unless you find that poorer pages are getting the rank, in which case focusing on sitemap issues, using nofollow, noindex and robots.txt can help). Google really does seem to know that customer-oriented reasons can result in internal duplication. However, putting all the link weight behind one page will probably help.

3/ work on resolving your problems with MinCPC and AvCPC by focusing on the way in which the AdGroup is constructed, and the CTR.

More information about the PPC problems might help better diagnosis.

Cheers, JeremyC.

[Peopleunit]

Hi Aurthur,

If I understand you correctly, you have positioning in the natural listings (finally), but the PPC costs have went through the roof (again). In the past when this happened (without the support of good natural listings positioning), your response was to modify the URL for your domain, and perhaps a reason you never achieved good "natural" positioning before. Then when the PPC costs slid back down, you went back to pursuing PPC traffic.

And if I understand you correctly, its happened again, except now you have good "natural" positioning, but you're still wanting to continue pursuing the PPC traffic as well(?), without upsetting the boat.

Ok, this leads me to ask, "Why?" Are you not satisfied with your traffic from the natural listings? The way I see it, you should be pleased that you're no longer reliant on PPC. And you CERTAINLY wouldn't want to tip the boat over by putting up another site with duplicate content.

Sure, you might be able to do some PPC on another site and go about turning away Google's spiders... sidestepping the duplicate content issue, but that's a big maybe and it sounds pretty RISKY as well as difficult, since you'd have to block the spiders from every possible entryway to the new site (every single page that can be linked to externally, I think).

Instead, I can see putting up a supporting site, one where you pursue PPC traffic, but I wouldn't fill it up with duplicate content and hope Google didn't figure it out. Instead, I'd place some unique and supportive content on the new site, with ample supporting links to your existing site. I don't know the nature of your site or your visitors interests, but it would seem to me you should be able to have both sites offer something UNIQUE, worthwhile, and stimulate traffic between the two without blasting yourself out of the water.

As for what's really happening, it sounds to me like your competition is waging a PPC war, causing your PPC costs to skyrocket, trying to kick you out of the PPC arena. I suspect it might just be a temporary thing, and the costs will soon decline again all on their own, once your competition relaxes a bit, just has it has in previous cases. I could be wrong on that, but I think its a reasonable conclusion. What do you think?

[datetopia]

You could use an alias domain for PPC and ban all crawlers from that one. Could be a domain that uses a 404.php file to redirect to the main one for each request.

[Alan-Hart]

i have two sites (cardpos.co.uk and cardposeurope.com) both are identical but both rank highly in google.

[TechEvangelist]

I do get a kick out the denial that duplicate pages get penalized. Tell that to anyone who does affiliate marketing using merchant data feeds.

Getting tagged for duplicate content is a random process. Sure, Google says it is not a penalty, but is really a filter. I can understand that. But if a duplicate content filter reduces the ability of a page to rank well, it sounds like a penalty to me.

If it walks like a duck, and quacks like a duck. . .

Beyond the risk of duplicate content issues, there is a much larger issue when you duplicate entire sites. That sets you up for a mirror site penalty. Over the years I have seen numerous situations where mirror sites have dropped out of the search engines. Sometimes the original site disappears when a pirate duplicates an entire site under a different domain. IMHO, the same site should never show up under multiple domains.

Just read the search engines' webmaster pages:

Webmaster Guidelines
Google: Don't create multiple pages, subdomains, or domains with substantially duplicate content.

Yahoo! Search Content Quality Guidelines. - Yahoo! Search Help
Pages Yahoo! Wants Included in its Index: Original and unique content of genuine value
What Yahoo! Considers Unwanted: Multiple sites or pages offering substantially the same content, Pages that rely heavily on content or links to content created for another web site, such as affiliate content

[saxman1973]

After reading all of these responses, I don't think that any one person here really understands EXACTlY what your goal is. However if I understand you correctly I agree that you should use robots to block the search engines from indexing your new domain, however, I don't get how you expect your minimum CPC is going to be any less with the new domain. If the content is the same then your Quality Score with remain the same. What it sounds like you need is someone to run your PPC campaign for you. Your PPC campaign should be set up with multiple Ad groups that have different keywords and keyword phrases that are relevant to specific content on differnet individual pages, not just the home page. People often make the mistake of sending all of their PPC campaign Ad Variations to the home page of their site when it just isn't relevant to every keyword that youa re trying to target. Tkae the time to build you campaign correctly and you will see that $10 CPC go down immediately. I hope I understood you correctly and inturn I hope this helps. If you do require some help you can visit NetSource Technologies, Inc. : Web Design, Web Hosting, Interactive Mutimedia, ecommerce website design in the Ocala, Jacksonville, Daytona Beach, Tampa, and Orlando areas and contact me, we would be glad to help you with your campaign. Have Fun - Saxman

[Webnauts]

I think if you read these news, you will better understand the whole concept: New Google Process for Detecting Near Duplicate Content -SEO by the SEA

[JezC]

Originally Posted by TechEvangelist

I do get a kick out the denial that duplicate pages get penalized. Tell that to anyone who does affiliate marketing using merchant data feeds.

The original poster for this thread appears to be working for the originator of the content. For a content originator, duplicate postings are mostly a problem if a duplicator achieves higher rank - or if they do it to themselves.

Affiliates often (not always) duplicate content from originators. These duplicates will, and should, be penalised in listings. Why? Because *users* find pages of duplicate listings to be tedious and annoying. The value of a search engine *to a user* lies in finding the best diverse range of resources, not a lot of listings of the same information. Google's popularity was based on providing results that were best for users, not affiliates.

However, the real problem appears to be the PPC. A $10.00 minCPC is a signal that the AdsBot has found a poor match between keywords, advert and landing page. It has nothing to do with competitive activity and only affects the Average Paid Price (AvCPC) if a substantial fraction of other bidders also have a poor Quality Score, or if the "natural price" is already in that range (e.g. certain types of business consultants may pay more than $10.00 to get a page 1 listing). The problem of poor matching is solvable. I regularly resolve this kind of issue

Cheers, JeremyC.

[activeco]

So now the question is, how do I leave the OLD site live (the one that is getting natural listings) and do PPC on a new URL.

I assume you don't mean a new domain, but simply a new landing page on the old, recognized domain (which takes care of higher quality score and lower ppc).
If so, the simple answer would be to put ppc pages in a new directory and disallow the directory in robots.txt for all robots EXCEPT 'AdsBot-Google', which should be able to crawl the page(s).
You can achieve the same per individual page by using meta tags.

[moredial]

What I find difficult to understand with this thread is 'What did the SEO expert you used do to the site that you have ended up with duplicate content?'

This should all have been cleaned out when they did the work for you. It rather leaves me thinking that the SEO you used was not quite the expert you thought they were.

If it were me, I would ask them to come back and finish the job - or send them the bill you get from using a real SEO consultant to sort out the mess.

You can use the robots.txt file to disallow the bots from the duplicate content - only partially successful.
Additionally, use the robots meta tag - generic and bot specific - with the noindex,nocache option.
Using the rel=nofollow on links to the duplicate pages also helps.

By far the easier and more secure is to replace all the duplicate content with empty pages with links to the new content - I can see many reasons for this not being a practical option, based on earlier comments.