Duplicate content and php includes

[texxs]

Ok we want to intentionally distribute duplicate content on our quickly growing network of web sites. Sound crazy? Well it's about efficiency. we're talking about storing our price/shipping time list in a .php file that we can include into any webpage on any of our other websites.

What are the search engines going to do about it? Especially Google.

Think we'll get banned? penalized? Maybe nothing at all?

[wige]

I think it depends on how much of the page content will be duplicated. A site that has a two paragraph copyright message on the bottom of every page would not be likely to be penalized for it: at worst, the copyright message itself would be ignored and give no keyword benefit to any of the pages. This is assuming, of course, that each page has more than two paragraphs of unique content. The same with navigation elements and other common content included in pages.

I guess the best way to explain it, is that in the same way users get desensitized to ads after seeing them in the same place on every page on a web site, Googlebot is smart enough to become desensitized to the same content snippets repeated on every page on a site.

[texxs]

Hmmm, it's a good theory.

I've heard Matt Cutts say some similiar things.

On the pages with the the include, they will be mostly duplicate content.

But my brilliance just struck me!

I can just put these "pricing pages" in a seperate directory on all the different sites and "no index" them with robots.text and meta tags.

That should prevent duplicate content penalties?

[Dubbya]

Wige, you're right on target. I recall Matt Cutts explaining this in a video a few months back.

My understanding was that Google is smart enough to evaluate navigation panels, common header text, ads, footer text, and repetitive page links.

They're not flagged as duplicate content and no penalty ensues, the repetitive occurrences are simply ignored.

As I said, that's just my interpretation but it seems pretty logical.

.02

[intelinside]

Yes, the best route will be to use noindex or the robots.txt to avoid the duplicate content issue.

I have recently done the same and so far so good :)

[texxs]

Sweet, thanks all.

[SemAdvance]

Actually Google is so smart they will realize the nature of your business is an e-commerce one and will not apply any filters for duplicate content.

There are a few 1,000 e-commerce sites which sit at the top of results and are not hammered for dupe content issues, Amazon is chief amongst them.

Duplicate content is scrutinized when looking at content based sites. Even then there is not much of a filter, so it's not much to worry over.

[PhilipDunn]

i actually have several sites with duplicate content, and so far, Google is the only one that has seemed to care..
and with one site, the majority of the duplication is book previews with links to purchase..

in my case, Google treated the two duplicating sites as one, giving one great positioning, while throwing the other back several pages...

but after 'undoing' the duplication, Google has mostly forgiven the offending site, and given back the higher ranking it originally had.....almost...i think it would be higher for a few phrases had the duplication not occurred..

[rudy102]

Wige, you're right on target. I recall Matt Cutts explaining this in a video a few months back.

My understanding was that Google is smart enough to evaluate navigation panels, common header text, ads, footer text, and repetitive page links.

They're not flagged as duplicate content and no penalty ensues, the repetitive occurrences are simply ignored.

As I said, that's just my interpretation but it seems pretty logical.

.02

I concur with Dubbya. Google's Webmaster Central blog has a number of posts on duplicate content and how Google's sophisticated algorithms deal with it. For more, please read: What does Google do about duplicate content? (http://r-rwebdesign.com/blog/?p=104)

[texxs]

Thanks for the link Rudy