How to overcome duplicate content problem?

[sangeeta_k]

I am fairly new to this forum and I need some help. I work as an assistant to a copywriter. My employer often experiments with content on his sites, offering one version to one set of prospects and another version to another set of prospects. He then decides which copy version is more effective. As a result his website has several versions of the same page with minor variations. Will he be penalized by search engines for duplicate content? Is there any limit for duplicate content on a site?
As his experimentals are vital for his studies is there a way out?

[nnacho]

In this previous Thread you can find more information about duplicated content.

http://www.webproworld.com/viewtopic.php?t=70815

[bhartzer]

Will he be penalized by search engines for duplicate content?

I wouldn't call it a penalty, but the duplicates will be removed. So, the first copy will be indexed and the others will most likely be ignored.

If you're testing copy, then you should post it in an area that is off-limits to search engine spiders (stop them from indexing certain areas of the site using the robots.txt file). Then, when the copy is ready to publish, I would then put in an area that is okay for the search engine spiders.

[bjbtexas]

How are syndicated stories handled?

Major syndication like AP releases? and Ezine Articles?

[sangeeta_k]

[quote="bhartzer"]

If you're testing copy, then you should post it in an area that is off-limits to search engine spiders (stop them from indexing certain areas of the site using the robots.txt file). Then, when the copy is ready to publish, I would then put in an area that is okay for the search engine spiders.

bhartzer, I appreciate your advice but can you tell me how to do it or suggest a link? I am a novice at such things.

[buster]

You can use your robots.txt file to bar engines from certain directories or pages.

Can you not test the pages offline? And only put them on your site when you are happy with them?

[tim2005]

As I know Google can interpret duplicating content as search engine SPAM and as a result your site will be penalized and removed from index. If design of your site is changed frequently, it can be not good for your site ranking as well. It is wise to have a single version of the web site which accessible for search engine robots and deny indexing of other versions.

Information about robots.txt file you can find there
http://www.robotstxt.org/wc/faq.html

You can use meta-tag “ROBOTS” with content "NOINDEX, NOFOLLOW" to disable indexing of exact pages. You can read about it on the site specified abow.

[sangeeta_k]

As I know Google can interpret duplicating content as search engine SPAM and as a result your site will be penalized and removed from index. If design of your site is changed frequently, it can be not good for your site ranking as well. It is wise to have a single version of the web site which accessible for search engine robots and deny indexing of other versions.

Information about robots.txt file you can find there
http://www.robotstxt.org/wc/faq.html

You can use meta-tag “ROBOTS” with content "NOINDEX, NOFOLLOW" to disable indexing of exact pages. You can read about it on the site specified abow.

Thanks a lot, this link is goooood!
No, buster the pages have to be tested online, but the noindex, nofollow should work fine for me.
Thanks once again.

[factoring]

I had a problem like yours on a mirror site of mine. I used the robots.txt file to tell the se bots not to look at my other site. This has worked so far. I am waiting for the bots to visit my site again.