2 pages containing duplicate content - Good or Bad?

[fxci]

I have a question,

On my site, I have created the entire site using dynamic pages and lots of mod rewrites.

I have an article index so search engines can find and index articles alot easer.

The downside is it creates a duplicate of the content on a different page.

I have been working very hard for the past few months on this site and am worried these duplicates could get me in trouble w/ search engines.

The pages are simular, except one doesnt have all the tables and extra eyecandy while the other is mainly a header and text and footer.

All the content is good content, just each article has it's "evil" duplicate...

What should I do?

Any comments would help greatly and ease my worries.
Mike

[jestep]

Im having a little of a hard time understanding what is going on, but to be on the safe side, I would block one set from the search engines. This is really the only way to be safe. I hope it wouldn't ruin your navigation or anything. The dup penalty is a little weird. I dont know anyone that can ever say for sure if it is going to be applied, so its kind of a guess. But like i said, the only way to be really safe is to block one set.

[fxci]

Thanks for the reply,

I went ahead and recoded the article index, now there should be no duplicate sites... if there is, then it will only be just a couple vs the thousands I had.

The question now is how do I remove the already indexed duplicates from google and yahoo ?

Will they drop on their own over time?

Thanks,
Mike

[jestep]

If you use a robots.txt they will eventually drop on their own. The most important thing is to make sure that the SE's cant find the duplicate content still.

[fxci]

I have added the directory to my robots.txt file.

and did a link crawl of my site and the dups are not found :)

Thanks for all the advice,

Mike

[bhartzer]

Duplicate content is not good. Either use a robots.txt file to disallow the spiders from getting to it, or make one page so it's not a dupe, which shouldn't be too difficult.

You could add a few topic-related news feeds, change the header area (meta tags, title, etc.), as well the page layout.

[fxci]

Thanks bhartzer :)

I removed the dup pages from my site...

They still work, but robots.txt wont let search engines near them any more.

I also removed all the links from the site pointing to the pages and added the robots no index tag to the pages.

I hope they drop off... I didnt want to remove the pages entirely because they get alot of hits, and people getting 404 errors is not good.

Google seems to be happy with my changes, it took went to all the right pages and was "eating" my site alive yesterday.

I hope this works :)

Mike

[bhartzer]

If you want to remove them faster, then use the Google removal tool.

[fxci]

I'm good :) I emailed google and they said dont worry about it...

They looked at my site said the content is good... and that the duplicates will drop off eventually because i nolonger link to them and changed the page that used to link to them.

And also because the old pages have the no index tag in the headers now.

Thanks
Mike