Word Press Duplicate Content Issue and Fix

[incrediblehelp]

I am sure most of you are familiar with many of the duplicate content issues on Google. Well if you use a Word Press blog and also translate the URLs using permalinks you are more than likely giving Google 2 or 3 different versions of the same page to crawl. now let me stop right here and say I am not saying that this is or is not affecting your blogs performance on the SE's. This is just a fix that I am comfortable with making to avoid any issues if they indeed do exist.

To avoid duplicate content on your own WP blog I found a great plug-in to use that will handle all the htaccess mod-rewrite code for you and I am have just installed it on my blog below and it works great.

http://fucoder.com/code/permalink-redirect/

For example these 2 URLs as the same:

jaankanellis.com/putting-seo-on-your-dashboard-by-paul-bruemmer/
jaankanellis.com/2006/01/27/?p=184

it works for URL’s missing the trailing slash, like:
/putting-seo-on-your-dashboard-by-paul-bruemmer

and also for dynamic URL’s like these:
/?p=184
/2006/01/27/?p=184

using the plug-in, they’ll all be 301′ed to
/putting-seo-on-your-dashboard-by-paul-bruemmer/

Right now Googlebot is spidering both and most of the time the numeric based one more. Of course I want them to spider the one using the article title only. Installing the plug-in above does the trick.

The reason this came up for me personal is when I looked deeper at my spider activity on my SEM blog. I started seeing Google and other bots crawling the original dynamic based URLs more than the rewritten ones. This is obviously no good if I ever want to permalink based URLs to rank higher than the dynamic ones! I was wondering why spiders like Googlebot still see my original URL structure in WP instead of the permalink setups I have installed? Who knows, it is just the way WP currently works, but to fix this I used the plug-in above.

More info from the Wolf Howl Blog

[bignm]

Thanks for the info on the permalink-redirect plugin. I have already installed in on my blog, only because it seemed a cool thing to have. But now I see it's true value and I'm glad I did thanks to your post...

[incrediblehelp]

Your welcome bignm!

[incrediblehelp]

OK found some more duplication issues. Any idea why WP is producing these duplicate URLs all over the place?

http://www.jaankanellis.com/diet-cok...y-eepybirdcom/ (original and right one)
http://www.jaankanellis.com/diet-cok...irdcom/page/1/
http://www.jaankanellis.com/diet-cok...irdcom/page/2/

[internet-marketing-cr]

incrediblehelp so we get the same duplicate content issue on blogger? It produces a duplicate content of the original post and also on the archives, check this:

http://www.pokerallstar.com/poker-articles/index.html

http://www.pokerallstar.com/poker-ar...egends_12.html

http://www.pokerallstar.com/poker-ar...r_archive.html

The same content in three different urls?

[incrediblehelp]

I just cant figure out why the software is making these other pages, I must be missing something here.

[jaydilla]

can't you go in and put a no index no follow into 2 of those pages and that will relieve the dup content issue

[incrediblehelp]

It is not just hapening to those to two pages. WP creates these types of dup pages for just about every post I make.

[aaron2005]

Hi honey i'm back!!! :)

Wordpress is very easy to fix, here are some steps:

1.) In your Wordpress console under "permalinks" set them to end in "/" by adding /%postname%/

2.) Remove the code in you sidebar for the calendar and lame archives.

3.) Instead link a sitemap in your footer using Dagon Sitemaps or some other cool plugin.

If you want to see an example of clean non duplicate code checkout my webmaster blog here: http://www.webmaster404.com

And don't worry about changing permlink structure, all three engines have no problem picking up your changes.

Also there is no evidence that Google and the other two have any problem with duplicate blog content but if you are like me and prefer minimal stuff, go for it.

Wordpress can be cut down to the most efficient CMS yet in easy steps.

Will have to write up a turorial on this, I could use a few links.

Take care,

Aaron

Update: Damn sorry guys I have not been posting in here in a long time, must have clicked the wrong button, doh!

This was in reference to incrediblehelp's wordpress duplicate content issue located here: http://www.webproworld.com/viewtopic.php?t=66110

[wednesday]

For not heavily linked sites excerpts,flat site structure and robots.txt could save you from the supplementals. For larger sites you could increase the number of articles shown in the categories.

Simple excerpt hack (not very user friendly but):

<?php if(is_single()) { ?>
<?php the_content(); ?>
<?php } else { ?>
<?php the_excerpt(); ?>
<?php } ?>

[incrediblehelp]

Originally Posted by incrediblehelp

OK found some more duplication issues. Any idea why WP is producing these duplicate URLs all over the place?

http://www.jaankanellis.com/diet-cok...y-eepybirdcom/ (original and right one)
http://www.jaankanellis.com/diet-cok...irdcom/page/1/
http://www.jaankanellis.com/diet-cok...irdcom/page/2/

What is the code or plugin that specifically fixes this issue?

[wednesday]

in the template header

<? if (strstr($_SERVER[REQUEST_URI], '/page/')){ ?>
<meta name="robots" content="noindex,follow">
<? } ?>

or alternatevlly you can put this in your worpress index file and do 301 but depending on your template yuo can have problems with the feed's http headers.

[khurramali]

i am using word press and the first thing i changed after installing it was to change the permalink option to Date and name based.

I though that the duplicate content problem should have be solved.

I was not aware of this problem, I will install this plugin and do some damage control.

[aaron2005]

Google actually has no issue with duplicate content in blogs now, they smartly get the good data and put the duplicate content in supplementals.

I for one like to hotrod my wordpress cms, a little hard to describe but if anyone wants to send me their template I could fix it for them and remove all duplicates. The three URL's below are all edited wordpress blogs with no duplicate URL strings.

Check the link to webmaster404 below and contact me there or here.

[incrediblehelp]

Originally Posted by aaron2005

Google actually has no issue with duplicate content in blogs now, they smartly get the good data and put the duplicate content in supplementals.

I understand this, but when they are crawling and ranking the dup URLs instead of the normal ones I don't like it.

Aaron can you email me your solution at jaan @ jaankanellis.com

[aaron2005]

You see there is not a "solution" you have to play around with your wordpress template code, there is also no book on this. I have been meaning to write something up on this but other projects call me.

I have data including screenies of the great job Google is doing with duplicates from wordpress but even then smart admins. take it further and hack at the code a bit to make it easy to crawl lightning fast...

[incrediblehelp]

I know Aaron, I was hoping for someone that had fixed these issues before to post an easy plug-in they may have found. I was being lazy on it because I am trying to rollout a new KBK website and SEO projects are also keeping me busy as well.

[xinfo]

well i found from tom blog that having

/%postname%/ give good result or search engine but there is minor bug in wp

its alway good to use
/%category%/%postname%/ to et maximum benifit

when i visited wordpress.org/support people say there

permalink is not a el to get search rank google never consider permalink, and google consider only title ,content , describ ....

any suggestion

[aaron2005]

People at Wordpress no nothing about "search rank", this is why they didn't care about duplicate content, look at my blogs below and follow me, they a optimized 100% and extremely search engine friendly. I had to remove just a few lines of code.

You know how people complain about not getting crawled? It has never happened to me, I believe it is because the blogs can be crawled fast and light.

[xinfo]

i like to know it some thing seo optimised themes in the market , if so how we can optimised the themes