Opinion? Would changing your Wordpress theme have an effect on optimization?

[ExtravagantMedia]

Here is the challenge.
I am trying to determine through actual testing, whether or not certain Wordpress themes are getting Google penalties.
I was wondering if anyone here has already looked into this?

[incrediblehelp]

Sure it could. A theme controls all different types of things like internal linking of pages. Out bound links through blog rolls. How content is shown on the page. etc etc etc. All of these things are vital to SEO.

[ExtravagantMedia]

Thanks for the response.
Right, but I'm not meaning from a canonical point of view, that would be a very newbish thing for me to ask.
What I meant was if the theme was changed but all other structure remained intact, do you think it would have an impact?

[chowell]

I would think that just changing a stylesheet would have no impact on your results, but a lot of theme designers put a backlink (or two) in the footer.

If that designer's site is of poor quality or if they're spamming the engines somehow, the outward link could have a negative effect on your site.

[mjtaylor]

Originally Posted by ExtravagantMedia

Thanks for the response.
Right, but I'm not meaning from a canonical point of view, that would be a very newbish thing for me to ask.
What I meant was if the theme was changed but all other structure remained intact, do you think it would have an impact?

Tough to answer without looking at a case in point. There may be quite a bit of structural difference between one theme and the next, so it *could* as inc'help correctly suggested.

I recently updated a site (not a blog) that had enjoyed top 5 SERPs for more than 5 years. Nothing was changed in terms of text or links, but we went from an old, terribly outdated HTML clogged with font code to a clean CSS driven HTML ... nothing else changed but the site has dropped an average of 4 positions.

So, depending on the differences between the themes.

BTW, I didn't see inc'help's response as related to canonical issues, nor do I think canonical issues are unsophisticated. Quite the contrary.

[tenkiya]

It could have a huge impact on SEO. It could have none. You should be able to tell from reading the code of the template.

[mjtaylor]

Originally Posted by tenkiya

It could have a huge impact on SEO. It could have none. You should be able to tell from reading the code of the template.

How can you tell from the template code? What exactly would we look for?

[Webnauts]

It depends how good you are in editing robots.txt.

[mjtaylor]

Originally Posted by Webnauts

It depends how good you are in editing robots.txt.

What would you recommend on editing robots.txt? I have been told to disallow archives ... do you agree? What else?

[Webnauts]

Originally Posted by mjtaylor

What would you recommend on editing robots.txt? I have been told to disallow archives ... do you agree? What else?

There are several folders/pages that should be disallowed. I just created for our members here a robots.txt that can be implemented right away for Wordpress without problems. And it will help you to get rid of most, if not all of pages found in Googles the supplemental results, which cause a harmful leak of PR.

User-agent: Googlebot
Disallow: /*?*
Disallow: /*?
Disallow: /*.php$
Disallow: /*.js$
Disallow: /*.inc$
Disallow: /*.css$
Disallow: /*.gz$
Disallow: /*.wmv$
Disallow: /*.tar$
Disallow: /*.tgz$
Disallow: /*.cgi$
Disallow: /*.xhtml$
Disallow: /category/*/*
Disallow: */comments
Disallow: /feed/$
Disallow: /*/feed/$
Disallow: /*/*/feed/$
Disallow: /*/*/*/feed/$
Disallow: /*/feed/rss/$
Disallow: /*/*/feed/rss/$
Disallow: /*/*/*/feed/rss/$
Disallow: */trackback
Disallow: /*/trackback/$
Disallow: /*/*/trackback/$
Disallow: /*/*/*/trackback/$
Allow: /wp-content/uploads

User-agent: *
Disallow: /cgi-bin
Disallow: /wp-
Disallow: /wp-admin
Disallow: /wp-includes
Disallow: /wp-register.php
Disallow: /wp-login.php
Disallow: /wp-content/plugins
Disallow: /wp-content/cache
Disallow: /wp-content/themes
Disallow: /feed/
Disallow: /trackback/
Disallow: /rss/
Disallow: /comments/feed/
Disallow: /page/
Disallow: /date/
Disallow: /comments/
Disallow: /search
Disallow: /stats/
Disallow: /contact/
Disallow: /about/
Disallow: /archives/
Disallow: /dh_

There are probably more folders or files that might need to be disallowed. For example as you mentioned, the archives.

I hope this is useful for you and others here.

[mjtaylor]

Thanks! You are da' bomb!

And that is for the robots.txt for the entire site, or is there a special one for wordpress?

[Webnauts]

Originally Posted by mjtaylor

Thanks! You are da' bomb!

And that is for the robots.txt for the entire site, or is there a special one for wordpress?

The one I posted above is special for Wordpress. Every site has its owns needs, so you cannot just edit a default robots.txt.

[ExtravagantMedia]

Webnauts thanks!! This is a big help!

[Webnauts]

Originally Posted by ExtravagantMedia

Webnauts thanks!! This is a big help!

You are always welcome.

[mjtaylor]

Originally Posted by Webnauts

The one I posted above is special for Wordpress. Every site has its owns needs, so you cannot just edit a default robots.txt.

Thanks ... my question was not clear, but you seemed to understand me. So, a site which also has static pages might need additional elements, but this example you generously gave us would be a good base for a site that is a wordpress blog only?

[Webnauts]

Originally Posted by mjtaylor

So, a site which also has static pages might need additional elements, but this example you generously gave us would be a good base for a site that is a wordpress blog only?

That's right MJ.

[lukkyjay]

Very helpful Webnauts. If my blog is http://mysite.com/blog

would I just put

/blog

in front of all those?

[Webnauts]

Originally Posted by lukkyjay

Very helpful Webnauts. If my blog is http://mysite.com/blog

would I just put

/blog

in front of all those?

If you use that, and you will disallow SE to index your blog. How did you come up with that? Can you just add there what I posted above?

[lukkyjay]

Originally Posted by Webnauts

How did you come up with that?

I figured you were using the blog as the root. So if the blog was a folder like www.mysite.com/blog/ - I figured putting /blog in front of them like this:

User-agent: Googlebot
Disallow: /blog/*?*
Disallow: /blog/*?

User-agent: *
Disallow: /blog/cgi-bin
Disallow: /blog/wp-
Disallow: /blog/wp-admin

....would be the same. If not, how should those be written for a blog setup like that?

[Webnauts]

Lukkyjay that would be correct if you would add that in your root robots.txt.

[lukkyjay]

Webnauts,
In my question about having the blog as a subfolder, it makes sense that those you have beginning with a "/" would look like this:

Disallow: /blog/*?*
Disallow: /blog/*?

but what about those 2 not beginning with a "/" in your example:

Disallow: */comments
Disallow: */trackback

?

Would they go like this:

/blog*/comments
/blog*/trackback

or this:

/blog/*/comments
/blog/*/trackback

?

This is a HUGE help. Thanks Webnauts!

[hitsusa]

I'd highly recommend using the robots.txt exclusions that Webnauts has shared (customized to your own specifics of course).

Google has recently become unhinged with their deep crawl tactics. Since December 1st, my blog has had 47 older posts that lost their top 10 rankings because Google replaced the real URLs with URLs to category and archive pages.

Even worse, they started spidering brand new posts wrong as well, indexing new posts as the index page or using the temporary internal search result pages that Wordpress creates.

And that's despite having an existing sitemap and no prior indexing problem.

I've had to use the Google URL removal tool to get rid of hundreds of junk pages, but I still haven't gotten all those pages back to where they were.

For example, one new post that had three number one spots in the SERPs on December 14th proceeded to disappear completely when Google switched it a week later to using an internal search results page as the primary URL.

Then, it got replaced in the SERPs by a Blogspot spam copy that didn't even show the accompanying photos (image hotlinking was blocked) and that still had an internal link within the post back to my blog.

After a Google spam report and flagging the culprit in Blogspot failed to get results, I had to go the DMCA complaint route and even the removal of the offending blog from Blogspot didn't fix my original post's disappearance from Google.

Switched the site from Wordpress search to Google site search and then searched for that post. Amazingly, it showed up as the 16th of 16 results.

Top two results were page 5 and page 21 from a category folder that just showed snippets of that particular post. And of course, neither page contained that post snippet since each new post changes what's on those category pages.

The other results were old cached versions of other pages in the blog that showed really old "recent" posts links in the the sidebar of the page.

Talk about bad indexing!

Follow Webnauts advice or face untold hours of damage control because Google no longer spiders Wordpress blogs correctly.

Even now after implementing all those exclusions and other changes, I still see Blogspot spam blogs that autocreate junk pages from RSS feeds show up above me by using snippets of my own content mixed in with other feeds.

Bottom line? Google's lost control of their indexing due to an overemphasis on deep indexing that began on December 1st. It may work great for indexing Blogspot junk, but it's wreaking havoc on Wordpress blogs.

You might want to search for some of your own posts within Google by post title + site name and see what pops up.

A word of caution on the robots.txt issue:

If you're blocking access to /category, /archives, and /page in robots.txt, then you must also include a link to your sitemap in either robots.txt or on your index page.

Example sitemap link in robots.txt:

Sitemap: http://yoursite.com/blog/sitemap.xml.gz

Hope this helps!

Oh yeah, back to the original question from Extravagant Media.

Yes, different Wordpress themes can greatly affect your rankings. Look for themes where your content is up high in the html code. Use tags and keep unrelated content to an absolute minimum. Delete the blogroll completely as Google sees these as sitewide links and discounts them. And pay great attention to onpage SEO. It's more important than most people think...

[Webnauts]

I also recommend installing this plugin: Wordpress Plugin, Dilution Factor | Simon Emery

[fxman]

Hitsusa and webnauts,

word of caution on the robots.txt issue:

If you're blocking access to /category, /archives, and /page in robots.txt, then you must also include a link to your sitemap in either robots.txt or on your index page.

Example sitemap link in robots.txt:

Does this still hold water. I have those items blocked, but I am still getting indexed well without the sitemap.. Any ideas?

Thanks,

Fxman

[Webnauts]

Originally Posted by fxman

Hitsusa and webnauts,

word of caution on the robots.txt issue:

If you're blocking access to /category, /archives, and /page in robots.txt, then you must also include a link to your sitemap in either robots.txt or on your index page.

Example sitemap link in robots.txt:

Does this still hold water. I have those items blocked, but I am still getting indexed well without the sitemap.. Any ideas?

Thanks,

Fxman

The info I provided was not customized to your own specifics. You have to check that yourself.

[bobitza]

is your robots file suitable for wordpress 2.3+ ?

[Webnauts]

Originally Posted by fxman

Hitsusa and webnauts,

word of caution on the robots.txt issue:

If you're blocking access to /category, /archives, and /page in robots.txt, then you must also include a link to your sitemap in either robots.txt or on your index page.

Example sitemap link in robots.txt:

Does this still hold water. I have those items blocked, but I am still getting indexed well without the sitemap.. Any ideas?

Thanks,

Fxman

Can you tell the URL?

[fxman]

THis site is a forex site, using the latest Wordpress. I used the robots.txt file you posted Webnauts. It was based on what you wrote. Thanks for the help.