Best Robots.txt wordpress

[deepsand]

You need to first identify which directories/files you do and do not want indexed.

Bear in mind that directories such as those that are used for segregating non-contextual files, such as CSS and JS, need not be blocked, as they've no effect on the ability of an SE's indexing engine to discover and have a crawler fetch copies of those files which you do want to be indexed.

[Webnauts]

Trying to prevent bots to access pages via robots.txt is not the ultimate or best solution. You can prevent the bots access to the chosen pages, but you can not prevent them to show snippets in their search results if someone is linking to your blocked pages. Also you cannot prevent a leak of PageRank either.

The best advise is to install and configure "SEO Ultimate" wordpress plugin: http://wordpress.org/extend/plugins/seo-ultimate/

Good luck.

[kgun]

Best Robots.txt wordpress

Recently I've got a wordpress blog at www.domain.com/blog/
With the recent changes in Google algorithm, I would like to take concrete steps in making it more SEO friendly by removing the low quality pages with tags.

My bolding.

The best advisor I can recommend for you is this http://www.packtpub.com/wordpress-3-...imization/book fairly up to date book and it is cheap in PDF format.

I have tried Antispam Bee and am very satisfied with that plugin. It should be even better for English / German speaking people since there is an option to restrict comments to English or German.

[mjtaylor]

It's also worth taking a look at what Yoast has to say:

http://yoast.com/prevent-site-being-indexed/ and Yoast's SEO plugin is another good choice for you: http://yoast.com/wordpress/seo/.

Webnauts is right, and his plugin suggestion is also a good one.