Error in robot metatag: consequence in Google search?

[slainthe]

I am desperately trying to find out why my Google traffic dropped so dramatically. My page is generally found after 10 or 15 pages on Google (for common keywords) since about 2 weeks.
The effect is that only people who know my site are coming back, and they form now about 90% of my traffic. I must say I'm happy to have such a faithful public. It reinforces mmy feeling that my website is good...
I discovered something yesterday, and as I'm not a SEO specialist, I'd like to get some feed-back by specialists on this topic.
1. The first version of my website did not use any Server Side Includes, and all my pages had the extension HTM.
2. A few months ago, I began using SSI and had to rename all my pages in .shtml.
3. I kept both versions for several months, just to avoid some error messages (404 errors), and systematically linked from the .htm pages to the corresponding .shtml pages. To be sure Google does not keep indexing my .htm pages, I added the metatag "Robots: noindex, nofollow".
4. After a while, I dropped my .htm pages, after having checked that my pages were listed on Google with the shtml extension, with the exception of index.htm (which links to index.shtml).
5. ONe day I also dropped my index.htm. This was nefast of course.
6. I still had an old version of this file, and restored it.
7. Yesterday I discovered by chance that this file also contains the keywords "robots: noindex, nofollow".


My question is: can this be the reason why Google decreased my PR from 4 to 2 and why my pages are hard to find?

I put the question, because I could imagine Google just dropping my whole site in this case. But if this could cause my current troubles, I could go back to the enhancing of the content of the site, rather than trying to get all kinds of new incoming links, and other trikky things, like repeating my keywords in the first lines of each page, usint <h1> tags (which is not evident with the use of CSS in combination with templates) etc...

There is however one good thing about the decrease of my traffic: I am now listed on dmoz.org and Zeal (which seems to be important for good rankings).

Please let me know what you think about this possible explanation.

Slainthe
http://www.whisky-distilleries.info/index_EN.shtml

[cbp]

can this be the reason why Google decreased my PR from 4 to 2

No. Meta tags have nothing to do with PR.

Have you removed the "robots: noindex, nofollow" - I can't see it there.

CBP

[Mel]

If you do this search in Google:
site:http://www.whisky-distilleries.info you will see that Google has 283 of your pages in its index, but more than 90% of are listed with no title and description, which normally means that Google knows about the link but has not spidered or indexed the page yet. This situation should improve with time.

If you have links from third party sites which are to specific pages and then you change the page names, subsequently dropping the pages with the old names in place of the new names, then these links no longer point to your sites pages, and naturally this would cause a drop in PR. In the case of links just pointing to your URL instead of a specific page this would not be the case, of course.

There would have been two solutions to this problem that could have been used at the time you made the change:

First there is really no need to change the extension to shtml in order to use server side includes. You can use your .htaccess file to have your server process html pages for includes.

Second, you could have put 301 redirects on your old pages pointing to the new pages, which would inform the search engines that the pages had moved to the new pages.

[slainthe]

Thank you cbp for having checked the site.
Indeed I removed the keywords immediately when I saw the problem.

[slainthe]

Thank you very much Mel,

I guess this is the problem I have.
But, my site has been up and running since nearly one year now, and I am sure all this pages were indexed.
I used to have between 150 and 200 daily unique visitors, and if I look back to my statistics (from Webstat - very detailed), I see that till 2 week ago, I had about 100 of them coming throug Google, based on searche on the non indexed pages.
Is that not the confirmation of the problem I pointed? I can imagine if Google encounters a tag asking not to index nor to follow the links, Google does this.
I guess this is the reason why this tag exists.
Probably that in the next visit of the spider (it used to be after 2 weeks - now may be some longer -) all those pages will be indexed again and my traffic can be normal again.
I have checked most of the incoming links (all of them are content-related), and none of them uses a particular page. All point to www.whisky-distilleries.info
I tried to use SSI without shtml but it simply crashed on the server. May be my code was not correct in .htaccess, but I dared not trying it again. By the way, the changes occured somewhere in March. I had expected negative effects from this operation much earlier, if this was the reason.

Thank you for this very good tips.

[Mel]

Hi Slainthe
I am sure that all your pages were indexed, but as mypage.html. Those pages are gone, replaced by mypage.shtml, which is a totally different page as far as Google is concerned, hence your old pages were dropped and only a few of the new pages are indexed. Not sure why the time lag, but very few of your pages are fully indexed by Google.