Crawlers and https???

[Niko Holopainen]

I'm getting a tad anxious and know that I should just wait and see...

But, just to check - does SSL (https://) prevent spiders from crawling a site? I assume not but with changing frmo http to https my rankings were lost.

Also, regarding the online tools that have been mentioned here, on WPW. The keyword, size etc. analyzers as well as tools that would present the site as the robots see it all turn up fatal errors and can't get anywhere near the site?

Does anyone have any idea what's going on?

[Niko Holopainen]

In response to blackicicles statement about url - my post was meant to be on general level and since the site is quite bad currently, didn't want to post the url but here it is (the one that has the spider problem). I'm redoing the site completely and didn't think a review of the statements was in question.

The site I'm having difficulties with is (in finnish only, sorry):
https://www.lomaosake.fi

*never trust a cms to do a person's job*

Yours truly, as always,

: NikoH

[Niko Holopainen]

Sorry for pushing this message upwards...

I'm really in need of help, does anybody know whether ssl hinders crawlers somehow or has my server been configured in a manner that stops crawlers?

Thanks in advance =)

[Mel]

Hi NikkoH

When you changed your pages from http to https you changed the URL and so lost the rankings for the old pages which the spiders now cannot find. It would have been better to have the http pages coexist for a while with a 301 redirect pointing to the new https pages to inform the spiders that the content was moved.

If your https pages do not require passwords or cookies to access them you should in time get your rankings back.

In order to help with the spidering it might be a good idea to set up your server to return a response to the if-modifed-since GET which most spiders seem to prefer.

Regarding the spiderability of your site, two out of three sim spiders could not spider your site, but I can't tell you why since the site is in Finnish.