Can The Spiders Eat?

[rickanderson]

Can the spiders eat when I add unique content to my site yet password protect the information?

Best
Rick

[thindenim]

No, they will not be able to access the password protected area.

Of course you don't want them to as if google accessed then the page would be cached and freely available for everyone to download and your protection would be negated.

[rickanderson]

thin...thanks for the reply


is there a clever way to add a ton of content that will feed the spiders yet not be too 'available' to steal from my competition?

rick

[alhefner]

Originally Posted by rickanderson

thin...thanks for the reply


is there a clever way to add a ton of content that will feed the spiders yet not be too 'available' to steal from my competition?

rick

Not really. The search engine spiders are looking for publicly available content only. This makes perfect sense because the search engines want to deliver sites to searchers that are most relevant to the search.

One thing you can do is provide "intro" content that does disclose a portion of the items you want members only to access. That way, you can intice people to join your site to get the full content as a benifit for membership. The "intro" content can contain the keywords or phrases you want to rink highly for.

One example is blogs where only the first part of a post is displayed on the index page and the rest is on the individual post pages. The index page will still rank well in a search if the first portion of a post is well optimized.

[Janna122003]

Originally Posted by rickanderson

Can the spiders eat when I add unique content to my site yet password protect the information?

Best
Rick

Spiders can only eat html file and any text in a page.

[subhzash]

Originally Posted by rickanderson

Can the spiders eat when I add unique content to my site yet password protect the information?

Best
Rick

No. Spiders can not read the content which is held in password protected area.

[wige]

Yes, spiders can access password protected content. Basically, you would need to set the site up so that it recognizes traffic from the spiders, and serves them the content, only presenting human users with the username/password prompt. You would then add a no-cache meta tag so that the page does not show up in Google's public cache. I see this a lot, especially with forums that require registration to read the posts, but that want the traffic from Google.



I report them to Google as spam.