PDA

View Full Version : how do passworded websites get indexed by search engines?



haef
01-12-2004, 07:00 PM
I've noticed some webpages will show up in search engines but they are password protected. For example, if you type "how to choose a pos system" on google, this page comes up.

http://www.restaurantowner.com/members/login.cfm?hpage=224.cfm

How did they do that? How can I do that?

I have a website with password protected areas. I use IIS 6 and asp. I also have access databases that are indexed. How can I get google to index them? Here's my website:

http://www.possoftwareguide.com

Any help would be appreciated.

Jeff

Mel
01-13-2004, 01:19 AM
In a word they don't Jeff, unless you take some very special and possbily questionable steps to accomodate them.

The page you referred to is not password protected, it is the regular page to log into the protected areas.

Many protected areas do not get spidered due to the fact that a form needs to be filled in to access the pages and spiders can't do that. Other pages don't get spidered because the directories themselves are password protected. I suspect you may have both problems on your site.

Think a moment about the results of successfully allowing spiders to see your pages - they are at best going to return links to pages that viewers will not be able to see, probably resulting in some 500 errors, - or they are going to return results linking to pages which viewers can see but which you want to restrict access to.

If do want to do this PM me. I'm don't want to get this discussion sidetracked into what you should or should not do,as the only ways I know of involve cloaking and redirects.

dmcgill
01-13-2004, 02:45 AM
The page that got spidered is http://www.restaurantowner.com with out the extention to the login page.. I do agree there was some illegal stuff going on there and I am not too sure why they would do that.