Urgent Question, PLZ help me

[alexcl]

The question is:
I know Google robot will crawl web pages with this kind of URL "http://www.google.com/xxx.htm(html, etc)".
I want to ask whether Google robot will crawl web pages with this kind of URL "http://www.google.com/xxx" (no ".htm" ".html" ".php" or other extended names)

[Sanjo]

If you have a URL that's:
http://www.yoursite.com/whatever

Google will think that "whatever" is a filename and look for:
http://www.yoursite.com/whatever/index.html, htm, php, etc.

[jestep]

I've seen lots of high ranking sites use the chmod technique for this so I believe that they are still indexable in google. Im not too sure about Yahoo or any other but I think you're OK in Google.

[oagady]

Actually, Google is using this technique itself. For exsamle, the following url:

http://www.google.com/ads/

It's not necessarily has to be "index.html". You can set up a web server to display any file as a defaul file. Any file name, any extension (which your server recognize). If Google set up its server to display "index.html" as a default, then when you type in a browser "http://www.google.com/ads/", the file "http://www.google.com/ads/index.html" is displayed (but it can be anything - "whatever.php", "whatever.jsp", "index.asp", "myfile.asp".

[oagady]

Sorry, I clicked a wrong button, so my posting was displayed a second time.

[Conficio]

Originally Posted by Sanjo

If you have a URL that's:
http://www.yoursite.com/whatever

Google will think that "whatever" is a filename and look for:
http://www.yoursite.com/whatever/index.html, htm, php, etc.

Sorry,
this is not really correct. Google will take what ever one's webserver returns for any URL. If you return HTML Google will take that, if you return XML, they will take that, if you return PDF they'l take that - provided the MIME encoding is set correctly.

By the way .PHP, .ASP, .pl, etc. do return usually HTML, as they are server scripting languages and the browser does not understand any of them.

http://www.yoursite.com/whatever returns the content of a file, if the webserver maps its URL space into a file system (default). If it does some dynamic page serving, it will do what ever is defined.

Only if you write your URL as http://www.yoursite.com/whatever/ (see the extra "/" at the end?) will your web-server (not Google) return a directory listing. If your webserver is configured that way (most are by default) it will attempt to return an index page (default index.html, etc.) instead of the list of files in the corresponding directory.

For more information read the Documentation of the most popular web server.

Hope I could be helpful

K<o>