Use of ? marks in URLs

[PastorRick]

Hi Folks,
I have two questions please. One involves extensions and how changing them affects SE's re-spidering your site and the other has to do with how sites with ?'s marks get spidered.

Will changing ".htm to .php" affect search engine optimization? I'm helping someone with a site and they have a lot of pages. They are not using any templates or includes so the navigation is a bit of a mess. I want to use includes to make the navigation the same on each page but they are worried about me changing the extensions. Will it have an affect?

I am using aardvark topsites as a topsites list for someone and they are worried about the question marks in the URLs. They want to make sure each page including those after the ? get spidered. Does anyone know how I can remove them or change it so that it will be spidered correctly by search engines?

[cyanide]

The extension of pages... php, asp, htm, html, etc. has nothing to do with page rank. The question mark won't be a problem either

[PastorRick]

OK, now I am confused. I have read and heard that Google robot for example, doesn't parse URL's past a ? which is why replicated sites cannot get into the SE's.

I know that google can index pages with all those extensions. The problem I see is that if google has page.html and you change it to page.php when google goes to check your site, it will not be found. This seems to me that you would have to start the whole promotion all over again. This hurts when you already have a page rank of 5 or higher.

Other Thoughts?

[cyanide]

Originally Posted by PastorRick

OK, now I am confused. I have read and heard that Google robot for example, doesn't parse URL's past a ? which is why replicated sites cannot get into the SE's.

You are either listening to someone who is not informed or read an article that is way out of date.

If you are changing the page names, then you will have to put some re-direct lines in your htaccess file

such as:

redirect 301 /oldpage.html http://domain.com/newpage.php

[AndrewX]

Some have theorized that non-cgi extentions like .html are preferred over cgi, because cgi scripts can be like a bottom-less hole, the ?arg can simply increase by one and the software could return a page that is not really unique. This is easier to understand if you've ever tried to implement a spider for a search index.

Look into mod rewrite. This will let you keep your pages as they are, but they can be rewritten by your server into a nicer name, something that looks like a normal static HTML page URL.

[ringofire]

If you are creating dynamic pages with a ? in the url, Google typically will not spider beyond 10 characters past the ?. They recommend that you put items like session ids in a cookie instead. It really depends on the composition of the entire page.

Another option is to use mod rewrite to create search-friendly urls. (as AndrewX also recommended)

Bernadette~

[savvymedias]

Trying to make dynamics static is the biggest waste of time out there. Time better spent making the site full of relevant content.

Make sure you have valid code, meta tags, relevant text, a site map and try to get a least a few new permanent links from people every week and you will be fine!

Check out lingerieatlarge.com and babysunprotection.com on google -- same shopping cart.

[savvymedias]

Trying to make dynamics static is the biggest waste of time out there. Time better spent making the site full of relevant content.

Make sure you have valid code, meta tags, relevant text, a site map and try to get a least a few new permanent links from people every week and you will be fine!

Check out lingerieatlarge.com and babysunprotection.com on google -- same shopping cart.

[Jayms]

Another option is to use mod rewrite to create search-friendly urls. (as AndrewX also recommended)

I third this as making your urls search engine friendly makes all of the difference in the world and there will never be a doubt as to whether your pages will be indexed.

[newmarket]

I don't know about ? marks, but you could just set your web server to parse .htm or .html files with php.

That way you wouldn't necessarily need to change the file extensions and you would still be able to use php.

Off the top of my head, I can't remember, but it is a simple entry in the .htaccess file.

I hope that this helps.

[hal]

Hello PastorRick,

The people over at Google have learned to never say never, because there always seem to be exceptions. So, what they do say tends to be rather generic.

At http://www.google.com/webmasters/2.html they say

We are able to index dynamically generated pages. However, because our web crawler can easily overwhelm and crash sites serving dynamic content, we limit the amount of dynamic pages we index

At http://www.google.com/webmasters/guidelines.html they say

If you decide to use dynamic pages (i.e., the URL contains a '?' character), be aware that not every search engine spider crawls dynamic pages as well as static pages. It helps to keep the parameters short and the number of them small.

And, upon doing a search recently for a "similar to" using as a seed a URL with a "?" in it, I remember seeing a note that they only check the first 10 ... was it "words?" "Elements?" <sigh> I forget the exact phrase.

In any case, there is a limit. The result is that the shorter the better. And many people have translated this into "difficulty with ? means don't do it at all."

Hope this helps.

Hal

[godzilla]

From personal experience I can tell that google spiders have no problems indexing files with *.php?page=*. As example: http://godzilla.bounceme.net/~godzil...p?page=soen357 is easily indexed. Note also that the rankings of such pages do not suffer in any fashion.

But as all previous users have mentionned, any crawler google or not may have difficulty to index long url's with "?". As an example:http://godzilla.bounceme.net/~godzil...JtXzIwMDIuZG9j is sometimes indexed, sometimes not.

[alienzhavelanded]

Changing the page extension could affect your rankings, but there are ways around it. For example, with my current project I'm converting an ASP site into PHP. I will have to leave a 301 redirect in the ASP pages for the spiders to get the new address. The only catch in my situation is the pages will have duplicate content (for a short while)if I do it that way. However the new design will greatly improve the site so I'm willing to take a hit on the ranks. I can also safely assume the new pages will be indexed soon since the spiders regularly visit the site.

You can also do things with custom 404 pages and .htaccess to point anyone linking to the old page to the new one as well as cyanide pointed out.

[Tamera Billings]

Can anyone tell me if a ? in a reciprocal link is counted as a back link?

For instance this is the link I got back from trading with a competitor:
http://www.leathertree.com/otherl.as...A016672DD54%7D

Does Google count this as a back link to my site?

[BobbyLocksville]

Inbound links to your site should really be direct links e.g Link to this cool site

However, if the page you are linking from has page rank, and the link can be followed by the Google spider, then yes, it will be included.

In my experience Google spider tends to stay away from links with more than two variables.

http://www.domain.com/viewpage.asp?id=234&side=2134
Would be indexed.

Where
http://www.domain.com/viewpage.asp?i...bv234hxd34dsff

Would probably be avoided.

The spider tends to avoid these kind of links as it will assume the site is highly dynamic and it could possible crash the site.

Keep your inbound links simple where possible.

- Bobby

[cryptblade]

Hi Pastor Rick,

I checked your site. The parameters in URLs that MAY affect your rankings don't just include "?". Currently, you have a lot of parameters.

Like others have said, a mod rewrite is a good way to go. After all, you are concerned with this issue, and a solution is available.

Always err on the side of caution when in doubt. A mod rewrite makes the URLs search-friendly - that is a good thing. Therefore, it doesnt hurt to spend the time doing that.

One poster said just focus on content. That poster might be a "disciple" if Jill Whalen of the SEO camp that contend that content and only content is king. That isn't necessarily true - as black hat seos prove time and time again. Besides that, it would be unfair for you to have to keep adding content after content and even spend time on the web building tenure before you got any rankings.

Cleaning the code up, url mod rewrite, titles, even meta tags are all good to do.

[DrTandem1]

I think a more important question is whether or not the content on a dynamic page such as .php will be indexed. Basically, the spiders don't see the content derived from the PHP script. So, worrying about the identifying characters after the "?" is really missing the point.

In other words, whether or not the spider will read beyond the "?" has absolutely no relation to it being able to index what you think is on the page.

[tsarouhan]

I think "savvymedias" is correct that mod writing is a waste of time if you properly design the data driven site. You can use parameters for most search engines but keep them short and keep them to no more than 2 parameters (example: www.yoursite.com?pid=23&uid=5). I have recently redesigned a dynamic site using asp.net and we went from 30 pages indexed to 14,000 pages indexed. They are an ecommerce site with about 4,000 products. It does work if you do it right. Check out the site for some tips (www.optcorp.com)

Good Luck