Canonicalization Prevention Guide

[wige]

Canonicalization, or more specifically the creation of duplicate content due to the way web server software handles variations on URLs, has become a much discussed topic here. Specific code snippets to resolve the problem in different situations have been widely discussed, but there is no single place that contains a list of different methods. I created this thread to list several of the more common ways of eliminating the two main types of canonicalization, www vs non-www duplication, and / vs /index.html canonicalization.

All of the examples I post will handle subdomain issues (www vs non-www), directory root issues (/ vs /index.html) and secure server (http vs https) issues if possible.

If you have any suggestions for other methods, please let me know and add them to the list.

[wige]

Apache Server Specific

Requirements:
Apache Server with mod_rewrite enabled
The ability to modify server settings using either .htaccess or access to the server configuration files.

www vs non-www
Add the following code to the .htaccess file in the root folder of your web content, or to the appropriate area of your server configuration, after the RewriteEngine on directive:

RewriteCond %{HTTP_HOST} !^www\.yourdomain\.com
RewriteRule (.*) http://www.yourdomain.com/$1 [R=301,L]

Remove /index.html or index.php from requests for the root of a folder
Add the following code to the .htaccess file in the root folder of your web content, or to the appropriate area of your server configuration, after the RewriteEngine on directive:

RewriteRule ^/index\.(php|html)$ http://www.yourdomain.com/ [R=301, L]
RewriteRule ^(.*)/index\.(php|html)$ http://www.yourdomain.com/$1/ [R=301, L]

Handling secure content, securely
To prevent duplication of content due to the use of a secure connection (https), the content that should be available as secure should be in a seperate folder. Unfortunately, this is generally not possible. For the sake of completeness, however, here is how to do it. Generally, you would put the secure content in a subfolder of the root folder of your web site. For simplicity, you could name this folder "secure". You would then add the following directives to the .htaccess file in the root directory of your server.

<Directory /secure/>
Order Deny,Allow
Deny from All
</Directory>

This will prevent anything from crawling your secure content over an http connection. However, if you want some files to be available on both connections (/style.css, /favicon.ico and all the images in /img/, for this example) you will need to create a second htaccess file in the /secure folder, with the following directives:

Alias /favicon.ico /absolute/path/to/favicon.ico
Alias /style.css /absolute/path/to/style.css
Alias /img/ /absolute/path/to/img/

[wige]

PHP Code

If you do not have the ability to create or modify your server settings, and use PHP to generate your pages, you can accomplish the same thing by adding a code snippet to the beginning of your scripts. The following code must be the first thing in the script, before any output is sent to the browser. Note that this code should work even if there is an internal mod_rewrite or other URL mapping or aliasing in place.

PHP Code:

<?php
if ($_SERVER['HTTP_HOST'] != 'www.yourdomain.com') { // First correct the domain issue
   header('Location: http://www.yourdomain.com'.$_SERVER['REQUEST_URI'], 301);
   exit();
}
if (eregi('/index.(html|htm|php)$', $_SERVER['REQUEST_URI'])) { // Then correct the directory root issue
   $redirect = 'http://www.yourdomain.com'.eregi_replace('/index.(html|htm|php)', '/', $_SERVER['REQUEST_URI']);
   header('Location: '.$redirect, 301);
   exit();
}
?>

For simplicity, this could be added to a remote library script, and simply called by each PHP page on your site.

Handling HTTP and HTTPS
If you have HTTPS on your server, and do not want all of your content mirrored on both the HTTP and HTTPS versions, you can add the following lines to the top of every script, below the code I lay out above:

PHP Code:

<?php
$SHOULD_BE_SECURE = true; // This should be true if the file should be available over HTTPS, false otherwise.
require_once('/path/to/file/below.php');
?>

Elsewhere, create a PHP file with the following lines. This is the file that the require_once will point to.

PHP Code:

<?php
if (($_SERVER['HTTPS'] == 'off' && $SHOULD_BE_SECURE) || ) {
   header('Location: https://www.yourdomain.com'.$_SERVER['REQUEST_URI'], 301);
   exit();
}
if (($_SERVER['HTTPS'] == 'on' && !$SHOULD_BE_SECURE) || ) {
   header('Location: http://www.yourdomain.com'.$_SERVER['REQUEST_URI'], 301);
   exit();
}
?>

You could also add the code at the top of this post to this file, and have every file on your site call this script to check the URL and do the appropriate redirections.

[Dubbya]

Here's a thorough guide to setting up 301 redirects under IIS.

IIS 301 Redirect for SEO - McAnerin International Inc.

[jboeckman]

There's a pretty good article here http://www.webconfs.com/how-to-redirect-a-webpage.php

[mjtaylor]

Great idea, wige. Someone give him some more rep points!

Originally Posted by jboeckman

There's a pretty good article here http://www.webconfs.com/how-to-redirect-a-webpage.php

That is a good article, thanks.

And Jaan's page has lots of resources for 301 redirects. I found this one most useful: How to Create Redirects.

How about making this a Sticky?

Cheers,

MJ

[wige]

Some great suggestions and links already, thanks!

Anyone have any ideas on how to accomplish this type of redirect on ASP .net sites? Generally if your site is on a shared Windows host you won't be able to access the IIS control panel, so this would need to be done programatically. I have seen instructions on doing the redirect:

<%@ Language=VBScript %>
<%
Response.Status="301 Moved Permanently";
Response.AddHeader("Location","http://www.new-url.com/");
%>

But I am not sure what the variables are that contain the host name, requested file name and query string so that these items can be tested. Any thoughts?

[wige]

PERL CGI Method

This is more commonly going to be used with older scripts and shopping cart systems if the server does not support htaccess. The following code should work with most PERL processors. This code must precede any other output to the browser.

if ($ENV{"HTTP_HOST"} != "http://www.yourdomain.com") {
   $q = new CGI;
   print "Status: 301 Found\nLocation: http://www.yourdomain.com".$ENV{"REQUEST_URI"}."\n\n";
   exit();
}

Unfortunately, it has been quite some time since I have done regular expressions in PERL. Any suggestions for the syntax to check if a request ends with "/index.php", "/index.cgi" or "index.html"?

[Dubbya]

This ASP redirect method allows you to pass along querystrings if you need to.
It fires back valid HTTP Status codes with "HTTP/ 1.1 200 OK"

*Caveat: To prevent hacking, It's a good idea to parse querystrings for illegal characters.

Check your headers: Check Server Headers Tool - HTTP Status Codes Checker

<%@LANGUAGE="VBSCRIPT"%>
<%
'*****************************************
' 301 Redirect for non-www domain entries
'*****************************************
 host = Request.ServerVariables("HTTP_HOST")
 page = Request.Servervariables("URL")
 pageData = Request.ServerVariables("QUERY_STRING")

'allow passing of querystrings to the redirected URL
 if pageData <> "" Then
	pageData = "?" & pageData
 end if

'substitute yoursite.com for your own URL.
 if host = "yoursite.com" then 
	host = "http://www.yoursite.com"
	newUrl = host & page & pageData
	Response.Status = "301 Moved Permanently"
	Response.AddHeader "Location", newUrl
	Response.End
 end if
'*******************
' End 301 Redirect 
%>

[lukkyjay]

Originally Posted by wige

Apache Server Specific
www vs non-www
Add the following code to the .htaccess file in the root folder of your web content, or to the appropriate area of your server configuration, after the RewriteEngine on directive:

RewriteCond %{HTTP_HOST} !^www\.yourdomain\.com
RewriteRule (.*) http://www.yourdomain.com/$1 [R=301,L]

I've tried it a few times, but it hasn't worked yet. Maybe it has something to do with what is in my .htaccess already:

ErrorDocument 500 http://www.insuranceshoppers.net/500.html

Should I put it before the "ErrorDocument" line? Including the RewriteEngine on directive, how should it look?

[DoneInStyle]

Very helpful, and I learned something useful. Thank you.

I will mention though that it's considered more forward thinking to redirect www.domain.com into domain.com rather than the other way around since even Tim Berners Lee considers the addition of www to any URL to be "the old web".

[edhan]

Thank you very much for sharing this as I was looking for how to go without both the index.html and index.php.

Great topic.

[Webnauts]

Allow me to add mine here too:

########## Require the www to avoid cannonicalization issues ###
RewriteCond %{HTTP_HOST} ^yoursite.com [NC]
RewriteRule ^(.*)$ http://www.yoursite.com/$1 [L,R=301]

########## Require to add trailing slash if not present to avoid cannonicalization issues ###
RewriteCond %{HTTP_HOST} !^www\.yoursite\.com [NC]
RewriteCond %{HTTP_HOST} !^$
RewriteRule ^/(.*) http://www.yoursite.com/$1 [L,R]

########## Redirect index.html to / ##########
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.*index\.html?\ HTTP/
RewriteRule ^(.*)index\.html?$ http://www.yoursite.com/$1 [R=301,L]

########## Redirect https to http ###
RewriteCond %{SERVER_PORT} ^443$
RewriteRule (.*) http://www.yoursite.com/$1 [R=301,L]

If you have https pages indexed because you have done a mistake, you can create an additional robots.txt calling it for example robots-secure.txt disallowing the indexed files and this in your .htaccess file:

########## To get rid of https files and cannonicalization issues ###
#RewriteCond %{SERVER_PORT} ^443$
#RewriteRule ^robots.txt$ robots-secure.txt

[edhan]

Hi wige

After I add:

RewriteRule ^/index\.(php|html)$ http://www.yourdomain.com/ [R=301, L]
RewriteRule ^(.*)/index\.(php|html)$ http://www.yourdomain.com/$1/ [R=301, L]

It gave me Error 500.

I have no problem when I use this:

RewriteCond %{HTTP_HOST} !^www\.yourdomain\.com
RewriteRule (.*) http://www.yourdomain.com/$1 [R=301,L]

Any idea?

[Bookmauritius]

Hi,

Since i am not a programmer, i cannot add my pinch to this forum which nontheless speaks out to me since i am having a new version of my website under development and rewriting / redirection / duplicate content issues are being considered at the moment.

All i can say is THANKS for sharing your knowledge and i look forward to get to a topic where i will be able to bring savy advice (SEO)

Nice day to all

[Webnauts]

Originally Posted by Bookmauritius

Hi,

Since i am not a programmer, i cannot add my pinch to this forum which nontheless speaks out to me since i am having a new version of my website under development and rewriting / redirection / duplicate content issues are being considered at the moment.

All i can say is THANKS for sharing your knowledge and i look forward to get to a topic where i will be able to bring savy advice (SEO)

Nice day to all

You do not need to be a programmer to copy and paste the above mods. I am not a programmer either.

[imsickofwebpro]

nice.

[ursfehr]

Very helpful, and I learned something useful. Thank you.

[wige]

I am glad people have already found the information useful. To address some of the comments....

edhan, I suspect the issue you are experiencing is arising because of a problem in the .php script itself, possibly because the script in question can't handle the filename not being in the REQUEST_URI. If you post (or PM me with) a URL where I can test it and replicate the issue, I may be able to confirm. The order of mod-rewrite directives vs other directives in your .htaccess file should not have any impact on the functionality. Error 500 though indicates an error in the script itself.

Webnauts, I have a question for clarification about one of the snippets you posted, specifically,

 ########## Require to add trailing slash if not present to avoid cannonicalization issues ###
RewriteCond %{HTTP_HOST}   !^www\.yoursite\.com [NC]
RewriteCond %{HTTP_HOST}   !^$
RewriteRule ^/(.*)         http://www.yoursite.com/$1 [L,R]

The comment indicates that this will add a trailing slash if it is missing, but the tests only look at the HTTP_HOST field, which would not contain a trailing slash. Only the domain name is contained in that field. The request (the portion after get or post in the request) is contained in REQUEST_URI. Did I misinterpret this?

Please note, when doing redirects, R uses a 302 redirect (at least in Apache 2.0-2.2), using R=301 will force the server to respond with a permanent redirect, which is generally preferable so that search engines properly process the redirect.

I'll also add, as DoneInStyle stated above, intellectually, yourdomain.com is preferred over the older, traditional www.yourdomain.com. However, I used www in all my examples because, well, I'm pandering to user expectations. Most servers and hosting companies up to a few years ago forced hosted sites to use www, www was also used to "brand" a company's Internet presence in offline marketing. This has ingrained www into the consciousness of the web-going consumer. It is second nature for users to enter www before a URL when trying to remember an address (as indicated in numerous usability studies, as well as the AOL Personal Information Giveaway - I mean leak) and redirecting a user, even seamlessly, could have the potential of disrupting the user experience, simply by defying the expectations of the user. Also, as most security certificates are issued for the www subdomain, some browsers may issue a warning when users change subdomains when entering or leaving the secure connection. This is all my own opinions, and not really the place to debate which form is better, I just wanted to address the comment and point out that yes, I did use www in all my examples for this reason, and if you prefer the non-www form be aware of this and modify the code accordingly.

[villageloop]

Originally Posted by DoneInStyle

Very helpful, and I learned something useful. Thank you.

I will mention though that it's considered more forward thinking to redirect www.domain.com into domain.com rather than the other way around since even Tim Berners Lee considers the addition of www to any URL to be "the old web".

Yes this is a very helpful thread! Thank you all.

With regards to using www, from my understanding, it does not make a difference to the search engines. And although it may appear to be "old web" it still makes sense with all of the new TLCs coming out, this way, when your url is used in print or any other medium, it is automatically known that it is a website.

I know it is hard to believe but there are many people who still dont know the difference between and email address and a website address.

[edhan]

Originally Posted by wige

I am glad people have already found the information useful. To address some of the comments....

edhan, I suspect the issue you are experiencing is arising because of a problem in the .php script itself, possibly because the script in question can't handle the filename not being in the REQUEST_URI. If you post (or PM me with) a URL where I can test it and replicate the issue, I may be able to confirm. The order of mod-rewrite directives vs other directives in your .htaccess file should not have any impact on the functionality. Error 500 though indicates an error in the script itself.

I was doing for my site: Thai Buddha Amulets | Tibetan dZi Beads | Swarvoski Crystals | Membership Club | Directory Search | SEO | Ebooks. What do you need to test my URL?

Thanks!

[webmax]

So if i understood correctly, to be able to have just a part of the site be https without touching the .htaccess file then all pages have to be php. Is this correct?

[wige]

Originally Posted by webmax

So if i understood correctly, to be able to have just a part of the site be https without touching the .htaccess file then all pages have to be php. Is this correct?

Correct, unless you have a Control Panel that lets you accomplish the same thing.

[lukkyjay]

On my interior pages, like www.mysite.com/help.html, my website shows an error if the user leaves off the .html (www.mysite.com/help). Is there something I can add to the .htaccess to fix that problem?

[wige]

It is possible. There are two situations you may encounter: the trailing slash may be omitted, or the file extension may be omitted. You would need to look at your logs and determine which happens more often and correct for that situation, and let the other situation be resolved through custom error pages.

If you want to add a trailing slash / if no file name is specified (domain.com/file becomes domain.com/file/) use the following:

RedirectMatch 301 ^/([a-zA-Z0-9/]*)$ http://domain.com/$1/

If you want to automatically add the extension .html, on the other hand, you would use the following:

RedirectMatch 301 ^/([a-zA-Z0-9/]*)$ http://domain.com/$1.html

[webmax]

Originally Posted by wige

Correct, unless you have a Control Panel that lets you accomplish the same thing.

Thank you, Wige... although I doubt my Control Panel lets me do that. We host all our clients in GoDaddy. But we've come across something else - a little piece of javascript that you put into the html files that you want to keep as http. Would you mind checking to see that I didn't mess up again? The site is spauno.com. Only the "Reservation" pages and the contact sheets need to be https. What was happening was that when you navigated out of an https page and went to an http page, it would show this last page as also being secure, even if you had previously navigated it in http. This was creating the duplicate content. For some 6 months G wouldn't touch this site not even with a 10 foot pole. I hope this will get my client ranked.

[wige]

The Javascript you have in use seems to work, however this type of redirection is not generally supported by the search engine spiders. You could run the risk of the https version of the page being indexed or, worse, dropped if the search engines see the Javascript redirection as sneaky/suspicious.

[webmax]

Thank you so much for your help... and unfortunately you are right about being dropped. Over the weekend G un-indexed all pages except for the php and one html where we forgot to put in the "magic" code. Ouch. Like I said, the site has no htaccess file, goes crazy every time we try to put one in, and the Control Panel does not have an option for singling out pages, or much of an option for controlling anything really. Would you suggest we change everything into php? Or is there something else we can do?

[wige]

In your case I would recommend contacting your web host to find out what they recommend to get this working. You would get the best results from setting this up through .htaccess, but the hosting company may need to change or enable something to get it to work for you.

[webmax]

Again, thanks a lot for your help. I'll call GoDaddy to see what they can do. In the meantime, we've gone the way of all php files.

[bj]

Webmax, Godaddy is notorious for this sort of problem. My best suggestion to you is to simply google "Godaddy Hosting" and then, having had your eyes truly opened, search for a real host!

I'd start your search here:

Web Hosting Talk - The largest, most influential web hosting community on the Internet

[webmax]

Is it smart to expect more, for $43.05 a year?

[bj]

You get what you pay for.

[webmax]

So I just called them and they said that yes, their $43.05 shared hosting supports htaccess files. They checked a sample of the htaccess we had written and said it needed fixing. So the mistake is ours, not theirs... (It's against their policy to rewrite code supplied by the client, otherwise the guy would have fixed it for me.)
For the sake of Wige's compilation of solutions to canonicalization problems, if ever we come up with that file and it works, we'll copy it here.
Thanks to all for your help.

[webmax]

Hello Wige, it's me again, back from Google hell with total forgiveness of all my sins...

We put this into an htaccess file outside of the secure folder:
RewriteEngine On
RewriteCond %{SERVER_PORT} !80
RewriteRule ^(.*)$ http://www.spauno.com/$1 [R,L]

We put this into another htaccess file inside the secure folder:
RewriteEngine On
RewriteCond %{SERVER_PORT} 80
RewriteCond %{REQUEST_URI} secure
RewriteRule ^(.*)$ https://www.spauno.com/secure/$1 [R,L]

You can now navigate back and forth between secure and unsecured content without creating duplicate content and Google has reindexed the pages. So to be able to say that this works for sites in shared hosting, specifically GoDaddy Linux hosting, all we need is Wige's blessing.

[wige]

Looks good to me, and it should work in most Linux setups. The only thing I would change is from
RewriteCond %{REQUEST_URI} secure
to
RewriteCond %{REQUEST_URI} ^/secure/

That should prevent issues if you have another page on the site that includes the word secure in the filename from triggering an endless loop of redirects.

[australianfranchises]

Does flash content creates conical issue?

[Webnauts]

Originally Posted by wige

I am glad people have already found the information useful. To address some of the comments....

edhan, I suspect the issue you are experiencing is arising because of a problem in the .php script itself, possibly because the script in question can't handle the filename not being in the REQUEST_URI. If you post (or PM me with) a URL where I can test it and replicate the issue, I may be able to confirm. The order of mod-rewrite directives vs other directives in your .htaccess file should not have any impact on the functionality. Error 500 though indicates an error in the script itself.

Webnauts, I have a question for clarification about one of the snippets you posted, specifically,

 ########## Require to add trailing slash if not present to avoid cannonicalization issues ###
RewriteCond %{HTTP_HOST}   !^www\.yoursite\.com [NC]
RewriteCond %{HTTP_HOST}   !^$
RewriteRule ^/(.*)         http://www.yoursite.com/$1 [L,R]

The comment indicates that this will add a trailing slash if it is missing, but the tests only look at the HTTP_HOST field, which would not contain a trailing slash. Only the domain name is contained in that field. The request (the portion after get or post in the request) is contained in REQUEST_URI. Did I misinterpret this?

Please note, when doing redirects, R uses a 302 redirect (at least in Apache 2.0-2.2), using R=301 will force the server to respond with a permanent redirect, which is generally preferable so that search engines properly process the redirect.

Wige my apologies for responding too late. Looks like I lost the track.

Well, I added that option with the trailing slash, since it is technically different than without it.
SEO advice: url canonicalization

Also many try to get IBLs adding the trailing slash in their URLs like http://www.justanexample.com/, aiming to flow the PageRank only to their homepage. If you probably have noticed, many web directories forbid that already. Besides, if you have such IBLs, do you exclude the possibility that Google or other SE will not see that as a canonical issue?

[wige]

Originally Posted by Webnauts

Also many try to get IBLs adding the trailing slash in their URLs like http://www.justanexample.com/, aiming to flow the PageRank only to their homepage. If you probably have noticed, many web directories forbid that already. Besides, if you have such IBLs, do you exclude the possibility that Google or other SE will not see that as a canonical issue?

I do exclude that possibility, because that is not how the spiders work. In order for a spider (or a web browser or any other system) to request a file, it breaks the link into two parts, the hostname (which will be either a domain name or IP address) and a request URI, which always starts with a "/". So, to give a few examples of how a spider looks at different links:

http://domain.tld/somepage.html
Spider sees
Protocol: http
Hostname: domain.tld
Request URI: /somepage.html

http://domain.tld/
Spider sees
Protocol: http
Hostname: domain.tld
Request URI: /

http://domain.tld
Spider sees
Protocol: http
Hostname: domain.tld
Request URI: /
(The request URI can NEVER be blank, and MUST ALWAYS start with a /, so if no URI is included in the URL, a slash is used by default.)

Even when the spider stores the information about the retrieved page in the index, (think giant relational database) that slash is always added, simply because the field containing the request URI can't be blank. The same happens when storing a list of links - the slash is added if not already present.

Specifying a leading slash at the beginning of the Request URI is also expected by the server. If a request reaches your server without that leading slash, the server may simply give a bad request message, or ignore the request, depending on how your server is set up.

I have also gotten indications from Google, Yahoo and MSN that their systems always add a leading slash if it is not already present, as does every malbot and spider system I have ever worked with. Even wget, which was the foundation for many spiders, automatically adds the slash. It is simply a default part of the HTTP protocol.

Regarding the link you mentioned, I take it you are referring to the following:

Originally Posted by Matt Cutts

• www.example.com
• example.com/

I have seen this used in numerous subsequent articles on canonicalization, used as the basis of an argument for taking steps to handle missing slashes. However, what he was highlighting was the absence of the subdomain in the second version, not the presence of the slash. In the rest of the article, he makes no mention of the slash at all, which leads me to believe this was only a typo. It was addressed in the comments, where Matt suggested selecting a preferred format for links, but it was not addressed beyond that.

Above all, it is important to remember that to a server, the requests for www.example.com and www.example.com/ both look identical (GET / HOST: www.example.com) so anything you do on the server to redirect from one to the other is pointless anyway.

[Conficio]

[QUOTE=wige;358772]
If you want to add a trailing slash / if no file name is specified (domain.com/file becomes domain.com/file/) use the following:

RedirectMatch 301 ^/([a-zA-Z0-9/]*)$ http://domain.com/$1/

"domain.com/file/" does make little sense to me, because the trailing slash in a URL indicates the index of the directory (See Apache DirectoryIndex directive).

The simplest use of a webserver is to point it to directory tree and let it serve the files there. Typical conventions are that you do specify a default file extension, for example .html so that domain.com/abc serves domain.com/abc.html. The second common convention is that the domain.com/edf/ shows the list of files available, unless the DirectoryIndex directive (or equivalent for non Apache) is set and the file specified is present, such as domain.com/edf/ actually serves domain/.com/edf/index.html.

To clarify when I say serve, I mean it returns thes specified content and not a redirect. This is evident by no change in the URL entry field of the browser.

In the context of this thread, this can lead to duplicate content, as for example domain.com/edf/ returns the same content as domain.com/edf/index.html (However, if no one ever links to .../index.html the search engine would never discover the URL)

By the way your redirect script would also change domain.com/abc.html into domain.com/abc.html/

K<o>

[wige]

Originally Posted by Conficio

By the way your redirect script would also change domain.com/abc.html into domain.com/abc.html/

It shouldn't, but I will take another look. The pattern I am using:
RedirectMatch 301 ^/([a-zA-Z0-9/]*)$ http://domain.com/$1/

contains ([a-zA-Z0-9/]) which should only be true if the string does not contain a .. As a result, if a file extension is specified, the user should not be redirected. (The request URI must only contain those characters shown in brackets for the user to be redirected.)

This is really intended to counteract a server setting that can allow the server to respond to requests omitting the trailing slash as though the slash was there, although it does also remove a possible error condition that would need to be handled with a 404 by redirecting the user to what is most likely the most desired location.

[edhan]

Originally Posted by edhan

Hi wige

After I add:

RewriteRule ^/index\.(php|html)$ http://www.yourdomain.com/ [R=301, L]
RewriteRule ^(.*)/index\.(php|html)$ http://www.yourdomain.com/$1/ [R=301, L]

It gave me Error 500.

I have no problem when I use this:

RewriteCond %{HTTP_HOST} !^www\.yourdomain\.com
RewriteRule (.*) http://www.yourdomain.com/$1 [R=301,L]

Any idea?

Finally, I tried using Webnauts

RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.*index\.html?\ HTTP/
RewriteRule ^(.*)index\.html?$ http://www.yoursite.com/$1 [R=301,L]

And it works fine! So now I have no problem of getting it displays as:

www.mydomain.com

for index.php or index.html

Thanks Webnauts!

[google consultant]

WIGE,
great post. Very useful for a client I have. I have gone into the site and used full domain to homepage rather than .index.html. But adding the code into the htaccess file is a great backup. Keep up the good work.

[ericajoieake]

The domain with www and without www are two different domain so there's nothing for you to worry about, but I suggest to own both of the domain and redirect to your chosen main url, to be able to gather all the traffics coming from both domains.

[edhan]

Originally Posted by ericajoieake

The domain with www and without www are two different domain so there's nothing for you to worry about, but I suggest to own both of the domain and redirect to your chosen main url, to be able to gather all the traffics coming from both domains.

But both with www and without www are the same content. I have read somewhere that it might be taken as duplicate content so that is why I am using canonicalization.

[Webnauts]

Originally Posted by ericajoieake

The domain with www and without www are two different domain so there's nothing for you to worry about, but I suggest to own both of the domain and redirect to your chosen main url, to be able to gather all the traffics coming from both domains.

How can someone own a domain with www and not without www, and the way around?

[deepsand]

Originally Posted by ericajoieake

The domain with www and without www are two different domain so there's nothing for you to worry about, but I suggest to own both of the domain and redirect to your chosen main url, to be able to gather all the traffics coming from both domains.

The prefix "www" is not part of the Domain Name, but rather an identifier which serves to denote which Internet system or service, in this case "World Wide Web," is being used to communicate with whatever resides at that domain.

Other commonly used services include File Transfer Protocol (ftp) & Internet Relay Chat (irc).

A Domain Name, on the other hand, serves as an identifier of a specific resource that exists within the Internet.

[webmax]

Wige,
Thank you for your help. Excellent post. We come back to it often to check for solutions.

[nashville]

How can I tell if I have Canonicalization issues at pilotjourney.com

[Dubbya]

Originally Posted by nashville

How can I tell if I have Canonicalization issues at pilotjourney.com

You do.

You want pilotjourney.com to resolve to www.pilotjourney.com. Presently, it doesn't.

Essentially, you want the server to check all queries to ensure that they include the "www." portion of the URL. If they don't, you want the server to rewrite the URL to include the "www.".

Ya follow?

Considering that you're running a .php site, you should be able to find the help you need to set up the rewrite script properly.

Good Luck.