robots.txt and htaccess for multiple domains on one hosting account

**Tarzan2** · 02-05-2009, 03:26 AM

I know there are several of you that are running several domains from a single hosting account, and I am thinking about doing the same. I have spent the last 2 days reading and reading, and I see that it is recommended to set up a robots.txt file to keep the spiders from lumping all domains in the hosting account into one.

I don't understand that thought... Don't the spiders follow links? If there are no links between the different domain names, then they stop spidering when they have finished spidering the one domain that they came in on. Is that not so? Do they actually go into the site directory itself and root around looking for all pages within a hosting account? If that is the case, then how do you tell a spider to only look at the one section (domain name) and then at the same time have them spider all your other domain 'sections' because you want those included in the search engines as well? I'm confused big time.

All the stuff I have been reading also suggests that you create a .htaccess file in the root directory of the hosting account to sort out what pages to display when a certain URL is specified. My host has what they call a domain pointing tool that will send visitors to the appropriate folder of your account depending on which domain name was typed in the browser. I imagine this is done by that very same .htacces file, so in essence, there already exists such a file and to create another would likely be suicide. I'm sure this file could also be edited, but I understand that certain commands in that file need to be in a particular order, and if I mess that up, I'm in trouble again!

So... what I am asking for is some clear guidance as to what I need to do to host several domain names that are each associated with a different web site, all under a single hosting plan? And, will visitors be able to tell that they are looking at a site that is hosted under another primary account?

To better understand what I am talking about, in my signature file, you can see that I have 2 distinct topics (sites) on the same host account. If I were to purchase another domain name for the second site (lets say 'friends.com'), can I make it look as if it is hosted on its own account (both for SEO and visitors)? If so, how exactly do I accomplish that?

Thanks in advance for your input.

**speed** · 02-05-2009, 05:26 AM

The following assumes you are on a cPanel based server and that the host is keeping it current.

When you are in cPanel there's the option Addon domains, click it and you enter the information for the domain you wish to add to your account.

If you enter example.com then by default cPanel will set the home account to public_html/example.com that is it creates the web space for the new domain inside the web space for the accounts domain, this allows you to access it as both http://example.com and http://yourdomain.com/example.com

The simple solution is when adding the domain edit the Document Root field and remove public_html from it so it just becomes example.com now your new domain has it's web space outside of the accounts web space and so can ONLY be accessed by its domain and not that of the account.

As a side note this is also true when creating sub-domains.

Now as you can't access your new domains web space via your main account domain there's no difference to hosting 2 or more domains on the same account or having 2 or more accounts on the same server.

If however your host is using a control panel which will not allow you to create web space for the addon domains outside of your main web space then either change host or you have to use rules in a .htaccess file placed within the folder assigned to each addon domain.

So if you have the addon domain example.com and it's web space is public_html/example.com then in public_html/example.com you'd place an .htaccess containing the following:

RewriteEngine On
RewriteCond %{HTTP_HOST} !^www\.example\.com [NC]
RewriteRule (.*) http://www.example.com/$1 [R=301,L]

That rule simply states if the domain name used to access the site is not www.example.com the do a permanent redirect to http://www.example.com/

Now you can't access the content via the main account domain again, its not as elegant as keeping the web space separate but it works and again there's no difference between 2 or more domains on one account or many accounts on one server.

I hope that helps if not ask away and I'll do my best to explain.

**kgun** · 02-05-2009, 05:49 AM

I think you are talking about what is called add on domains and not sub domains. I have an add on domain solution by two different hosters. On the web server the add on domains are organized as subfolders of the main domain. There are two different solutions:

The simplest, one ftp password. The add on domains are uploaded as to the main domain. The various domains can share one central CSS styling file located in the sub folder of the main domain.

Example:

Main domain: Kjell Bleivik

Add on domain: DigitalStart.net: The starting point for English speaking surfers and webmasters
Each domain has its own unique ftp password. The add on domains are seemingly uploaded to its own URL, but it is organized on the web server as the above, that is as a sub folder to the main domain.

Example

Main domain: ForumNorway.com :: Index

Add on domain: Meta Search Engine and search resources: SurfToolbar.com

The first is of course easiest.

I use a unique robots.txt file for each domain regardless of solution. Regarding .htaccess files you need to be very careful. .htaccess settings on the main domain carries through to the add on domains (subfolders). Example:

On the main domain: You disallow access from IP region Y.

On one of the add on domains you want access from IP region Y. Then you have to make a separate .htaccess file for this domain (folder) where you allow access from IP region Y.

To sum up:

Exept for styling, I treat each add on domain as a unique domain, even if there may be overlap. That is also easiest if I later want to move an add on domain to its own unique Ip or sell the site. You can upload the code as is.

**SemAdvance** · 02-05-2009, 06:22 PM