will virtual directories affect SEO

[thebloke]

Guys

I've been asked by our marketing department to set up a number of urls for ad campaigns.

I've started doing this using virtual directories in IIS which all point to one file in a folder on the webserver /marketing/redirect.asp . Now, I've disallowed this folder in my robots.txt file and there are no direct links to this folder (all the ads will be off-line).

Is this likely to have any affect on my SEO work? I've put a lot of work into SEO over the last couple of months and it has paid dividends but I don't want to scupper it over something stupid!

Cheers


Peter

[ctabuk]

Hi, the link is down, could you check it out please?

[thebloke]

Originally Posted by ctabuk

Hi, the link is down, could you check it out please?

Hi. No, that aint the actual link. That was just as an example. One of the links is:

http://www.choicequote.co.uk/catnw

[mktplace]

If there are no links to the /ads1 folder from your site, and you are blocking it via the robots.txt file, there should be no effect to your SEO efforts, as the major search engines will not see the file, and thus not consider it duplicate content.

Remember, search engines know nothing about "virtual directories", just as they know nothing about ASP or PHP code - in the end, all they see is the HTTP headers and HTML returned when they do a GET, POST or HEAD on an URL.

[effisk]

Originally Posted by thebloke

all the ads will be off-line

What do you mean by this?

Originally Posted by thebloke

Is this likely to have any affect on my SEO work? I've put a lot of work into SEO over the last couple of months and it has paid dividends but I don't want to scupper it over something stupid!

It won't affect you SEO work as long as you ad pages are in that "no-robots" directory. They won't see it, it will be as if these pages don't exist to the bots "eyes".

[effisk]

hehe, next time I'll try to be quicker ;)

[greeneagle]

Your question about "virtual directories" isn't very clear.

The url you gave:http://www.choicequote.co.uk/catnw
redirects to: http://www.choicequote.co.uk/van/van-insurance-uk.asp

My immediate reaction without knowing more is, that you are playing dangerous games with what the SEs will view as doorway pages.

Surely I have misunderstood and you are not calling "doorway pages", "virtual directories"!

Also, Would that be a 301 or 302 redirect?

Ken

[effisk]

Originally Posted by greeneagle

Your question about "virtual directories" isn't very clear.

The url you gave: -http://www.choicequote.co.uk/catnw
redirects to: -http://www.choicequote.co.uk/van/van-insurance-uk.asp

That's true, I was too quick in my reply. I am not familiar with IIS servers, but I understand that virtual directories are directories that appear in the urls but are not actual directories on your server.
When I check your robots.txt file (-http://www.choicequote.co.uk/robots.txt) I see that neither catnw/ nor van/ are listed, so search engines robots will be able to follow the path to the actual page (-http://www.choicequote.co.uk/van/van-insurance-uk.asp). Don't assume search engines don't find pages when there is no link to those pages - very often you'll end up with these pages listed on search engines results because there was a leak somewhere (well now there are links anyway).

If you don't want to see catnw/ indexed in search engines, add it in your robots.txt file.

Same for van/

Originally Posted by greeneagle

Also, Would that be a 301 or 302 redirect?

302:

Originally Posted by Check Server Headers Tool

#1 Server Response: -http://www.choicequote.co.uk/catnw
HTTP Status Code: HTTP/1.1 302 Object Moved
Location: -http://www.choicequote.co.uk/catnw/
Server: Microsoft-IIS/5.0
Content-Type: text/html
Content-Length: 158
Redirect Target: -http://www.choicequote.co.uk/catnw/

#2 Server Response: -http://www.choicequote.co.uk/catnw/
HTTP Status Code: HTTP/1.1 302 Object moved
Server: Microsoft-IIS/5.0
Date: Wed, 27 Jul 2005 08:28:04 GMT
X-Powered-By: ASP.NET
Location: ../van/van-insurance-uk.asp
Connection: Keep-Alive
Content-Length: 121
Content-Type: text/html
Set-Cookie: ASPSESSIONIDCAQBCQAB=MDAHNJDDLAGCMMIIHNMPOEOP; path=/
Cache-control: private
Redirect Target: ../van/van-insurance-uk.asp

(tool is at -www.seoconsultants.com/tools/headers.asp )

I guess Google will index catnw/ in its listing with the content of van/van-insurance-uk.asp

[thebloke]

Ok, let me clarify.

I have set up a number of virtual directories in IIS. One being catnw. This, and the rest, points to a file, redirect.asp, in a folder called marketing. The marketing folder is disallowed in robots.txt and the redirect.asp file contains only vbscript/asp/ado type stuff. The file extracts the information it needs from the PATH_INFO servervariable, logs the visit in a sqlserver database then redirects to the relevant page on the site, in the case of catnw to /van/van-insurance-uk.asp. This van insurance page is linked to from the main site. I have done it this way as I didn't want to duplicate the van insurance page and didn't want to have multiple copies of it which would mean I would have to update multiple copies of it should anything change.

Does this make things clearer?!

Cheers

[thebloke]

ps, I've now disallowed all of the virtual directories in robots.txt. Will this help!?

[effisk]

Originally Posted by thebloke

ps, I've now disallowed all of the virtual directories in robots.txt. Will this help!?

no.
If I understant correctly, only the virtual directories such as catnw/ (basically all the pages that don't exist to the eye of visitors and that redirect to other pages that are linked from the homepage) should be listed in the robots.txt file, because you want to have the content of van/van-insurance-uk.asp and so on to be indexed in Google but not catnw/

What I don't get though is why you created catnw/ ?
Is it just for a simpler and shorter URL to remember?

[thebloke]

Originally Posted by effisk

What I don't get though is why you created catnw/ ?
Is it just for a simpler and shorter URL to remember?

Marketing wanted me to do it so they can put it on off-line ads. Theres a whole host of them so they can track the effectiveness of each ad or leaflet. And yes, the reason it is just something short rather than having something like ?campaign=catnw is so it is something short and simple for the customer to type in.

So then. Do you reckon what I've done will be ok? I think I'll leave the virtual directories in the robots.txt file, just in case. Can't see that will do any harm?

[effisk]

Originally Posted by thebloke

Do you reckon what I've done will be ok? I think I'll leave the virtual directories in the robots.txt file, just in case.

Yes. I first thought there were also "real" directories in there (didn't realize it was scooters/ and not scooter/ etc.)

[thebloke]

Originally Posted by effisk

Yes. I first thought there were also "real" directories in there (didn't realize it was scooters/ and not scooter/ etc.)

Ah, yes! scooter/ is the real directory whereas the virtual directory is scooters/ (and taxi/taxis, motorcycle/motorcycles etc).

I think I'll keep an eye on things and if anything starts taking a nosedive then I'll review it then.

Thanks for your help guys.


Peter

[thebloke]

Call me paranoid but since I posted the link above (the one with /catnw at the end) my van insurance listing has gone into free-fall. I need to edit the post to remove this but can't seem to do it.

It may just be coincidence but I'd rather take the link off, just to be on the safe side. Any idea how I can do this? Moderators?!

Cheers

Peter

[effisk]

Originally Posted by thebloke

Call me paranoid but since I posted the link above (the one with /catnw at the end) my van insurance listing has gone into free-fall.

paranoid ;)

Your page is nbr 11 for "cheap van insurance" on google.com
That's not too bad given the competition.

I think Google doesn't check the robots.txt file every time it visits your site. So it might have seen the catnw/ page although it's not indexed. Wait a few more days...

Originally Posted by thebloke

Any idea how I can do this?

never realized we can't edit posts on this forum. bizarre...

[effisk]

one more thing ...maybe Google crawled your site just before you updated your robots.txt file... (ok it's improbable but still possible)

edit: looks like you can edit messages you've recently posted and not older ones.

[thebloke]

Originally Posted by effisk

Your page is nbr 11 for "cheap van insurance" on google.com
That's not too bad given the competition.

We're now bottom of page 5 for "van insurance". We were heading upwards on page 4 but we seem to have done a U-turn.....

[effisk]

Originally Posted by thebloke

We're now bottom of page 5 for "van insurance". We were heading upwards on page 4 but we seem to have done a U-turn.....

I see you there (middle of page): -http://www.google.com/search?q=van+insurance&start=50

If your site has only moved down 2 or 3 positions, it's not you, it's probably one of your competitors who's working harder than you :)

[thebloke]

We're now half way down page 6. Now I'm worried. We have indeed gone into free fall.

[kgun]

Have you seen this GUI before?

http://www.wisconsinbusinessdirectory.com/?

My Spyware Doctor OnGuard blocks the site at my computer, so load it at your own responsibility.

A lot of sites on the net redirects or pop up this

http://www.buydomains.com

site.

Some useful resources here:

http://www.garykeith.com/browsers/downloads.asp

If a Bot visits a folder blocked in robots.txt, it is

1. A Bad Bot or
2. A spiderbot that "any" SE company may have.

A better place to block the Bot, may be in .htacess
http://www.aaronlogan.com/downloads/htaccess.php

Some other useful links:

http://www.modrewrite.com/
http://forum.modrewrite.com/

Kjell Gunnar Bleivik
http://www.multifinanceit.com/
http://www.blognorway.com/

[greeneagle]

GOOGLE especially doesn't like 302s. They index 301s much better.

They generally consider 302s as spammy.

You would most likely see a big difference in indexing with a system of 301s.

Ken

[cspelts]

Peter, I am far from being an expert at what you are describing, but I think it's far more likely that the 302 redirects are causing your freefall - and not the link in this thread.

[thebloke]

Originally Posted by cspelts

Peter, I am far from being an expert at what you are describing, but I think it's far more likely that the 302 redirects are causing your freefall - and not the link in this thread.

Thanks guys, I've changed the redirect code so it should now return a 301 response.

I'll keep my fingers crossed and hope this arrests the slide.

I'm sure my paranoia will be eased pretty soon!

[effisk]

So how are things going now?

[thebloke]

Hi! I'd forgotten about this thread!

We seemed to level out and, after my initial paranoid passed, the ship appears to have steadied!

Our newly designed site (www.choicequote.co.uk) went live on 16th August so I am now patiently waiting for us to rocket to the top of the listings! :-)

I guess this is going to take a while to happen so I'll just sit and twiddle my thumbs in the meantime!

[mike]

That's a nice looking site by the way...

[effisk]

Very nice site indeed.
Sleek presentation, and well optimized for search engines.

Some minor improvements are possible though.

It's good that all the JavaScript is at the bottom of the page, however it would be better I you put it in a external .js file, or at least the last script that's very long.

Also maybe you can find something a bit more adapted to a homepage for the description metatag; you already have pages optimized for the words you use in the description, try to describe the activity of your company instead.

just my two cents ;)

[thebloke]

Thanks guys!

I have played around with the javascript and moved some of it to external files. I'll see if I can get it all out but some of it's a bit messy (mostly affiliate tracking, Google adwords, overture etc).

As for the homepage description...I thought I'd done that already! Well spotted. I'll get on the case!

Thanks again.

:-)