adwords robot.txt

[markhallgate]

hello,
i have recently signed up for adwords. I was told that my robots.txt should be changed to:

User-agent: Mediapartners-Google*
Disallow:

Some of the spiders semm to be stoping after reading the txt file, is this because my file is now blocking them?
Any advice appreciated
The site is www.shedsonline.co.uk
thanks, Mark

[David16]

I also have just signed up but I left my robots.txt as
FULL access (All Spiders)

User-agent: *

Disallow:

As it allows all spiders it should also allow google

[David16]

I also have just signed up but I left my robots.txt as
FULL access (All Spiders)

User-agent: *

Disallow:

As it allows all spiders it should also allow googleMediapartners I would have thought but I'm no expert. The particular site that its on is showing adds fine.

[gworld]

Originally Posted by David16

I also have just signed up but I left my robots.txt as
FULL access (All Spiders)

User-agent: *

Disallow:

As it allows all spiders it should also allow googleMediapartners I would have thought but I'm no expert. The particular site that its on is showing adds fine.

You are right, I have adwords and I haven't changed my robots.txt and it works fine.

[wrmineo]

I honestly forget where/who this came from, but when I was not getting indexed by Google at all, I was advised to use the following.

User-agent: *
Disallow: /azr94v2hh2lg/

I won't pretend to be able to briefly or intelligently explain the rationale, but basically during "flux" periods when Google changes its formulas, it may fool itself into being disallowed even though the string allowing all, is present ... I see Googlebot/2.1, Googlebot-Image/1.0, and Mediapartners-Google/2.1 in my web stats at regular intervals now.

Good luck.

[Bridge]

I use this:

User-agent: Mediapartners-Google*
User-agent: *
Disallow: somethingyouwantleftunindexed.php
Disallow: /foldernotforindexing
Disallow: etc etc

I did it this way as I have a script installed that catches spiders NOT obeying the robots.txt disallows. However MSN was getting trapped,(without the inclusion of User-agent: *) after the google reference.

The script bans the nasty spiders from the site adding a line into the .htacces