View Full Version : Block Conversion page in robots.txt ?
jimmy_floyd
04-20-2011, 05:22 AM
Hello all !
Been trying to find an answer to that question, but so far nothing.
I am wondering if I should block my conversion / goal page in robots.txt to avoid those pages being indexed in search engines. What I am scared is, whether this block will prevent google analytics and adwords from giving me accurate reports.
Thank you for your kind answers !
morestar
04-20-2011, 11:08 AM
Hey Jimmy, Welcome to WebProWorld.com - good to see ya!
Good question too. I'm not too sure how some of the other members would deal with this kind of situation. I personally have never thought of blocking a conversion page from robots.txt. If you a page in robots.txt, you understand that only robots/spiders will not crawl/use the page for their purposes but regular human users can still view the page if they find it.
What's your reason for blocking the conversion page?
Also, if you block a page in robots.txt Google will still provide you the traffic/etc. analytics for those pages if you have your Analytics code in them.
Stay tuned as a few more responses to your question should come through shortly.
;)
jordanmcclements
04-20-2011, 02:08 PM
You dont need to block conversion pages (at least I cant think of a scenario where you would need to...)
Though as morestar points out, if you did, it would not affect analytics or AdWords conversion tracking.
I'd give a longer explanation, but this phone is getting on my nerves now :-)
jimmy_floyd
04-21-2011, 12:23 AM
What's your reason for blocking the conversion page?
Well the main reason is to avoid the page from being indexed in search engines. Even though there are little chances that they will be.
So did any of you actually tried it ?
dburdon
04-21-2011, 03:44 AM
Just robots no follow in the head. Despite this you will still get the odd false conversion.
jordanmcclements
04-21-2011, 04:24 AM
Well the main reason is to avoid the page from being indexed in search engines. Even though there are little chances that they will be.
So did any of you actually tried it ?
I have never actually tried it, to be honest.
But with regards to AdWords conversions - it uses cookies - so there is no chance of an AdWords conversion being registered unless someone actually clicks on one of your ads and then visits your conversion page within 30 days (as far as I know) (I always like to say this these days after it was pointed out that I don't know everything :-).
MrGamm
04-24-2011, 02:37 AM
It is an interesting question, have you considered that some anti-malware and anti-virus software will sometimes crawl the pages you just fetched? I'm not sure if there are many that do that, but I have seen crawlers start crawling pages that weren't accessible to anyone without privileged information as to where the web pages addresses were.
Have you considered making the page which is measuring a conversion accessible only through POST, instead of GET?
If not... blocking something with robots.txt by definition should stop automated crawlers who obey it, it won't block a human from visiting the page and triggering third party analytics scripts from recording information, and I can't see why it would effect advertisements.