Quote:
I personally do not need to ask Matt/Google official stance on the noindex robots.txt directive. By the way nofollow robots.txt does not exist.
I do not need to ask anybody since I already implemented on all my sites and on all sites of my customers and works as it should. I hope that is fair enough, or?
|
I meant noindex robots.txt directive. It just seems odd that Google or Matt C never mention it? Why would that be the case if it was such a useful tool that would benefit everyone including Google?
Quote:
|
It really does the same job as the noindex meta robots tag on a server level.
|
Can i just clarify... If i block a page in robots.txt with the NOINDEX directive will that stop the page being crawled, indexed and also stop it building or leaking pagerank?