View Single Post
  #1 (permalink)  
Old 06-13-2006, 04:26 AM
AjiNIMC's Avatar
AjiNIMC AjiNIMC is offline
WebProWorld Pro
 
Join Date: Aug 2004
Location: India
Posts: 279
AjiNIMC RepRank 0
Default Is the robots.txt header imp? Html or plain text?

The pages under domain X is not getting cached and the header for its robots.txt says,

Code:
HTTP/1.x 200 OK
Date: Tue, 13 Jun 2006 07:06:47 GMT
Server: Apache/1.3.36 (Unix) mod_fastcgi/2.4.2 mod_auth_passthrough/1.8 mod_log_bytes/1.2 mod_bwlimited/1.4 PHP/4.4.2 FrontPage/5.0.2.2635.SR1.2 mod_ssl/2.8.27 OpenSSL/0.9.7a
X-Powered-By: PHP/4.4.2
Content-Type: text/html
Connection: close
The pages under domain Y is getting properly cached and the header for its robots.txt says,
Code:
HTTP/1.x 200 OK
Date: Tue, 13 Jun 2006 07:07:19 GMT
Server: Apache/1.3.36 (Unix) mod_fastcgi/2.4.2 mod_auth_passthrough/1.8 mod_log_bytes/1.2 mod_bwlimited/1.4 PHP/4.4.2 FrontPage/5.0.2.2635.SR1.2 mod_ssl/2.8.27 OpenSSL/0.9.7a
X-Powered-By: PHP/4.4.2
Content-Type: text/xml
Connection: close
The difference that I can see if
Content-Type: text/html and Content-Type: text/xml

Can this be a possible reason for no cache with Google? What are the other possibilities for the pages not getting cached (rather loosing the caches from Google)? I am assuming no penalty and no filters from Google as these sites were doing good and are very helpful ones with no SEO done.

Thanks,
AjiNIMC
__________________
Be web educated - Client education series - Let not your SEO company cheat you. When they screw your website, they lose one client and you everything.
Reply With Quote