View Single Post
  #5 (permalink)  
Old 01-08-2008, 02:45 PM
dmeakings dmeakings is offline
WebProWorld New Member
 
Join Date: Jan 2008
Location: Burlington, Ontario
Posts: 5
dmeakings RepRank 0
Default Re: Google Sitemap Statistics: Total URLs Indexed

Quote:
Originally Posted by wige View Post
No. The absence of a robots.txt file should not cause you any problems.

If there is nothing blocking the spiders from crawling to the pages in question, and they are linked from pages that the spiders already crawl, most likely the factor is simply time. It will take a while (again, this seems to depend in large part on the number of inbound links you have and the PR of those linking pages) for the spiders to completely crawl your site. It could be up to a month, although depending on factors ranging from your subject (rarity of the topic) to size of your site, etc. it could be anywhere from a week to a month for Google to do a full crawl.

EDIT: You may also want to check for canonicalization issues - make sure everything is forced to one subdomain (www, or not, but not both).


I have check the canonicalization issues and they have been resolved in my .htaccess file. With respect to time the site has been indexed in google for over 6 months now with many pages missing. There was a time when the title was the same on every page but that has been resolved as well.

I'll have to give it another two weeks I suppose.
Reply With Quote