iEntry 10th Anniversary Forum Rules Search
WebProWorld
Register FAQ Calendar Mark Forums Read
Search Engine Optimization Forum SEO is much easier with help from peers and experts! The WebProWorld SEO forum is for the discussion and exploration of various search engine optimization topics. Any non (engine) specific SEO or SEM topics should go here.

Share Thread: & Tags

Share Thread:

Reply
 
LinkBack Thread Tools Display Modes
  #1 (permalink)  
Old 10-20-2005, 04:44 PM
WebProWorld New Member
 
Join Date: Oct 2005
Location: Argentina
Posts: 5
cela_carlos RepRank 0
Default How to get the bots to index my whole site?

Two weeks ago, I did post my site to dmoz (http://www.basedecoracion.com.ar/), and today it's listed when I search for it in Google. However, the only page of the site that gets listed is "index.htm". My question is: how do I do to get the bots to index all the inner pages (none of them are hidden)? Sorry if this question is very fool.
Thx!
Reply With Quote
  #2 (permalink)  
Old 10-21-2005, 07:00 AM
WebProWorld Veteran
 
Join Date: Jul 2005
Location: London - Cheswick
Posts: 669
JuniorOnline RepRank 0
Default

i am interested to know the response to your question. as far as i am aware, the spiders index pages every month - do you have links on your index page? posting your site url would be helpful - i could have a look at the coding and see whats missing.

it does take time though for spiders to find anew website..and you need to build your link exchange list to enhance your listing. This is how google tend to rate your site - they look at other people linking to you - each link is like a vote that your site is worth visiting and would be helpful for people.
Reply With Quote
  #3 (permalink)  
Old 10-21-2005, 08:45 AM
WebProWorld New Member
 
Join Date: Oct 2005
Location: Lancashire, UK
Posts: 17
ste_dean RepRank 0
Default

Hi,

I have taken a look at the pages indexed within google and you are correct it has only indexed your home page. This is probably becuase it has taken the link directly from DMOZ and it hasnt yet searched your site with its spider.

Have you submitted your site directly to Google? If you have it may take a long time before it comes up in its rankings. You can get Google to spider your site by posting a sitemap:

https://www.google.com/webmasters/sitemaps/login

If you want anymore advice on SEO or getting your site indexed you can email me at seo@sda-solutions.co.uk
__________________
Stephen Dean
SEO Solutions
http://www.seo-firm.co.uk
Reply With Quote
  #4 (permalink)  
Old 10-21-2005, 06:14 PM
WebProWorld New Member
 
Join Date: Oct 2005
Location: Argentina
Posts: 5
cela_carlos RepRank 0
Default Re: How to get the bots to index my whole site?

Quote:
Originally Posted by cela_carlos
Two weeks ago, I did post my site to dmoz (http://www.basedecoracion.com.ar/), and today it's listed when I search for it in Google. However, the only page of the site that gets listed is "index.htm". My question is: how do I do to get the bots to index all the inner pages (none of them are hidden)? Sorry if this question is very fool.
Thx!
Thank you very much, both of you. That's useful information.
ste_dean: I'll be in touch with you.
Reply With Quote
  #5 (permalink)  
Old 10-21-2005, 06:42 PM
WebProWorld New Member
 
Join Date: Oct 2005
Posts: 2
csorensen RepRank 0
Default My problem was no robots.txt file

When I first created my website (it was a Yahoo store), I couldn't get Yahoo and MSN to index my pages. A couple months went by and I had a site map and Google had hundreds of my pages indexed and with PR. I finally figured out that Yahoo Store does not create even an empty robots.txt file. I guess they figure everybody is a tech guy and knows that they have to do this. As soon as I created the robots.txt file, Yahoo and MSN indexed everything I had within a few weeks.

Also, if you don't have a site map, be sure to create one and link to it from your home page.
Reply With Quote
  #6 (permalink)  
Old 10-21-2005, 10:25 PM
WebProWorld New Member
 
Join Date: Oct 2005
Location: Argentina
Posts: 5
cela_carlos RepRank 0
Default Re: My problem was no robots.txt file

Quote:
Originally Posted by csorensen
Also, if you don't have a site map, be sure to create one and link to it from your home page.
Thank you! I'll take that piece of advice!
Reply With Quote
  #7 (permalink)  
Old 10-22-2005, 11:39 AM
WebProWorld Veteran
 
Join Date: Oct 2005
Posts: 528
aaron2005 RepRank 0
Default

I notice that google finds a new site if you link it from an exisiting site or directory.

It then lists just the index of the site for a few weeks or longer but visits it a few times during this time to look for changes.

Then it comes back on the same day and goes thru the entire site, it appears to have a time stamp on new sites and a predictable pattern if some geek wants to write it down.

Just be patient, link the site from another site and distract your mind by taking out the garbage or throwing a ball to your dog ;)
__________________
SEO Blog
Reply With Quote
Reply

  WebProWorld > Search Engines > Search Engine Optimization Forum

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On



All times are GMT -4. The time now is 07:22 PM.



Search Engine Optimization by vBSEO 3.3.0