Submit Your Article Forum Rules

Results 1 to 6 of 6

Thread: Multiple Sitemap Generators Missing 80% of pages. Why?

  1. #1
    Junior Member
    Join Date
    May 2004
    Posts
    7

    Multiple Sitemap Generators Missing 80% of pages. Why?

    Hello all,

    I have been using multiple sitemap generators (I have tried 6 in total) on a site I have been asked to SEO, and they all pick up only 9 pages out of 50. And the pages they pick up seem to all be media files (.wma)

    I could code this sitemap by hand, but I would rather figure out what is inhibiting these generators. They have worked very well on many other sites I have contributed to.

    Does anyone have any input? Am I missing something obvious, like maybe a tag that is interfereing? Much appreciated.

  2. #2
    WebProWorld MVP incrediblehelp's Avatar
    Join Date
    Jan 2004
    Posts
    7,567
    Maybe post an example website? Just an obvious questions...do you have link structure so the generators can crawl and find these other pages?

  3. #3
    Junior Member
    Join Date
    May 2004
    Posts
    7
    The link structure appeared to be indexable. Upon closer examination, however, it appears that even though the links in a dynamic drop down menu appear static and non-javascript - only links off the main page are showing up in the sitemap.

    It is totally the navigation. Thank you very much for your answer. I guess the take away lesson here is - if your sitemap generator isn't crawling your site properly due to fancy navigation - then neither are the indexing spiders.

  4. #4
    WebProWorld MVP TrafficProducer's Avatar
    Join Date
    Jul 2003
    Posts
    1,958

    multiple sitemap generators

    Multiple sitemap generators

    You will find, (if not already tried), this about the best. May be set to limit file types, individual links etc and is able to make most types of maps, xml, html and more....

    Projects may be saved etc... and more, and it's free :)

    GSiteCrawler will help you generate the best Google Sitemap file for your website. The GSiteCrawler uses different ways to find all the pages in your website and can generate all sorts of files, statistics and more. The sitemaps file format has lately been also adapted by Yahoo! - even MSN/Live.com is pledging it's support.

  5. #5
    WebProWorld MVP Orion's Avatar
    Join Date
    Sep 2003
    Posts
    716
    it isn't too difficult to create your fancy drop downs using pure CSS and xhtml keeping all the links crawlable in the code.

    barring that a sitemap page with pure html links in it works too just have that linked from the footer or someplace on the front page of the site.
    Ron Boyd
    website consulting - design • optimization • marketing • [url=http://owhosting.com]Hosting[url] :: Follow Me: @boydrw

  6. #6
    Senior Member texxs's Avatar
    Join Date
    Jul 2005
    Posts
    528

    Great advice!

    It is totally the navigation. Thank you very much for your answer. I guess the take away lesson here is - if your sitemap generator isn't crawling your site properly due to fancy navigation - then neither are the indexing spiders.


    What a piece of obvious (although I had never thought of it) and great advice to be remembered by all!

Similar Threads

  1. Replies: 3
    Last Post: 09-02-2009, 04:37 AM
  2. Missing Pages & Supplemental Listings
    By kevsta in forum Google Discussion Forum
    Replies: 7
    Last Post: 07-27-2007, 01:38 PM
  3. Recommendation for Sitemap Generators
    By sgsmorgan in forum Search Engine Optimization Forum
    Replies: 14
    Last Post: 10-18-2006, 06:38 PM
  4. Missing pages
    By onelife in forum Google Discussion Forum
    Replies: 16
    Last Post: 07-28-2005, 11:42 PM
  5. Googlebot quit visiting. Pages missing!
    By Ed_S_1 in forum Google Discussion Forum
    Replies: 6
    Last Post: 03-10-2005, 04:24 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •