Multiple Sitemap Generators Missing 80% of pages. Why?

[Pyeman73]

Hello all,

I have been using multiple sitemap generators (I have tried 6 in total) on a site I have been asked to SEO, and they all pick up only 9 pages out of 50. And the pages they pick up seem to all be media files (.wma)

I could code this sitemap by hand, but I would rather figure out what is inhibiting these generators. They have worked very well on many other sites I have contributed to.

Does anyone have any input? Am I missing something obvious, like maybe a tag that is interfereing? Much appreciated.

[incrediblehelp]

Maybe post an example website? Just an obvious questions...do you have link structure so the generators can crawl and find these other pages?

[Pyeman73]

The link structure appeared to be indexable. Upon closer examination, however, it appears that even though the links in a dynamic drop down menu appear static and non-javascript - only links off the main page are showing up in the sitemap.

It is totally the navigation. Thank you very much for your answer. I guess the take away lesson here is - if your sitemap generator isn't crawling your site properly due to fancy navigation - then neither are the indexing spiders.

[TrafficProducer]

Multiple sitemap generators

You will find, (if not already tried), this about the best. May be set to limit file types, individual links etc and is able to make most types of maps, xml, html and more....

Projects may be saved etc... and more, and it's free :)

GSiteCrawler will help you generate the best Google Sitemap file for your website. The GSiteCrawler uses different ways to find all the pages in your website and can generate all sorts of files, statistics and more. The sitemaps file format has lately been also adapted by Yahoo! - even MSN/Live.com is pledging it's support.

[Orion]

it isn't too difficult to create your fancy drop downs using pure CSS and xhtml keeping all the links crawlable in the code.

barring that a sitemap page with pure html links in it works too just have that linked from the footer or someplace on the front page of the site.

[texxs]

It is totally the navigation. Thank you very much for your answer. I guess the take away lesson here is - if your sitemap generator isn't crawling your site properly due to fancy navigation - then neither are the indexing spiders.

What a piece of obvious (although I had never thought of it) and great advice to be remembered by all!