How critical is a sitemap to google?

[bruceldr]

I am having a heck of a time getting my .xml sitemap to WORK for Google. It was fine, now it's dead. All my pages are available through crawling but I have huge blocks of pages and information that never show up in search results. Does this mean that Google hasn't indexed them?

I have two sites with sitemaps:

Information About Camping In Arizona (http://www.PitchYourTent.com) (this one has a "valid" XML sitemap in the main directory under sitemap.xml)
Welcome to EatStayPlay.com - America's Recreation Connection (http://www.EatStayPlay.com)

I want to get the smaller site, PitchYourTent.com working FIRST and then do the bigger site. But, I can't seem to make the xml format work. I've been all through Google's help but it just doesn't actually help.

What am I doing wrong?
Should I be worried about sitemaps or is there a better SEO technique for me to focus on?

Thanks!

[Webnauts]

If you watch these videos you will understand that Google will not necessarily will crawl and / or will index all your pages, even if you have an XML sitemap.

1. YouTube - Which is better: an HTML site map or XML Sitemap? (http://www.youtube.com/watch?v=hi5DGOu1uA0)

2. YouTube - How many pages can Google index from a single site? (http://www.youtube.com/watch?v=gW7AUnZvSAE)

3. YouTube - Does the size of a website affect its authority in Google? (http://www.youtube.com/watch?v=Mha9q2aAfdM)

Good luck. :)

[danlefree]

I'm a big fan of sitemaps - both XML and HTML (though I would recommend organizing your HTML sitemaps such that people can use them, first and foremost).

Webnauts is correct RE: Google assigning lower priority to some pages (and possibly leaving them out of search results, though a site: operator search may turn up some surprises if you've been verifying with keyword searches in the past).



I can't seem to make the xml format work. I've been all through Google's help but it just doesn't actually help.

What am I doing wrong?

Depending upon your hosting platform and site setup, I may be able to provide some code to churn out sitemaps automatically.

Here's what's wrong with your present setup:


<?xmlversion="1.0"encoding="UTF-8"?>
<urlsetxmlns="http://www.sitemaps.org/schemas/sitemap/0.9">

... should be:


<?xml version="1.0" encoding="UTF-8" ?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">

(just needed a little adjustment on spacing)

[bruceldr]

Thanks. I'm resubmitting it to Google now and will see what happens.

Thanks a TON for those video links. I never would have figured that an HTML sitemap would be "better" than an xml. The only issue is that the main site has 35,000+ pages so it's not like it would really help PEOPLE at all.

If I created an HTML sitemap for it, where should I link to it from? It wouldn't be a help to users without a TON of work and even then, I'm not sure.

Just a thought!

Thanks again.

[mjtaylor]

Link from the home page, for sure, for the spider and it can also be linked from all pages to for the convenience of the visitor.

It's not important, IMO, for ranking if your site is can be spidered effectively without it.

[Webnauts]

The only issue is that the main site has 35,000+ pages so it's not like it would really help PEOPLE at all.
About that issue I posted a while ago an solution: Bot Herding: The Ultimate Tool for PageRank Sculpting | Search Engine Journal (http://www.searchenginejournal.com/bot-herding-pagerank-sculpting/10352/)

[Webnauts]

Link from the home page, for sure, for the spider and it can also be linked from all pages to for the convenience of the visitor.
I think I could add a video here for a better understanding: YouTube - What's a preferred site structure? (http://www.youtube.com/watch?v=Hx68PlCTuIU)

[bruceldr]

Okay, I tried to fix the xml sitemap for Information About Camping In Arizona (http://www.pitchyourtent.com) and google is STILL saying they can't read it. What am I missing?

[Webnauts]

Remove this </urlset> at line 8.

Don't rely on validators like validome.org, etc. They are not always accurate.

[bruceldr]

Thanks. I just couldn't FIND the error. I really appreciate you looking at it for me!

[bruceldr]

So, HTML sitemaps are good... Is there a preference if I was choosing between text & xml? What does yahoo and MSN/Bing prefer?

[Webnauts]

So, HTML sitemaps are good... Is there a preference if I was choosing between text & xml? What does yahoo and MSN/Bing prefer?
They all prefer XML Sitemaps.

Yahoo also accepts text files, simply making a list of the urls one beneath the other, and you should call the file urllist.txt

Clear enough?

[ciidoctor]

from my experience with sitemaps , there is no difference whether you submit sitemap or not one of my blogs indexed regularly after minutes of posting without sitemap i also read a strange article that says going without sitemaps is better for indexing your site.

[ChiefLee]

It's not important, IMO, for ranking if your site is can be spidered effectively without it.

This last summer I realized that our google sitemap XML file wasn't working. And probably hadn't been in months. (Long story.) Got it back up and running. Spent some time getting it just the way we wanted it. And I don't think it mattered a bit. Still getting crawled at same rate. And nothing noticeable in the SERPs. So if your site is structured well (which ours is) I wouldn't be all that concerned about it.

[JasonJersey]

In my opinion, sitemap.xml usage is very important between my sites relationship with Google.

[twt]

I found exactly the same optical burst.

Back to the original question. I too was having a similar problem then I found a free website map generator at auditmypc (http://www.auditmypc.com/free-sitemap-generator.asp) - helped me just nicely.

It did not restrict the number of urls and gave me some other cool features & it was free.

I then uploaded it and resubmitted via Google webmaster tools. Solved my problems.

Oh I should mention it gave some great info that I thought was well worth reading.