Sitemaps -- Google does it again!

[ReviewGolf.com]

Ok, I admit it, I've had some issues with Google in the past, and honestly not quite settled on all account yet, but I must give credit to their new brainchild: Google Sitemaps (https://www.google.com/webmasters/sitemaps/login).

The sitemap (single file) must be formatted in a particular, XML friendly way for Googlebot to pick it up, but once you manage to get it right, it works great. I noticed my results just days after I submitted my map. Now, if you like tinkering with raw code, do it manually. There are a few scripts out there that will help you in the process. I personally opted out for a $15 package that turns the hours of labor in a matter of a few minutes.

So if you have not done so, give sitemaps a try, chances are you won't regret it!

[MtraX]

Has anyone posted sitemaps after Jagger 3 and noticed dramatic results? I'm not sure whether this post is related to that...

MtraX

[biju]

well i have heared about this in other forum and i donot how far it is working

[pagetta]

do you have to continually reupdate and adjust and re-submit google sitemaps? how often should you do this? what is the best way to update them?

[MtraX]

As far as I know you only submit it once, but re-upload to your own server after updates. Many developers writes scripts to automatically update the sitemap upon anything changing.

[ReviewGolf.com]

You can resubmit the map when you change/add pages. I set mine to run monthly through a CRON job. I put the first map last week, I see my pages on Google spidered this week.

[Faglork]

Originally Posted by pagetta

do you have to continually reupdate and adjust and re-submit google sitemaps? how often should you do this? what is the best way to update them?

Best way is to generate sitemaps with a script and use a cron job to re-generate on a regular base - so it's just "set it and forget it". Googe offers a free sitemap-generator written in python (reqires python 2.2 or higher).

After re-generation, you have to inform Google about the change with an http request:
http://www.google.com/webmasters/sit...ap=sitemap_url
Again, you can automate this with a cron job.

Extensive information about all this can be found at
https://www.google.com/webmasters/si.../en/about.html


faglork

[joer80]

i posted 400 pages durning jagger 1 and havnt seen them spidered yet...

[Map Books 4 U]

Originally Posted by pagetta

do you have to continually reupdate and adjust and re-submit google sitemaps? how often should you do this? what is the best way to update them?

It is my understanding that you do have to re-submit, (can' t remember if it is monthly or not), and you have to keep it up to date.

If you are changing things in your site or store, make sure you update it so that it is not removed.

[wmrobwl]

So, let me get this straight. Is Sitemaps designed to get your site/pages into their index or to rank better if you are already in their index.

Would anyone suggest this, if googlebot visits your site every day?

[computergenius]

Originally Posted by pagetta

do you have to continually reupdate and adjust

You should should update and upload every time that you change a page

Originally Posted by pagetta

and re-submit google sitemaps? how often should you do this?

You don't need to re-submit. Google collects mine regularly - the times show 6 hours ago, 3 hours ago, 8,8,8 and 5 hours ago, on the various sitemaps that I have up there.

Originally Posted by pagetta

what is the best way to update them?

That would depend on how you created them.

As some of the sites that I handle are 10k pages, I am slowly amending the software that I write to change the pages, so that it amends the sitemap at the same time. This is useful, because I can also amend feeds at the same time, without a total re-write of the feed.

[MtraX]

I've just created a sitemap (as a 'couldn't hurt' resolution) for the one site that died in Jagger. Will report if anything changes, which I doubt...but you never know.

MtraX

[ReviewGolf.com]

I just generated site map for another site. Submitted to Google and see that it was picked up by it an hour later... Probably too early yot see any results still :)

[MtraX]

I definately got an active crawl through the site and the cache seems to be updated, but still no real change in SERPs.

MtraX

[brian.mark]

The only thing I've noticed with using Sitemaps is that when I make changes I want to test, I can generate a new file and ping them and they spider almost on call (ok, so there is a day or so delay at times). But that sure beats the old "wait for them to come, then hope they see the updated page" approach.

It's not designed to help rankings, just to help in indexing.

Brian.

[Faglork]

Hi,

does anybody know of a google sitemap generator using http protocol? Googles own python-based generator travels the file system or uses the access logs or an existing list.

- file system: I have a lot of files sitting there, so it would be much fuss to filter them
- access logs: this is what I am currently using, but it is a hog on resources and still a lot to filter
- existing list: too much work

A spider-like http approach would be best. There are online Google Sitemap Creators which do that, but I am looking for a perl/php/python solution.

Any suggestions?

faglork

[seolion]

I still feel it's a mixed response for sitemaps.

[Graf1771]

I made a sitemap last night for one of my sites, uploaded it, got it checked by Google and am now waiting for a response.

I loaded about 3,000 pages into it using a little bit of php code added to the content management system. To do it manually on a site any bigger than 50 pages would take a long time, although you may get better results with a more tuned approach.

The actual sitemap file is a very simple .xml file, a basic template if shown on Google for you to copy. As far as I know, having made one of these for a large grocery company, you just drop it in to the root directory of your site and leave it (unless you need to keep manually adding new pages).

Google keeps track of the sitemap and will stop by to read it when it crawls your site. It's main use is to give the spider a quicklink access to every page on your site, which could very well help with ranking / results - at least it's not going to hurt!

[greeneagle]

faglork,

This one seems to have the most "bells and whistles" that I have seen:

http://johannesmueller.com/gs/

Have you looked at it yet?

Ken

[ReviewGolf.com]

I bought this one and highly recommend:

http://www.xml-sitemaps.com/

I set it via CRON job and now do not have to worry about it at all.

[send2paul]

xml-sitemaps - nice one :)

I used the freebie online version and loaded the sitemap with Google, (and registered with them as well).

What is in the paid for version that isn't available from the initial set-up of the site map with the freebie online version? I mean once the sitemap is created and loaded up onto your server - isn't the rest then up to Google once you've registered with them?

[incrediblehelp]

Originally Posted by greeneagle

faglork,

This one seems to have the most "bells and whistles" that I have seen:

http://johannesmueller.com/gs/

Have you looked at it yet?

Ken

I use this one when I "need" to, pretty easy. If you have a PHP based website their are other better options than this software.

Originally Posted by wmrobwl

So, let me get this straight. Is Sitemaps designed to get your site/pages into their index or to rank better if you are already in their index.

Would anyone suggest this, if googlebot visits your site every day?

No I don't find it necessary to use sitemaps, just to use it. If your website is being fully crawled already it is not useful.

Are we forgetting why this was released? This tool is there to help Google crawl dynamically based websites that choose or cant implement URL rewriting and possibly websites that are brand new to the web. It is not there help get get "better" ranking results in Google. It is there to help you get crawled.

If they are not crawling you then you may not be inline with their sitemap or content guidelines.

[Faglork]

Originally Posted by incrediblehelp

Are we forgetting why this was released? This tool is there to help Google crawl dynamically based websites that choose or cant implement URL rewriting and possibly websites that are brand new to the web. It is not there help get get "better" ranking results in Google. It is there to help you get crawled.

Just to add: I have sites where some of the pages do not get spidered, only heaven (a.k.a. Google) knows why. Out of 70 pages, the GOOG choses to index just about 40, with no apparent difference between the pages. In another case it is about 700 pages/400 spidered. So I decided to try the Google sitemap route. I keep you informed how well it works.

@Ken: Thanks for the link, it is very nice, but I am looking for a server-based solution, because I want to run it from a cron job.

@ReviewGolf.com: Thanks, this is what I am looking for ...

@send2paul: The paid version is a standalone program which can run on the server, activated by a cron job. So you do not have to manually create a new sitemap and then ping google, the program does that on a regular time frame.


Thanks for all your input,
faglork

[incrediblehelp]

New sitemap info from Matt Cutts:

http://www.mattcutts.com/blog/pubcon-notes-part-1/

Now you dont need a full site map to enjoy some feedback stats from Googlebot. Trying it now, pretty cool and very easy.

You upload a empty HTML file to verify it is your website (no log access yeah!) and they give you crawl feedback.

[ReviewGolf.com]

send2paul,

Not sure what it had as a free trial, but the paid version works beautifully. It also allows to save sitemap in .gz format, quite handy if you have a lot of pages. Also sitemap in txt format compatible with Yahoo! sitemap version.

It's worth every penny!

[greeneagle]

What about the new "Webmaster Panel" announced?
Ken

[Faglork]

Originally Posted by greeneagle

What about the new "Webmaster Panel" announced?
Ken

Goes along with the sitemap panel. Not bad, as it gives you a general idea how well your site is doing especially with googlebot. Could be a bit more, though. But for a free service, it's ok. Just try it, it is easy.
https://www.google.com/webmasters/sitemaps/

faglork

[incrediblehelp]

Yes that is what I am referring to ken. Here is a write up on it:

http://www.site-reference.com/articl...s-For-SEO.html

[ReviewGolf.com]

Some links on the panel did not work for me at all. Not broken, just not hooked up it seems. I'll try tomorrow again.

[MtraX]

I posted about this in another sitemap thread. They should have at least given us top 10 keywords, but at least we're getting something... I already picked up one error page which I didn't previously know about, so kudos to Goog for the upgrade ;-)

MtraX

[ReviewGolf.com]

Say what you want, but I seem my site being spidered very often and the results are significant. Now, I still see some sort of discrepancies. For example I have a #1 position on Google, Google UK and Google China, but #2 in Germany. Why is that?

Also, I see the date stamp next to my link, which I did not see before and do not see next to other sites. Is that a sign that Googlebot is using sitemap to navigate?

[aaron2005]

Someone may have mentioned this already (very busy no time to read this whole thread) BUT isn't the title of this thread a bit humerous? "Sitemaps -- Google does it again"! They sure did! LOL! ;-)

Ok, I admit to not liking Google this week, is this ok?

[ReviewGolf.com]

aaron2005,

You got it. The title does have some sarcasm in it. You can check my posts of a year or so ago. I'm one of the Google's anti-fans. I had had it with Google when they lost one of my sites from #1 position to oblivion and the following weeks of getting a straight answer from Google just confirmed my suspitions of their support's total incompetence. The explanation could've been something as mondane as some disgruntled employee pulled a plug on his way out in one of the third-world-countries, where Google keeps a lot of their servers, while paying Nike's type wages. Obviously nobody from withing the corporation would allow such a bad PR. It's so much more "professional" to keep sending irrelevant canned messages to you until you get tired and just leave them alone.

However, I gotta give them credit when the credit is due. Sitemaps, or some other coinsidence got us right up where we were some time ago. I think Sitemaps helps me with having a better control of where I want the googlebot to wonder, in a much better, precise manner.

[aaron2005]

Originally Posted by ReviewGolf.com

aaron2005,

You got it. The title does have some sarcasm in it. You can check my posts of a year or so ago. I'm one of the Google's anti-fans. I had had it with Google when they lost one of my sites from #1 position to oblivion and the following weeks of getting a straight answer from Google just confirmed my suspitions of their support's total incompetence. The explanation could've been something as mondane as some disgruntled employee pulled a plug on his way out in one of the third-world-countries, where Google keeps a lot of their servers, while paying Nike's type wages. Obviously nobody from withing the corporation would allow such a bad PR. It's so much more "professional" to keep sending irrelevant canned messages to you until you get tired and just leave them alone.

However, I gotta give them credit when the credit is due. Sitemaps, or some other coinsidence got us right up where we were some time ago. I think Sitemaps helps me with having a better control of where I want the googlebot to wonder, in a much better, precise manner.

Here is something new to fire you up again then! ;0

http://www.seobuzzbox.com/google-usi...tics-data.html

[ReviewGolf.com]

Interesting stuff, thanks.

[ReviewGolf.com]

OK, as I promissed here's an update on the effects of sitemap. I took a site with very low visibility and created a sitemap. That was a week or so ago. Today I noticed a spike in referrals from Google. Coinsidence? Heck knows, but it almost had no traffic from Google before today. I'll keep an eye on it. I think I'm a believer in this tool!