Importance of Site Map

[ksu]

Please, explain why Site Map is important?

[macdesign]

The theory says by having a site map at the second level of your site hierarchy, then all links to your site get indexed by search engine crawlers that limit the depth of searching.

[Andilinks]

If you use JavaScript navagation on your site the site map is a good place to provide non-JS links to everything. This also provides access to non-java browsers and covers other accessibility issues.

And as a bonus if you design it well all your visitors may find it helpful in understanding your site..

Andi

[cbp]

Google in their guidelines say to use a site map - therefore it must be important :-)

CBP

[fathom]

Originally Posted by macdesign

The theory says by having a site map at the second level of your site hierarchy, then all links to your site get indexed by search engine crawlers that limit the depth of searching.

That's a big point - "a page where everything is one click (link) away".

If however, you have a small website where every page can be accessed from the mainpage - a sitemap is redundant.

[Andilinks]

a sitemap is redundant.

Redundant navigation is good design. A good interface will have several ways to do the same thing so each user can pick the most comfortable option.

Far too many web sites (and applications and operating systems) offer a single nav option--"my way or the highway." This discourages users, a fine thing if you don't like users.

Andi

[Kenai Alaska]

Just thought I would throw my two cents in here.
I am in the process of rebuilding one of our sites and thus built the site map first as a tool for me to keep focused on a layout that has taken nearly three weeks to finish. It’s built to be easy to add on to as new information is requested. This site map will be very helpful to the visitors as a quick reference for information and trip planning. The site map is being designed as a tool for design layout and visitors reference. Every page will be listed within the category it represents and followed by a short one sentence page topic. We hope to complete the site by late fall and then add to it each year. The top folder we are currently working in is simply labeled “2004”. Inside will be all the information and photo indexes from our “Alaska flying adventures” for the 2004 year.
I guess what I am trying to say is the site map is a tool where the structure is easily portrayed to the visitors as well as the webmaster as time goes on. Every page of the site is only one click away and on a site that is starting to get rather large, this can be a blessing.
When we upload the site this fall, I will post it in the site design arena for all to review.

[Mel]

Don't forget that a properly constructed sitemap can provide relevant anchor text links to each of your pages in addition to all the other advantages.

[Rajeev Ranjan Singh]

Site map is a Atlas for the vistor as well as the crawler.

[mawells]

I have several sites that use JavaScript navigation and they all have a site map for the search engines to follow. However, it just came to my attention that some of my sites were getting indexed and others were not. So, I took a look at my <noscript> tags and realized that they were using Meta refresh tags in the <noscript> area. (This is put in by the software I use to create the navigation - AllWebMenus). In one case, I removed that meta information and replaced it with the contents of my site map. Yahoo was then able to grab all of the pages of the site, but Google has only added two more pages.

The other site, I replaced that meta refresh tag with a link to the site map page. Time will tell if Google indexes more pages. Currently they only have 4 pages indexed of this site.

The ones that seem to have been indexed okay, I used a transparent GIF, which I needed as a spacer, and linked it to the site map page.

For those who don't already know, you can see how many pages Google has indexed of your site by typing in "Site:mydomain.com" where "mydomain" is your domain. Note there is no space between the colon and your domain name.

[Jeremy21b]

A sitemap can be very useful for both visitors and search engine spiders. I ran into a problem with my sitemap though. I used keywords in the anchor text on the sitemap links. Now the sitemap shows up in search results instead of the optimized content pages. It's nice that we at least show up in results, but with the wrong page it doesn't attract the targeted market and it ranks lower than the other page would. Anyone know how I can fix this? Is the only way to get rid of the keywords on the sitemap or is there something else I should try? Perhaps I should optimize the page for just the word sitemap with some larger fonts and also decrease font size for the terms that I don't want the page to show up for. Any thoughts on this? thanks.

[Dave Hawley]

I used keywords in the anchor text on the sitemap links. Now the sitemap shows up in search results instead of the optimized content pages

The anchor text would boost the pages that the links point to, not the other way around.

Is the only way to get rid of the keywords on the sitemap or is there something else I should try?

You are kidding right? You have really thought about de-optimizing a page in hope others will rise???? Again, you have it all back-to-front.

[Mac 5]

How about a sitemap (list of links) to external sites that link to you, so the engines can find them?

[mawells]

>>How about a sitemap (list of links) to external sites that link to you, so the engines can find them?<<

Hmmm.... now I think you're bordering on what is called a "link farm." Could be risky - depends on the purpose. If the links are of genuine value to site visitors, then it's probably okay. If the intent is just to build up PageRank - it will probably backfire.

[Andilinks]

now I think you're bordering on what is called a "link farm." Could be risky

No, not bordering, that would be a full-fledged link farm, and almost certain to cause problems.

A one-for-one correspondence of reciprocal links all on a single page could not be a coincidental occurance.

Andi

[Jeremy21b]

No actually anchor text does affect the page that the link is on. Text within a 'a href' tag is considered important text on a page to Google. Anchor text affects both the page it links to and the page that the link is on. De-optimizing our sitemap would improve rankings because other pages are optimized to certain key phrases. So although it has a lower page rank, it has better keyword density and better optimization techniques implemented. A listing in a search result just titled 'sitemap' would get very few clicks as well. I don't have anything back-to-front, but would appreciate some advice.

[Mel]

Hi Jeremy
If you "deemphasize" your sitemap your other pages that its links point to will rank lower since they will lose the benefit of the anchor text linking to them.

Each page stands on its own, if individual pages are not ranking well the only way to help them rank higher is to optimize the pages and links pointing to them better, not the other way round.

[Dave Hawley]

Hi Jeremy21b

Honestly, you do have your logic back-to-front.

What you are think of doing is aiming for the lowest common denominator. Aim to bring the other pages up, not lower the pages that are ranking well.

[Andilinks]

yeah, what Mel and Dave said.

[rex_b]

is there anything that can generate a sitemap automatically?

[flood6]

Originally Posted by rex_b

is there anything that can generate a sitemap automatically?

Here is a search on HotScripts for "Sitemap". They have several applications that can generate sitemaps. Most are free.

While you most likely won't have a problem with most of the scripts, it might be better to use pure standards-compliant hand-written html, though. No javascript, dynamic links, or other stumbling blocks for your users or spiders to overcome.

[Andilinks]

Yes, quite a few. I've never used any, but the I have used the Xenu Link Sleuth for extensively for over a year as a link checker and it claims to have a site map generation feature. I have heard from others that it works well, there are other such programs as well. Xenu has the one most sought after feature though: it's freeware.

http://home.snafu.de/tilman/xenulink.html

It does use server resources heavily for link checking, probably will for site map generation too. It would be best to use a local copy of your site if you use shared servers or if your server resources are maxed out.

Andi

[poab]

I disagree, and without meaning to cause any offence my opinion is as follows:

If you add a site map to your site the message you're giving your customers is "You're about to get lost because our site, like our company is a confused mess".

Build sites with strong architecture, create navigation using hyperlinks styled with CSS and spend time on your content and you'll not need a site map. (The exception being very large sites with 1000+ pages.)

If a site map really does make your website easier to navigate then you've done a poor job of building the primary navigation.

[atimmins]

Rex_b:
I use a freeware program I found that does a very good job. It's called XENU Link Sleuth, I do not recall where I found it, but I'm sure if Google it you will find it. If you can't email me and I can send you the zipped file.

Anchor Text and Site Maps:
I have a client whose site was in .asp. I did my normal optimization, including using Xenu to create the site map. When I sent him my 8 week report he was concerned by the fact that the site map was indexed. Xenu uses the Title tags for naming the links on the site map, so the keyword heavy page ranked well for neaqrly all his phrases. The client actually wanted it de-optimized so that it would'nt rank. Once I explained that the site map wasd actually adding relevance for the pages it linked to, by describing what they about, he had a great question. He wanted to know if he could add content to the page. I thought it was a great idea and felt bad that I did'nt think of it myself. Such is life in SEO, we learn in un-expected ways!

Site Maps and Redundancy:
In no uncertain terms let me say that in my opinion site maps are required and not redundant. Well more to the point they are but for important reasons. As Andilinks points out many of the best OS's and software packages allow users to do things in multiple ways. This is one of the reasons M$ Windows became so popular, any idiot can figure out how to do what they need to do. Treat internet users like children, not because they are, but you never know who will be using you site, and possibly add to your bottem line. Redudant navigation is needed for those vistors who are not internet savy, and do not realize that to see your other links they have to hold their mouse over really COOL Java nav links.

[Andilinks]

If you add a site map to your site the message you're giving your customers is "You're about to get lost because our site, like our company is a confused mess".

No, that is not the message at all. Good design is redundant and allows multiple ways to acheive the same goal because people have varying styles of surfing--some prefer a site map.

Additionally, Googlebot and other agents like site maps.

Though I do agree that the site map shouldn't be offered as the primary navigation, just as an alternative.



Andi

[braknews]

Originally Posted by poab

If you add a site map to your site the message you're giving your customers is "You're about to get lost because our site, like our company is a confused mess".

In my opinion a site map says the opposite. It says that this site is structured in this manner.

Originally Posted by Andilinks

Googlebot and other agents like site maps.

If you are using any kind of menu that is not "plain-old" HTML the bots may not be able to read it. I would suggest you put a link to your site map on your home/main page so that you can make sure that the bots can find it.

[Canadian101]

[quote="Dave Hawley"]

I used keywords in the anchor text on the sitemap links. Now the sitemap shows up in search results instead of the optimized content pages

Back to this comment. This is something that we have been noticing as well. I find this very strange since the links on the sitemap only make a brief reference to the actual page that is optimized for the kw's. You would think that the page optimized for kw =x would achieve a top ranking, rather than the site map sitting in a #1 position for the kw. Especially when there are so many other terms present on the sitemap (in link descriptions and anchor text). Perhaps it has to do with the overall theming of the page???

[webiff]

Well I would love to take part in this topic because i have a city portal website of Pakistan known as www.apnahyderabad.com and i have created a SITE MAP of this website, the best part is that i have redirect 404 page to site map page, if users not find the appropriate page or get an error they will automatically redirect site map.

visit www.apnahyderabad.com

Best Regards,
Furqan Durvesh

[Andilinks]

...i have redirect 404 page to site map page,

I like this, but I have to wonder, do the hackers/crackers probes that normally generate 404's get redirected to the site map?

Does this have any ill effect?

I usually have very few of these but occasionally I get hit with hundreds at a time. Wouldn't that cause some problem if redirected to an actual page?
I have my 404 page down to a lean 400 bytes for that reason.

Andi

[Conficio]

Originally Posted by Jeremy21b

A sitemap can be very useful for both visitors and search engine spiders. I ran into a problem with my sitemap though. I used keywords in the anchor text on the sitemap links. Now the sitemap shows up in search results instead of the optimized content pages. It's nice that we at least show up in results, but with the wrong page it doesn't attract the targeted market and it ranks lower than the other page would. Anyone know how I can fix this? Is the only way to get rid of the keywords on the sitemap or is there something else I should try? Perhaps I should optimize the page for just the word sitemap with some larger fonts and also decrease font size for the terms that I don't want the page to show up for. Any thoughts on this? thanks.

Did anyone try

<meta content="NOINDEX,FOLLOW" name="robots">

for the sitemap page? This should get the sitemap page out of the search results. But the question is, does it still help your page rank? Any decent bot should still find all pages in your site.

I usually send the sitemap only to the search engines for publishing.

K<o>

[braknews]

Originally Posted by Conficio

Did anyone try

<meta content="NOINDEX,FOLLOW" name="robots">

for the sitemap page? This should get the sitemap page out of the search results. But the question is, does it still help your page rank? Any decent bot should still find all pages in your site.

I usually send the sitemap only to the search engines for publishing.

My site map has that meta tag & while I do not have a good a PR as some of you I am still a 5.

[flood6]

Originally Posted by Andilinks

I have my 404 page down to a lean 400 bytes for that reason.

I hate to go off topic for a minute, but I just wanted to add that IE will use the default 404 error page instead of the custom one if the custom one is less than 512 bytes.

Two articles about it:

http://websiteowner.info/tutorials/s...stomerrors.asp
(poorly formatted, the content is below menu, but good information)

http://www.404-error-page.com/404-er...osoft-ie.shtml

I went to http://andilinks.com/sdfgfdsg in IE on Windows ME and got the default 404. Mozilla, of course gives your custom page.

I don't know if you knew this or not, Andi, I just wanted to pass that along in case someone else did not realize it.

[TrafficProducer]

Site Map info:-

http://www.webproworld.com/viewtopic...ghlight=#88439

[Andilinks]

I don't know if you knew this or not, Andi,

Actually I had heard this but thanks for letting me know anyway. :)

The default page also gives a link to my root directory so it doesn't make much difference.

Thanks again,

Andi

[atimmins]

I actually have all my error pages point to my site map. The therory being is you really never know if someone who decides to link to you, mistalenly may use the wrong page URL. On the other hand if you make changes to your site, as in a redesign, and pages get renamed, moved, or just removed, anyone with a bookmark to that page will at least see your new site map.

[Jörn Malek]

Our main portalhttp://www.1-costaricalink.comhas over 18'000 pages. I made 16 site-maps for the most important ones. A few weeks ago I took the no index comand off and every now and than I find a site mpa as a search result. I don't really know if I like that. How should the head of a site-map look like?
Here is a site map:http://www.1-costaricalink.com/site_map_10.htm

Have a happy day
Jörn Malek

[Andilinks]

Well, my site map has the same PageRank as my main page and I have no robot instructions on it.

It is referred as a search result often. But as a site map it is well ordered and easy to understand so the searcher should have no trouble then linking to the appropriate page and will also get some flavor of the rest of the site along the way.

I have no problem with this.

But then I'm not selling anything except the site itself as search destination.

Andi

[Andilinks]

How should the head of a site-map look like?

The site map should be one schematic which can direct the visitors to the page they need to go to easily and transparently, no external links, nothing confusing... I think having 16 site maps for one site is confusing. Perhaps one overall site map with subsidiary pages would be better.

Andi

[Mel]

Originally Posted by poab

I disagree, and without meaning to cause any offence my opinion is as follows:

If you add a site map to your site the message you're giving your customers is "You're about to get lost because our site, like our company is a confused mess".

Build sites with strong architecture, create navigation using hyperlinks styled with CSS and spend time on your content and you'll not need a site map. (The exception being very large sites with 1000+ pages.)

If a site map really does make your website easier to navigate then you've done a poor job of building the primary navigation.

I disagree. A site map is an index to your site, just like the index of a book, and helps viewers find the content they are looking for quickly and easily. It can be of even more assistance if you say organize your navigation menu system by topic and your sitemap by alphabet there are two different ways of finding things.

[Mel]

Originally Posted by Andilinks

The site map should be one schematic which can direct the visitors to the page they need to go to easily and transparently, no external links, nothing confusing... I think having 16 site maps for one site is confusing. Perhaps one overall site map with subsidiary pages would be better.

Andi

In many cases it could be Andi, but in this case the site maps are organized as alphabetized groups of hotel whichresults in a convient reference to finding information on a specific hotel.

[Dave Hawley]

for the sitemap page? This should get the sitemap page out of the search results

I just don't get this line of thinking. Why would anyone NOT want one of their site pages ranking well?

I disagree, and without meaning to cause any offence my opinion is as follows:

If you add a site map to your site the message you're giving your customers is "You're about to get lost because our site, like our company is a confused mess".

Build sites with strong architecture, create navigation using hyperlinks styled with CSS and spend time on your content and you'll not need a site map. (The exception being very large sites with 1000+ pages.)

If a site map really does make your website easier to navigate then you've done a poor job of building the primary navigation.

Like Mel, I do not agree with this. A Site map offer choice and if nothing else, another page for Google to eat up.

It's a bit like having a brick and mortar business with a front on 2 Streets. You have the choice of a door on both Streets, or only one. I know which I would go for.

[Andilinks]

If the analogy is of a web site to a book, then the Site Map is the "Table of Contents."

An alphabetical listing is not a site map, it is an index. This is unfortunately confused by the academic minds who originally named the default page "index," but I would maintain that a site map should not be an alphabetical listing.

A "map" is more of a visual analogy, not an index or alphabetical listing.

Calling an alphabetical listing a site map is confusing.

Andi

edited for clarity

[Jörn Malek]

First I want to thank you all for the different imput. I just changed my robot file again to allow site-maps to be indexed. Thanks !

Well my site-maps were never intended to be an alphabetical listing. But since one should not put more than 100 links on one page if it should get spidered or indexed, I had to create some kind of order. So far I always had the impression, that the sitemap fits mainly the purpose, to help spiders not to have to search several levels down. As I mentioned before my 16 site-maps only list the most important pages of the site. The Site Map nr. 1 for instance http://www.1-costaricalink.com/site_map_1.htm contains my most important first level pages and have nothing to do with alphabetical order. With so many hotels and every day one or two new ones that need to be added to the site map (tody for instance hotel cala luna on the site map 11) I figured that this is the best way to go about. What I am not sure is my head. What should go into the head of a site-map (revisit, <META NAME="ROBOTS" CONTENT="FOLLOW"> is this all correct?

Have a happy day
Jörn

[Dave Hawley]

Hi Jörn

No need to use:
<META NAME="ROBOTS" CONTENT="FOLLOW">

By default SE spiders will follow all HTML links.

[Jörn Malek]

Originally Posted by Dave Hawley

Hi Jörn

No need to use:
<META NAME="ROBOTS" CONTENT="FOLLOW">

By default SE spiders will follow all HTML links.

Thanks Dave,
I always wondered why people put this just on site-maps and not on other pages.

Have happy day (night)
Jörn

[DrTandem1]

Site maps are important, as they are one link to the rest of your site for search engines. The other reason to have one, especially on a large, corporate site, is these types of sites are usually put together by a number of people or teams. Many times a human visitor can not find what they want, until they visit the site map.

It never ceases to amaze me how often I can't find something that should be readily available on a site. I liken it to using the "Help" feature on Microsoft OSs.

[rubbadubbin]

you said:

"The theory says by having a site map at the second level of your site hierarchy, then all links to your site get indexed by search engine crawlers that limit the depth of searching."

how do i set the site map page to the second levle of my site hierarchy? and what levle should my pages all be whats this all about could you expound on this hierarchy you spoke about. i would realy apreciate it seing as how i just spent over an hour so i could post this. forgot my user name and pasword i read in here all the time and you should see and hear more from me now.
thanks guys,]
respectfully,
Bryan Laughead

[DrTandem1]

rubbadubbin--

That simply means to have a link to your site map on your Home (index.html) file.

[lucks]

Add title tags to links in your sitemap. Then submit the SiteMap to the Search Engines instead of the home page. and See all your pages getting indexed.

If any visitor lands up on your sitemap page instead of the home page, it isn't a wrong thing either, as the visitor will have a chance to view and decide which page he wants to go.

Treat the site map as an INDEX of a book. Won't u like to go thru the index before u go to the relevant topic?? I think this will answer ur doubt!

Thanks
Lucks

[Mel]

Originally Posted by Andilinks

If the analogy is of a web site to a book, then the Site Map is the "Table of Contents."

An alphabetical listing is not a site map, it is an index. This is unfortunately confused by the academic minds who originally named the default page "index," but I would maintain that a site map should not be an alphabetical listing.

A "map" is more of a visual analogy, not an index or alphabetical listing.

Calling an alphabetical listing a site map is confusing.

Andi

edited for clarity

See your point in some case Andi,but in many cases such as Jorns there alphbetization of the site map lends order to a large chunk of links and lets your viewers find things easier. But its still a site map which is just an ordered listing of links to every page in your site.

Whether that order is hierarchal or alpahbetical is a moot point IMO.