Site links and google indexing

[sleeparchive]

Quick question for you guys,

I've been asked by my directors why when you google our competitors they have site links and when you google us we have none.

I think I have found out the reason but I don't know how to go about fixing this.

If you google the company name it brings up a page within the site not the homepage itself. The most powerful page is showing instead because of a university link that I acquired last year. I've changed the title tags to include the keywords that I want the homepage to pop up for and I know that most of our links point directly at our homepage.

Does anyone have any suggestions?

Also is it right to expect another link on your site to pop up in the webmaster tools as a site link if you block one of the assigned urls? Or will the site suffer by only having 7 links?

[sleeparchive]

Note: I've discovered that the homepage may not be indexed. I assume this the problem but a.) how is it indexing the rest of the site without the homepage b.) is there to fix this!?

[seomagician]

Is it possible for you to post the site so we can take a look? It would make it much easier to figure out what's going on.

[vblni]

As per Google's explanation, GoogleDOTcom/support/webmasters/bin/answer.py?answer=47334&topic=8523 it analyzes the link structure of a website to find shortcuts that will save users time and allow them to quickly find the information they're looking for.

[SEObyCanz]

I have found that site links are a reward for getting a higher pagerank, PR4 and above seem to have site links, but not for all keyphrases, and google seems to change the rules on sitelinks, I have had them for a few weeks, then they disappear, then they are back....

[BluePlanet]

Google does some strange things. I did a site with two iFrames on the index page for displaying photos. They had names like rightpics and leftpics. Google indexed those pages and not the index. I do not understand why they did that. This case seems similar. I had to deny access to those pages in the robots file.

[sleeparchive]

Website in question is:

wwwDOTlabmedrecruitDOTcoDOTuk

I'll read the google guidelines now. Thank you.

[Clint1]

Originally Posted by SEObyCanz

I have found that site links are a reward for getting a higher pagerank, PR4 and above seem to have site links, but not for all keyphrases, and google seems to change the rules on sitelinks, I have had them for a few weeks, then they disappear, then they are back....

You cannot request Site Links and there's nothing you can do to get them, it's something G does to a site "when they feel like it". I don't think PR has anything to do with it. If it does, the threshold is definitely not PR4 because I have had PR4 and PR5 sites for years that don't have them.

[Terry Van Horne]

I agree with Clint it likely isn't PR related but... trust that the site is most relevant for the query, IMO, is likely a very important factor. Getting Site Links for a company name or website should be almost automatic or was after the Site Links "feature" was fully implemented in the Universal Search algo.

The links included in the Site Links OneBox is where the mystery is for me.

[Clint1]

Originally Posted by Terry Van Horne

......Getting Site Links for a company name or website should be almost automatic or was after the Site Links "feature" was fully implemented in the Universal Search algo.

How do you mean? Do you mean a search for company name should yield the website listed with site links?

[seomagician]

Originally Posted by sleeparchive

Note: I've discovered that the homepage may not be indexed. I assume this the problem but a.) how is it indexing the rest of the site without the homepage b.) is there to fix this!?

I took a very quick look and think your issue is the homepage is not indexed.

The rest of the site is indexed becasue there are external links coming in to other pages.

Do yo have a xml sitemap submitted to google? Have you checked the webmaster tools? I can't see any obvious reasons in my quick look. You have enough links and decent links at that. Sometimes these things happen and then right themselves.

[Terry Van Horne]

Originally Posted by Clint1

How do you mean? Do you mean a search for company name should yield the website listed with site links?

For an established site...Yes.

A new site is much harder to find with a site OneBox which is why I think trust is a factor in the algo and likely why some people believe it is "PgRank" related. If you have no site links I wouldn't be surprised to see a "site: search" doesn't list the root page first.

[Clint1]

Originally Posted by Terry Van Horne

For an established site...Yes.

A new site is much harder to find with a site OneBox which is why I think trust is a factor in the algo and likely why some people believe it is "PgRank" related. If you have no site links I wouldn't be surprised to see a "site: search" doesn't list the root page first.

My main site is over 10 years old and it's never had any site links. And my homepage has always been indexed. (Of course that doesn't include all the countless G screw-ups over the years that delete homepages for X weeks at a time for no reason other than yet another algo screw-up). I'm not alone in that area, many long time well established sites don't have them.

It could be they may be PR-related after all, but a high PR is required. ? That may be interesting to find out more about that. Maybe a PR6+ or 7+ is required.

Does anyone here have site links with a medium to low PR?

[Terry Van Horne]

Clint1, Googles' algos usually don't screw up over the long term, they may not be doing what you think they should be doing, but, IMO, what you say is an algo screwup I'd say is an issue with what is likely a complex site that is experiencing crawling hiccups. 15 years doing this and lots of times I see what looks like Google screwups turning into an undetected 404/500 error or other crawling obstacle. They work themselves out.

[Terry Van Horne]

World music supply PR4

[Clint1]

Originally Posted by Terry Van Horne

Clint1, Googles' algos usually don't screw up over the long term, they may not be doing what you think they should be doing, but, IMO, what you say is an algo screwup I'd say is an issue with what is likely a complex site that is experiencing crawling hiccups. 15 years doing this and lots of times I see what looks like Google screwups turning into an undetected 404/500 error or other crawling obstacle. They work themselves out.

Aww Terry, spoken like a G employee. Well we don't need to get into that here , but I'm certainly not alone in thinking "they may not be doing what you think they should be doing" when they delete not just the homepage but even entire white-hat legit non-complex plain basic HTML websites of only a few hundred or less pages, all without a single http error, nav issue, etc., just because of yet another algo screwup. No one can think that's acceptable behavior, except of course G because they continue to allow it.

If you've been doing it that long, then you should have seen (as I have) across dozens of forums, countless thousands of posts of victimized site owners, all fitting the above criteria, all in the same unfortunate boat after an algo update that went afoul.

I'm certainly not saying that every site that gets deleted or has pages deleted has done nothing wrong (either intentional or not intentional), and doesn't have themselves to blame for any page deletions. Sure, that happens. I agree with what you say, just to a much lesser degree. What I'm saying is there is an ever-growing number of victimized site owners that does not have themselves to blame for any G "penalties" incurred. For one not see that, deny it, or turn a blind eye to it, is being short-sighted and looking at the www through blinders or rose colored glasses.

If G thinks that's "what they should be doing", then they are more (do no) "evil" than I previously thought. But I don't believe they think that's what they should be doing. What I do believe, is they are totally irresponsible in their treatment of white-hatters and those who made them what they are today. I believe they put out new overzealous algo's without any kind of testing whatsoever, which would ensure these new algo's don't have any negative repercussions to the white-hatters, and not throw the baby out with the bathwater. I believe they should be MUCH more responsible with the power they wield.

But, that's just my opinion. Actually, it happens to also be the opinion of quite a large growing number of others that have also seen it first-hand. I wish it didn't exist, but it's most unfortunate for all (of the non-Amazon, non-Ebay, non-Dealtime, non-Bizrate et al types of sites which enjoy special impunity, i.e. enjoy "algo insurance"), that it does exist. Hey I wonder if anyone actually sells algo insurance? They'd make an absolute killing on that. LOL.

[Clint1]

Originally Posted by Terry Van Horne

World music supply PR4

Interesting, thanks. Anyone else see one with a PR lower than 4?

[SisterSledge]

Here's a Google link to info of why/what/where of site links in their search results:
How does Google compile the list of links shown below some search results? - Webmasters/Site owners Help

Here's an example of one of my clients - PR3 site with sitelinks: search Google for discountwatchstore

[Terry Van Horne]

Clint1, here's why I have to believe differently.

To believe it's Google's screwup leaves me with nothing to do. Thinking... hmmm... maybe I should check this that or the other... just to be sure it aint me. When I'm fairly sure... I tell the client it will return "Just the way she goes boys!", your business is based on a blackbox if you want stable traffic then.... do some PPC like I recommended to begin with.

Lost sites and pages happens to be "part of the seo busines" however that has improved 10 fold or more over the years. Why think all those who complain are absolutely correct that G is whacked...

why bother chasin' algos? And how is it Google's problem/screwup since they are the ones who decides when it's functioning correctly? They owe us nothing all that should matter is their users. If they F that up... then... your Google worries are gone because they've joined Lycos, InfoSeek and all the other dead soldiers that didn't meet users expectations.

[cocoon]

Clint - we have site links on site homepage PR 3. I wonder if it's to do with the PR of the pages on the site links? Or maybe just backlinks? No idea how we got them, but I do know they change - eg. men's gifts features at the moment, didn't before Christmas.

[full house]

is the site links that shows in the google serp? like the links are the sub pages of the main site. Well google gives weight to that. It depends on the quality of your site, google show it to help users to see what they want easily.

[Old Welsh Guy]

Normally when the homepage disappears from the index, it is down to bad nameserver setup. often I have comeacross where the hosting company is using their own name servers when it is not configured properly. This is a REAL common issue!

I have seen poorly written redirects (I note that the site is on IIS so presumably using ISAPI? this can be configured wrong also.

Neither the homepage root nor the default.asp page is indexed, although the site can be accessed using either www or non www. this can lead to canonical issues.

Also the links on the site are relative " /" try changing them to hardcoded http:// www domain.com that might help (assuming this has been going on for some time)

Normally it is a technical glitch some place along the line. a little thing like a 404 returning a 200 header can spook the search engines.

Sometimes though it just happens. then it rights itself. I put it down to server glitches causing the domain root page to be dropped. if it hasn't come back after being spidered though, then be worried, and take action.

Oh & make sure you do all the usual stuff like Google WMT verify the domain, upload a sitemap, check for errors etc.

[Clint1]

Originally Posted by cocoon

Clint - we have site links on site homepage PR 3. I wonder if it's to do with the PR of the pages on the site links? Or maybe just backlinks? No idea how we got them, but I do know they change - eg. men's gifts features at the moment, didn't before Christmas.

Ok I see you and "SisterSledge" state a PR3 site with site links. So perhaps it's safe to say PR may have nothing to do with it, very little if at all. Some of my internal pages are PR3 as well. But after 10+ years and PR's of 4 and 5, no site links. Like I mentioned earlier, it's something G does "when they feel like it".

I see SisterSledge mentioned How does Google compile the list of links shown below some search results? - Webmasters/Site owners Help and they say it's totally "automated". Once again, contradictions:

"We only show sitelinks for results when we think they'll be useful to the user.".

When would they NOT be useful???

"If the structure of your site doesn't allow our algorithms to find good sitelinks, or we don't think that the sitelinks for your site are relevant for the user's query, we won't show them."

That says a lot. Maybe a specific "structure" is required, but, just what is that "structure"??

"At the moment, sitelinks are completely automated. "

If they are "completely automated" as they claim, then why and how can they say: "We only show sitelinks for results when we think they'll be useful to the user" and ".....or we don't think that the sitelinks for your site are relevant for the user's query, we won't show them". Those statements prove they are NOT "completely automated"!!

[Clint1]

Originally Posted by Terry Van Horne

To believe it's Google's screwup leaves me with nothing to do.

Hee hee. I hear ya!

Thinking... hmmm... maybe I should check this that or the other... just to be sure it aint me.

Sure, I totally agree with that. It's stupid not to be sure everything is ok, I never said one should not do that. What I'm saying is about the people that DO check and are sure everything is ok, yet it still happened to them.

Lost sites and pages happens to be "part of the seo busines"

Sure, I agree.

Why think all those who complain are absolutely correct that G is whacked...

Never said that either: "I'm certainly not saying that every site that gets deleted or has pages deleted has done nothing wrong (either intentional or not intentional), and doesn't have themselves to blame for any page deletions. Sure, that happens. I agree with what you say, just to a much lesser degree. What I'm saying is there is an ever-growing number of victimized site owners that does not have themselves to blame for any G "penalties" incurred. For one not see that, deny it, or turn a blind eye to it, is being short-sighted and looking at the www through blinders or rose colored glasses."

And how is it Google's problem/screwup since they are the ones who decides when it's functioning correctly?

"If G thinks that's "what they should be doing", ("functioning correctly"), "then they are more (do no) "evil" than I previously thought. But I don't believe they think that's what they should be doing. What I do believe, is they are totally irresponsible in their treatment of white-hatters and those who made them what they are today. I believe they put out new overzealous algo's without any kind of testing whatsoever, which would ensure these new algo's don't have any negative repercussions to the white-hatters, and not throw the baby out with the bathwater. I believe they should be MUCH more responsible with the power they wield."

They owe us nothing all that should matter is their users.

But we are their users. They owe people responsible behavior. With great power comes great responsibility. To ignore that, is immoral, and IMO rather "evil".

If they F that up... then... your Google worries are gone because they've joined Lycos, InfoSeek and all the other dead soldiers that didn't meet users expectations.

Wouldn't that be great, no more G to worry about.

[sleeparchive]

I'm sure it's a algorithmic decision based on several factors. It maybe amount of higher PR / nicely linked internal pages. This is what they mean by pages deemed useful. We all know that google heavily relies on link profiles to gauge SERP rank. I suspect that the relationship between the homepage and internal pages link profiles let's the algorithm decide.

[Old Welsh Guy]

Rather than pull out slices of others posts this is what i have been told by google people, and what i have from my own experience.

Sitelinks are given by phrase, once you become an authority in the Serps for that phrase, you will get sitelinks based on the phrase. This is why they always appear for your business name (unless you have a generic business name).

When google say 'structure' they mean just that. If your site has a proper structure (a pyramid), then google knows instantly that the index pages in that folder are the sitelinks as they are the most important in that section of your site.

You can also alter the sitelinks from within WMT once you have been allocated sitelink status.

[Terry Van Horne]

Most businesses site links include the contact page so... that's a no brainer, since if you are searching on the company name contacting them would be a good reason to search on the name. I've also noticed that most of the site links are among the pages with the highest pageviews.

Look at the site links for SeoPros
Here's what I know:
1. the title at the top "SEOpros.org" likely isn't found anywhere on the site but... is found in links to it
2.The RFP service (RFPgenerator) is often recommended on forums etc.
3.Some of the site link pages are linked to from authority sites
3. Most of the site links are to Org info and included in the bottom and right margin Nav Menu of every page on the site. Note that most of the site link text comes from the right nav menu, however, the About us link is from the bottom menu, perhaps, Contact and About are "keywords" they look to match. Providing site links that match site link text aids in navigation of the site
4.The Google site link text is a close match to the link text used in the site navigation mentioned above.

Link text is likely a factor but not in a way that PR (quantity & quality) would indicate much if anything. IMO, it's about link text and authority of the page the IBL resides on. I wouldn't be surprised to find the IBL with the most authority uses SEOpros.org as the link text.

What will be interesting is that we're doing some work that will change the nav menu. It will be interesting to see if that changes the site link text.

[Clint1]

Originally Posted by Old Welsh Guy

Sitelinks are given by phrase, once you become an authority in the Serps for that phrase, you will get sitelinks based on the phrase.

That makes sense, but.......

This is why they always appear for your business name (unless you have a generic business name).

That can't be it because they still don't show for my unique non-generic company name, but my website does show up in the index at #1 when searching for the name.

[Old Welsh Guy]

Originally Posted by Clint1

That makes sense, but.......


That can't be it because they still don't show for my unique non-generic company name, but my website does show up in the index at #1 when searching for the name.

Keep in mind it is algorithmic, do you have sitelinks for other phrases? It is unusual to have sitelinks for other phrases but NOT for your company name (unless as I said above). If however your internal and external link footprint doesn't uses your company name, and your company name isn't your domain name, then that can cause what you are talking about. I was assuming that a website for abcplanning, would have a matching url .

[full house]

yup! site links that shows on the serp are algorithmic and google always update its algo so its hard to figure out.

[Clint1]

Originally Posted by Old Welsh Guy

Keep in mind it is algorithmic, do you have sitelinks for other phrases? It is unusual to have sitelinks for other phrases but NOT for your company name (unless as I said above). If however your internal and external link footprint doesn't uses your company name, and your company name isn't your domain name, then that can cause what you are talking about. I was assuming that a website for abcplanning, would have a matching url .

No, I have no sitelinks for anything. My company name is the domain name.

[Old Welsh Guy]

Originally Posted by Clint1

No, I have no sitelinks for anything. My company name is the domain name.

Ah then my statement about sitelinks doesn't apply to you as you have none When I said 'this is why sitelinks always appear for the name' I meant if you actually have sitelinks in the first place. When you DO get them, the chances are that it will be for your name and possibly one or 2 others.

[Clint1]

Originally Posted by Old Welsh Guy

Keep in mind it is algorithmic...

I don't think it's all algo, if any. Look at my post #24. They claim it's "automated" but they contradict that.

[Clint1]

Originally Posted by Old Welsh Guy

Ah then my statement about sitelinks doesn't apply to you as you have none When I said 'this is why sitelinks always appear for the name' I meant if you actually have sitelinks in the first place. When you DO get them, the chances are that it will be for your name and possibly one or 2 others.

You said "do you have sitelinks for other phrases?" and I was just answering that (and mentioning my company name is the domain).

[Old Welsh Guy]

Originally Posted by Clint1

I don't think it's all algo, if any. Look at my post #24. They claim it's "automated" but they contradict that.

Clint, they aren't contradicting anything. When they say 'we think' they are probably talking about articifial intelligence, and/or manually intervention AFTER the links have been algorithmically created. (in much the same way you can alter them once algorithmically awarded.

Not sure what the second post about company name and sitelinks is about though. I was just pointing out that when i stated that they always appear for the company name, I meant once they are awarded, they are normally awarded for the company name (Which was why i asked if you had ANY sitelinks)

[Terry Van Horne]

Originally Posted by sleeparchive

I'm sure it's a algorithmic decision based on several factors. It maybe amount of higher PR / nicely linked internal pages.

IMO, browser PR likely doesn't tell or indicate anything in this case. PR can be dressed up/mistaken for trust because that is seemingly part of the PR algo. IMO, if it's trust... then if a authority site links to you then your PR may not budge or budge much... but it could end up in a site link being awarded... or possibly on your way to getting them. There are likely/could be more than 1 criteria. IMO, some of the text in the Site Links comes from IBLs and that would be a reason to enable editing them In the Webmaster Tools.

[sleeparchive]

I'm still having a issue with getting my index page reindexed.

I've resubmitted my sitemap, it's been crawled. Google still seems to be not indexing my homepage.

I'm perplexed.

Any suggestions?

[Clint1]

Originally Posted by Old Welsh Guy

Clint, they aren't contradicting anything. When they say 'we think' they are probably talking about articifial intelligence, and/or manually intervention AFTER the links have been algorithmically created. (in much the same way you can alter them once algorithmically awarded.

My point is, even if there is "manual intervention", then it's not "completely automated" as they claim.

Not sure what the second post about company name and sitelinks is about though. I was just pointing out that when i stated that they always appear for the company name, I meant once they are awarded, they are normally awarded for the company name (Which was why i asked if you had ANY sitelinks)

I gotcha.

[Clint1]

Originally Posted by sleeparchive

I'm still having a issue with getting my index page reindexed.

I've resubmitted my sitemap, it's been crawled. Google still seems to be not indexing my homepage.

I'm perplexed.

Any suggestions?

Have you/are you:

Submitting your homepage URL to the G DC's everyday?

Checked with a validator to see if there are any major errors on the page that may prevent Gbot parsing?

See if you link to any bad neighborhoods, or checked to see who links to you? (Yes, people that don't like you can setup "bad" websites and link to you, and G will penalize you for that). Bad Neighborhood - Link Exchange Tool

Why are there two meta name="verify-v1" tags on your homepage?

I'm not saying that it will help for sure, but you may want to try putting your title tag first in the source code, just below the <head> start.

Have you gotten a G Webmaster Tools account? If not, you need to do this because that can tell you things about what's going on.

[xuchunfeng1984]

cool for this post. thank you all my friend.

[Clint1]

Originally Posted by xuchunfeng1984

cool for this post. thank you all my friend.

If that's meant for my post #39, you're welcome. Please click it as helpful.