Keyword Competition Analysis: KEI and Beyond

[Garrett]

A poster at the ihelpyou forum has an idea for a site but before he launches he wants to determine the competition for his chosen keywords. Dan Thies, a "Super Moderator" at ihelpyou, offers some fantastic advice that applies to those analyzing business model viability or hunting for some less competitive keywords to optimize for.

Dan presumes that the very least your competitors are going to do is to put keywords into their page title. So you want to see if your new turtle ski resort site will have much competition you should do an "allintitle" search like this:

allintitle: turtle ski resort

This will give you a rough idea of the number of pages in competition with the ones you're planning.

Now if you want to see who's targeting exact keyword phrases in their page title then you can search this way:

allintitle: "turtle ski resort"

This will deliver a smaller set of sites, but it will show you who's in more serious competition with you.

Those, as Dan says, are the "on page" factors you can measure with search engine operators. Now lets look at the off page factors - namely, how many links the top ranked sites for those keywords have.

Here you'll simply perform a link:domain search (link:www.turtleskitrips.com) to see how many links your competitors have to your site.

If you're a wordtracker user, which you should be if you're serious about your SEO efforts, then you're familiar with the Keyword Effectiveness Index, or KEI.

According to Wordtracker, "the KEI compares the Count result (number of times a keyword has appeared in our data) with the number of competing web pages to pinpoint exactly which keywords are most effective for your campaign."

Their formula, which they compute for you in the Wordtracker program, works this way: "Let P denote the popularity of the keyword and C the competitiveness.

"The formula that we have chosen is KEI = (P^2/C), i.e. KEI is the square of the popularity of the keyword and divided by its competitiveness."

Dan has taken the original KEI formula and tweaked it:

s = matches for "allintitle"
p = popularity (search count)
KEI = p^2/s = p / s * p

He thinks that "Intitle+Inanchor is a better measure of the serious competition."

According to the Wordtracker site, any KEI of 40 or higher is worth targeting. However, as Dan points out, "KEI is most effective in making the initial selections for an SEO campaign. Over time you want to target any popular search term that's highly relevant."

Read Free Keyword Research Tools.

Thanks again to Dan's SEO Research Labs as well as the ihelpyou forum for an excellent thread.

[bhartzer]

This is some great stuff, I will definitely be using it in the future when optimizing sites and determining the potential competition for keywords I'm researching.

Why am I not surprised that this awesome formula and info comes from Dan Thies?

[Mel]

While Dans revision of KEI is much more useful than the original it still neglects one important factor:

The number of pages that use an exact phrase in their title is not nearly so important as the proportion of those who rank in the top ten who use that exact phrase in thier title.

If you are serious about ranking in the top ten for any term you are ONLY in competition with those sites that rank top ten for that keyword. Analyzing those pages will give you more answers than any formula ever will.

But if you want a fast and easy answer to the difficulty of ranking on a keyword I suggest that you look at the cost of a top bid for that term at Overture.

[Dave Hawley]

s = matches for "allintitle" will often be way off.

Results 1 - 10 of about s for allintitle: "term here" is often out by 50%

[DanThies]

Mel,

The notion that you're only competing against 10 sites is a bit of oversimplification. You're also competing against everyone else who is trying to get into the top 10.

In my opinion, KEI is absolutely, fundamentally flawed. Revising it as we have helps, in one sense, to make it more realistic, but it's still a flawed approach.

Garrett's post and article sort of stopped at the "intitle" matches, and we do look at that, but you can actually narrow it down much more by adding "inanchor" searches to find out who is using the search terms in link text as well.

If you want to know how much competition there is for a search term, it's not really about "how many" competitors there are, but how effective they are, and how motivated they might be.

To that end, we also look at the number of domains linking to top ranked sites, as well as pay-per-click bids.

As you pointed out, Mel, the PPC bid price is the best "rule of thumb" indicator of competition - if people are willing to pay $5 a click for advertising, you can be certain that they're also willing to spend a lot targeting organic listings.

[Mel]

Hi Dan, basically we are in agreement that KEI is a flawed approach to the difficulty of ranking on any given keyword, and I agree that you modification to KEI is an improvement, but my experience is that if you can beat any of the sites who currenlty rank top ten, then you will be ranking top ten. ;-)).

While theoretically you are competing against all the 3 million or whatever pages who are found for a particular term, the toughest competition comes from those who made top ten, and therefor IMO those are the pages you should compete with.

As I said in my last post:

If you are serious about ranking in the top ten for any term you are ONLY in competition with those sites that rank top ten for that keyword. Analyzing those pages will give you more answers than any formula ever will.

(empnasis mine)

A bit of time spent analyzing those pages will give you a far better evaluation of the optimization required to rank well on any particular term than any formula.

[Dave Hawley]

I cannot see much point in it at all. If you are in the business of say "Web Hosting", then that's what you must compete for regardless of how tough it is.

[DanThies]

I cannot see much point in it at all. If you are in the business of say "Web Hosting", then that's what you must compete for regardless of how tough it is.

I dunno, Dave. There are a whole lot of more specific search terms that will convert better, with a lot less competition.

It depends on your resources. If you're Rackspace or Verio, and you just discovered SEO, go ahead and target the biggest most generic term. If you're bootstrapping with a low marketing budget, "LAMP hosting" is probably more realistic as a starting point.

[Dave Hawley]

I dunno, Dave. There are a whole lot of more specific search terms that will convert better, with a lot less competition.

Yeah agree. I was was probably too vague in my statement.

Hypothetical
Let us say that "Web Hosting" is the best term to convert from hits to $$, then "Web Hosting" is one to shoot for. While not a golden rule, it stands to reason that the tough terms are often the $$ ones. KEI is not going to help idendify the best terms/keywords to use, only the less competitive ones and cannot see how that it is helpful.

Then there is the major flaw in it's calculation argumnet s;
KEI = p^2/s = p / s * p
where s is Results 1 - 10 of about s for allintitle: <term here> These "about" from google are rarely even close.

[DanThies]

Then there is the major flaw in it's calculation argumnet s;
KEI = p^2/s = p / s * p
where s is Results 1 - 10 of about s for allintitle: <term here> These "about" from google are rarely even close.

Although it's not possible to examine the case where s is greater than 1000, we've found that the Google API delivers very good numbers.

But who cares about KEI anyway? :D