Protection of Listings vs Indexability

[cfinlayson]

In a previous post, I received some great advice about helping our local stores get ranked.

Helping our retailers rank in their town

It turns out the best solution was to have a single html page for each store with optimized text, proper META tags, etc.

In order to have them indexed, we need to have all the pages within two clicks of our homepage, (as per our SEO professional). The two stores that we put in the .xml site map, and on our .asp sitemap were indexed and ranked almost immediately, so we know that the page level tactics are working well.

This is a bit of a problem for us, however, as we don't want our competition to be able to go in to the .asp site map, which is linked in the footer of every page of our site, and pull down our entire dealer list in a matter of minutes.

So far, none of the other pages, (except the two mentioned above), have been indexed by Google, even though they all were added to the .xml sitemap, and have been resubmitted. Likely because they are not on the .asp sitemap and therefore not linked internally within the site.

Does anyone have a creative suggestion to solve the problem? Do we absolutely have to put them onto the .asp sitemap in order for them to show up in the organic results?

Chris

[imvain2]

If you want the search engines to spider your content, then visitors will be able view the content that is spidered.

Even if you don't put the dealers in your asp site map, anyone can goto into google and type in site:yourdomain.com and see all the pages that were spidered. So, it doesn't matter where you put the links, anyone can see the pages.

[OpticBurst.com]

Originally Posted by imvain2

If you want the search engines to spider your content, then visitors will be able view the content that is spidered.

Even if you don't put the dealers in your asp site map, anyone can goto into google and type in site:yourdomain.com and see all the pages that were spidered. So, it doesn't matter where you put the links, anyone can see the pages.

This is very true. You are always going to be visible to your competition. Focus more on making sure you are indexed.

[cw1865]

Originally Posted by cfinlayson

This is a bit of a problem for us, however, as we don't want our competition to be able to go in to the .asp site map, which is linked in the footer of every page of our site, and pull down our entire dealer list in a matter of minutes.

Why worry about them? They should be worried about you, right?

You just found out that anything that is viewable by the public is viewable by your competition as well.

Anybody generating leads working sales with your competition will undoubtedly troll through your 'store finder' function.

This looks like a niche industry something tells me the major players have found just about every legitimate distributor in North America as it is.

Unfortunately what you're looking to do is mutually exclusive. You cannot adveritise on behalf of your distributors and conceal it from your competition at the same time. You'll have to choose one or the other. My feeling is that your distributors will be happy anytime the phone rings and they find that they're receiving leads because of your website.

[ami_iss]

1. I agree that once it's out there, it's out there. Nothing can be done about that.
2. For purposes of better conversion (getting people to the link from your main page) - Local Listings link should probably be on the main page and not just in the site map. That's one less click. "Where to buy Beachcomber Hot Tubs"
3. For purposes of better search engine visibility, you can create several regional pages if you have a lot of outlets and list the ones for each region. That would make it a tiny bit harder for competition to find the outlets, but it won't stop them.
4. Another alternative is to create a Web log and then put articles in it that link to each of those store sites. The Web log is useful for publicizing other aspects of your site as well. Since it has an RSS feed, search engines should pick up the listings pretty quickly.
5. You raise the question however of whether a search engine or Google in particular will list an orphan page (one with no links to it from anywhere on the site. My experience is that somehow or other they do, even when you don't want them to do so, though it may take a long time. That seems to be true regardless of listing in XML sitemaps.

6. Don't know if you tried this - submit the XML sitemap again (manually) to Google in Webmaster central and note if any errors are returned - they may flag the orphan pages. If your site map is huge, try breaking it up and submitting several. Google doesn't like sitemaps with over 100 entries or so - or so they say.

7. It is not a good idea to jump to conclusions from behavior of Google or other search engines in 1 instance. They are erratic and may do or not do things for reasons that are not related to anything we did or did not do. You can do the rain dance every day. Some days it rains, some days not. Depends on the whims of the great father in Google Central.

8. I am surprised there is not a regional Google Onebox local business listing placed at the top of such searches. Did you try registering each business in the google map? Or did Google give up on those?

<please add your link to your signature>

[seopo]

I agree with the above that you'd be spending a lot of time trying to protect what is effectively public information.... however, if it's very important to you there is an option open to you.

You can serve up different content based on the IP addresses of your competitors. Not cloacking, just dummy data to a few competitors.

All you need do is get the IP's. To do this, I suggest emailing your competitors (probably on you email lists?) with something too enticing to not click on and then keep a log of those IPs.

The above isn't full proof but does add an extra hurdle.

[RichAtVNS]

Originally Posted by ami_iss

Google doesn't like sitemaps with over 100 entries or so - or so they say.

Where is this written?
I have a sitemap.xml with over 16000 pages...
Are you saying I am being penalized because of it?

Thanxs

[imvain2]

Originally Posted by RichAtVNS

Where is this written?
I have a sitemap.xml with over 16000 pages...
Are you saying I am being penalized because of it?

Thanxs

You are right, Google actually states the number is 50,000

Creating a Sitemap - Webmaster Help Center

[imvain2]

Originally Posted by seopo

I agree with the above that you'd be spending a lot of time trying to protect what is effectively public information.... however, if it's very important to you there is an option open to you.

You can serve up different content based on the IP addresses of your competitors. Not cloacking, just dummy data to a few competitors.

All you need do is get the IP's. To do this, I suggest emailing your competitors (probably on you email lists?) with something too enticing to not click on and then keep a log of those IPs.

The above isn't full proof but does add an extra hurdle.

This is actually very easy to bypass using Google, amoungst other tools:

site:yourdomain.com and click on the cached versions, or just read the title tag and description that is brought back for those pages.

[RichAtVNS]

Originally Posted by imvain2

You are right, Google actually states the number is 50,000

Creating a Sitemap - Webmaster Help Center

Thanks I thought that was way off base....

People need to be more careful when they post. Misinformation can lead to a lot of wasted time and energy.....

If you don't know don't make it up....

the old adage applies I guess..

Better to be thought a fool then to open ones mouth and remove all doubt!

[imvain2]

Originally Posted by RichAtVNS

Thanks I thought that was way off base....

People need to be more careful when they post. Misinformation can lead to a lot of wasted time and energy.....

If you don't know don't make it up....

the old adage applies I guess..

Better to be thought a fool then to open ones mouth and remove all doubt!

Funny and true, another that fits is...

If you can't dazzle them with brilliance, baffle them with bullsh*t.

[adtastichosting]

Originally Posted by RichAtVNS

Thanks I thought that was way off base....

People need to be more careful when they post. Misinformation can lead to a lot of wasted time and energy.....

If you don't know don't make it up....

the old adage applies I guess..

Better to be thought a fool then to open ones mouth and remove all doubt!

Well lets not harangue the person too much that made the post about not putting more than 100 links into a sitemap because I can tell you where they got that information and it came directly from Google:

"Offer a site map to your users with links that point to the important parts of your site. If the site map is larger than 100 or so links, you may want to break the site map into separate pages."

This is from the google design and content guidelines in webmaster guidelines and I can see where it might be miscontrued to think it is referring to an xml sitemap when in reality it is referring to a user friendly sitemap so I think we can say they didn't "make it up" but were just slightly off-base:
Webmaster Guidelines - Webmaster Help Center