WebProWorld Part of WebProNews.com
Page One Link To Us Edit Profile Private Messages Archives FAQ RSS Feeds  
 

Go Back   WebProWorld > Search Engines > Insider Reports
Subscribe to the Newsletter FREE!


Register FAQ Members List Calendar Arcade Chatbox Mark Forums Read

Insider Reports Anyone is welcome to reply and discuss but starting new topics is reserved for WebProWorld staff and MVPs.

Reply
 
LinkBack Thread Tools Display Modes
  #1 (permalink)  
Old 05-18-2004, 08:32 AM
Garrett's Avatar
WebProWorld Veteran
 

Join Date: Jun 2003
Location: Lexington, KY. USA
Posts: 449
Garrett RepRank 0
Default Risky Technique For Luring Yahoo Spider Into Your Site

If you've got one or a few pages in Yahoo's index but are having trouble getting more pages indexed one thing that might work is putting links to your deeper, unindexed pages high in the code of the pages that are indexed.

EGOL in an SEOChat forum discussion said, "I am getting new pages in the index by making them one of the first links on one of my already indexed pages."

He goes on to say that he's not putting links to hundreds of pages up there - just the ones he considers the most important at the time. And he's doing this only in similarly-themed sites so that the links don't seem out of place.

"I have rotated them in and out of my top-of-page navigtation - and I don't think that it stunk up my design or content - these are all similar-theme sites."

The rotation allows him to get more than a few pages crawled from the already existing page.

Another poster in the thread suggested that those concerned with hiding these links to maintain the sites current layout could try hiding your links in a noscript tag.

Scott Harris, our head designer, suggested a more efficient, and definitely risky, method of hiding your links using CSS. Don't try this on your main site - you could kill your business. If you're going to use high risk techniques try them on a test site to see what happens.

His first example was simply to set the visibility of your links to "hidden."

Here's the code as he sent it to me:
Risky Technique

That link would only show up in the html and not on the site itself.

As Yahoo could possibly check links for the phrase "visibility: hidden," you might want to make an external style sheet and name your tags so that your intentions aren't evident to the spider.

Your hidden link would appear to the spider this way:
Risky Technique

The search engine won't know that "risky" means:

.risky {
position: absolute;
visibility: hidden;
z-index: -1;
}

If this technique works in Yahoo it would, in all likelihood, work in Google as well.

While this is probably an ancient high risk concept to some SEOs, I've not read much of this technique for hiding links thus far, and therefore don't know if it's currently an effective high risk technique or not. If you're considering using this or any high risk optimization method be sure to test it out on a sandbox site first. And remember - you're optimizing at your own risk.

If you notice that your competitors have links and text in their source code that doesn't appear on their site you could look for the .css link (do a ctrl + f and type in ".css"). In that style sheet do a ctrl + f and type in "visibility: hidden" to see what stands for hidden text in their source code. And brush up on your CSS here because there's probably more ways than that to hide text and links.

This may help you uncover some hard evidence of high-risk techniques you can take to the search engine officiators and get your competitor's banned.
__________________
Garrett French
Editor, WebProNews.com
http://www.WebProNews.com
Reply With Quote
  #2 (permalink)  
Old 05-19-2004, 04:35 AM
Mel Mel is offline
WebProWorld 1,000+ Club
 

Join Date: Jul 2003
Posts: 1,919
Mel RepRank 0
Default

What happened to using good old fashioned site maps to insure that all pages get indexed?
__________________
Mel Nelson
Expert SEO
Reply With Quote
  #3 (permalink)  
Old 05-19-2004, 04:58 AM
effisk's Avatar
WebProWorld Pro
 

Join Date: May 2004
Location: Biarritz, France
Posts: 153
effisk RepRank 0
Default

Hi,

I'm not sure google or yahoo don't visit the .css files. Can anyone tell us if they do? and what about menus stored in .js files? If google doesn't visit these files, then the rest of the website will never be indexed if links to the various pages aren't displayed anywhere else.
and .swf flash animations? Can google read the text contained in the animations?

cheers
[edit]ok, I've got the answer for the flash animations:
http://www.webproworld.com/viewtopic.php?t=18475
[/edit]
Reply With Quote
  #4 (permalink)  
Old 05-19-2004, 05:01 AM
effisk's Avatar
WebProWorld Pro
 

Join Date: May 2004
Location: Biarritz, France
Posts: 153
effisk RepRank 0
Default

Quote:
Originally Posted by Mel
What happened to using good old fashioned site maps to insure that all pages get indexed?
The risk is that your sitemap page gets a higher ranking than any other page on your website. I've seen that happen more than once. It's probably not such a bad thing though...
Reply With Quote
  #5 (permalink)  
Old 05-19-2004, 08:49 AM
Mel Mel is offline
WebProWorld 1,000+ Club
 

Join Date: Jul 2003
Posts: 1,919
Mel RepRank 0
Default

If your sitemap gets a better ranking than you pages constructed to rank well on a topic, then the optimization must be pretty poor.
__________________
Mel Nelson
Expert SEO
Reply With Quote
  #6 (permalink)  
Old 05-19-2004, 08:40 PM
WebProWorld Member
 

Join Date: Mar 2004
Location: United Kingdom
Posts: 93
spidermonkey RepRank 0
Default

Where did all this "ethical" crap come from?

Let's get back on topic!

CSS can be used to hide stuff that the Google Algo has no chance of seeing - fact!

If you are really sneaky you can put your "dodgy" CSS files in a folder that is excluded in your robots.txt file - fact!

So the only way the Google Algo can spot you is to break the robots exclusion protocol - fact!

If some "white hat" lamo reports you it will take months to be penalised - if at all - Google have expressed a preference for letting the Algo find the cheaters - fact!

Thanks Garrett for this introduction.

I am planning an "advanced" posting sometime soon.

Mike
Reply With Quote
  #7 (permalink)  
Old 05-20-2004, 02:44 PM
WebProWorld New Member
 

Join Date: Dec 2003
Location: Edinburgh, United Kingdom
Posts: 4
GilesGuthrie RepRank 0
Default

Quote:
Originally Posted by spidermonkey
Where did all this "ethical" crap come from?

Let's get back on topic!

CSS can be used to hide stuff that the Google Algo has no chance of seeing - fact!

If you are really sneaky you can put your "dodgy" CSS files in a folder that is excluded in your robots.txt file - fact!

So the only way the Google Algo can spot you is to break the robots exclusion protocol - fact!

If some "white hat" lamo reports you it will take months to be penalised - if at all - Google have expressed a preference for letting the Algo find the cheaters - fact!

Thanks Garrett for this introduction.

I am planning an "advanced" posting sometime soon.

Mike
The problem here is that the robots could then decide to exclude any <a> tage that has a class attached to it. This would be a very bad thing for people (such as myself) who are using CSS to control the behaviour of different styles of links in different areas (i.e. navigation vs content) in the same page.
__________________
Take a break on the web at giles-guthrie.com
Reply With Quote
  #8 (permalink)  
Old 05-20-2004, 04:47 PM
eightfifteen's Avatar
WebProWorld Veteran
 

Join Date: Jan 2004
Location: Des Moines, IA
Posts: 489
eightfifteen RepRank 0
Default

If you want to add links to your page, but keep the integrity, why not use spacer images with links? Does Google consider that an evil thing to do?
Reply With Quote
  #9 (permalink)  
Old 05-20-2004, 06:06 PM
WebProWorld New Member
 

Join Date: May 2004
Location: Sweden
Posts: 3
carnail RepRank 0
Default

Hold on now; this thing might be new to me: are you guys saying that bots (or rather the software "looking at the html") is indexing the parsed and "rendered" output? I thought most of them ran through the raw html that was outputed to the client (the bot in this case) before it was rendered.

See it like this: If I would disable CSS in my webbrowser, I would see your hidden link, right? The answer is Yes.
So who says that a bot that doesnt care about makeup first renders the page (and enabling CSS) and then scrapes data from the result? If you have a blue background and make the text blue, would the algo care? Is the algorithm blind???? Come on ...!
Reply With Quote
  #10 (permalink)  
Old 05-20-2004, 06:21 PM
WebProWorld New Member
 

Join Date: May 2004
Posts: 9
sri_gan RepRank 0
Default

I'm not sure how far this is worth a testing.

'cause it is pretty clear in the CSS statement it is a hidden object.

The Googlebot (Crawler) might not find it but the Page Parser might track this down very easily.

I'm sure if Google catches this, they will definetely enforce a ban 'cause its mentioned in their guidelines, they don't encourage hidden links.

Not sure of Yahoo yet. It is not that impossible to track.
__________________
Online Education Portal - Portal Dedicated for Online Education
Reply With Quote
  #11 (permalink)  
Old 05-20-2004, 11:13 PM
Mel Mel is offline
WebProWorld 1,000+ Club
 

Join Date: Jul 2003
Posts: 1,919
Mel RepRank 0
Default

Quote:
Originally Posted by carnail
Hold on now; this thing might be new to me: are you guys saying that bots (or rather the software "looking at the html") is indexing the parsed and "rendered" output? I thought most of them ran through the raw html that was outputed to the client (the bot in this case) before it was rendered.

See it like this: If I would disable CSS in my webbrowser, I would see your hidden link, right? The answer is Yes.
So who says that a bot that doesnt care about makeup first renders the page (and enabling CSS) and then scrapes data from the result? If you have a blue background and make the text blue, would the algo care? Is the algorithm blind???? Come on ...!
Bots don't parse or render anything they simply crawl and report the results of the crawl back to the search engine.

While google is a very intelligent engine, there is evidence to support the concept that their normal algo cannot recognize even the most elementary hidden text created by using text the same color as the background.

They can and do however make a special effort from time to time to ferret out hidden text (and other spam)and at that time pass out penalties to thousand of sites.
__________________
Mel Nelson
Expert SEO
Reply With Quote
  #12 (permalink)  
Old 05-21-2004, 01:45 AM
WebProWorld New Member
 

Join Date: May 2004
Location: Sweden
Posts: 3
carnail RepRank 0
Default

Quote:
Originally Posted by "spidermonkey
CSS can be used to hide stuff that the Google Algo has no chance of seeing - fact!
K, so this is bullox then.
If the <A> link is in the output to the bot it will be used in calculations by the algo wheter or not its hidden for the human eye, bcause it doesnt understand CSS (as the top poster has proven by trying to cloak the "hidden" value with a call to a CSS file instead)

A "hiding" made on the clientside wont protect algorithms from seing stuff in an unparsed document retrieved from the server. This is only used for making pages look nice even with x numbers of link at the top.
Reply With Quote
  #13 (permalink)  
Old 05-21-2004, 02:17 AM
WebProWorld 1,000+ Club
 

Join Date: Feb 2004
Location: Australia
Posts: 1,263
Dave Hawley RepRank 0
Default

Agree with Mel (and that doesn't happen everyday ;o).

Why carry on bad site design by placing band-aids all over it? Create a site map if you have problems with pages getting indexed.

Quote:
The risk is that your sitemap page gets a higher ranking than any other page on your website. I've seen that happen more than once. It's probably not such a bad thing though...
"risk"? Funny way to look at having a page ranking well. If my site map ranks higher than another page, I'm NOT going to get rid of the site map! I will try and get the page, I would rather above it,ranking better. If no matter what I try I doesn't work, so what?

If your handbrake on your car is better than your disk brakes, would you work on your handbrake or disk brakes?
Reply With Quote
  #14 (permalink)  
Old 05-21-2004, 02:23 AM
WebProWorld 1,000+ Club
 

Join Date: Feb 2004
Location: Australia
Posts: 1,263
Dave Hawley RepRank 0
Default

Quote:
Originally Posted by spidermonkey
Where did all this "ethical" crap come from?

Let's get back on topic!

CSS can be used to hide stuff that the Google Algo has no chance of seeing - fact!

If you are really sneaky you can put your "dodgy" CSS files in a folder that is excluded in your robots.txt file - fact!

So the only way the Google Algo can spot you is to break the robots exclusion protocol - fact!

If some "white hat" lamo reports you it will take months to be penalised - if at all - Google have expressed a preference for letting the Algo find the cheaters - fact!

Thanks Garrett for this introduction.

I am planning an "advanced" posting sometime soon.

Mike
WOW! Each to their own but why chance getting banned from Google. For many, it would mean the end of their business.

You sound like the sort of guy that would burn/hide the garage sale sign of your neighbour if you had one on the same Week-end.
Reply With Quote
  #15 (permalink)  
Old 05-21-2004, 10:50 AM
OneMoreBite's Avatar
WebProWorld Pro
 

Join Date: Aug 2003
Location: Vancouver, Washington / Portland, Oregon
Posts: 288
OneMoreBite RepRank 0
Default

At the risk of being considered prudish, I'm confused as to why the WebProWorld forum would suggest a technique with dubious benefit and possibly adverse affect? This seems less than prudent, IMO.

It brings to mind the recent postings by a fellow in the http://weightloss.about.com/mpboards.htm forum who was promoting his "fast food only" diet. Unlike Spurlock who produced the currently showing documentary, Super Size Me, this fellow claimed to have eaten nothing but McDonalds and lost weight. It may have worked for him (he claims) but on the whole it seemed a ridiculous thing to promote to a group of people seeking answers and/or advice in regards to gaining better health.

I visit and participate in these forums to discuss and debate innovative ideas, thinking and theories, not scrape the seedy underbelly of "what's on the bleeding edge of acceptable use."

Kathryn
__________________
Free E-book & Instructions in Using EFT & NLP for Weight Loss
OneMoreBite-Weightloss.com
Reply With Quote
  #16 (permalink)  
Old 05-21-2004, 12:21 PM
WebProWorld New Member
 

Join Date: May 2004
Posts: 9
sri_gan RepRank 0
Default

Quote:
Originally Posted by spidermonkey
Where did all this "ethical" crap come from?

Let's get back on topic!

CSS can be used to hide stuff that the Google Algo has no chance of seeing - fact!

If you are really sneaky you can put your "dodgy" CSS files in a folder that is excluded in your robots.txt file - fact!

So the only way the Google Algo can spot you is to break the robots exclusion protocol - fact!

If some "white hat" lamo reports you it will take months to be penalised - if at all - Google have expressed a preference for letting the Algo find the cheaters - fact!

Thanks Garrett for this introduction.

I am planning an "advanced" posting sometime soon.

Mike
I strongly disagree with the bolded part of the quote, I have my own reason to say so.

If you guys have time, spend some time in Googlelabs, you will know why I say this.
__________________
Online Education Portal - Portal Dedicated for Online Education
Reply With Quote
  #17 (permalink)  
Old 05-22-2004, 04:00 AM
Mel Mel is offline
WebProWorld 1,000+ Club
 

Join Date: Jul 2003
Posts: 1,919
Mel RepRank 0
Default

IMO anything that is publically viewable in googlelabs has not been added to the algo.
__________________
Mel Nelson
Expert SEO
Reply With Quote
  #18 (permalink)  
Old 05-22-2004, 04:43 AM
WebProWorld Member
 

Join Date: Mar 2004
Location: United Kingdom
Posts: 93
spidermonkey RepRank 0
Default

Quote:
Originally Posted by Dave Hawley

You sound like the sort of guy that would burn/hide the garage sale sign of your neighbour if you had one on the same Week-end.
Your burning analogy fits in as much as my making a posting to light a fire under this thread was entirely deliberate.

I spotted something in the original posting that I felt wasn't getting thoroughly examined.

Just because I make a posting that is a little contentious doesn't mean I am recruiting for the dark side of the force. I just wanted to get things moving before the topic got buried.

Best regards to all

Mike
Reply With Quote
  #19 (permalink)  
Old 05-22-2004, 04:59 AM
WebProWorld Member
 

Join Date: Mar 2004
Location: United Kingdom
Posts: 93
spidermonkey RepRank 0
Default

Incidentally, though this topic references Yahoo, I figured it was really relavent to all search engine robots and the state of artificial intelligence being used to find hidden content at the moment.

Mike
Reply With Quote
  #20 (permalink)  
Old 05-22-2004, 09:58 AM
WebProWorld Pro
 

Join Date: May 2004
Location: USA
Posts: 101
morpheus.100 RepRank 0
Default

I linked a site map with links dynamic and static to the top of my homepage with
and Googlebot indexed the entire site from site map. Also why do people deem it necessary to place a sitemap link at the bottom of a page. I had this self same link at the bottom of my homepage for over a month and googlebot was well on its way to another page before it got anyway near. It remains to be seen as to whether yahoo will index well as they really dont spider very often , but several other bots have all crawled my site with great sucess. As for CSS files being found by bots the answer is yes. Once bots found their way around my site they were taking an interest in all links even those in the headers.
Reply With Quote
  #21 (permalink)  
Old 05-22-2004, 09:27 PM
WebProWorld 1,000+ Club
 

Join Date: Feb 2004
Location: Australia
Posts: 1,263
Dave Hawley RepRank 0
Default

Spidermonkey, it was your
Quote:
Where did all this "ethical" crap come from?
that I was making reference to mostly. I, and I suspect many others, would read that statement as you having no ethics.

People without ethics are best ignored.
Reply With Quote
  #22 (permalink)  
Old 05-23-2004, 04:24 PM
WebProWorld Member
 

Join Date: Mar 2004
Location: United Kingdom
Posts: 93
spidermonkey RepRank 0
Default

Quote

All men are frauds. The only difference between them is that some admit it. I myself deny it.
H. L. Mencken (1880 - 1956)
Reply With Quote
  #23 (permalink)  
Old 05-23-2004, 09:52 PM