PDA

View Full Version : SPIDERING



DavidGarfield
07-14-2003, 08:38 PM
Hey guys, it's my first time on here, but I gotta say, I've been hackin, crackin, and stackin pages trying to get Google (or yahoo or anyone else) to spider my pages automatically.

How come someone like GameFaqs, that has scripts generate their pages GETS THOSE PAGES LISTED?

My site is: WonderdogSoftware.Com (only go if your curious, I get enough bandwidth)... but my question is... I have several links throughout the site that all lead to 'Game Titles' on a specific page (both within the page description, title, AND texts) Why are these pages not individually coming up, but rather the link lists that hold them?

Just curious how those big guys get all their pages listed and I can only get like 5 or 10.

Anyways, try entering 'Zelda Wind Walker Walkthrough' into Google and you'll see what I mean.

Thanks for any suggestions you may have.

Sincerely

David J Garfield
ceo@wonderdogsoftware.com

le_gber
09-07-2003, 06:55 AM
Hi DavidGarfield,

The reason why your pages don't come up is because they are behing a membership/password protected page. Spiders can't get in hence no listing ofr these pages

Leo

DavidGarfield
07-07-2004, 11:11 PM
I have a database and various other methods with a huge bunch of content on various video games that doesn't at all have to be presented through the 'paying member contributors'.

Any other ideas?

OneMoreBite
07-07-2004, 11:50 PM
I went and took a look using the search terms you suggested and I found a couple links to your site, both of which took me to a page that required registration. So, I went backwards to the domain itself and did find what I think you are referring to when you say you have pages with content, etc., but the problem appears to be the framesets.

I looked at the source for the main frame and it doesn't appear to contain any text and/or content that search engines would find appealing, and the content frame likewise. You may have better success if the pages (all pages within the frameset) contained meta tags and content.

Perhaps just add a navigation bar at the bottom of the interior frame with links to the other parts of the site, i.e. FAQ's etc.

Kathryn

sudhani
07-08-2004, 09:15 AM
Simple:

You have hidden text on your pages. You are lucky. Google didn't notice it (or) even it did you didn't get heavy penalties.

Remove the hidden text ... before Google decides to drop you from their index or some of your competetors go and suggest the same to google!!!

Pi Squared Technologies
07-09-2004, 10:01 AM
Hi, You might also want to use robot tags to guide the spiders through your site.
http://www.searchengineworld.com/metatag/robots.htm
"The Robots META tag is a tag to tell a robot if it is ok to index this page or not."

DavidGarfield
03-14-2005, 12:09 PM
It's been almost a year now since you've helped me with your suggestions. I'd just like to say thanks since all your ideas are working and have paid off.

Thanks again!

David Garfield
ceo@wonderdogsoftware.com