Adsense spidering

andyholt · #1 (**permalink**) 07-22-2003, 12:41 PM

Hi
I am in the process of testing out adsense from google and have a question.

What exactly does the spider spider when deciding what ads to serve? Does it read meta tags, title, content - or all 3.

If its all 3 what gets priority meta tags or content?

Any info would be appreciated on this as I would like to know if I have to tweak my meta tags for best results

Andy

sspry · #2 (**permalink**) 07-22-2003, 07:35 PM

Hi

While I can't answer the specific questions asked, I can give some feedback on my experience to date - which is about 10 days :)

I was a little concerned about some of the ads that first appeared on a couple of my sites - adsense seemed to have grabbed an inappropriate keyword from the CONTENT and pumped in ads related to that.

However, it seemed to sort itself out after a couple of days - the spider must've visited - and now, in most cases, the ads seem to be related to the specific topic of each page. Check the category pages at www.comeonaussie.com or the main sections at www.ozsmallbiz.net to see what I mean...

What IS interesting is that the IP address of the visitor seems to determine the ads being shown - for example, seeing I'm an Aussie, when I visit US based sites I see a mix of Australian ads!

As far as testing it is concerned - just DO IT! I have a range of sites that have earnt next to nothing for years (they don't sell anything, just provide information), and now with adsense in place, I've earned good dollars since I started.... especially with the conversion to Aussie dollars... and I'm already well over the minimum payment threshold!

With all the other affiliate programs I've been involved in for the past six months, I've earned a grand total of around $4.00. I got almost US$2 an hour from adsense yesterday.

Put it on all the pages you possibly can! If you've got good traffic to your site (I've got around 40k unique visitors a week to all mine) you'll find that it should be worth it!

Cheers :)
Stephen Spry

Shawn Collins · #3 (**permalink**) 07-23-2003, 02:52 PM

Quote:

Originally Posted by andyholt

I am in the process of testing out adsense from google and have a question.

What exactly does the spider spider when deciding what ads to serve? Does it read meta tags, title, content - or all 3.

I've been wondering if putting AdSense on sites that were previously listed poorly on Google would increase the presence of that site on Google.

Anybody know? It seems possible that since they are going to spider the pages where the AdSense code appears, those pages might be newly added to Google.

I'm crossing my fingers, at least.

JMac · #4 (**permalink**) 07-23-2003, 02:56 PM

I completely agree with Stephen - Google AdSense has made more for me on one site in ten days than affiliate programs have on 3 sites in 6 months. Long live AdSense!

With regard to meta tags vs. content priority, all I can say is that the spidered games page at mostlymommies.com generates toy and games ads while the home page generates handmade children's clothing ads with the occassional toy ad. When they were originally spidered (10 days ago) I believe my keywords tags were the same on both pages, and all the other tags I have are the same even now. (I have changed the keywords for both pages) The only difference was/is content.

Leads me to believe that with Google, yet again, content is king.

Please take a moment to review my site:
http://www.webproworld.com/viewtopic.php?t=1686

Thanks!

JMac

andyholt · #5 (**permalink**) 07-23-2003, 09:03 PM

Hi
Thanks all for the replies - I am beginnning to notice that ads seem to be more content based that meta based - which suits me just fine - got enough to do without rethinking some tags :)

As for shawns comments about better ranking for AdSense site - here's hoping :) I am already well indexed and ranked on google but every little helps :)

Ta :)

Andy

sspry · #6 (**permalink**) 07-23-2003, 09:39 PM

Another advantage (I think) I can see to taking part in Adsense is that it WILL get the google spider to visit your site and index it.

A couple of clients have been waiting for ages to get their sites listed on google, so a few adsense ads on their pages might speed that process up :)

Just a thought.

sellportal · #7 (**permalink**) 08-01-2003, 03:49 PM

Hi Andy,

According to sources from many places about adsense will adsense not,

* Improve you ranking as the adsense spidering is "for the ads only"

But as others have said before me does the ads come from the text of the webpage where you put the ad code.

And yes, I can also verify the ad selection as being connected to your geographical location. It's even easier for me to see as I'm from a non English spoken country (well native spoken at least, I did manage to get this together:-)

But so much can I say, adsense rock!

I have a clickthrough rate of almost 9% after 16 days! I'we never seen anything coming close to this.

Kenth

Ps I've not doen anything special with the ads, more then pasting the code to see if it worked!

azwar1 · #8 (**permalink**) 08-05-2003, 11:04 PM

Hi Everybody

To have Adsense spidering:
your website must have robot.txt on main file than write...

User-agent: Mediapartners-Google*
Disallow:

This isn't only help Mediapartners Google & GoogleBot to spider / crawl, but several search engine got permission from you to spider /crawl your side.

If your robots.txt is blank, Mediapartners Google and Googlebot will not crawl / spider your website.

Regards,
Azwar AM
www.tamakoo.com

alsoran · #9 (**permalink**) 08-06-2003, 09:52 PM

For all that you guys have been telling about AdSense, I was dismayed when Google didn't approve my website for displaying AdSense.

Is it due to my geographic location (India) or traffic rank (Alexa 131707 presently) or something else..!

I'm more flummoxed 'cause Google says AdSense is for every site. Well it is not....ask me.

E-MAC · #10 (**permalink**) 08-07-2003, 06:06 AM

Quote:

Originally Posted by ezypost

For all that you guys have been telling about AdSense, I was dismayed when Google didn't approve my website for displaying AdSense.

Is it due to my geographic location (India) or traffic rank (Alexa 131707 presently) or something else..!

I'm more flummoxed 'cause Google says AdSense is for every site. Well it is not....ask me.

This is just a guess, but I would say it either has to do with the type of content on the pages, or the traffic count.

I don't know how much traffic your site receives, but I would guess they probably do look at the numbers (as a way of determining which sites can participate). So maybe that is part of it?

The other thing, might be the content. I see that you have one page that has links to articles about web development and such, and the rest sells your templates/services. Could the content/product advertsing (your templates) ratio be why they didn't want to participate? I'm not sure, just a thought.

I also noticed a couple of other things. Maybe they just didn't like the other links to affiliate programs you are currently participating in?

And one more thing, maybe not related, but worth looking at. You have a lot of dark text on dark backgrounds, almost like "hidden" text, which is something that has been abused in the past to "dupe" search engine robots. So, I would consider changing the text/background colors in some places, so the text is more visible.

Hope any one of these might help you. I am curious if Google Adsense gave you a reason why they didn't accept the site?

alsoran · #11 (**permalink**) 08-07-2003, 08:45 PM

Hi E-Mac,

Thanks for your suggestions which will surely help me.

Google did not inform why my website wasn't considered for AdSense. I inquired, but there wasn't any reply. Google is a big entity and to expect a reply is perhaps not practical.

But I wrote in other forums, but yet to get a satisfactory reply. Does anyone in WebProWorld know..!

Denise · #12 (**permalink**) 08-10-2003, 05:45 PM

Hello EzyPost!

You don't have a lot of banners or advertising on your site, which Google will quickly deny you for, and you do have plenty of good content at your site, so I would take a look at the exit pop-up. Google has strict rules against popup ads. Wouldn't hurt to try removing that and re-applying to the AdSense program before you spend a lot of time redoing your site.

alsoran · #13 (**permalink**) 08-10-2003, 09:16 PM

Thanks Denise for soothing my nerves.

Yes, I do have exit pop-ups from TrafficZap hoping traffic would zoom. That's not happening though, but that's a different topic.

I'd remove the pop-ups and see if Google agrees..

allbiz · #14 (**permalink**) 08-13-2003, 02:24 PM

If I'm not mistaken, perhaps it's better NOT to have a robots.txt file on a site IF you want the entire site indexed. I think I remember that the absence of the file is akin to the noindex,index,nofollow,follow META tag in that in it's absence the default is to index and follow.

alsoran · #15 (**permalink**) 08-13-2003, 09:30 PM

Allbiz,

True, a robots.txt file excludes crawling web-files you do not want indexed. Like I have for my image files, CSS file, certain zip files and some html files (e.g. 404.html, thanks.htm, etc.).

Do you mean that would make my website disqualify for AdSense..! This is a new suggestion. I'm not so sure if that could be the real reason...

Any suggestion out there!

klyve1 · #16 (**permalink**) 08-14-2003, 09:26 AM

Adsense may use content to deliver relevant ads and so far everything delivered to my sites are highly relevant to their core subjects.

What I find interesting is that when I work on my development pages on my machine's webserver I get web hosting and reseller ads!

GSO · #17 (**permalink**) 08-20-2003, 12:09 AM

I would suggest that you read the Google Adsense FAQ and Terms of Service which explains all of the criteria for selection and exclusion factors. I had several pages that were already listed in Google before my site was accepted into their program but there has been no addional pages listed because of my participation. I have been involved in several major affiliate programs and have spent hours working for very little. With Adsense, it was copy and paste and the first day with several pages displaying public service ads, I had a click through rate of 3.6% and it has steadily increased for the last week.

dcdavis · #18 (**permalink**) 08-21-2003, 03:55 PM

Google adsense excepted my site in one day altho I recieve very little trafic.so click thru must not play a big part in it.

edhan · #19 (**permalink**) 08-31-2003, 05:51 AM

Hi

I have submitted one of my PPC Search Engine for Google Adsense and got a reply saying it was not accepted.

I am trying to figure out what are the factors they are judging to accept or reject a site.

Any ideas?

Regards
Edward Han

Diana · #20 (**permalink**) 08-31-2003, 06:46 PM

I'm going to take a long shot and say, they might not like PPC search engines?

You don't say what you're site is, but since you say PPC, I'm going to assume it has very little content, besides search results. Google seems to prefer content rich sites to affiliate and other sites.

Diana

bhopkins · #21 (**permalink**) 09-11-2003, 09:01 PM

Quote:

Originally Posted by Shawn Collins

Quote:

Originally Posted by andyholt

I am in the process of testing out adsense from google and have a question.

What exactly does the spider spider when deciding what ads to serve? Does it read meta tags, title, content - or all 3.

I've been wondering if putting AdSense on sites that were previously listed poorly on Google would increase the presence of that site on Google.

Anybody know? It seems possible that since they are going to spider the pages where the AdSense code appears, those pages might be newly added to Google.

I'm crossing my fingers, at least.

Shawn, I would say you have a very good chance of increasing your position. I work at a college and our web site did not have great position in Google. I did need a search solution for the site so I signed up for google's university search and our pages shot to the top for almost all our key words. I'm not sure if this was because google had to crawl the site more to perform accurate searches or if it was just the prescence of the google code on the pages, but adsense may work the same way. At least we can be sure thats it not going to hurt your rankings.

Bruce

jnrjrsongs · #22 (**permalink**) 09-27-2003, 02:51 AM

Adsense's spider always seems to catch up to the content onf the page within a few hours or so.

JuarezTourGuide · #23 (**permalink**) 10-08-2003, 08:34 PM

Quote:

Originally Posted by ezypost

For all that you guys have been telling about AdSense, I was dismayed when Google didn't approve my website for displaying AdSense.

Is it due to my ge