Yahoo works hard, delivers little
Yesterday, I extracted crawler calls from my logs over a 12 hour period. This is a typical day, nothing special. This table shows the name of the crawler and the number of page requests. Twiceler may also include some image requests.
The table shows Yahoo fetches almost double the number of pages that google does, three times what MSN fetches. This is no problem, my system can easily handle it.
The disappointment comes when I look at the number of readers returned from the search engines. Yahoo sends 1/16 of the number of readers that Google sends, a little less than three times what Ask sends and eight times what MSN sends.
How can I help Yahoo? Are our stories so terrible for them?
Obviously, there is no problem getting Yahoo to visit.
Any ideas?
Yahoo slurp 17066
Googlebot 10756
msnbot 3085
AMZNKAssocBot 1325
Twiceler 1110
Teoma 855
MJ12bot 611
Exabot 397
Gigabot 213
'MS Search 4.0 Robot' 150
CCBot 124
SWSBot 80
Spinn3r 45
Yeti 20
Wasabot 18
W3CRobot 9
'Grub.org crawler 8
jetbot 4
|