Gee guys, & gals,

I feel a little outclassed here, with Slurp only gobbling 35.23 Megs. last month (per site - on average), but thats more than all 13 of the other, regular SE bots combined. I got just over 7,000 uniques from Google, for their 2.74 Megs. of crawler bandwidth as opposed to 778 uniques from Yahoo.

While I don't have the dynamic shopping cart problem, I do have a lot of websites which seem to get trawled way too heavily by the hungry Inktomi spider in relation to the amount of traffic I get from Yahoo. Hasn't gotten so bad that I want to exclude Yahoo traffic altogether, but there's got to be a way they can get the info they need to do a creditable job of indexing without raping a site every time they come to get it.

Will keep an eye to this thread to see what others have gleaned or suspect.

Cheers,
Doc