Thread
:
Robots/Spiders uses very much traffics
View Single Post
#
3
(
permalink
)
07-10-2008, 06:32 PM
incrediblehelp
WebProWorld 1,000+ Club
Join Date: Jan 2004
Location: Live in Cincy Now
Posts: 7,575
Re: Robots/Spiders uses very much traffics
Slurp has always been a hungry spider. Have you tried changing the crawl delay in your robots.txt?
__________________
KBK Marketing
|
Search Marketing Blog
incrediblehelp
View Public Profile
Send a private message to incrediblehelp
Visit incrediblehelp's homepage!
Find all posts by incrediblehelp