|
|
||||||
|
||||||
| Index Link To US Private Messages Archive FAQ RSS | ||||||
| Search Engine Optimization Forum SEO is much easier with help from peers and experts! The WebProWorld SEO forum is for the discussion and exploration of various search engine optimization topics. Any non (engine) specific SEO or SEM topics should go here. |
Share Thread: & Tags
|
||||
|
![]() |
|
|
LinkBack | Thread Tools | Display Modes |
|
||||
|
Hi All,
Recently got hit hard from a crawler with a user_agent of Helmdall (webmaster@crawler . com) This crawler wasn't very well behaved and completely bombarded our server with requests. As it's dynamic and uses MySQL it caused problems because it consumed all of the allowed connections. Probably not the best way I know but it's been OK until this crawler came along. Anyway, I decided to block it with an Apache RewriteRule - My question is this: Is it wise to block this crawler, does anybody know about it? Thanks for any help you can provide Cheers, Andy
__________________
http://www.milonic.com/ Free website menus for non profits and free licenses for Open Source Projects |
|
|||
|
I've never heard of it, but I doubt that blocking it would cause any problems.
|
![]() |
|
| Thread Tools | |
| Display Modes | |
|
|
|
WebProWorld |
Advertise |
Contact Us |
About |
Forum Rules |
MVP's |
Archive |
Newsletter Archive |
Top |
WebProNews
WebProWorld is an iEntry, Inc. ® site - © 2009 All Rights Reserved Privacy Policy and Legal iEntry, Inc. 2549 Richmond Rd. Lexington KY, 40509 |