iEntry 10th Anniversary Forum Rules Search
WebProWorld
Register FAQ Calendar Mark Forums Read
Search Engine Optimization Forum SEO is much easier with help from peers and experts! The WebProWorld SEO forum is for the discussion and exploration of various search engine optimization topics. Any non (engine) specific SEO or SEM topics should go here.

Share Thread: & Tags

Share Thread:

Reply
 
LinkBack Thread Tools Display Modes
  #1 (permalink)  
Old 04-05-2005, 07:27 PM
milonic's Avatar
WebProWorld Member
 
Join Date: Feb 2005
Location: UK (Telford)
Posts: 88
milonic RepRank 1
Default Helmdall - crawler dot com

Hi All,

Recently got hit hard from a crawler with a user_agent of Helmdall (webmaster@crawler . com)

This crawler wasn't very well behaved and completely bombarded our server with requests. As it's dynamic and uses MySQL it caused problems because it consumed all of the allowed connections. Probably not the best way I know but it's been OK until this crawler came along.

Anyway, I decided to block it with an Apache RewriteRule - My question is this:

Is it wise to block this crawler, does anybody know about it?

Thanks for any help you can provide

Cheers,
Andy
__________________
http://www.milonic.com/
Free website menus for non profits and free licenses for Open Source Projects
Reply With Quote
  #2 (permalink)  
Old 04-06-2005, 11:52 AM
WebProWorld 1,000+ Club
WebProWorld MVP
 
Join Date: May 2004
Location: Austin, TX
Posts: 1,229
jestep RepRank 0
Default

I've never heard of it, but I doubt that blocking it would cause any problems.
Reply With Quote
Reply

  WebProWorld > Search Engines > Search Engine Optimization Forum

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On



All times are GMT -4. The time now is 10:21 AM.



Search Engine Optimization by vBSEO 3.3.0