iEntry 10th Anniversary 
Forum Rules 
Search
|
|
||||||
|
||||||
| Index Link To US Private Messages Archive FAQ RSS | ||||||
| Search Engine Optimization Forum SEO is much easier with help from peers and experts! The WebProWorld SEO forum is for the discussion and exploration of various search engine optimization topics. Any non (engine) specific SEO or SEM topics should go here. |
Share Thread: & Tags
|
||||
|
 |
|
|
LinkBack | Thread Tools | Display Modes |
11-16-2004, 05:30 AM
|
||||
|
||||
Grub Crawler
I opened a free account today and started crawling the web with looksmarts distributed crawler "grub". I'm wondering if anyone has any experience with their local crawl utility or inclusion into wisenut.com with these methods. I'va also read some posts that indicate that some webmasters consider the grub crawler a "virus" of sorts (w/ site bandwidth?) Any comments?
|
|
11-16-2004, 08:09 AM
|
||||
|
||||
|
Waht are you intending to do with the results? If a spider spends hours on site just chewing up bandwidth for fun, then I will block it. If it is a visit permonth or so and the results are doing something useful, then the door is open!
|
|
11-16-2004, 12:00 PM
|
||||
|
||||
|
Hi
Well the results are sent to them to help build www.wisenut.com and indirectly www.looksmart.com
By helping them crawl the web there is an option that you can crawl your own websites. (a little payment for my time.)
|
|
|
| Thread Tools | |
| Display Modes | |
|
|
|
WebProWorld |
Advertise |
Contact Us |
About |
Forum Rules |
MVP's |
Archive |
Newsletter Archive |
Top |
WebProNews
WebProWorld is an iEntry, Inc. ® site - © 2009 All Rights Reserved Privacy Policy and Legal iEntry, Inc. 2549 Richmond Rd. Lexington KY, 40509 |
Linear Mode
