View Single Post
  #8 (permalink)  
Old 04-05-2008, 08:14 PM
MursPlace MursPlace is offline
WebProWorld Member
 
Join Date: Jan 2008
Posts: 27
MursPlace RepRank 1
Default Re: MSN links--Thousand of pages linking from all different domains but seem the same

Quote:
Originally Posted by chandrika View Post
When you say tracking them, do you mean finding out where they originated from? If so...
No not really like tracking them down and stalking the bot to his house and cutting the power lines.

I think you are going with the honeypot of known bots while I was going to reverse the process and treat everything as a bot but good known User Agents (browsers).

If all holds true you should be spammed (scrapped) from unknown or new bots while I will be notified of bots not listed in the list. Between the two tests the results I get should match yours plus unknown but only Al Gore and his Internet really knows.

It's like Data Mining to see what bots visit your site. Block them all and alert yourself of their visit so you can add them to your "Good Bot List". They always return.
Reply With Quote