Oh that Googlebot Story
Googlebot taking random stabs at Atom/RDF files
I usually skim probably about 100 or so stories around the net a day ... never in one particular area either. One day I will be into what is happening with Google (which I try to avoid...sheesh) and the next I will be flipping thru some CSS blogs and tutorial sites. My focus wanders.
Mostly I write a couple of "readers digest" type of paragraphs. Which is what most of the stories are about anyway before they go off into the tail-end Sergio and Larry life story which consumes 80% of the article ... but I will leave a link for that for those readers who have not heard it for the 100th time yet.
Sometimes though, I will combine three or four seperate articles that I have been reading and try to put some more background to the story. That is what happened with the "googlebot story". That one in particular went beyond the initial story that was in the WPNews article that Garrett wrote about.
I use one newsreader called
Awasu which is a desktop application. You should give that one a whirl. I think you will like it. The other one I use is the
new service at My Yahoo which you can add RSS feeds to your personal pages (i have different pages broken down by category -- channels) and that is very convenient and works pretty well. Of course, I have mine in there as well and ping Yahoo about twice a day to pick up on it. Those pages are now listed in the search results with RSS feed next them now (kewl).
I saw your entry today (or was it yesterday) about your new program you are trying out -- the Directory Extractor. Here is another one that you may want to look at. I have been playing around with it a little bit and it is pretty kewl. Loop Improvements
Net Research Server which crawls, indexes, and searches websites. They have a free version for you play with too at
http://www.loopimprovements.com/download.html. It only stores 1000 documents, but it is enough to get a feel for what it does. I noticed that you only have around 125 or so pages on your site ... so you could probably use this to completely index it and have your own built in Search Engine for it. Just a thought.