http://www.mnogosearch.com/
It comes with an excellent spider written in C so it's very fast. It has front ends for perl and PHP. It is incredibly configurable. It is free, except if you want to use the Window$ version.
Two drawbacks: you need to be able to compile stuff on your server, and the front-end template system is not the easiest thing in the world to figure out. I have a working example here:
search among local websites in Parry Sound.
If you only want to search among entries in your own directory, you're better to write a quick search script yourself.