If mysql full text queries are taking that long it sounds like you've got something incorrectly configured. Did you create an full text index? Is your match using the identical columns to the full text index?
You could try sphider quite easily and see how that goes.
If not then you'd need to do some experimenting with Zend Frame work to see how the performance of search is. Indexing is just travelling though the pages extracting useful information from fluff and sending it to the engine, if it's all in MySQL then the extraction is already done for you.
You might get more speed by using C/C++, Java etc,
Apache Lucene - Overview is a search library for Java.