free internal site search code ?

[anandnadaar]

Can anyone provide with a free internal site search code that does not use DB and indexes the site. Maybe I am would pay a few bugs for it. I have tried couple of free ones available but they did not work.
Can anybody please ?

[arvana]

You can try Swish-e (it's open-source software):

http://www.swish-e.org/

I haven't used it myself, but it looks very good. I'd like to hear if it works for you!

[davebarnes]

www.freefind.com

I use it in paid mode for 15+ sites.
Use the free version on 2 websites.

[pagetta]

we use the Fluid Dynamics Search Engine. $45 on off fee for the licence. you can see it at work on all of our sites.

http://www.xav.com/scripts/search/

[Vectorman211]

Very good info guys! Bumped to sticky :D

[alextj]

I can second Pagetta's recommendation of Fluid Dynamics - it has been working brilliantly for us and can handle re-directs, PDFs and DOCs with a plug-in, as well as opening with search terms highlighted. Definitely recommended.
Take care

[hawkwind dave]

great post's people!

after spending a hour or two trawling through the web, and searching different forums for some site search scripts, I find just what i need in a nice handy sticky (good spot vectorman211)

dave

[kgun]

I wrote this thread:

Google site operator and forum site search

before I read this post. It is explained how to implement a simple site serach function in the XML book mentioned in my first post. The code follows with the book.

If you structure your site very well, you can make a very efficient site search engine in my view. It is not so very difficult to modify the code to your own needs. That site search function will retrive content by:

• keywords,
• titles,
• and description

and display those pieces that have a status of live.

It is a programming task to retrive content by n features (eg. elements and attributes). The method is there.

[kgun]

Here is a related thread that goes into more detail:

An XML powered site search engine.

[Dubbya]

I get asked about this all the time and finally came across ZoomSearch a few weeks ago. After trying it out, I've decided to include it in a new site I'm working on. The free solution is perfect for little sites that don't have big budgets but still want a simple search feature.

ZoomSearch is easy to set up and free for small sites up to 50 pages. The "Standard" version will index 100 pages for $49 US, the "Professional" solution is only $99 US and will search up to 100,000 pages. The unlimited "Enterprise" license is only $299 US.

You run their software from your desktop and specify file types and locations. The tool indexes your site and compiles a few files. You then upload the files to your server and you're good to go.

The search box, extended search and results pages are simple to integrate with your site too.

WrenSoft Website Search Engine Software

Enjoy.

[kgun]

But the important question is what the SE index and find. Does it scan every page in your site for:

"bug"

" bug "

for example?

The code I mention above is so simple that there should be no problem to modify it to find whatever you are looking for in your documents, especially if you have structured your documents and files well.

If you buy an engine, it scans documents based on another programmers chosen criteria. If you program it yourself, it scans on what you specify. Not least for large companies that make their own XML meta language this may be very important. It can be made very effecient choosing the correct tags and attributes.

How many have used engines like freefind, yahoo, google etc. sitesearch on their sites and searched for words you know are in your documents without finding them? The reason is that the engine does not scan the complete document as a string and match the word(s) you specify.

There is no theoretic limit on the number of documents that can be scanned. The engine goes in a loop using XMLReader or another parser, loads one and one file into an object /variable, whose tags and attributs are scanned for the chosen criteria.

Here is the most important and difficult part of the code:

$handle = opendir($fileDir);
 $items = array();
 while (($file = readdir($handle)) !== FALSE) {
  if (is_dir($fileDir . $file)) continue;  
  if (!eregi("^(news|article|webcopy).*\.xml$", $file)) continue;
      $xmlItem = simplexml_load_file($fileDir . $file);
       if ((stripos($xmlItem->keywords, $term) !== FALSE or
       stripos($xmlItem->headline, $term) !== FALSE or
       stripos($xmlItem->description, $term) !== FALSE) and
       (string)$xmlItem->status == 'live') {
        $item = array();
   $item['id'] = (string)$xmlItem['id'];
   $item['headline'] = (string)$xmlItem->headline;
   $items[] = $item;
   } //The if test ends here.
 }  // The while loop ends here.

Note that $items is an array of $item arrays. The part in red is the part you can modify to your own needs. As you will see, this special engine only scans documents with a status of live. You can choose whatever you want. simplexml_load_file loads the document into a variable that may be scanned as a string. For example "stripos($xmlItem->headline, $term)" scans the document for headlines that contains the $term you choose. Stripos automatically casts the first argument in the function to a string. That is not done in the last read line, where you have to cast the $xmlItem->status object to a string. Casting is everyday work in OOP.

If you looked up the two related posts at the W3 Schools forum, you may have noted that my signature there is:

$MyProfile = simplexml_load_file(myprofile.xml);
echo $MyProfile->name;
:: Kjell Gunnar Bleivik
echo $MyProfile->xpath('/personalinfo/profile[@profileID=W3Schools]');
:: Why learn the bad dialect of HTML when you get it free when tagging XML?
Learn to tag with XML, program with XSL, then improve using Ruby on Rails, PHP...
Later you may learn to fly using BETA.
I liked the Borland C++ Builder with the possiblity to use inline ASM {statements, but I am an economist}.
Overall principle, make it simple, as simple as possible but no simpler.

as of 30. august 2007. It is when you combine the parsers with XPath (red line above), XPointer, XLink, XSL(T) the power of XML comes to its right.

[kgun]

If there is something called a web 2.0 site search engine / function, that is especially well suited for XML and its family of techologies and XML parsers. There should be no problem for a clever programmer that knows:

1. XML XPath XPointer and XLink
2. PHP or another server side language with good XML Parsers.
3. AJAX.

to build a fairly advanced site search engine with Google suggest functionality. May be there are one already.

You find more information in my Web 2.0 static link collection of resources. Especially the links with anchor text:

• Improving Web linking using XLink.
• XML linking language.

These technologies make it possible to have dynamic and generic links, multi-source, multi-destination links and much much more. Using link bases, location sets, archs and assiciations between these sets in your XML documents, the search engine may be fairly advanced and flexible.

Here is a link base example:

"Example: Annotating a Specification

Following is a non-normative set of declarations for an extended link that specializes in providing linkbase arcs:

<!ELEMENT basesloaded ((startrsrc|linkbase|load)*)>
<!ATTLIST basesloaded  xlink:type      (extended)      #FIXED "extended">
<!ELEMENT startrsrc EMPTY>
<!ATTLIST startrsrc  xlink:type      (locator)       #FIXED "locator"  xlink:href      CDATA           #REQUIRED  xlink:label     NMTOKEN         #IMPLIED>
<!ELEMENT linkbase EMPTY>
<!ATTLIST linkbase  xlink:type      (locator)       #FIXED "locator"  xlink:href      CDATA           #REQUIRED  xlink:label     NMTOKEN         #IMPLIED>
<!ELEMENT load EMPTY>
<!ATTLIST load  xlink:type      (arc)           #FIXED "arc"  xlink:arcrole   CDATA           #FIXED "http://www.w3.org/1999/xlink/properties/linkbase"  xlink:actuate   (onLoad                  |onRequest                  |other                  |none)          #IMPLIED  xlink:from      NMTOKEN         #IMPLIED  xlink:to        NMTOKEN         #IMPLIED>

Following is how an XML element using these declarations might look. This would indicate that when a specification document is loaded, a linkbase full of annotations to it should automatically be loaded as well, possibly necessitating re-rendering of the entire specification document to reveal any regions within it that serve as starting resources in the links found in the linkbase.

<basesloaded>
 <startrsrc xlink:label="spec" xlink:href="spec.xml" /> 
 <linkbase xlink:label="linkbase" xlink:href="linkbase.xml" /> 
 <load xlink:from="spec" xlink:to="linkbase" actuate="onLoad" />
</basesloaded>

Following is how an XML element using these declarations might look if the linkbase loading were on request. This time, the starting resource consists of the words "Click here to reveal annotations." If the starting resource were the entire document as in the example above, a reasonable behavior for allowing a user to actuate traversal would be a confirmation dialog box".

<basesloaded>  
<startrsrc    xlink:label="spec"    xlink:href="spec.xml#string-range(//*,'Click here to reveal annotations.')" /> 
 <linkbase xlink:label="linkbase" xlink:href="linkbase.xml" />  
<load xlink:from="spec" xlink:to="linkbase" actuate="onRequest" />
</basesloaded>

Source: ML Linking Language (XLink) Version 1.0

For those that only know HTML, there should be nothing revolutionary in the above markup. The difficult part is to learn the new markup languages and the important concept of XML name spaces that you use to bind different resources. Note that Internationalized Resource Identifiers (IRIs) is a generalization of an URI that is an generalization of an URL.

So to sum up. To make a very efficient Web 2.0 XML powered site search engine with AJAX functionality (like Google suggest) the technology is there already. It is very important to think thoroughly when you structure your XML (CMS) site. Clever and smart use of tags, nodes, attributes, link bases and location sets etc. etc. may make your site search engine stand out. If you take the time to write one based on the above ideas, please cite this source and give me an example for free. Google suggest (AJAX) functionality will be much appreciated. Preferrably, use PHP paresers to make the code compact. Use streaming parsers like XMLReader to make it efficient.

[mamola]

Hi All,

I wrote my own internal site search for the website i work on, derived from the project i worked on for my Msc. I used a ranking based on natural language and stem formation of words. (For example if someone writes reporting, reported or report these would all be converted to their stem form which is 'report'). I could then compare words easily when i then set a reference within a database table to each page on my site including reference to the key-phrases on each page.

This method allows me to rank pages based on what and how i wanted them to rank and appear - and of course cause i built it it's free - woo-hoo! The only bummer is i havent written anything to automattically index new pages so i have to update the external page search params via mssql. If anyone wants to know anymore/access the source drop me a line.

cheers

mamola

[kgun]

Should like to see the code.

[mikmik]

Originally Posted by kgun

If there is something called a web 2.0 site search engine / function, that is especially well suited for XML and its family of techologies and XML parsers. There should be no problem for a clever programmer that knows:

1. XML XPath XPointer and XLink
2. PHP or another server side language with good XML Parsers.
3. AJAX.

to build a fairly advanced site search engine with Google suggest functionality. May be there are one already.

You find more information in my Web 2.0 static link collection of resources. Especially the links with anchor text:

• Improving Web linking using XLink.
• XML linking language.

These technologies make it possible to have dynamic and generic links, multi-source, multi-destination links and much much more. Using link bases, location sets, archs and assiciations between these sets in your XML documents, the search engine may be fairly advanced and flexible.

Here is a link base example:

"Example: Annotating a Specification

Following is a non-normative set of declarations for an extended link that specializes in providing linkbase arcs:

<!ELEMENT basesloaded ((startrsrc|linkbase|load)*)>
<!ATTLIST basesloaded  xlink:type      (extended)      #FIXED "extended">
<!ELEMENT startrsrc EMPTY>
<!ATTLIST startrsrc  xlink:type      (locator)       #FIXED "locator"  xlink:href      CDATA           #REQUIRED  xlink:label     NMTOKEN         #IMPLIED>
<!ELEMENT linkbase EMPTY>
<!ATTLIST linkbase  xlink:type      (locator)       #FIXED "locator"  xlink:href      CDATA           #REQUIRED  xlink:label     NMTOKEN         #IMPLIED>
<!ELEMENT load EMPTY>
<!ATTLIST load  xlink:type      (arc)           #FIXED "arc"  xlink:arcrole   CDATA           #FIXED "http://www.w3.org/1999/xlink/properties/linkbase"  xlink:actuate   (onLoad                  |onRequest                  |other                  |none)          #IMPLIED  xlink:from      NMTOKEN         #IMPLIED  xlink:to        NMTOKEN         #IMPLIED>

Following is how an XML element using these declarations might look. This would indicate that when a specification document is loaded, a linkbase full of annotations to it should automatically be loaded as well, possibly necessitating re-rendering of the entire specification document to reveal any regions within it that serve as starting resources in the links found in the linkbase.

<basesloaded>
 <startrsrc xlink:label="spec" xlink:href="spec.xml" /> 
 <linkbase xlink:label="linkbase" xlink:href="linkbase.xml" /> 
 <load xlink:from="spec" xlink:to="linkbase" actuate="onLoad" />
</basesloaded>

Following is how an XML element using these declarations might look if the linkbase loading were on request. This time, the starting resource consists of the words "Click here to reveal annotations." If the starting resource were the entire document as in the example above, a reasonable behavior for allowing a user to actuate traversal would be a confirmation dialog box".

<basesloaded>  
<startrsrc    xlink:label="spec"    xlink:href="spec.xml#string-range(//*,'Click here to reveal annotations.')" /> 
 <linkbase xlink:label="linkbase" xlink:href="linkbase.xml" />  
<load xlink:from="spec" xlink:to="linkbase" actuate="onRequest" />
</basesloaded>

Source: ML Linking Language (XLink) Version 1.0

For those that only know HTML, there should be nothing revolutionary in the above markup. The difficult part is to learn the new markup languages and the important concept of XML name spaces that you use to bind different resources. Note that Internationalized Resource Identifiers (IRIs) is a generalization of an URI that is an generalization of an URL.

So to sum up. To make a very efficient Web 2.0 XML powered site search engine with AJAX functionality (like Google suggest) the technology is there already. It is very important to think thoroughly when you structure your XML (CMS) site. Clever and smart use of tags, nodes, attributes, link bases and location sets etc. etc. may make your site search engine stand out. If you take the time to write one based on the above ideas, please cite this source and give me an example for free. Google suggest (AJAX) functionality will be much appreciated. Preferrably, use PHP paresers to make the code compact. Use streaming parsers like XMLReader to make it efficient.

This is incredible. i have to learn xlink and xforms, but I use ajax and of course the basics of xslt and schemas (and dtd).
I am getting a new website this weekend, with JSP. This is xml at it`s most long developed api.
Dynamic search responses are excellent using javascript, and xml files.

[letraspedia]

thanks guys.
i like this site because i get usefull infos for me

[mikmik]

Fluid Dynamics, written in PERL, is the best I have ever used.

[kgun]

I am sure there is an exceedingly fast out there written in todays "assember", C++ where my preferred platform is the platform just above ASP.net on this site.

<digression>
Why is Russian Academy of Sciences ranked just above Havard on that site? The reason is the following told by one of my professors (that I rely on) of mathematics.

1. An American mathematican (don't ask me whome and about what) claimed that he had proved an unproved mathematical "theorem", that was 400 pages long.
2. In the USA they laughed at him.
3. He travelled to Russia and held a lecture on his proof.
4. The audience was silent listening for days.
5. When he finished, they applauded.
6. Even if rocket scientists some times crash out of orbit, mathematics is very important in economics and finance.

Fluid dynamics is solution of partial differential equations (Conservation laws, used in the oil industry are parabolic as far as I remember. Are those used in flued dynamics elliptical? They can be discontionous too, if the discontinuity is not too wild.). The most advanced are non-linear where differntial operators are defined on distributions like the Dirac Delta function or more genereally, generalized functions (scroll down to the lecture by Michael Oberguggenberger, with the title ""Nonlinear SDEs: Colombeau solutions and pathwise limits")

Evolutin is not always continuos, but may evolve in steps, known from Genetic algorithms as mutation.

This is not linear or continous reality. (look in the right column).

Reccomended deeper search.

Today I got an idea,

Project for a student.

Write a book with the following title: "From tagging, via XSL rule based programming, procedural and object oriented programming in C++ to distributed object programming and patterns in Beta".

There are some treads in the C++ sub forum of my forum, that shows how compact, generic and flexible C++ is.

This is definitely not W3 Schools stuff at present.

P.S. Is Beta still ahead of its time, or are there some true pattern based languages out there.
</digression>

Note: Perl and PHP are scripring languages. PPH is interpreted. I do not know Perl, so that may be compiled machine code on the server.

The version of Borland C++ that I have, could combine C++ code with inline assembler code. I once saw an example of a fractal programmed in C (that may be faster than C++ because of overhead in C++) and the same program written in assembler (you may use conditional compilation to solve probemes with various assemblers). The assembler version was 100 times faster.

H. W. Stockman (Sept-Oct 1988 : "Fast fractals" Micro Cornucopia #43 page 22 - 29.

<cite>
This is one of our trick articles. The fractals are really just a sneaky way to get our attention. After all, how many of you would read an article about speeding up 386 software by a factor of 100? (One second thought ...)
</cite>

Fast computers or fast code or both?

Example chess program written in Borland C++ Builder in 1995. Click on the link "sjakkprogram".

mikmik, do you beat that program 10 of 100 times when the computer has 5 seconds to take the next move, you are definitely a better chess player than me. I am an unhappy amateur

Pm me the code / link if you find one in C++

[canzoni32]

Ok guys, nice to see this post, but:
Is that code working for non databse search engine?
I have all my pages static on the server / folders / etc.
Can i use it?

[johnxuster]

"If you structure your site very well, you can make a very efficient site search engine in my view. It is not so very difficult to modify the code to your own needs. That site search function will retrive content by:

• keywords,
• titles,
• and description

and display those pieces that "-that's good idea

[Anamika]

Originally Posted by arvana

You can try Swish-e (it's open-source software):

Swish-e :: Home Page

I haven't used it myself, but it looks very good. I'd like to hear if it works for you!

Hello Arvana,
I used your given url it's not work properly . Please suggest me the another site search engine code.

[designtodevelop]

Can someone pls suggest simple script for an internal site search without any database?

[cpbhagtani]

Hi, I have simple javascript to search static pages in a website

var item = new Array();

/* Enter the information of your static pages in item array as each entry
    following are the 2 sample entries I have made.
    The entries should contain
    item[c] = new Array("Page Name","path","Page Title","Many,Key,Words","Descriptive Comments")
*/
   
c=0; item[c]=new Array("index.htm","","info - Everything you ever wanted to know about Sikkim","homepage","info, Everything you ever wanted to know about Sikkim.");

c++; item[c]=new Array("maps.htm","","Maps","content,main,focus","maps of Sikkim.");

page="<html><head><title>Search Results</title><link href='somecss.css' rel=stylesheet type=text/css></head><body bgcolor='white'><span style='background-color: #000000'><p><b><center><font color='#FFFFFF' face='Tahoma' size='2'>&nbsp; Search Result </center></b></p></font></span><center><font face='verdana' size='2'><table border=0 cellspacing=10 width=80%>";

/* search is the function to search contents in website */
function search(frm) 
{
    if(frm.srchval.value=="" )
         alert("please enter the search word");
    else
    {
        win = window.open("","","width=400,height=340,left=200,top=80,scrollbars");
        win.document.write(page);
        txt = frm.srchval.value.split(" ");
        fnd = new Array(); total=0;
        for (i = 0; i < item.length; i++) 
        {
            fnd[i] = 0; order = new Array(0, 4, 2, 3);    
            for (j = 0; j < order.length; j++)
            for (k = 0; k < txt.length; k++)
            if ((item[i][order[j]].indexOf(txt[k]) > -1 || item[i][order[j]].toLowerCase().indexOf(txt[k]) > -1 || item[i][order[j]].toUpperCase().indexOf(txt[k]) > -1) && txt[k] != "")
            fnd[i] += (j+1);
        }
    
        for (i = 0; i < fnd.length; i++) 
        {
            n = 0; w = -1;
            for (j = 0;j < fnd.length; j++)
            if (fnd[j] > n) { n = fnd[j]; w = j; };
            if (w > -1) total += show(w, win, n);
            fnd[w    ] = 0;
        }

        win.document.write("</table><b><hr>Total found:</b> "+total+"<br></font></body></html>");
        win.document.close();
    }
}

function show(which,wind,num) 
{
    link = item[which][0] ;
    line = "<tr><td><a href='" + link  + "'  target= '_blank' >" + "<b>" + item[which][2]  + "</b></a> " +"<br>" ;
    line += "<font face='verdana' size='1'>" +item[which][3] + "</font><br>"+"</td></tr>";
    wind.document.write(line);
    return 1;
}

I think it helps

[sushil]

Fluid Dynamics Search Engine

[cvos]

this provides some good help.

is there an opinion of hosted vs. owning code for site search? my preference is hosted.

these seem like good options

picosearch.com
fusionbot.com
swish-e.org/
xav.com/scripts/search/

[mikmik]

Fluid Dynamics
Fluid Dynamics Search Engine

the best I ever used, ever

[cvos]

Originally Posted by mikmik

Fluid Dynamics


the best I ever used, ever

thats high praise - do you know if it works with dynamic sites?