Upgrade search on your site with Google Custom Search Engine

[danlefree]

Warning: This will turn into a series of posts, though I should (hopefully) be able to update this post and keep an index of the contents.

• 1.0 Sign up for a Google Custom Search Engine
• 2.0 Integrate the Google Custom Search Engine code with your CMS
• 2.1 Basic Integration (Javascript Required)
• 2.1.1 wordpress Integration
• 2.1.2 MediaWiki Integration
• 2.1.3 vBulletin Integration
• 2.2 Advanced Integration (No Javascript Required)

I noticed the free internal site search code ? sticky on this subforum and I figured I would share some of the Google CSE implementations which have worked for me in the past.

Let's face it, the search functionality packaged with most content management systems (I'm looking at you - MediaWiki, wordpress, and vBulletin) doesn't always live up to the expectations set by Google - AKA the de facto search destination/inevitable ruler of the known universe.

If you have a lot of content and a CMS that isn't so great for search - and you don't mind the fact that nothing is really, truly free when you use Google - here are the blueprints to sacrifice some autonomy (and possibly $$$) for the sake of your site's users:

[danlefree]

1.0 Sign up for a Google Custom Search Engine

1.1 Log in to your Google/Gmail account and go to Google Custom Search Engine - Site search and more

1.2 Click Create a Custom Search Engine

1.3 Complete the Basic information fields as desired

1.4 Select Only sites I select from the What do you want to search? options

1.5 In the Sites to search field, enter the URL's, domains, and subdomains you would like the custom search engine to index or enter wildcards to indicate that all URL's on all subdomains should be included

Examples

domain.com/index.htm
domain.com/*
sub.domain.com/*
*.domain.com/*

1.6 Select whether you wish to use the "free" option (ads will display with results) or the Business edition ($100/year) of Google Custom Search Engine

1.7 Finalize the setup process and you will arrive at the My search engines page - follow the control panel link to manage settings for your custom search engine

1.8 Click the Look and feel option from the left-hand navigation menu and select Iframe under Choose a hosting option

1.9 Update and customize the search results style to your preferences, then click Save to store your customizations

1.10 Click Get Code and save a copy of the Search box code and the Search results code for use on your site


2.0 Integrate the Google Custom Search Engine code with your CMS

Your Google Custom Search Engine relies upon two blocks of code for integration into your site:

• Search box code - Draws a form and calls a Javascript file which will draw the Google logo into the form (amongst other things)
• Search results code - Draws the search results IFRAME (amongst other things)

2.1 Basic Integration (Javascript Required)

You can copy the search box code into your CMS template so it appears on every page and create a new page devoted to the search results code (follow the instructions at the Google Custom Search Engine > Control Panel > Get code page) and you'll have a working custom search engine for site visitors who have Javascript enabled.

For the purposes of the examples below, the code Google provides isn't exactly ideal (it doesn't provide a Javascript-free search method) - I'll be working with modified code blocks, so wherever you see the following lines you will want to replace them with the similar lines from your code blocks:

Search box code

<input type="hidden" name="cx" value="000000000000000000000:abcd_efghi" />
<input type="hidden" name="cof" value="FORID:10" />

[danlefree]

2.1.1 wordpress Integration

The wordpress plugin - Google Custom Search Plugin by Aleem Bawany - probably works... but I'm not a big fan of relying upon wordpress plugins, particularly for simple features which should never need an update (even if this feature does, it'll be a matter of copy-and-paste).

We'll be keeping the default wordpress search behavior for people who have Javascript disabled (no harm in that) - you should use FTP to complete this process (you will be uploading a new file if you are not working with a legacy wordpress theme), otherwise you can use the wordpress control panel under Appearance > Editor

The instructions below will assume a default wordpress 2.9.1 installation.


2.1.1.1 searchform.php

wordpress's get_search_form() function will generate your search form unless a file named searchform.php is located in your /wp-content/themes/<theme name> directory - so let's create that file:

<!-- Google CSE Search Box -->
<form action="http://domain.com/uri/" id="cse-search-box">
  <div id="googleCSE" style="display:none;">
    <input type="hidden" name="cx" value="000000000000000000000:abcd_efghi" />
    <input type="hidden" name="cof" value="FORID:10" />
    <input type="hidden" name="ie" value="UTF-8" />
    <input type="text" name="q" size="31" />
    <input type="submit" name="sa" value="Search" />
  </div>
</form>
<script type="text/javascript" src="http://www.google.com/cse/brand?form=cse-search-box&lang=en"></script>
<script type="text/javascript">
/*
    Javascript is enabled: Display the Google search form
*/
if ( document.getElementById('googleCSE') ) {
    document.getElementById('googleCSE').style.display = 'block';
}
</script>
<!-- EOF Google CSE Search Box -->

<noscript>
<!-- Legacy Search Box -->
<form method="get" id="searchform" action="<?php bloginfo('url'); ?>/">
<div><input type="text" value="<?php the_search_query(); ?>" name="s" id="s" />
<input type="submit" id="searchsubmit" value="Search" />
</div>
</form>
<!-- EOF Legacy Search Box -->
</noscript>

2.1.1.2 search.php

This file should exist with any wordpress theme - we'll be editing it to detect whether the Google search or the legacy wordpress search box was used and display results accordingly:

<?php
/**
 * @package wordpress
 * @subpackage Default_Theme
 */

get_header(); ?>

    <div id="content" class="narrowcolumn" role="main">

<?php
    /*
        Wrap search results in a conditional
        and detect whether to display Google CSE
        or legacy WP search
    */
    if ( $_GET['s'] ) {
?>
    <?php if (have_posts()) : ?>
        <h2 class="pagetitle">Search Results</h2>
        <div class="navigation">
            <div class="alignleft"><?php next_posts_link('&laquo; Older Entries') ?></div>
            <div class="alignright"><?php previous_posts_link('Newer Entries &raquo;') ?></div>
        </div>
        <?php while (have_posts()) : the_post(); ?>
            <div <?php post_class() ?>>
                <h3 id="post-<?php the_ID(); ?>"><a href="<?php the_permalink() ?>" rel="bookmark" title="Permanent Link to <?php the_title_attribute(); ?>"><?php the_title(); ?></a></h3>
                <small><?php the_time('l, F jS, Y') ?></small>
                <p class="postmetadata"><?php the_tags('Tags: ', ', ', '<br />'); ?> Posted in <?php the_category(', ') ?> | <?php edit_post_link('Edit', '', ' | '); ?>  <?php comments_popup_link('No Comments »', '1 Comment »', '% Comments »'); ?></p>
            </div>
        <?php endwhile; ?>
        <div class="navigation">
            <div class="alignleft"><?php next_posts_link('&laquo; Older Entries') ?></div>
            <div class="alignright"><?php previous_posts_link('Newer Entries &raquo;') ?></div>
        </div>
    <?php else : ?>
        <h2 class="center">No posts found. Try a different search?</h2>
        <?php get_search_form(); ?>
    <?php endif; ?>
<?php
    } else {
?>
<!-- Google CSE Search Results -->
<div id="cse-search-results"></div>
<script type="text/javascript">
  var googleSearchIframeName = "cse-search-results";
  var googleSearchFormName = "cse-search-box";
  var googleSearchFrameWidth = 600;
  var googleSearchDomain = "www.google.com";
  var googleSearchPath = "/cse";
</script>
<script type="text/javascript" src="http://www.google.com/afsonline/show_afs_search.js"></script>
<!-- EOF Google CSE Search Results -->
<?php
    } // EOC: if ( $_GET['s'] )
?>
    </div>

<?php get_sidebar(); ?>

<?php get_footer(); ?>

2.1.1.3 Test

Test your configuration (with and without Javascript enabled) to ensure everything is in working order.

[kgun]

This is excellent information. Hopefully we don't see code repitition i further threads / post.

So if it is possible for you

1. DRY = Don't Repeat Yourself, by trying to identify design patterns.
2. Lift duplicate code to the interface and / or abstract class level.
3. Reuse code by putting code in interfaces, abstract classes and concrete classes where the code is implemented via inheritance and object overloading.

[danlefree]

So if it is possible for you

1. DRY = Don't Repeat Yourself, by trying to identify design patterns.
2. Lift duplicate code to the interface and / or abstract class level.
3. Reuse code by putting code in interfaces, abstract classes and concrete classes where the code is implemented via inheritance and object overloading.

Changes at the class level would require modification to the core codebase or the development of plugins, otherwise the modifications will be lost on the next upgrade.

Links to plugins and extensions are provided (where they exist) - I am not a fan of them but that shouldn't stop anyone else from using them if they prefer.

I wouldn't try to convince an open source development community to include reliance upon a commercial entity in their core codebase and, likewise, I wouldn't trust a lone developer or small development house to provide me with secure, reliable code so I am going with the simplest solutions and including explanations in the interest of providing like-minded users with a starting point for their own implementations.

[danlefree]

2.1.2 MediaWiki Integration

I've long been a fan of MediaWiki - it has all the features I need, a strong track record as with Wikipedia.org, and (with enough tweaking) it makes for the perfect CMS for clients who want to handle their own updates but are not particularly proficient with HTML.

There are a variety of ways to go about adding a custom search engine to MediaWiki and, while the Extension:Google Custom Search Engine method recommended by User:BiGreat does appear to be perfectly sufficient (the better choice if you are using your MediaWiki installation as an actual wiki), I prefer to keep all modifications in the MediaWiki skin so I don't have to worry about exporting pages to preserve functionality.

Note: This example assumes that you (a) have MediaWiki installed under the /w/ directory in your web root and (b) are using mod_rewrite directives to support the $wgArticlePath = '/wiki/$1' setting.


2.1.2.1 MonoBook.php

The /w/skins/MonoBook.php skin is the default for MediaWiki - it makes a for a good working example, however, you will probably have your own skin in place if you are using MediaWiki as your CMS.

Search box - Can be placed wherever you would prefer - I'm adding it at the top of the sidebar and hiding the legacy search box with CSS:

Replace:

		<div id="column-one">
	<div id="p-cactions" class="portlet">

... with:

		<div id="column-one">
	<div id="p-googleCSE" class="portlet" style="display:none;">
	<!-- Google CSE Search Box -->
	<form action="http://domain.com/uri/" id="cse-search-box">
	  <div>
	    <input type="hidden" name="cx" value="000000000000000000000:abcd_efghi" />
	    <input type="hidden" name="cof" value="FORID:10" />
	    <input type="hidden" name="ie" value="UTF-8" />
	    <input type="text" name="q" size="31" />
	    <input type="submit" name="sa" value="Search" />
	  </div>
	</form>
	<script type="text/javascript" src="http://www.google.com/cse/brand?form=cse-search-box&lang=en"></script>
	<script type="text/javascript">
		/*
			Javascript is enabled: Display the Google search form
		*/
		if ( document.getElementById('p-googleCSE') ) {
			document.getElementById('p-googleCSE').style.display = 'block';
		}
	</script>
	<style type="text/css">
		/*
			Hide the legacy search box
		*/
		div#p-search {
			display:none;
		}
	</style>
	<!-- EOF Google CSE Search Box -->
	</div>
	<div id="p-cactions" class="portlet">

Search results - Nested in a conditional to avoid reliance on article creation

Replace:

	<div id="content">
		<a name="top" id="top"></a>
		<?php if($this->data['sitenotice']) { ?><div id="siteNotice"><?php $this->html('sitenotice') ?></div><?php } ?>
		<h1 id="firstHeading" class="firstHeading"><?php $this->data['displaytitle']!=""?$this->html('title'):$this->text('title') ?></h1>
		<div id="bodyContent">
			<h3 id="siteSub"><?php $this->msg('tagline') ?></h3>
			<div id="contentSub"><?php $this->html('subtitle') ?></div>
			<?php if($this->data['undelete']) { ?><div id="contentSub2"><?php     $this->html('undelete') ?></div><?php } ?>
			<?php if($this->data['newtalk'] ) { ?><div class="usermessage"><?php $this->html('newtalk')  ?></div><?php } ?>
			<?php if($this->data['showjumplinks']) { ?><div id="jump-to-nav"><?php $this->msg('jumpto') ?> <a href="#column-one"><?php $this->msg('jumptonavigation') ?></a>, <a href="#searchInput"><?php $this->msg('jumptosearch') ?></a></div><?php } ?>
			<!-- start content -->
			<?php $this->html('bodytext') ?>
			<?php if($this->data['catlinks']) { $this->html('catlinks'); } ?>
			<!-- end content -->
			<?php if($this->data['dataAfterContent']) { $this->html ('dataAfterContent'); } ?>
			<div class="visualClear"></div>
		</div>
	</div>

... with:

	<div id="content">
<?php
	if ( $_SERVER['REQUEST_URI'] != '/wiki/Search:Google' ) {
?>
		<a name="top" id="top"></a>
		<?php if($this->data['sitenotice']) { ?><div id="siteNotice"><?php $this->html('sitenotice') ?></div><?php } ?>
		<h1 id="firstHeading" class="firstHeading"><?php $this->data['displaytitle']!=""?$this->html('title'):$this->text('title') ?></h1>
		<div id="bodyContent">
			<h3 id="siteSub"><?php $this->msg('tagline') ?></h3>
			<div id="contentSub"><?php $this->html('subtitle') ?></div>
			<?php if($this->data['undelete']) { ?><div id="contentSub2"><?php     $this->html('undelete') ?></div><?php } ?>
			<?php if($this->data['newtalk'] ) { ?><div class="usermessage"><?php $this->html('newtalk')  ?></div><?php } ?>
			<?php if($this->data['showjumplinks']) { ?><div id="jump-to-nav"><?php $this->msg('jumpto') ?> <a href="#column-one"><?php $this->msg('jumptonavigation') ?></a>, <a href="#searchInput"><?php $this->msg('jumptosearch') ?></a></div><?php } ?>
			<!-- start content -->
			<?php $this->html('bodytext') ?>
			<?php if($this->data['catlinks']) { $this->html('catlinks'); } ?>
			<!-- end content -->
			<?php if($this->data['dataAfterContent']) { $this->html ('dataAfterContent'); } ?>
<?php
	} else {
?>
			<!-- Google CSE Search Results -->
			<h1 id="firstHeading" class="firstHeading">Search Results</h1>
			<div id="cse-search-results"></div>
			<script type="text/javascript">
			  var googleSearchIframeName = "cse-search-results";
			  var googleSearchFormName = "cse-search-box";
			  var googleSearchFrameWidth = 600;
			  var googleSearchDomain = "www.google.com";
			  var googleSearchPath = "/cse";
			</script>
			<script type="text/javascript" src="http://www.google.com/afsonline/show_afs_search.js"></script>
			<!-- EOF Google CSE Search Results -->
<?php
	}

			<div class="visualClear"></div>
		</div>
	</div>

[danlefree]

2.1.3 vBulletin Integration

Unlike wordpress and MediaWiki, vBulletin is a commercially-licensed product - the source is open for licensed customers, however, and there are a variety of modifications from the vBulletin user community.

The integrated vBulletin search features are generally adequate (and it can be very useful to have the option to select a specific user or subforum to search) - this example will simply add the custom search engine to the list of options without affecting the existing search options.

I am a fan of vBulletin's plugin system and I would recommend the Google Custom Search Engine Integration plugin by SunLab.de as a plugin, however, the same effect can be achieved with the vBulletin Style Manager and a custom PHP file uploaded to vBulletin directory.

Note: This example assumes a vBulletin 3.8.4 PL2 installation and access to the vBulletin.org forums (which will require an active vBulletin license). Code examples do not include vBulletin source, though it should not be hard to figure out where to put your code blocks.


2.1.3.1 Style Updates

1. From your forums' admin control panel, go to Styles & Templates > Style Manager and select Edit Templates
2. Select the Navigation / Breadcrumb Templates option > navbar
3. Insert the code (below) wherever you would like the search box to appear (typically under the "<!-- header quick search form -->" section)

		<tr>
		 <td class="thead">Google Site Search</td>
		</tr>
		<tr>
                 <td id="googleCSE" class="vbmenu_option" title="nohilite" style="text-align:center;">
               <!-- Google CSE Search Box Begins  -->
					<form action="http://domain.com/uri/" id="cse-search-box">
					  <div>
					    <input type="hidden" name="cx" value="000000000000000000000:abcd_efghi" />
					    <input type="hidden" name="cof" value="FORID:10" />
					    <input type="hidden" name="ie" value="UTF-8" />
					    <input type="text" name="q" size="31" />
					    <input type="submit" name="sa" value="Search" />
					  </div>
					</form>
					<script type="text/javascript" src="http://www.google.com/cse/brand?form=cse-search-box&lang=en"></script>
               <!-- Google CSE Search Box Ends -->
                 </td>
		</tr>

2.1.3.2 Custom Search Results Page

See the How to create your own vBulletin-powered page guide at vBulletin.org - you can insert the search results code into your existing template (this method is preferable as your search results page will always reflect the most recent template updates).


2.2 Advanced Integration (No Javascript Required)

Visitors without Javascript enabled will be presented with no search results unless you are defaulting back to your CMS search results (either way, they're not getting the Google results and that is far from ideal).

There is another way to integrate the search results with your site and it's not particularly tricky if you're familiar with programming for the web - but I'll leave it up to you to figure out how to void the warranty on your custom search engine.

[tangeray]

1.0 Sign up for a Google Custom Search Engine

1.1 Log in to your Google/Gmail account and go to Google Custom Search Engine - Site search and more

1.2 Click Create a Custom Search Engine

1.3 Complete the Basic information fields as desired

1.4 Select Only sites I select from the What do you want to search? options

1.5 In the Sites to search field, enter the URL's, domains, and subdomains you would like the custom search engine to index or enter wildcards to indicate that all URL's on all subdomains should be included

Examples

domain.com/index.htm
domain.com/*
sub.domain.com/*
*.domain.com/*

1.6 Select whether you wish to use the "free" option (ads will display with results) or the Business edition ($100/year) of Google Custom Search Engine

1.7 Finalize the setup process and you will arrive at the My search engines page - follow the control panel link to manage settings for your custom search engine

1.8 Click the Look and feel option from the left-hand navigation menu and select Iframe under Choose a hosting option

1.9 Update and customize the search results style to your preferences, then click Save to store your customizations

1.10 Click Get Code and save a copy of the Search box code and the Search results code for use on your site


2.0 Integrate the Google Custom Search Engine code with your CMS

Your Google Custom Search Engine relies upon two blocks of code for integration into your site:

• Search box code - Draws a form and calls a Javascript file which will draw the Google logo into the form (amongst other things)
• Search results code - Draws the search results IFRAME (amongst other things)

2.1 Basic Integration (Javascript Required)

You can copy the search box code into your CMS template so it appears on every page and create a new page devoted to the search results code (follow the instructions at the Google Custom Search Engine > Control Panel > Get code page) and you'll have a working custom search engine for site visitors who have Javascript enabled.

For the purposes of the examples below, the code Google provides isn't exactly ideal (it doesn't provide a Javascript-free search method) - I'll be working with modified code blocks, so wherever you see the following lines you will want to replace them with the similar lines from your code blocks:

Search box code

<input type="hidden" name="cx" value="000000000000000000000:abcd_efghi" />
<input type="hidden" name="cof" value="FORID:10" />

1.6 Select whether you wish to use the "free" option (ads will display with results) or the Business edition ($100/year) of Google Custom Search Engine

Can I earn revenue off of the ads Google displays on search results.

[danlefree]

1.6 Select whether you wish to use the "free" option (ads will display with results) or the Business edition ($100/year) of Google Custom Search Engine

Can I earn revenue off of the ads Google displays on search results.

Google CSE > My search engines > Control panel > Make money

If you have an existing AdSense account you can associate your custom search engine with it (or you can create a new AdSense account if you do not already have one).

There is a warning on the Custom Search Engine page which states that your custom search engine will be broken if you have an existing AdSense account but you attempt to create a new AdSense account (a somewhat surprising validation fail for Google).

[Clint1]

Dan will this give you an advanced site search and without the G logo anywhere? Does it have to be used with CMS or WP?