AJAX and SEO.

[kgun]

AJAX (Asynchronous JAvascript + Xml) or empovered JavaScript increases the functionality of a webpage. In a nutshell, AJAX is a mix of technologies that lets you get rid of the page reloads. AJAX is not new. The heart of AJAX is the JavaScript XMLHttpRequest object invented by Microsoft in 1999.

Because JavaScript is used when the client talks with the server, you have to fight with browser inconsistencies. AJAX-specific pages don't work on browsers without JavaScript, they can't easily be bookmarked and search engines don't always know to parse them.

The pages are not unique, since parts of them are updated in the background. That implies obvious problems for the SE bots. Yous should be aware of this since more and more companies use AJAX technologies on their web pages. XML is an important element in the AJAX technology and there are a lot of technologies that empovers XML. Some of them are:

• The document object model (DOM).
• Document Type Definition (DTD).
• Recommended DTDs to use in your Web document.
• XSL
• XSLT
• XML Path Language (XPath)
• XML Linking Language (XLink)
• XML Pointer Language (XPointer)

CSS is also an important element in the technology, even if it is not needed. A lot use PHP as server language and MySQL as database platform, but any server language and SQL database plattform may be used to create AJAX functionality.

[kgun]

Here are some relevant links:

AJAX and SEO

SEO Considerations for AJAX Development

Implementing AJAX? SEO and Accessibility Considerations

[kgun]

In the interview:
Matt Cutts Interview on Personalization and the Future of SEO

Matt Cutts also mentions AJAX and SEO.

Matt:
"I think there still is a place for you for a pure SEO consultant but it's also true that over time those consultants have to keep adding to their skill set. A few years ago no one would have even thought about the word Ajax and now people have to think about Ajax or Flash and how do I handle some of these new interfaces to still make sites crawlable? So I definitely think there will still be places for consulting and improving crawlability of sites and advice on keywords and personalization will add some wrinkles to that, but I have faith that, over time we’ll see the benefit to users and if you make good site for your users, you will naturally benefit as a result. Some people spend a lot of time looking at data centers and data center IP addresses and if people want to have that as a hobby they're welcome to it but a lot of people don't do that anymore and they're just worried about making good results and yet, everything still comes out pretty well for them".

Also note the start of the interview where Matt says:

"I think that it’s natural that some people would be worried about change, but some of the best SEO’s are the SEO’s that are able to adapt, that are able to look down the road 4 or 5 years and say, “What are the big trends going to be?” and adjust for those trends in advance, so that when a search engine does make a change which you think is inevitable or will eventually happen, they’ll be in a good position."

So I think Matt agrees with me that SEO is a moving target.

[WebGeek182]

Here's another article on AJAX and SEO.

I've noticed a lot of misconceptions about AJAX and SEO. It seems that 2 of the biggest problems are:

• No content or navigation on initial page load.
• Lack of unique URL's.

[kgun]

Originally Posted by WebGeek182

Here's another article on AJAX and SEO.

I've noticed a lot of misconceptions about AJAX and SEO. It seems that 2 of the biggest problems are:

• No content or navigation on initial page load.
• Lack of unique URL's.

There are older techniques using hidden frames (zero height and with) where browser history is maintained, examples: Gmail and Google maps.

For this reason some AJAX applications use a combination of hidden frames and the XMLHttpRequest object.

There is a third technology using iFrames. The main difference between a hidde fram and an iFrame is that an iFrame can be placed inside of a non-frameset HTML page. In that way any part of a page can become a frame. An iFrame can even be created on-the-fly in Javascript.

The main advantage with using the XMLHttpRequest object is that it is created for the purpose of asynchronous communication, is a newer and a much cleaner technology.

You also need to understand how XMLHttpRequest is supported in different browsers, especially IE, since in that browser it is based on ActiveX controls. If you are in a particular security zone that doesn't allow ActiveX controls, you cannot access the XMLHttpRequest object.

You should also know what is meant by the same origion policy, but that is another story.

Soruce:
Zakas, McPeak and Fawcett (2006 or later) : "Professional AJAX" ISBN: 0-471-77778-1

There is an AJAX library zXML where you can download code that can be implemented in your AJAX applications.

[dburdon]

Originally Posted by kgun

There is a third technology using iFrames. The main difference between a hidde fram and an iFrame is that an iFrame can be placed inside of a non-frameset HTML page. In that way any part of a page can become a frame. An iFrame can even be created on-the-fly in Javascript.

Kjell,

since you directed me here I shall ask you a question on the legitimacy of using iFrames. The sites I've seen show one thing to the human visitor and something else entirely to the search engines!!

[kgun]

Yes, I know what Google writes about cloacking and non unique sites etc.

Do you know, when Niklas Zennström, creator of Skype enters a conference room, CEO's of the large telecom companies shivers.

I don't think the CEO's of the SE companies are shivering when they hear about AJAX (more generally about web applications) since they were the first to use the technology. But one thing I am sure of is that web applications are the future, and SE's have to prepare for it.

Originally Posted by kgun

Example of AJAX technology in use:
CNN Money brings updated stock quotes without reloading the page.

Source: WPW thread: Ajax Puts Web Ad Sales in Peril

Should the CNNMoney site be downgraded by the SE's because it uses one ore another AJAX technology? Then there may be a disagreement between human surfers and SE Bots, since a human being may prefer a site with stock quotes that updates without refreshing the page.

There are very strong arguments for a financial, AJAX driven web site (application) with dynamic (even some seconds delayed) quotes to have higher rank than a similar static.

Note: In the beginning XML was a buzz word. Today it is a very important technology.

Theoretically, if stock quotes are in free fall, a static page could ruin you.

May be not important for a web master, but ask a trader.

[dburdon]

Kjell,

the context you quote regarding CNN is obviously bona fide. CNN are not trying to fool anybody.

However, I've seen iFrame based pages that show one thing to the search engine - keyword rich content - and another, more aesthestic page to the human visitor. The question is how the search engine establishes intent.

[kgun]

Originally Posted by dburdon

The question is how the search engine establishes intent.

David, yes with AJAX and more generally web applications (it is not important what the technology is called) that is very important.

The heart of the problem is that content are not unique any longer. The newest AJAX technology uses The XMLHttpRequest Object that is worse regarding page indexing than AJAX technology based on hidden (zero hight and width) frames. The hidden frames AJAX technology is based on asynchronous communication with the server taking place in the hidden frame and updating part of the page in (a) visul frame(s). Because the last technology is more SE friendly than that based on the modern XMLHttpRequest Object model, some companies like Google have as far as I know combined the two technologies in GMail and Google maps. Google suggest is an other AJAX based website. Yahoo news another.

Source:
Nicholas C. Zakas, Jeremy McPeak, Joe Fawcett (2007 second edition or later): "Professional Ajax."

Important related link: Zakas home page.

P.S.
I think the next generation forums will have AJAX technologies that remids you that a thread you posted to have been updated with a new post. That technology is described in the above mentioned book.

[WebGeek182]

The CNN stock quote example isn't the best example because the stock quotes have no relevance to SEO for that page, so by all means use AJAX. The trouble comes when developers place ALL the meaty content in AJAX so that if you have JavaScript turned off, there is no content. They should uses a server-side script to dynamically generate the initial content, and then use AJAX to manipulate it from there. Keep in mind it is important to us unique URL's as well. That can be a challenge as it's tempting to just have all the content on one page. This is similar to SEO challenges faced by Flash designers.

[kgun]

Originally Posted by WebGeek182

The CNN stock quote example isn't the best example because the stock quotes have no relevance to SEO for that page, so by all means use AJAX.

• Sometimes a thread can go off topic. That may be good or bad.
• An AJAX driven website has to take SEO into account. Read through this thread once more if you do not understand that.
• Yes, JS turned off may be a problem. It is estimated that around 10 % of web surfers have JS turned off. (Another group use very old browsers or text only browsers). A well formed AJAX site shall degrade gracefully if the surfers browser has no support for AJAX. That is why external AJAX libraries are important. You can divide this group into two. The advanced surfers that do not like JS and popups. AJAX was launced in 2005 as a technology to bridge the gap between web applications and desktop applications, by getting rid of the need for page reload. These surfers will soon pick up the AJAX technology and turn JS on. Whether web applications are driven by AJAX or another technology is not important. The important aspect is the asynchronous communication between the client and the server. With technologies like AJAX, there has to be a connection between browser, (DOM) and server scripting.
  
  The other less technical group will not know the effect of turning off JS. Future browsers may / shall have a warning when you turn off JS, that you may get problems with important web pages by turning JS off.
  
  So there is a great probability that this 10 % group will decrease absolutely and relatively.

[kgun]

Here is a related WPW thread with identical title and additional information.

[Orion]

... a bit late to the table on this one, but it was quoted in a recent post also..

if you use iframes you should also be using the <noframe> tag to provide content for the SE's

Another way to do it is to use <div> with a scroll option and <!--#include> the xml document into the <div>.

Doesn't that solve at least some of the issue? Then in some instances using js to hide / reveal the <div> doesn't mess with the SE's cause the code is still on the page.

Dunno if I'm using way out of date ideas there, but it still seems pretty valid..

[kgun]

Note: There are two (three) AJAX technologies, the iFrame (hidden frame) and the The XMLHttpRequest Object technology. The last technology is worst for indexing, since it gives nonunique pages.

It has been assumed that frames are bad for SEO. But, as far as I know, Google themself, has used a combination of the two technologies because the first is better at giving unique content. It gives the same result when you use the back button in the browser.

Why should it be problematic for Se BOT to crawl and index a framed page? Zero frames have been related to spamming the index. But a zero frame is an essential part of the framed AJAX technology, since the asynchronous communication takes place in the zero (width and height) frame while content is presented in visible frames.

It should be possible for a Se BOT to identify the different "AJAX" frames as "separate pages" and index them as such without using the noframe tag. It should also be possible for an advanced Bot to identify and analyze the content on the hidden frame and raise a red flag if it is identified as spamming the Bots index.

Conclusion:
I am unsure of how important the noframe tag is. Is there anything about this in the SE companies guidelines? It should be related to AJAX and SEO.

[Orion]

I have a Q.. if Ajax relies on frames how do you load data into it since target= is invalid.. or if you use Ajax do you just forgo strict validation for your site?

[kgun]

Originally Posted by Orion

I have a Q.. if Ajax relies on frames how do you load data into it since target= is invalid.. or if you use Ajax do you just forgo strict validation for your site?

My bolding.

When using the noframe tag? I have never used it.

1. AJAX does not rely on frames. I think most AJAX applications rely solely on the XMLHttpRequest object that give nonunique pages. So they do not worry about uniqueness and validation of this part of the site.
2. Hidden frames is an old technique. It is used by Gmail and Google Maps according to Professional Ajax first edition page 32.
   
   "One of the biggest arguments for using hidden frames is that you can maintain the browser history and this enable users to still use back and forward in the browser.
   
   ...........................
   
   This technique is used in both Gmail and Google Maps for this very reason.
   
   Be careful with iFrames, because iFrames don't always store browser history. Whereas IE always stores the history of iFrames, Firefox does so only if the iFrame was defined using HTML (that is not created dynamically using JS). Safari never stores browser history for iFrames, regardless of how they are included in the page".
   
   Since the title of the book is "Professinal AJAX" and sold by Amazon", I personally think that the authors know what they are talking about.
   
   Related link:
   JavaScript and history navigation in Opera.

[kgun]

I let the authors from the book "Professional Ajax" speak:

"Advantages and Distavantages of XMLHttp.

Undoubtedly, you can see the advantage of using XMLHttp for client-server communication instead of hidden frames. The code you write is much cleaner and the intent of the code much more apparant than using numerous callback functions with hidden frames. You have access to request and response headers as well as HTTP status code, enabling you to determine if your request was successful.

The downside is that, unlike hidden frames, there is no browser history record of the calls that were made. The back and forward buttons do not tie in to XMLHttp requests, so you have effectively cut off their use. It is for this reason that many AJAX applications use a mixture of XMLHttp and hidden frames to make a truly usable interface.

Another disadvantage, wich applies to Internet Explorer only, is that you depend on ActiveX controls being enabled. If the user has your page set up in a particular security zone that doesn't allow ActiveX controls, you cannot access the XMLHttp object. In that case, you may have to default to using hidden frames."

My bolding.

First edition page 43. There is a related problem, the same origon policy that you can read about in the book that I can highly reccommend. Get the last version and edition and download and use the zXml 1.0.2 library from Nicholas C. Zakas home page.

Note: Wrox has also published a book, Beginning Ajax with ASP.NET

[imsickofwebpro]

Simplified Version:

Ajax uses <Div>. It's great for SEO. It's like loading multiple pages of content into one page. Well organized for front end users, and easy to implement on the the back on end when using external classes. Eg. Spry!!!!
-------------------------------------------------------------------------------------------------------------------------------------
Spry:
<script src="SpryTabbedPanels.js" type="text/javascript"></script>
<link href="SpryTabbedPanels.css" rel="stylesheet" type="text/css" />

<div id="TabbedPanels1" class="TabbedPanels">
<ul class="TabbedPanelsTabGroup">
<li class="TabbedPanelsTab" tabindex="0">

PHP Code:

<?php echo $row[AJAXtitle]; ?>

</li> <!--TAB 1 Title-->
<li class="TabbedPanelsTab" tabindex="0">

PHP Code:

<?php echo $row[SPRYtitle]; ?>

</li> <!--TAB 2 Title-->
</ul>
<div class="TabbedPanelsContentGroup">
<div class="TabbedPanelsContent">

PHP Code:

<?php echo $row['AJAXcontent']; ?>

</div>
<div class="TabbedPanelsContent">

PHP Code:

<?php echo $row['SPRYcontent']; ?>

</div>
</div>
</div>
<script type="text/javascript">
<!--
var TabbedPanels1 = new Spry.Widget.TabbedPanels("TabbedPanels1");
//-->
</script>

-------------------------------------------------------------------------------------------------------------------
Don't use iframes. Once a SearchBot enters the iframe link, where does it go from there???

Argument Guy: It could follow anchor text.

Me: I like non-forced website navigation, and SearchBots are people too.

[adultcinezone]

A necessary first step in letting your AJAX application come up for a breath of SEO.