What content does Google index?

[Global777]

When a GoogleBot visits a site, what code is actually being viewed and indexed? Is it the raw html, for example, or the html that is presented through a browser?

More to the point, are the bots indexing the css code, if the stylesheet is external, or what I would see if I did a View Source?

Thanks in advance...

[NateDesmond]

There is a tool called seo browser that will show you what the search engines see when they crawl your site. You can use it at Free SEO Software Tool & Text Browser, Search Engine Optimization Tools - SEO Browser.

[Global777]

Thanks Nate. That helps.

[erikko]

your tool Nate is amazing, i tested it and i love it, now i will now if a site used raw html

[incrediblehelp]

Another good one:

Rex Swain's HTTP Viewer

[Janna122003]

Nice tool buddy. thanks for sharing it.

[jabo]

Originally Posted by NateDesmond

There is a tool called seo browser that will show you what the search engines see when they crawl your site. You can use it at Free SEO Software Tool & Text Browser, Search Engine Optimization Tools - SEO Browser.

Great tool nate. Thanks for sharing that. I also found other useful tools in that site so it helped me alot..

[mit]

Originally Posted by incrediblehelp

Another good one:

Rex Swain's HTTP Viewer

Thanks for sharing the tool buddy...

[imvain2]

Google also suggests using the linx browser to see what the search engines see.

[wige]

It should probably be noted that the search engines do look beyond what the spider simulators indicate. These tools will remove the javascript, client side scripting and style sheets, which is helpful for estimating whether or not your content and navigation "work". However, the search engines do periodically check files such as your CSS stylesheets and the on page scripting to look for possible spamming.

Font sizes, the visibility of elements, color, and other factors are occasionally reviewed by the spiders, and while they may not directly impact the indexing of your site, they can cause sections of content to be ignored and can impact the ranking of a page. Most tools do not emulate this behavior, because no one outside the search engine engineers know exactly how these factors are determined or weighted. In fact, I have not seen any emulators that attempt to factor these elements into their analysis.

[Webnauts]

I do not know any better tool than this one: Browser Simulator/Emulator Tool, Web Page Tester, URL Source/Headers Viewer

[Peter (IMC)]

Originally Posted by Global777

When a GoogleBot visits a site, what code is actually being viewed and indexed? Is it the raw html, for example, or the html that is presented through a browser?

More to the point, are the bots indexing the css code, if the stylesheet is external, or what I would see if I did a View Source?

Thanks in advance...

Basically they index the html code and then process that data. For example they will extract all the text from the document, like those tools show. They will also look at what tags the various texts are in and the content of these various tags will be weighted differently. For example the content in the title tag has more weight.

But they´re smarter than that. They will also compare the content of various tags. For example, the page will be considered less useful (and thus rank lower) if the content of the title is unique and none of the words in it are found in the rest of the page.


It would be strange if it made a difference if the css is in the page or in a seperate file. If a simple browser can figure out where the css data is, a search engine should have no difficulty finding it too.

One thing that's important is that they do not execute code like a browser does.

And also realize that a bot is not the search engine it self. It doesn't do anything but grabbing data.

[benc007]

Great tools ... thank you for sharing.

[Global777]

Thanks to each of you for the information. I appreciate!

[2fk]

Originally Posted by Peter (IMC)

Basically they index the html code and then process that data. For example they will extract all the text from the document, like those tools show. They will also look at what tags the various texts are in and the content of these various tags will be weighted differently. For example the content in the title tag has more weight.

But they´re smarter than that. They will also compare the content of various tags. For example, the page will be considered less useful (and thus rank lower) if the content of the title is unique and none of the words in it are found in the rest of the page.


It would be strange if it made a difference if the css is in the page or in a seperate file. If a simple browser can figure out where the css data is, a search engine should have no difficulty finding it too.

One thing that's important is that they do not execute code like a browser does.

And also realize that a bot is not the search engine it self. It doesn't do anything but grabbing data.

Very understandable..very well explained... much better on the tools - learning the concept.. thanks peter

[JenKumar]

You can get your site spidered at SeoChat.com - they have many tools.

[sjk]

I use Web Sniffer quite often.