Crawler

[Allan12]

A crawler is also referred to as a robot or bot from a search engine. A crawler reads the content from a page and usually stores this in a search engines database, called an index. From this index the search results are extracted when one has entered a keyword in a search engine.

[nickoran]

A crawler is also referred to as a robot or bot from a search engine. A crawler reads the content from a page and usually stores this in a search engines database, called an index. From this index the search results are extracted when one has entered a keyword in a search engine.

What does this mean for those working in SEO and for the general public then?