I was poking around some website directories where I work, and found this robots txt file in one of the sites:
User-Agent: * Disallow: /
# No search engines allowed.
Doesn't this tell search engines to NOT crawl the site? Kind of strange, because the site holds a #2 position in Google for its targeted keyword. However, the Google listing has never included a title tag, description, or snippet of content -- just the url (which is what prompted me to investigate).