W3C has proposed a set of standards for web page coding. This is not their first version of the standards (publicly they are up to version 4.01) but it has been a slow process to attract sufficient interest in the wider internet community (i.e., beyond hobbyists, purists, and the most focused designers and
SEO people) so there are still a lot of pages, perhaps the majority of pages on the net, that do not conform to the standards and a lot of browsers that pay little attention.
I don't know, of course, but I would strongly doubt that spiders care much whether a site is W3C compliant. The relevant factors, where you seem to see a difference, are more likely to be recency and extent of changes to the site, which will bring back spiders more frequently, and in cases where W3C compliance produces cleaner and leaner code (which should be but isn't always the case) it may help the spiders to get to the relevant data they need before they get nored and take a coffee break (e.g., there are page size limitations beyond which the spiders won't read, although most pages won't get to that limit).