I think for most of us the site speed issue still isn't an issue to be hugely concerned about. I get the impression from Matts post that it's only going to be an issue for sites that are very slow. It's good to have the confirmation that this is now definitely a ranking feature but I think we need to do some more testing before we all run off and start changing our hosting etc.
If a client comes to me with a very slow site that is under performing then I'll address the issue and follow the excellent tips above but the most I've done so far is add WP-Supercache to my Wordpress clients sites. Am I behind the game? I just dont want to spend hours working to lower page load by amounts unnoticeable to the visitor.