None of us can be sure that removing those links would fix the problem, since we don't know that adding those links caused the problems. Unless someone here was a Google Engineer responsible for specifically removing your site from the directory, we couldn't honestly answer that question, as none of us can know the real reason that your site dropped.
Outside of the fact that you did these links, there is also a big discussion going on right now about a new Google update coming down. So while the timing may be interesting, it is unlikely that just those two links killed your site.
Google update anyone? - Webmaster Forums UK SEO SEM Webmaster Community Forum
Webmasters Report February 2008 SERP Changes
It would be nice if you didn't discredit the advice your being given. The overall likelihood is that those small links are not causing your drop, there are much more critical concerns regarding your site than those two things, things you have much greater control of. Google is doing a major update and your site could be a casulty, this is a much more likely concept than a couple of directory links killing your rankings.
Remember that search engines are not fans of the same old. They change and you have to change with them. That includes updating your site to follow their new whims.