And the whole discussion leads to a chicken and egg argument..
In the beginning there were links.. And only links.. And those links tended to be given solely to sites that you liked or were of similar content (webrings abound!).. All was good..
Along comes a SE and says, hey look, all these wonderful webmasters have given us a tool to use to help us choose the really good and important sites and move them to the top.. And again, all was good..
And then came the "eCommerce" monkey.. The eCommerce monkey himself is not an evil monkey, but he leads otherwise good people to do bad things in an effort to please the eCommerce monkey.. And the link farm was born.. And it was bad..
And since that day it has become a game between the monkey and the SEs to determine who will control the webspace..
I'm sure that relevancy plays a role in the rankings of links, but I'm not convinced that it plays a large role.. If it did then the hundreds, thousands, of generic link directories would not be providing such a large boost to SERP rankings.. Simply because the only relevant content is other links on that same page.. The rest of the directory site is simply random data..