January 21, 2011 / cdsmith

A Recap About Cabal and Haskell Libraries

Wow, it turns out I’m not the only person having issues with managing dependencies in Haskell! There has been a lot of discussion and sharing of ideas and information in a few different places, starting from my original article about “DLL hell” in Haskell three days ago. I’m going to try to collect all the relevant points together, along with some added reading I’ve done, and organize them into a coherent summary. I’m sure I’ll inadvertently leave some things out, and if so, I apologize.

Regarding “Blame”

First of all, I want to echo Johan Tibbe, who mentioned on Reddit that this is, in many ways, a sign of good things. Haskell isn’t running into this problem because something broke; it’s running into this problem because it’s being used for a remarkable variety of projects in a way that hasn’t happened before. I also agree with John Millikin, Mike Burns, and others who pointed out that the problem isn’t uniquely Haskell. Indeed, I think it’s fair to say that perhaps Haskell is one of the few environments where we’ve got a prayer at solving the problem.

The causes come down to two things, basically,

Haskell encourages the writing of many small libraries. This is in part because we do such a good job of managing the dependencies on all those libraries. Can anyone imagine that we’d have a lot of the stuff we have now on Hackage if we didn’t have excellent tools for working with it? If there’s any lesson to be learned here, it’s just that people will try to do more things until they hit the limits of the tools available to them!
Haskell is now being used to write “integration” type software; packages that pull together a lot of very different functionality and build something out of the pieces. I think this is a relatively new phenomenon, at least at the scale it’s happening. And it puts a lot more stress on systems like Cabal.

So I’d be unhappy if someone walked away from my article thinking “Haskell sucks at dependency management; I should use something else.” The thing is, something else probably sucks too, quite possibly a lot more! In many cases, it sucks so much that you wouldn’t even attempt to do the things we routinely do in Haskell. No one releases a Java library with only 50 lines of code – in part because it’s hard to do anything interesting in 50 lines of Java, but also because the effort of getting it installed and working with all your other libraries would be so great that it would swamp the effort of rewriting the functionality.

Regarding Defeatism

On the other side, several comments were made to the effect that it’s too much to hope that we can solve version dependency problems in Haskell. The argument goes that there are lots of other languages that also have these same problems; they haven’t found answers, and we probably won’t either. There’s mention that this is an “active research topic,” implying that it should be left to the researchers.

It’s good to inject some context into the discussion about the scope of the problem, but ultimately I think we should reject the idea that Haskell isn’t going to solve these problems, for three reasons:

We’re almost there! Despite the admittedly negative tone in the article that started this all, I think I’ve actually enumerated most of the issues with the current system, and they all look solvable. GHC and Cabal developers have put immense amounts of effort into working on these problems, and they have nearly gotten us to a place where Haskell dependencies just work in a way that’s not true in any other language. (A key point here: Isolation-based answers help tremendously in practice, but still leave possibilities of problems. In fact, any time something only works because of isolation, it’s also the case that someone trying to combine that code together into a bigger system is going to run into problems. So any environment that widely recommends an isolation-based answer is clearly short of the “just works” point, and instead settling for “works so far, and we hope no one tries to do something bigger.”)
These are Haskell’s problems to solve. While dependency management comes up in many different programming languages, the solutions in this case are Haskell solutions. If you look at the central two issues I pointed out – internal dependencies, and the Cabal butterfly effect – what’s ultimately holding up a solution is work on the compiler to contribute things that relate to its understanding of the Haskell programming language and the interfaces between modules that the compiler builds. If Haskell doesn’t solve these problems, then no one else is going to do it for us.
Since when does the Haskell community shy away from hard problems? Sure, this is a difficult problem. So are/were lazy evaluation in a general-purpose language, type classes, models for I/O in a pure functional framework, functional dependencies and type families, software transactional memory, vectorization of data parallel code, type inference for GADTs, and the construction of what is almost certainly the most advanced optimizing compiler in the world. Haskell did (and/or is doing) all of those.

I don’t mean to dismiss those who pointed out this is a hard problem; it may take a while to solve it; so those who are trying to use Haskell in practice right now are well-advised to find temporary workaround or partial answers, such as isolating dependencies of projects, for example. At the same time, though, part of the unique spirit of Haskell has always been the willingness to live with our problems all the way up to (but not past) our tolerance point, and take the time to find the right answer instead of the expedient one.

That’s always been my interpretation of the Haskell motto to “avoid success at all costs” – what we’re really avoiding is the temptation to take short-cuts with duct-tape and glue, and in the process compromise the effort to find the right answers to questions about Haskell. This isn’t the fault of the duct tape or the glue, which are useful short-term tools. But when keeping up with the duct tape and glue gets in the way of making correct decisions, then a programming language starts that deadly race wherein we try to get as popular as we can before the language gets too crufty to use any more, and people jump to something else.

Isolating Environments

Another very informative part of the conversation relates to isolation of different build environments. I’m not active in the Python or Ruby development communities, but several people (including John Millikin and Mike Burns, mentioned that they routinely solve these kinds of problems with sandbox or isolation tools. These tools maintain local package databases that are just big enough to build a particular piece of software, thereby guaranteeing that installing some unrelated library for a separate project won’t break your libraries for this one. Ruby’s tool for this purpose is rvm (Ruby Version Manager), while the Python community uses virtualenv.

It may come as a surprise to many people that Haskell has its own tool for isolated build environments, called cabal-dev. John Millikin wrote an in-depth introduction on building and using it on Reddit. Basically, it keeps a private package database for you in a subdirectory of your build directory. The idea is that you can install a minimal amount of stuff in the system-wide or user-wide databases, and let cabal-dev download and build most dependencies on demand for your specific project and store them locally. It’s not quite an rvm-level tool, in that it does not manage multiple versions of the compiler, but it sure helps with library version isolation.

As I mentioned above, I see isolation as a short-term good, but perhaps a premature casting off of a hair shirt. If I can only build all my Haskell stuff because of isolated package databases, then this means there are some integration projects that I could not embark on because they would be too large for the isolated package databases. So I’m of mixed minds about this; it’s no-doubt good that cabal-dev exists. On the other hand, I’d hope it does not obscure the need for a right answer to package dependencies.

A few other notes related to isolation:

Another idea that was brought up again, and that’s come up in many other places recently, is installing and using a local copy of Hackage for use in a collection of related projects. Michael Snoyman’s Yackage is a minimal example of this that looks like a piece of cake to install. It’s also supposed to (eventually? now?) be relatively easy to install hackage-server and customize the exact set of features you want. I have yet to do any of this, but it certainly looks appealing, especially if you’re trying to maintain an ecosystem of connected software.
Something else that came up with respect to cabal-dev and rvm, for example, is that rvm also isolates the version of the compiler you’re using, as well. It looks rather difficult, currently, to have multiple versions of the ghc compiler installed at the same time. Indeed, this is part of what turned me off from doing GHC work some time ago; it looks like it’s more work to keep several GHC versions in working order at once than it is to actually modify the code. It seems we’re a long way from ‘cabal install ghc-7.1’!
Finally, a sort of “partial isolation” that seems unambiguously to be a good thing was mentioned in the context of Ruby’s bundlr by Darrin Thompson. The comment was that when Ruby’s gem system resolves dependencies, it can be asked to write out the exact resolution to a file, and then other tools on other systems can bring exactly that same package dependency tree into being. I think to date, the Haskell community has largely avoided struggling with deployment issues and integration systems, but it doesn’t seem difficult to get Cabal and cabal-install to pick up the idea of fixing a specific solution to version constraints, to ensure that packages used on deployed systems exactly match their development counterparts, even if a rebuild is needed (e.g., different processor architecture). Of course, Ruby has a somewhat greater need for this, in that I can generally copy a compiled and statically linked binary in Haskell; but as the move to shared libraries continues, this may well become far more relevant.

Internal Dependencies

Nothing new here, except that there seems to be a general consensus that this needs to be one of the first changes made to the Haskell build infrastructure. As Duncan has pointed out before, this involves a coordinated change involving both GHC and Cabal.

Something I find myself in agreement with is that perhaps the best approach going forward would be to fix this, and the next point (the “butterfly effect”), and then take stock again of where we are. Fixing the internal dependencies issue would hopefully reduce the number of version constraints Cabal needs to deal with by an order of magnitude or so. That might make many of the other issues people are facing go away, or reduce them to the point that they are solvable by people just talking to each other and politely requesting bumps in version upper bounds. That seems sensible to me; there’s a legitimate hope that this fix would make everything else a matter of patching up the odd points here and there.

The Butterfly Effect

This was point number 3 in the “DLL Hell” article, but I’ve since written an expanded description of what is happening, and made up a name for it in hopes of making it easier to discuss. The idea is that, while Cabal and GHC are fine with multiple packages existing having different versions, they are not okay with multiple packages have the same name and version, but different dependencies. As a result, one can end up in a situation where installing a package breaks another package. This is the only issue in the list that I consider to be a bug (albeit probably a known and understood one) in Cabal.

A number of people have brought up GHC’s ABI hash… that long sequence of hexadecimal that GHC appends to your package when you rebuild it. I’ve spent some time doing a bit of reading into what GHC does here. While it’s certainly related, unfortunately this still doesn’t actually solve the problem. What it does do is help GHC to detect the problem. The idea is that GHC hashes together all the information about the ABI – that is, all the information that needs to be known to correctly link against a package. Then if some package gets recompiled and exposes a different ABI, the hask will differ, and GHC will notice that packages that depend on it are broken.

This raises the question of whether GHC could just keep around multiple copies of the same library, differing only in their ABI hash. The answer, as Simon Marlow pointed out, is no. Because the ABI hash is not included in the names of symbols in a library, trying to link two different packages with the same name and version but different ABI hashes would lead to problems later on. So currently, the ABI hash is used to detect when a dependency is recompiled, but it cannot be used to keep several instances of the same package version. The reason for not including the ABI hash in the symbol name seems to be related to avoiding unnecessarily recompiling things that don’t need to be recompiled. That’s also a valid goal; so something a bit more complex would have to happen to get this sort of thing working. Still, it doesn’t look undoable.

Several people mentioned the Linux package manager Nix as inspiration here. It does look very much like what ought to be done. Whether we would want deferred garbage collection or shared –user installed packages is an interesting question, but I think far less important than solving the immediate problems.

About the Package Versioning Policy (PVP)

One of the surprises for me was the response to my comments about the PVP (the Package Versioning Policy). This comes back to different people having different kinds of projects, I suppose. I personally have never witnessed a package that used to build breaking because of upgrades elsewhere and someone failing to put upper bounds on their package dependencies. Don’t get me wrong; I’ve seen a few Cabal build failures in my time, but generally they’ve always been traced to some other problem; I’ve just never seen it be the case that the package would have been fine with an appropriate upper bound, but failed to compile because there wasn’t. There’s always been something else involved; usually a compiler version difference and resulting changes in some package for which an older version doesn’t exist for the new compiler.

Apparently other people consider this a significant problem, though, and if others are having problems even today with the PVP not being followed tightly enough, then I certainly retract my suggestion that we consider weakening it. Suggestions were made to have Hackage enforce the PVP, but my personal feelings always come back to what I think we ought to consider an axiom of Hackage: anything we do that makes it less likely people would upload their packages is a step backward, not forward. The fact that practically all open-source Haskell code out there is contained in Hackage is an immense treasure of the Haskell community, and we’d be fools to jeopardize that in any way. Having Hackage enforce the PVP means requiring that a package be buildable in the Hackage environment before accepting it, and that seems like a non-starter.

One question I’d like to keep on the table is the possibility of telling Cabal to distinguish between strong bounds (“I know this won’t work”) and weak bounds (“I haven’t tested this, possibly because it doesn’t exist yet to test“). Perhaps new version bound operators (>>) and (<<) could be introduced to capture the strong bounds. Cabal could then be told on a case-by-case basis to override the weak version bounds on other people’s packages. Then one might imagine an interaction something like:

$ cabal install foo
Configuring foo...
Cannot resolve dependencies for foo (... for some reason involving upper bound on bar ...)
$ cabal relax-depends bar 'baz < 0.5'
$ cabal install foo
(successful build)

It’s also worth mentioning, here, Michael Snoymans packdeps package and its web front end. These are tools for alerting package authors when they are excluding other libraries because of upper bounds on their packages. This can help reduce the problem of keeping package dependencies up to date.

Interactions With OS Packaging

Finally, there were a few comments about using the operating system package manager instead of trying to “reinvent the wheel” with Cabal. All things considered, this doesn’t look like a reasonable idea. The amount of interconnection between Cabal and GHC, mentioned in several places earlier, is good proof that package management is somewhat intimately connected to the build tools of the language we’re doing it in. Add to this the fact that there are nearly 3000 different packages on Hackage (never mind all the different versions, where some old versions are still needed!), and the fact that several of them are updated several times per day. Packaging libraries for distribution with the operating system is a completely different model.

However, this brings up the question of what to do about OS-packaged Haskell libraries. Personally, what I do is just let the operating system manage my global package database, and manage my own user package database. Then if the operating system packaged libraries are updated, I may even just have to blow away and reinstall all of my user packages, but it’s infrequent enough to not be an issue. Maybe there’s something theoretically better we could do here, but I don’t see it as a serious issue.