On http-enumerator

This is going to be a short blog post.  I just wanted to point how lucky the Haskell community is to have the http-enumerator package, written by Michael Snoyman.

Back Story

Okay, so 5 years ago, there was a recognized problem that Haskell was short on libraries.  Today, there’s a tendency to see that problem as fixed.  After all, there’s Hackage, with its ridiculous number of packages.  There’s Cabal making it easy to install any library in a uniform way.  There’s even cabal-install, automating (okay, with hiccups) the process of chasing down dependencies to get your packages built successfully.  Indeed, today it is almost more common to hear the opposite complaint: that Haskell has too many libraries!  There’s intense discussion on how to pare down the the list to something manageable, how to let people more effectively filter out the many packages they don’t need.

It’s easy to forget sometimes that there are embarrassing gaps in the library picture.  One of these has been SSL.

My story: As a sort of half-hobby project, I took to building a client library for Linden Labs’ Second Life virtual world.  The bulk of this protocol is binary packets with UDP, a task that Haskell is definitely up to.  But a smaller bit of the communication was a lot more problematic.  There was login, which had to be done with an XML-RPC call to a web server at https://login.agni.lindenlab.com/cgi-bin/login.cgi.  Oops, while there was an aging XML-RPC implementation around (haxr), there was no way to talk to anything using SSL.  This wasn’t too hard to handle: I installed stunnel, set it up to listen from a port on localhost, and forward to login.agni.lindenlab.com.  Problem solved, right?  But then we run into part 2.  For certain tasks where a binary UDP-based protocol is poorly suited, the system has an alternative, called the capabilities system.  The way this works is: when you log in, part of the login response was a dynamically generated URL one some arbitrary server in a large server farm.  You could connect to that server through, you guessed it, HTTP over SSL, and make POST requests to it.  In fact, the first server you get is the “seed” capability, which just gives out more dynamically generated URLs to more servers!  Sadly, stunnel just wasn’t up to this task.  Discouraged, I ignored that portion of the protocol, and managed to get something functioning in a limited way using login through stunnel and the UDP-based protocol.

It’s not as if SSL is a minor, isolated corner of the universe.  In a variety of very common scenarios, SSL is used all the time, and unfortunately, we just didn’t have a very good way to deal with it in Haskell at the time I started.

Enter http-enumerator

Luckily for me, in September, Michael Snoyman ran into a similar problem.  I think in Michael’s case, he was working on implementing the OAuth protocol, for which — understandably — a lot of providers use SSL.  Michael, being more motivated to solve this problem, built http-enumerator, a package that implements the client side of HTTP, including SSL, within the enumerator framework.

To Michael’s great credit, there’s not really that much more to say!  This is a library which:

• Takes advantage of modern ideas about building good high-performance code with nice abstractions… but also doesn’t force them on anyone.  The package is a piece of cake to use without having to study a new way to organize code, at least as long as you are okay with holding requests and responses in memory.
• Is about solving an actual problem that pops up in the world all the time, and which was sufficiently intimidating that no one else tackled it for years.
• Works great!

So… what?  So, as a community, we owe Michael a bit of thanks for patching an embarrassing gap in library coverage.  Thanks!