— title: Simplifying Haskell development and deployment with Nix and Docker author: ire —
This should be easier. Things seem to get substantially more complicated over time. Sure, we’ve made improvements in lots of areas, but simplicity doesn’t figure into it. But that’s a rant for a different time. I’ve tried finding clearer documentation for doing this before, but it seems every information and tutorial assumes you already know way more than might be true or do crazy forward jumps. Like first explaining what Docker or Nix is, what containers or reproducible builds mean and jumping to setting up docker-composer and spewing a bunch of docker commands without any explanation of what’s going on, same for throwing out a bunch of Nix expressions. There’s a disconnect there. This is also a rant for a different time. And lastly, this took me way too much trial and error to get right. It shouldn’t be this way.
With that in mind, I’m documenting my current setup, because I think it’s a good setup, and thanks to the promise of reproducibility from Nix, should be an easy to replicate setup. I’m also publishing it in the hopes of being corrected on some points as well as getting suggestions on how to improve it and simplify it, as my current understanding of both Docker and Nix is surface level. Without further ado, the setup.
No more docker files
The first attempt at this setup had not one but three Dockerfiles; one used to install GHC itself as well as stack, cabal and dependencies (openssl still painful). This is certainly temporary, just until fpcomplete/commercialhaskell publish a docker image using LTS-17.4 (as of the time of writing, the latest LTS available on hub.docker.com was 16.31, from 2 months ago). The second one would actually build the app with dependencies, and the third one was for a very small image with just the necessary (or so I thought at the time, more on it later) to actually run the app. The current binary size for the Haskell application is circa 12mb, the final image from this process was around 200mb. This was also before I learnt that Docker had added built-in support for multi-stage Dockerfiles a while back, so this could all be a single file.
Now, whilst this all worked, it had some issues. It’s complicated, slow to use, and required a lot of trial and error to ensure that the Docker container had everything I needed and nothing more. Another issue is that I attempting all of this running on an M1 MacBook. Don’t get me wrong, I’m absolutely in love with this little machine, but it’s still early days, and Docker support is still slowly rolling out. I can run most things, at the time of writing, ghc-pkg (the low level API from the compiler to link dependency packages) fails miserably. It seems to be related to something on the qemu layer (Docker is a Linux-based technology, and as such, require emulation below containerisation on mac/windows).
Enter Nix
Nix, on the other hand, seemed to promise a better user story on the Mac. Most of the issues seemed resolved, when it comes to Arm compatibility, and it promised a few things Docker doesn’t. Using the power of Nix and Nix-Shell, you can guarantee not only reproducible running and deployment, but also development environments.
By configuring everything to run and build locally inside what’s called a pure nix shell, you prevent your whole stack from accessing anything from the host machine. Everything needs to be made explicitly available inside this shell to be accessible. Not only that, but Nix is also a fantastic package manager that provides, out of the box, most of hackage (Haskell dependency repository) out of the box. Sounds great. The downside? It’s very complex, not really the most beginner friendly, and documentation, despite being plenty available, is of limited help.
Part of the complexity is the term overloading. When we say Nix, we could mean Nix the package manager; the (lazy, pure, functional) programming language used to describe packages and the build process; the Linux-based operating system. Although the latter is usually referred to as NixOS, which helps when talking about it, doesn’t narrow it down enough for searching online. And part of the complexity is from Nix itself. Although that will get easier with time, it does require quite a lot of it, and most people can’t afford the investment.
Now, all that being said, there’s a reason it’s getting more and more traction, especially in the Haskell community. Once you get it to work, it’s fantastic. I mentioned the reproducibility before, there’s however a lot more, which I’ll deem outside of the scope of this article. I’ll refer you to Burke Libbey’s youtube channel for some overview of what it can give you.
I mentioned before about the Nix story being more mature when it comes to M1 support, and that’s true, but sadly that’s not true of every package. At the time of writing a surprisingly large number of dependencies flat-out fail to build on my machine. This isn’t however an M1 issue per-se, but rather, a Big Sur issue. With the latest operating system release, Apple changed how dynamic libraries work (for improved safety and security), but a lot of packages did not yet update their linking steps to use the new system. An example here.
In the end, it seemed nothing was going to work 100% on the Mac. Lucky me, I have a desktop PC running Linux at home.
Enter NixOS
It was time to give up. So, let’s install NixOS on my home PC and use it as a build machine. This turned out better than I expected. Installing NixOS was incredibly easy (for a developer very comfortable with the CLI). The whole system basically runs out of a single configuration.nix file, where you describe the system, and then tell it to create a new generation from it. You need to tweak a configuration? Edit the file, build a new generation. Want to install packages? Same thing. Installing and configuring the whole system was a matter of editing this file and cloning my dotfiles from gitlab. But if you use nix home-manager, even that last part can be handled by Nix itself.
With NixOS installed and running on my desktop without a hitch, I was able to really see the power of a nix-based development flow.
First, I migrated my current app from using stack to using Nix. I got a huge head start there thanks to this simple repository that offers a nix/haskell application skeleton. There, I hit my first bump.
A few of my dependencies simply wouldn’t build. They were not available on hackage, or were marked as broken on the Nix packages repository. I was stumped. How do I integrate them? Here, I again I was facing the documentation and expectations issue. I was supposed to just know it. After a lot of tinkering, and reading, and pulling out some hair, here’s my current development flow:
- Clone nixkell to create a nix-based skeleton for a haskell application.
- Add the development tools to the nix shell by editing nixkell.toml.
- Add dependencies outside nix using cabal2nix.
- Build, test and run my application in the linux build machine.
- Use Nix dockerTools to automate creating docker images out of my application automatically.
- Deploy to my linode-based server by pushing the docker image up, connecting to the server via ssh, pulling it there, and running.
The last item is the place that needs the most improvement, but I’m not there yet.
Just like my predecessors before me, I’ve made assumptions and jumped forward. I introduced things I haven’t mentioned before, assuming you knew them also. So let’s take a step back, and break it down. Again, I’ll try to highlight my particular pain points.
1 and 2 are easy. Cloning the repository, adding my development tools to a toml file, relying on direnv to reload my shell with them? Smooth. This gives me the power of reproducible development environment across machines. If I am working with other developers on the same app, we will all get the same development environment. No fiddling around.
3 was more complicated. I spent a lot of time trying to find out ways to add dependencies to a nix-based project, and most answers didn’t help me. They would, if I were more familiar with Nix the language, its APIs and inner workings. But I’m not there yet. In the end, I found a way to make it all work using cabal2nix. The nixkell skeleton setup provides us with a packages folder. Dropping nix derivations inside it get automatically loaded; and cabal2nix makes it trivial to build derivations out of cabal files. Since every Haskell package has one, it trivialised a lot of the work. Simply calling the following command did it for me.
bash $ cabal2nix cabal://package-1.0.3.0 > nix/packages/package-1.0.3.0.nix
This is not a cabal2nix tutorial, but as a quick overview it supports fetching packages from hackage itself by using the cabal:// URL scheme. It also supports local file paths, git, github, and so on.
First pain point: running that immediately spew out an error. It turns out, although not evidentely documented, that it requires a globally installed cabal with an up-to-date database. The following fixes it:
bash $ cabal v2-update
4 was smooth as well. Nixkell provides a few helper commands, described in the nix/scripts.nix file, where you can edit or add your own. This includes a build and a run command.
5 on the other hand, was not as smooth. There are still configuration options available that I don’t know what they do, or how to use them. There’s also a bit of hacking about to get everything to work as I wanted to. Let’s dive in.
Firstly, Nix provides APIs to describe a docker image. Here’s my current nix expression doing just that (edited for publishing):
```nix { pkgs ? import
with pkgs; # considered bad practice in most cases
let # import our sub-expressions needed to build the app app = import ../nix/. {}; # get the resulting binary we want to include in the image bin = app.pkgs.app-name.bin; in
This is the nix api to build images dockerTools.buildImage { # our
image name name = “registry.gitlab.com/elland/app-name”; # our image tag tag = “latest”; # creation date defaults to Unix epoch 0, for reproducibility # but that makes it harder for me to reason about it created = “now”;
# this is a list of the things we want to include # in the image. it’s incredibly bare by default. contents = [ bin # our app pkgs.cacert # so it can handle https/ssl network calls
# this evaluates to an expression adding that folder's contents
# to the root of our image. It doesn't seem to support copying
individual files so I moved everything I needed inside that
folder as files/subfolders. (../data)
# the following are only addeed if I need a shell into the image
for debugging # pkgs.coreutils-full # pkgs.bash # pkgs.wget ];# This exposes the Dockerfile commands you might be familiar with config = { Cmd = [ “${bin}/bin/app-name-exe” ]; ExposedPorts = { “8000/tcp” = {}; }; }; } ```
I put that inside a docker folder inside the project. Whenever I need to generate a new image, I can just run nix-build . and loading the resulting image to docker with docker load < result from inside that folder. Which I’ve automated using my Makefile.
```Makefile img-name = registry.gitlab.com/elland/app-name:latest
dockerise: ## Create docker image, load it up cd docker; nix-build . && sudo docker load < result
docker-upload: ## Upload docker image to registry sudo docker push ${img-name}
docker-run: ## Run docker image, using latest tag sudo docker run –expose 8000 -p 8000:8000 -v app.db:/db:rw –rm -it ${img-name} ```
And that is basically it. 6 is the basic Docker workflow, if you’re really bad a Docker and DevOps. I’ll improve on it at some point, but right now it’s quite early in development for me. The goal is to automate it using continuous delivery once I’m closer to an alpha release.
Finally, to ensure it all actually worked as assumed, I’ve made a point to clone my own repository in a different machine, and let nix/direnv take care of everything. Just cding into the directory, direnv allow and waiting a bunch, as Nix downloaded everything needed, I had my exact development environment replicated, minus a few things, that are quite individual, my neovim/emacs configurations for example. All in all, after all the sweat and blood, a worthy experience and a fantastic end-result. Hopefully the initial price of learning and setting this up for the first time is only paid once.
Caveat emptor: I was unable to actually build docker images from Nix on the Apple Silicon Mac. Everything else works.