Nix (Package Manager) Knowledge

Contents

• Overview
• Core Idea
• Key Concepts

Overview

Nix is a suite of tools for building packages in a declarative and reproducible way.

A package is a collection of software, which may contain:

• executables (e.g. scripts or binaries)
• libraries
• documentation
• data
• other files and metadata

Creating working versions of these packages involve solving two main problems:

• how to manage dependencies
• how to compile the source code

Maintainers of C-based packages have consistently reached for autotools and make as preferred solutions. When you run ./configure, autoconf checks your system for the presence, location, and information of dependencies, including the compiler. When you run make that information is marshaled to actually execute the build and compile source code.

What invariably happens is one (or many!) of:

• configuration fails because of:
  • a missing system dependency (like a correct compiler)
  • a dependency you know is present on your system but unable to be found (maybe in a non-standard location)
• compilation fails because of:
  • a missing package dependency
  • a package dependency present but two versions out of date compared to what is needed
  • the source code is written expecting a more modern compiler
  • the build platform architecture is not supported
• the build succeeds but:
  • executing the resulting binary fails with a runtime error
  • there are subtle (or not so subtle) differences in execution because of dependency variances

And all of these are repeated for each environment (my machine, my coworker’s, production...) that wants to use the package.

Let’s imagine you overcome all of these issues. One still has to contend with the particularly pernicious problem of having two packages A and B that each declare a dependency on different versions of package C. How should your system be configured such that both can exist?

Nix aims to solve these problems (and more) by providing an ergonomic harness around existing packages’ build processes.

This harness consists of:

• the Nix programming language, created to:
  • describe a package’s build process through expressions
  • evaluate those expressions to a derivation representing the dependency and build graph
• a standard library that makes it easy to:
  • define specific versions of dependencies as inputs
  • parameterize options to these dependencies
  • split the build into well-defined phases that may be modified or overridden
  • marshal tooling like bash, autoconf, and make (themselves each as versioned dependencies!)
  • define the published outputs for other packages to depend on
• the Nix software which:
  • creates a sandboxed environment to execute the build
  • ensures all inputs are themselves built at the required versions and visible as if exclusive
  • saves the resulting outputs for use by other packages in an immutable store
• a mechanism for decentralized package definition repositories (with a very comprehensive default in nixpkgs)
• a caching system to reuse previously built outputs

The additional architectural benefit here is that existing software’s build processes remain untouched: the work is with the Nix package maintainers to create the package expressions.

The end result are packages with:

• explicitly listed and versioned dependencies, including any necessary compilers
• a well-defined build process with convenient knobs and hooks for customization
• a mechanism to store and reuse previously built artifacts transparently

Core Idea

Nix the language is about functional programming with a build system.

It, and its standard library, are specifically created to:

• describe the build process of packages
• provide knobs and hooks within easy reach for customization
• evaluate to a build and dependency graph (including recursively for all inputs!)

Nix the software is about then executing that build graph to produce a package in a reproducible way.

It accomplishes this through:

• an immutable store of the output files, each with paths unique to hash of their inputs
• a sandboxed build environment that—through absolute paths pointing to the store—ensures that the build process executes with only the specified versions of dependencies available

By knowing the path ahead of time (it is based on the hash of its inputs) Nix provides a means to transparently reuse previously built portions of the build graph without recompilation.

Key Concepts

• package management
  • dependency version conflict
  • reproducible builds
  • standardization of build definition, process, and evaluation
• functional programming
  • lazy evaluation