Bazel Knowledge

Kip Landergren

(Updated: )

My Bazel knowledge base that evolves as I learn more.

Contents

Overview

bazel is an artifact-based build system. It offers the ability to distribute build jobs over remote instances and caches build artifacts for reuse. It is controlled by BUILD files that contain rules defining how the artifacts should be constructed.

Motivation

Build tooling started as an extension of what the developer was already doing manually: pass files to a compiler as inputs and generate a binary as outputs.

As projects grow in scope—e.g. in number of generated binaries, dependencies, different compilers / languages—the scope of what these manual build tasks must manage becomes more complex:

And the problems with their operation and maintenance become manifold:

Additionally, from an engineering organization perspective, there is an ever present desire for increased product velocity (which trickles down to developer velocity).

This results in an increased emphasis on:

And so to address these needs tasks grow even larger in scope and complexity and brittleness.

There is a fundamental issue at play here: because the tasks are so flexible—they offer no guarantees about their operation and output—the containing system is hamstrung in terms of what execution improvements it can confidently make.

Build systems that exhibit these characteristics are called task-based build systems.

In contrast, bazel is an artifact-based build system. An artifact-based build system instead focuses on declaring what the outputs should be and leave the “how” of creating them to the build system itself. By creating build primitives that developers can invoke to specify what their desired outputs are—including metadata like their dependencies— bazel forces the developer to give up some of that task freedom in exchange for powerful build system capabilities.

Those capabilities include:

bazel accomplishes this with the following setup:

This graph, in combination with the rules (of which it has guarantees about operation, like their lack of side effects), allow bazel to determine an optimal build sequence. Additionally, the reproducibility requirement makes it possible to introduce remote shared caching and remote execution.

Core Idea

bazel reduces the developer’s ability to define the “how” of how a target is built and instead directs them toward specifying the “what” of what needs to be built.

Conceptual Model of a Build

Think of bazel as constructing a build graph of files and actions:

It does not have a semantic understanding of the commands invoked within actions: just that there are inputs and the command is a means to transform them into outputs.

If an intermediate step in your build would benefit from caching performance ask yourself:

"What file and command is backing this?"

And trace performance from there.

Command Invocation

Let’s look at the command to build target //pkg:foo:

bazel build //pkg:foo

Breaking this down:

                    label identifying target
                    +----------------+
bazel     build     //     pkg  :  foo
-----     -----     --     ---  -  ---
  |         |       |       |   |   |
  |         |       |       |   |   \ target name
  |         |       |       |   \ separator
  |         |       |       \ package
  |         |       \ root of the main repository
  |         \ bazel command to build the given targets
  \ bazel client

The build command needs to broadly do two things:

Phases

The conceptual model (behind the scenes bazel interleaves these phases) for how bazel builds a target involves three phases:

Loading Phase

Evaluates all the BUILD files associated with a target and its dependencies to produce the target graph.

Analysis Phase

Targets are associated with rules. This phase then creates the build actions by evaluating the rules’ implementations with their invocations. This combines the rules with the build configuration to produce configured targets.

Execution Phase

Determine what actions actually need to run (and how). Then invoke the commands associated with those actions, in their dependency order, to produce the outputs. Parallelize and read results from caches where possible.

Caching

A partial picture:
package loading cache keys invalidated when the associated BUILD file changes
analysis cache the build graph of configured targets
action cache ?? hash of the rule (after variable expansion) to avoid re-evaluating the rule ??
?? local action cache ?? ?? combination of action and content (the input files) hashing to prevent additional work ??

Key Concepts

Tooling

Gazelle

Gazelle is BUILD file generator. It analyzes a target’s dependencies and ensures that they are available for bazel to find and use when building a target.