CI that keeps up
how-toby Kyle Penfound

How do I cache Rust builds in CI without rebuilding everything?

A Rust CI cache has to keep Cargo’s downloaded crates and the compiled dependencies in target/ between runs. It has to follow changes to Cargo.lock and the toolchain, and every machine that builds the project has to reach it. On GitHub Actions, Swatinem/rust-cache meets most of these requirements. On other CI platforms, a platform cache, sccache, or cargo-chef each meets some of them. Dagger meets the same requirements in one place. Its cache volumes keep both directories between runs, and the same checks run on a laptop and in Dagger Cloud.

What a Rust CI cache has to do

A Rust CI cache has four requirements. Each GitHub-hosted runner is a new virtual machine (GitHub-hosted runners). A job on a new machine starts without Cargo’s files, so Cargo downloads every crate and compiles every dependency again. A cache that meets these requirements lets Cargo skip that work:

  • Keep the downloaded crates. Cargo keeps them in its home directory, ~/.cargo, or /usr/local/cargo in the official rust image. Its registry directory holds the crates from crates.io, and its git directory holds the Git dependencies (Cargo home).
  • Keep the compiled dependencies. Cargo writes the output of a build to the target/ directory (build cache). When the previous target/ directory is present, Cargo recompiles only the crates that changed (Cargo source).
  • Follow changes to Cargo.lock and the toolchain. Cargo’s fingerprint of each crate includes the rustc version and the fingerprints of its dependencies (Cargo source). After a dependency update or a new toolchain, Cargo recompiles the affected crates. The cache then has to store the new output, and an older cache still saves the work for the crates that did not change.
  • Share the cache across CI machines and laptops. A cache helps only the builds that can read it. When jobs run on many ephemeral runners, or on more than one CI platform, each of them needs the same cache.

The sections below meet these requirements by hand, then with Dagger.

Keep Cargo’s directories on a long-lived runner

On a self-hosted runner that lives a long time, keep both directories on the runner’s disk. Set CARGO_HOME and CARGO_TARGET_DIR to directories on that disk. Nothing moves between runs, and Cargo’s fingerprints follow the changes to Cargo.lock and the toolchain. The cache serves only that runner. Cargo cleans its global cache in Cargo’s home directory automatically, but not build artifacts (Cargo configuration). Old artifacts stay in the target directory until you remove them.

Cache Cargo’s home and target/ with Swatinem/rust-cache

On GitHub Actions, the Swatinem/rust-cache action meets the first three requirements, and the fourth for jobs in GitHub Actions. Install the toolchain before the cache step, because the action puts the rustc version in its cache key. In a workflow file such as .github/workflows/ci.yml, the steps are:

- uses: actions/checkout@v6
- run: rustup toolchain install stable --profile minimal
- uses: Swatinem/rust-cache@v2
  with:
    shared-key: "ci"
    save-if: ${{ github.ref == 'refs/heads/main' }}
- run: cargo test --locked

The README says how the action meets each requirement:

  • Keep the downloaded crates. It caches ~/.cargo.
  • Keep the compiled dependencies. It caches ./target. Before it saves the cache, it removes the workspace crates and the incremental build artifacts. The cache keeps only compiled dependencies. The action also sets CARGO_INCREMENTAL=0.
  • Follow changes to Cargo.lock and the toolchain. Its cache key includes the rustc version, a hash of every Cargo.lock and Cargo.toml, the toolchain files, and compiler variables such as RUSTFLAGS. When the lockfile changes, it restores the cache of the previous lockfile. Then only the changed dependencies rebuild.
  • Share the cache. shared-key gives several jobs one cache. save-if limits saves to main.

The action stores its cache in the GitHub Actions cache, so GitHub’s cache limits apply to it.

Cache Cargo with a platform cache keyed on Cargo.lock

On a CI platform without a Rust-specific action, use the platform’s own cache with a key from Cargo.lock. List Cargo’s home and target/ as the cache paths to keep the downloaded crates and the compiled dependencies. The key from Cargo.lock makes a new cache when the dependencies change. GitHub Actions cannot change a cache after it saves it, so a new key is the only way to store new output (GitHub docs).

On GitHub Actions, the platform cache is actions/cache with hashFiles('**/Cargo.lock') in the key (GitHub docs). On GitLab CI/CD, it is cache:key:files, which makes a new key when the listed files change. GitLab cache paths are relative to the project directory, so set CARGO_HOME to a directory in the project. In .gitlab-ci.yml:

variables:
  CARGO_HOME: $CI_PROJECT_DIR/.cargo
cache:
  key:
    files:
      - Cargo.lock
  paths:
    - .cargo/
    - target/

If a rust-toolchain.toml file pins the toolchain, add it to the key files. Then a new toolchain also makes a new cache.

Share compiled crates across machines with sccache

sccache shares compiled output across machines. Cargo’s own docs name it as a way to “share built dependencies across different workspaces” (build cache). It caches the output of each rustc call, not a whole directory, so it does not keep the downloaded crates. It stores results on local disk or in a remote backend: S3, R2, Google Cloud Storage, Azure, Redis, or the GitHub Actions cache.

To turn it on, set RUSTC_WRAPPER=sccache in the build environment. For an S3 bucket, also set SCCACHE_BUCKET and SCCACHE_REGION (S3 setup). Many machines can read and write one bucket, so sccache fits a fleet of ephemeral runners, such as CI agents on Kubernetes.

The Rust notes of sccache list its limits for Rust:

  • Incremental compilation in rustc must be off.
  • sccache does not cache crates that call the system linker: bin, dylib, cdylib, and proc-macro crates. These crates compile on every run.
  • Procedural macros that read files from the file system may not cache correctly.

Cache Rust dependencies in Docker builds with cargo-chef

When CI builds a container image, cargo-chef keeps the compiled dependencies in a Docker layer of its own. cargo chef prepare writes a recipe, recipe.json, from the manifests and the lockfile. cargo chef cook builds only the dependencies from that recipe. Docker keeps that layer in its cache until the dependencies change. The README warns: “You must use the same Rust version in all stages”. The cache reaches the machines that can read the Docker layer cache.

Rust cache options against the requirements

Each option meets a different part of the requirements:

Option Keep the downloaded crates Keep the compiled dependencies Follow changes to Cargo.lock and the toolchain Share the cache
Long-lived runner Yes, in CARGO_HOME on its disk Yes, in CARGO_TARGET_DIR on its disk Cargo’s fingerprints Only that runner
Swatinem/rust-cache Yes, ~/.cargo Yes, dependencies only A key with rustc, the lockfiles, and RUSTFLAGS, and a fallback to the previous lockfile GitHub Actions jobs, with shared-key
Platform cache keyed on Cargo.lock The paths you list The paths you list A key from the files you list Jobs on that CI platform
sccache No The output of each rustc call, except linked crates Each rustc call caches on its own Every machine that reads the backend
cargo-chef In the dependency layer Yes, as a Docker layer A recipe from the manifests and the lockfile Machines that read the Docker layer cache

Limits of these caches as a project grows

The hand-built caches strain on the same requirements as the project grows:

  • Keep the compiled dependencies. The target directory is large. An archive takes longer to upload and download as the workspace grows. GitHub gives each repository 10 GB of cache by default. It removes entries that no run accessed in over 7 days. Above the limit, it removes the entries with the oldest last access. Pull request runs can restore caches from the default branch, and their own caches use the same 10 GB.
  • Follow changes to Cargo.lock and the toolchain. Keys miss. A key from Cargo.lock misses after each dependency update, unless a fallback key restores an older cache. A new toolchain or a new RUSTFLAGS value starts a new cache.
  • Share the cache across CI machines and laptops. Each setup belongs to one platform. rust-cache is a GitHub Action. GitLab, Buildkite, and Jenkins pipelines each need their own cache configuration. The CI platform caches do not apply when a developer runs the same steps on a laptop. sccache reaches every machine, but it does not cache linked crates. Binaries and proc-macro crates compile on every run.

Cache Rust builds with Dagger cache volumes

Dagger runs each Cargo command in a container and keeps Cargo’s home and target/ in cache volumes. A cache volume is a directory whose contents persist across Dagger sessions. It lives on the Dagger Engine, the program that runs the containers. No archive moves between steps. On the next run, Cargo finds its compiled dependencies in the same place and rebuilds only what changed. Dagger also caches each step by its inputs, so a rerun skips the steps whose inputs did not change.

The official Dagger modules cover tools such as Go, pytest, and ESLint, but not Rust. A search for a Rust module finds nothing:

dagger module search rust
No matching modules found.

So the Cargo steps go in the project’s own Dagger module: a package of functions and checks that build and test the project. Write it in Dang, Dagger’s native language for modules. Run these commands from the root of the Git repository:

dagger init
dagger module install dagger.io/sdk/dang
dagger module init dang

dagger init creates dagger.toml, the workspace configuration, or uses the file that is there (Quickstart). dagger module init without a name creates the module at .dagger/modules/<project>-dev. It also makes the module the workspace main module. The command returns a changeset, a diff of the proposed files, and Dagger shows it for review before it writes any file. Pass -y to apply the changeset without the prompt. A headless run, such as CI or an AI agent, needs -y.

The commands also write dagger.lock, which pins the images and Git refs that the workspace uses. After the first dagger check, it pins the rust image too. Commit dagger.lock with dagger.toml and .dagger/.

Write the Cargo checks in Dang

The module runs cargo fmt, cargo clippy, and cargo test as checks. A check is a function that validates the project, such as a test run or a linter. dagger check runs every check. Replace the starter code in .dagger/modules/<project>-dev/main.dang with the code below. ProjectDev stands for the type name that dagger module init generated, so keep your own type name.

"""
Build, lint and test the Rust workspace with Cargo's caches kept between runs.
"""
type ProjectDev {
  let source: Directory!

  """
  Rust version, used for the image tag and the target cache name.
  """
  pub rustVersion: String!

  new(ws: Workspace!, rustVersion: String! = "1.99") {
    self.source = ws.directory("/", exclude: ["target", ".git", ".dagger"])
    self.rustVersion = rustVersion
    self
  }

  """
  A Rust toolchain with the workspace mounted and Cargo's caches attached.
  """
  pub rustEnv: Container! {
    container
      .from(`rust:${rustVersion}`)
      .withExec(["rustup", "component", "add", "clippy", "rustfmt"])
      .withMountedCache("/cargo", cacheVolume("cargo-home"))
      .withEnvVariable("CARGO_HOME", "/cargo")
      .withMountedDirectory("/src", source)
      .withMountedCache("/src/target", cacheVolume(`cargo-target-${rustVersion}`))
      .withWorkdir("/src")
  }

  """
  Check formatting with rustfmt.
  """
  pub fmt: Void @check {
    rustEnv.withExec(["cargo", "fmt", "--all", "--check"]).sync
    null
  }

  """
  Lint every crate and target with Clippy, denying warnings.
  """
  pub clippy: Void @check {
    rustEnv
      .withExec(["cargo", "clippy", "--workspace", "--all-targets", "--locked", "--", "-D", "warnings"])
      .sync
    null
  }

  """
  Build and run the workspace's tests.
  """
  pub test: Void @check {
    rustEnv.withExec(["cargo", "test", "--workspace", "--locked"]).sync
    null
  }
}

rustEnv mounts two cache volumes, and all checks share them. The sections below show how each volume meets the requirements. The rustup component add step installs Clippy and rustfmt, because the official rust image has only cargo, rustc, and the standard library. Dagger caches this step like any other step.

Keep the downloaded crates in a Dagger cache volume

The cargo-home volume keeps the downloaded crates. All checks and all Rust versions share it. CARGO_HOME points to the volume, not to /usr/local/cargo in the image. This is the directory that rust-cache keeps on GitHub Actions.

Cargo’s docs advise CI to cache only parts of Cargo’s home. An archive of all of it holds the sources of each crate twice (Cargo home). A cache volume is not an archive and does not move between runs, so the module mounts all of Cargo’s home. Cargo keeps the lock for its package cache in Cargo’s home directory (Cargo source), so that lock is in the volume too.

Keep the compiled dependencies in a Dagger cache volume

The target volume, mounted at /src/target, keeps the compiled dependencies. It works like CARGO_TARGET_DIR on a long-lived runner. The directory stays where Cargo left it, and Cargo recompiles only the crates that changed. Unlike rust-cache, the module does not remove the workspace crates, so the volume also keeps the project’s own compiled crates.

The source excludes target, .git, and .dagger. Dagger does not upload your local build output, and changes to it do not invalidate Dagger’s cache.

The checks share one target directory and one Cargo home. dagger check runs all checks in parallel, and Cargo’s file locks in the two volumes make the checks take turns. When one check waits for another, Cargo prints Blocking waiting for file lock on package cache or Blocking waiting for file lock on build directory.

A target volume that lives a long time grows, as a local target/ directory does. Cargo cleans its global cache in Cargo’s home directory automatically, but not build artifacts (Cargo configuration). To start a target directory again on purpose, change the name of the volume.

Follow changes to Cargo.lock and the toolchain with Dagger

Cargo writes to the cache volumes on every run, so no cache key has to change when Cargo.lock changes. After a dependency update, Cargo finds the old artifacts in the target volume. It recompiles only the changed dependencies and the crates that depend on them (Cargo source). rust-cache gets the same result when it restores the cache of the previous lockfile.

The name of the target volume includes the Rust version. Cargo puts a hash that includes the compiler version in the file names of its artifacts (Cargo source). Artifacts from two compiler versions thus stay side by side in target/. A new name gives a new toolchain an empty target directory, so old artifacts do not collect in one volume. rust-cache also starts a new cache when rustc changes.

rustVersion is a constructor argument, so it is also a module setting. To change it, run this command:

dagger module settings <project>-dev rustVersion 1.98

The command adds the setting to dagger.toml. In a project named greeter, it adds these lines:

[modules.greeter-dev.settings]
rustVersion = "1.98"

Every check then uses the new toolchain and a new target volume.

Share the cache across CI machines and laptops with Dagger

Cache volumes live on the Dagger Engine that runs the checks. On a laptop, the engine keeps them between runs, until its garbage collector removes cache to free disk space. By default, the engine prunes its cache when free disk space falls below 20% (engine.json). The gc settings in engine.json change these limits.

In CI, Cloud Checks run the checks on Cloud Engines, the remote engines that Dagger Cloud manages for you. Cloud Engines have scale-out and a distributed cache. Developers, agents, and CI pool one cache, and the --engine=cloud flag runs a pipeline from a laptop on Cloud Engines (Cloud Engines). Dagger Cloud also removes stale cache volumes on Cloud Engines automatically (changelog).

An engine on a laptop keeps its own cache volumes. The module is the same on every engine, and the cache volumes have the same names. One module thus meets the requirements on a laptop and in Dagger Cloud, with no cache configuration for each CI platform.

Run the same cached checks locally and with Cloud Checks

Run dagger check on your machine and in CI, and the same checks run in both places. List the checks, then run them:

dagger check -l
dagger check

In a project named greeter, dagger check -l lists the module’s three checks and a staleness check from the Dang SDK:

MODULE       CHECK   DESCRIPTION
greeter-dev  clippy  Lint every crate and target with Clippy, denying warnings.
dang-sdk     stale   staleness check: re-generate modules and clients managed by the Dang SDK
greeter-dev  fmt     Check formatting with rustfmt.
greeter-dev  test    Build and run the workspace's tests.

This guide’s greeter project has two crates, with serde, serde_json, and anyhow as dependencies. On an engine with an empty cache, the runs took these times:

  • The first dagger check took 28.7 seconds. It pulled the rust image, installed Clippy and rustfmt, downloaded the crates, and compiled them.
  • A rerun with no changes took 1.9 seconds.
  • After an edit to one file in the greet crate, dagger check took 1.8 seconds. Cargo compiled only greet and cli, because cli depends on greet.

These times come from one run on one machine. They are an example, not a benchmark.

In CI, Cloud Checks run the checks. Cloud Checks run your checks on Cloud Engines, Dagger’s managed engines, after each push and pull request, with no CI workflow file. The repository must be on GitHub, with its origin remote set to it. Run this command from the project root:

dagger cloud checks on

To see the runs, use dagger workspace activity.

Bottom line: A Rust CI cache has to keep Cargo’s downloaded crates and the compiled dependencies in target/. It has to follow changes to Cargo.lock and the toolchain, and reach every machine that builds. On GitHub Actions, Swatinem/rust-cache meets most of them in a few lines of YAML. sccache shares compiled crates across a fleet of machines, and cargo-chef fits Docker builds. Dagger meets the same requirements with cache volumes, and the same dagger check runs on a laptop and through Cloud Checks.