CI that keeps up
how-toby Kyle Penfound

How do I share the Go build cache across CI runs and runners?

To share the Go build cache across CI runs, keep two directories between jobs: GOMODCACHE for downloaded modules and GOCACHE for compiled packages and test results. Key them so that the build cache gets saved again on new commits, and store them where every runner can reach them. On GitHub Actions, actions/setup-go does part of this. On GitLab CI, cache: with a go.sum key and fallback keys does it. For machines that do not share disks, a GOCACHEPROG program can serve the build cache remotely. Dagger meets the same requirements in one place. Its official Go module mounts both caches as cache volumes, and Cloud Checks run it on Cloud Engines that share their cache.

What a shared Go cache must do

A Go cache setup for CI has four requirements. Each one comes from how the go command and the CI platforms work:

  1. Keep both caches. The go command keeps downloaded modules in GOMODCACHE and build outputs in GOCACHE. It also “caches successful package test results” in GOCACHE. If you keep only the module cache, every job compiles every package again and runs every test again.
  2. Keep the build cache current. A build cache that CI saved once and never updated gets older with each commit. Each run then compiles more packages again.
  3. Stay correct. The go command’s own cache keys already account for “changes to Go source files, compilers, compiler options, and so on”, and the cache is “safe for concurrent invocations of the go command” (go help cache). The CI cache key only decides which snapshot a job gets. It does not make the build correct or incorrect.
  4. Reach every runner. A cache on one machine’s disk does not help a job that runs on another machine.

Cache Go with actions/setup-go on GitHub Actions

actions/setup-go caches Go modules and build outputs by default (cache: true). Since v6, it computes the cache key from go.mod in the repository root. To use go.sum, or to cover several Go modules, set cache-dependency-path, which accepts globs (setup-go README):

- uses: actions/setup-go@v7
  with:
    go-version-file: go.mod
    cache-dependency-path: '**/go.sum'

This meets “keep both caches”. By default, it does not keep the build cache current. The setup-go docs say that the GOCACHE cache “is not updated based on source changes”. You can add **/*.go to cache-dependency-path, but the docs warn that this “can create new caches on many commits”. When the key matches, the job restores the files and saves nothing new, because you cannot change the contents of an existing cache.

To save a new build cache on each run, turn off the built-in cache. Then use actions/cache with a unique key and a restore-keys prefix. setup-go gives the two cache paths as outputs:

- uses: actions/setup-go@v7
  id: go
  with:
    go-version-file: go.mod
    cache: false
- uses: actions/cache@v5
  with:
    path: |
      ${{ steps.go.outputs.GOMODCACHE }}
      ${{ steps.go.outputs.GOCACHE }}
    key: go-${{ runner.os }}-${{ hashFiles('**/go.sum') }}-${{ github.sha }}
    restore-keys: |
      go-${{ runner.os }}-${{ hashFiles('**/go.sum') }}-
      go-${{ runner.os }}-

restore-keys restores the most recent cache with the same prefix (cache key matching). Two rules decide which runs can share it:

GitHub stores the caches, so every runner in the repository can restore them.

Cache Go with cache keys on GitLab CI

GitLab caches only paths inside the project directory. Caches “can’t link to files outside it”. Move both Go caches into $CI_PROJECT_DIR, then give each cache its own key:

variables:
  GOMODCACHE: $CI_PROJECT_DIR/.go/pkg/mod
  GOCACHE: $CI_PROJECT_DIR/.cache/go-build

test:
  image: golang:1.26
  cache:
    - key:
        files: [go.sum]
      paths: [.go/pkg/mod]
    - key: go-build-$CI_COMMIT_REF_SLUG
      fallback_keys: [go-build-$CI_DEFAULT_BRANCH]
      paths: [.cache/go-build]
  script:
    - go test ./...

The module cache changes only when go.sum changes, so cache:key:files fits it. The build cache uses one key for each branch, and each job saves a new copy under that key. A new branch starts from the default branch’s cache through fallback_keys, which takes up to five keys.

Three settings decide who shares what:

Share the build cache remotely with GOCACHEPROG

Since Go 1.24, the GOCACHEPROG environment variable names a program that implements the build and test cache for the go command. The go command sends get and put requests for single cache entries over a JSON protocol. The program can store the entries anywhere, such as in remote storage. Depot’s depot gocache is one such program.

This meets “reach every runner” without archive files. Each job gets only the entries its build asks for, and it writes its new entries back, so the cache stays current. GOCACHEPROG replaces the build cache only. You still cache GOMODCACHE another way, and you run or buy the program and its storage.

Where the hand-built setup strains

The setups above work, but each requirement lives in a different place:

  • The keys are a design you maintain. A go.sum key stops build cache updates. A commit key needs restore-keys or fallback_keys, and one job that writes the cache.
  • Every job transfers the whole archive. GitHub and GitLab restore and save whole directories, so a larger build cache is a larger download and upload in each job.
  • The limits are the platform’s. GitHub removes caches that no run used in 7 days and limits a repository to 10 GB by default. GitLab separates protected branches, and it needs S3 or a network directory for several runners.
  • The CI caches stay in CI. The setup-go and GitLab caches do not reach the laptops where developers run the same tests.

Keep both caches with the Dagger Go module

The Dagger Go module runs go test and go generate for every Go module in the repository. A Dagger module is a package of functions that build and test a project. The Go module mounts GOMODCACHE and GOCACHE as cache volumes, which are directories whose contents persist across runs on the Dagger engine. The engine is the process that runs the module’s containers. You write no cache configuration.

Complete the Quickstart in the repository first. Then install the module:

dagger module install dagger.io/go
dagger.io/go ➡️ https://github.com/dagger/go#v0.3.1
Installed module "go" in /work/dagger.toml

Run its checks. A check is a function that passes or fails, such as a test run:

dagger check go
== CHECKS ==  ✔ 2 passed
✔ dag://?check=go/packages/generate/stale 0.5s OK
✔ dag://?check=test&go-package=. 4.8s OK
  == TESTS ==
    ✔ example.com/inventory  1 passed

    ✔ 1 passed

The module mounts the module cache at /go/pkg/mod and the build cache at /root/.cache/go-build, and it sets GOMODCACHE and GOCACHE to those paths. Every Go command that the module runs uses the same two volumes.

This guide’s project has one package that imports the etcd v3 client. The first dagger check go took 6.5 seconds, and its test check took 4.8 seconds. After a one-line change to the package source, dagger check go took 1.6 seconds, and the test check took 0.6 seconds. That is one run on one machine, not a benchmark.

Keep the cache current and correct with Dagger

The cache volumes have no key to design. Each run reads from the same volumes and writes its new entries back to them, so the build cache stays current on every commit. The go command’s own cache keys keep the build correct, the same as on a CI runner. Concurrent checks share the volumes, which the Go cache allows.

Dagger also caches each check’s result by exactly the files the check saw (Daggerize a Go Project). In this guide’s project, a rerun with no changes took 1.5 seconds for dagger check go, and the test check took 0.4 seconds.

Share the cache across runners with Cloud Checks

Cloud Checks run your checks on Cloud Engines on each push and pull request, with no CI workflow file. Cloud Engines are Dagger engines that Dagger Cloud runs for you, with a distributed cache. Turn on Cloud Checks from the project root:

dagger cloud checks on

The command needs a Dagger Cloud account and a connected GitHub repository, and it tells you how to complete what is missing. After that, each push runs the same dagger check that you run locally, with the same Go module and the same cache volumes.

According to the changelog, Cloud Engines share their cache through copies of a shared cache disk, and these copies are best effort. A version that shares single results is in the changelog’s Coming soon section and is not released.

To share one cache between developers and CI, run local commands on the same engines with dagger check --engine=cloud. To use your own machine instead, point every runner at one engine, for example with --engine=ssh://build-host (remote execution).

Requirement setup-go GitLab cache: GOCACHEPROG Dagger Go module
Keep both caches Yes, by default Yes, after you move both into the project directory Build cache only Yes, two cache volumes
Keep the build cache current Only with a custom key or source globs With a branch key and fallback keys Yes, entry by entry Yes, every run writes back
Reach every runner GitHub cache storage S3 or a network directory for several runners The program’s storage Cloud Engines, or one shared engine

Bottom line: Keep both GOMODCACHE and GOCACHE, and key the build cache so that it gets saved again on new commits. Store it where every runner can reach it: setup-go with a custom key on GitHub Actions, branch keys with fallbacks and S3 on GitLab CI, or GOCACHEPROG for remote entries. The Dagger Go module keeps both caches in cache volumes with no keys to maintain, and Cloud Checks run it on Cloud Engines that share their cache.