CI that keeps up
how-toby Kyle Penfound

How do I make Go builds faster?

Most slow Go builds have one cause: the go command starts with an empty cache. A fast Go build keeps the build cache (GOCACHE) and the module cache (GOMODCACHE) between runs. It also keeps the inputs of each cache key stable, lets go test reuse passed results, and builds cgo code with a C compiler. On a laptop, Go keeps its caches for you. On a CI runner that starts empty, you must restore both caches yourself. Dagger meets the same requirements in one place. The dagger.io/go module mounts both Go caches as persistent volumes and runs the tests of each package as a separate check, in parallel, locally and in CI.

What makes a Go build slow

A fast Go build meets these requirements. The go command documentation is the source for each one.

  1. Keep the build cache. The go command caches build outputs in GOCACHE, which is ~/.cache/go-build on Linux. Since Go 1.20, Go does not ship precompiled standard library packages. They are “built as needed and cached in the build cache”, so an empty GOCACHE also compiles fmt, net/http, and the rest of the standard library again.
  2. Keep the module cache. The go command stores downloaded modules in GOMODCACHE, which defaults to GOPATH/pkg/mod. An empty module cache downloads every dependency again.
  3. Keep cache keys stable. The build cache “correctly accounts for changes to Go source files, compilers, compiler options, and so on”. A change to the Go version, CGO_ENABLED, -race, or -tags gives new cache keys, so the build compiles again.
  4. Let go test reuse results. go test caches passed results only in package list mode, and only with the cacheable flags.
  5. Handle cgo on purpose. cgo needs a C compiler, and “the build cache does not detect changes to C libraries imported with cgo”.

Measure before you change anything

Time the build first, then ask the go command why it rebuilt. Use these tools from the go help cache and go help build docs:

  • time go test ./... gives the wall time. A second run shows how much the cache saves.
  • go build -x prints each command that runs. A warm build with no change prints no compile commands.
  • GODEBUG=gocachehash=1 prints the inputs of each cache key. Compare the output of two runs to find the input that changed.
  • GODEBUG=gocachetest=1 prints why go test did or did not reuse a cached result.
  • GODEBUG=gocacheverify=1 ignores the cache, rebuilds everything, and checks that the results match the cache entries.
  • go build -a rebuilds all packages, also packages that are up to date.

This guide used a project with six packages that import the AWS SDK for Go v2 and k8s.io/client-go, with 167 lines in go.sum. In a golang:1.26-bookworm container with empty caches, each command took this long:

Command Time
go mod download 4.9 s
go test ./..., first run 14.0 s
go test ./..., no change 0.18 s, every package (cached)
go test -count=1 ./... 0.8 s
go test ./... after a one-line change in one package 0.8 s

These numbers come from one run on one machine. In this project, the first download and compile took most of the time, so keep the caches first.

Keep GOCACHE and GOMODCACHE between runs

The build cache and the module cache help only if they exist at the next run. On a developer machine, they stay in the home directory. The go command “periodically deletes cached data that has not been used recently”, so you do not need to clean the cache. Use go clean -cache only to debug the cache.

On a CI runner that starts empty, save both directories at the end of a job and restore them at the start of the next job. Get the paths with go env GOCACHE GOMODCACHE. On GitHub Actions, actions/setup-go caches modules and build outputs by default. By default, it makes the cache key from go.mod, and the cache-dependency-path input changes the file it uses. For full control, use actions/cache with your own key.

Keep cache keys stable

Each change to a compiler option gives a new set of cache keys. In this guide’s project, a warm build took 0.8 s. The next build, with CGO_ENABLED=0, took 8.7 s, because the go command compiled each package again for the new keys. Use the same Go version, GOFLAGS, build tags, and CGO_ENABLED value everywhere that shares a cache. -race and -cover are build flags too, so their builds use separate cache entries.

A binary build has one more step after the compile: the link. In this guide’s project, a one-line change to the main package rebuilt the binary in 0.4 s. With an empty cache, the same build took 5.8 s. The go command compiled only the changed package, then linked the binary again.

Let go test reuse passed results

go test caches a passed result only in package list mode. The go help test docs define the two modes:

  • Local directory mode is go test with no package arguments. “In this mode, caching (discussed below) is disabled.”
  • Package list mode is go test with package arguments, such as go test ./... or go test .. A passed package prints (cached) the next time.

go test reuses a result only when the flags “come entirely from a restricted set of ‘cacheable’ test flags”: -benchtime, -coverprofile, -cpu, -failfast, -fullpath, -list, -outputdir, -parallel, -run, -short, -skip, -timeout, and -v. Any other flag turns off the test cache. “The idiomatic way to disable test caching explicitly is to use -count=1.” Search your Makefile and CI scripts for -count=1. Keep it only on tests that must run each time, such as tests against a live service.

Tests that open files in the module or read environment variables “only match future runs in which the files and environment variables are unchanged”. Run GODEBUG=gocachetest=1 go test ./pkg to see which input made a test run again. Run go clean -testcache to remove the cached test results and keep the build cache.

Handle cgo on purpose

cgo packages need a C compiler, and the build cache does not track the C libraries they use. If the C libraries on the system change, use go build -a or clean the cache, as go help cache says.

Since Go 1.20, CGO_ENABLED defaults to 0 when the go command cannot find a C compiler. The golang:1.26-alpine image has no gcc, so a file with import "C" drops out of the build there. In this guide’s run, the error named a missing symbol, not cgo. If the project does not need cgo, set CGO_ENABLED=0 everywhere, so that the cache keys stay the same. If it needs cgo, build in an image that has a C compiler, such as golang:1.26-bookworm.

Where the hand-built cache strains in CI

The requirements stay the same as a project grows, but the CI configuration that meets them gets harder to maintain:

  • The cache goes stale. “You cannot change the contents of an existing cache,” as the GitHub Actions cache reference says. With a key on go.mod, the saved build cache keeps old compiled packages until go.mod changes. A key on the source files gives a new cache for each commit, and needs restore-keys as a fallback.
  • The settings live in many places. The Go version, CGO_ENABLED, and build tags are in the Makefile, the workflow, and the shell of each developer. When one copy changes and the others do not, the cache keys no longer match.

Keep the Go caches warm with Dagger

Dagger runs the same go commands in containers on the Dagger Engine, which runs pipelines and stores the cache. A module is a package of functions that build and test a project. The official dagger.io/go module finds each go.mod. It runs the tests of each package as a check, a function that passes or fails. Complete the Quickstart first, in a Git repository.

Install the Go module:

dagger module install dagger.io/go
Installed module "go" in /work/dagger.toml

The command adds the module to dagger.toml. It also writes dagger.lock, which pins the module to an exact version. Later Dagger commands add each image that they use to dagger.lock. Commit both files. Then run the checks:

dagger check

Keep the build cache and the module cache. The Go module mounts two cache volumes, directories whose contents persist across Dagger sessions. One is at /go/pkg/mod, the module cache. The other is at /root/.cache/go-build, the build cache. You do not write a save step or a restore step.

Keep cache keys stable. Each Go module runs in golang:<version>-alpine, for the version that its own go.mod asks for. The version, goflags, and buildFlags settings in dagger.toml apply to every run, on every machine (Go module reference).

Let go test reuse results. Dagger caches the result of a check by the files that the check saw (Daggerize a Go Project). Each package is a separate check that runs go test on that package, and dagger check runs all checks in parallel (Checking your code). The second run with no changes printed this:

== CHECKS ==  ✔ 7 passed
✔ dag://?check=go/packages/generate/stale 1.1s OK
✔ dag://?check=test&go-package=cart 1.0s OK
✔ dag://?check=test&go-package=inventory 1.0s OK
✔ dag://?check=test&go-package=orders 1.0s OK
✔ dag://?check=test&go-package=payments 1.0s OK
✔ dag://?check=test&go-package=shipping 1.0s OK
✔ dag://?check=test&go-package=users 1.0s OK

The same six-package project gave these times on one machine:

dagger check run Time
First run, new dependencies 49.1 s
No change 4.7 s
One-line change in orders 5.6 s
Change to README.md only 3.8 s

Dagger keeps the caches on each engine, so a CI run that uses the same engine starts warm. To test only the package you changed, select it:

dagger check --go-package=orders

Run cgo packages in an image with a C compiler

If no package uses cgo, skip this section. The default Alpine image has no gcc, so a cgo package does not build there. For a test of a cgo function, dagger check printed this error:

  ┃ # example.com/shop/hash [example.com/shop/hash.test]
  ┃ hash/hash_test.go:6:5: undefined: Abs
  ┃ FAIL    example.com/shop/hash [build failed]

Give the Go module a container with a C compiler through its base setting. Install the Dang SDK. Then create the project’s dev module in Dang, the native language of Dagger modules:

dagger module install dagger.io/sdk/dang
dagger module init dang

dagger module init creates .dagger/modules/<project>-dev. Replace the contents of .dagger/modules/<project>-dev/main.dang with this function. ProjectDev stands in for the type name that dagger module init generated, so keep your own type name:

type ProjectDev {
  """
  The container the Go module runs tests in: Debian, with gcc for cgo.
  """
  testRuntime: Container! {
    container.from("golang:1.26-bookworm")
  }
}

Connect the function to the Go module:

dagger module settings go base test-runtime

dagger.toml now holds base = "dag://<project>-dev/test-runtime" under [modules.go.settings]. The image now sets the Go version, so the module ignores its version setting. This is the same fix as the hand-built version: an image with a C compiler. Run dagger check again, and the cgo test passes.

Run the same checks locally and in CI

Cloud Checks run dagger check on Dagger’s Cloud Engines for each push and pull request to a GitHub repository, with no CI workflow file. Connect the repository:

dagger cloud checks on

The checks in CI are the checks that dagger check runs on your machine, with the same image, the same GOFLAGS, and the same cache keys.

Bottom line: To make Go builds faster, keep GOCACHE and GOMODCACHE between runs, and use the same compiler options and Go version everywhere. Run go test in package list mode without -count=1, and build cgo code in an image with a C compiler. The dagger.io/go module keeps both Go caches in persistent volumes and runs the tests of each package in parallel, locally and with Cloud Checks.