CI that keeps up
how-toby Kyle Penfound

How do I run Go tests, lint, and go generate the same way locally and in CI?

To run a Go project’s tests, lint, and go generate the same way on a laptop and in CI, the setup has five requirements. It uses one Go toolchain and the same tools, and it tests every Go module. It starts the services that tests need, fails on stale generated files, and pins the linter. By hand, each laptop and each CI job repeats the go commands and installs the tools. Dagger meets the same requirements in one place. The official Go module and a linter module run them as checks, and dagger check runs the checks on a laptop and in Cloud Checks.

What a good Go CI setup needs

A Go setup that gives the same result on a laptop and in CI meets five requirements. Each one comes from how the go command works:

  1. Use one Go toolchain and the same tools. The go line in go.mod “declares the minimum required Go version” for the module. A //go:generate command must be “an executable file that can be run locally”, such as protoc. The go command turns cgo off when it cannot find a C compiler on the PATH. Different tools on two machines give different results.
  2. Test every package in every Go module. The ./... pattern does not match directories that hold their own go.mod. You reach those packages only when you change into that module’s directory.
  3. Start the services that tests need. A test that needs a database can skip at run time when the database is not there. A skipped test does not fail, so CI must start the service, or the test never runs.
  4. Fail on stale generated files. Go generate “is never run automatically by go build, go test, and so on”. A package that others import must have its generated files in the repository. Thus CI must check that the committed files match their source.
  5. Lint with a pinned linter version. The golangci-lint docs recommend “installing a specific version of golangci-lint” in CI. The Staticcheck docs give the same advice.

The examples come from a small project with the Go module example.com/greetings. It has a store package with tests, and a version package with a //go:generate line.

Run the Go toolchain and tests by hand

Most Go teams run the go commands on a laptop, and repeat them as steps of a CI job. The Go version comes from go.mod in both places. Since Go 1.21, the default GOTOOLCHAIN=auto setting “downloads newer toolchains as needed”. In this project, go.mod has go 1.26, and the laptop has Go 1.24:

go version
go version go1.26.0 darwin/arm64

In GitHub Actions, actions/setup-go reads the version from go.mod through its go-version-file input (setup-go). Tools outside Go, such as protoc or a C compiler, come from each laptop’s package manager and from an install step in the CI job.

Run the tests of the module:

go test ./...
ok  	example.com/greetings	0.005s
ok  	example.com/greetings/store	0.005s
?   	example.com/greetings/version	[no test files]

The run passes, but TestDatabase did not run. It skips itself when DATABASE_URL is not set, and only -v shows the skip. On a laptop, start Postgres in a container, then give the test its address:

docker run --rm -d --name greetings-db -p 5432:5432 -e POSTGRES_USER=test -e POSTGRES_PASSWORD=test -e POSTGRES_DB=test postgres:17-alpine
DATABASE_URL="postgres://test:test@localhost:5432/test?sslmode=disable" go test -v -run TestDatabase ./store/
=== RUN   TestDatabase
--- PASS: TestDatabase (0.00s)
PASS
ok  	example.com/greetings/store	0.009s

In GitHub Actions, the job gets the same database from a service container in the workflow file. A repository with several Go modules runs go test ./... once in each module directory, often in a shell loop or a job matrix.

Check go generate output and lint by hand

The usual drift check runs go generate, then fails when Git sees a change. git diff --exit-code exits with status 1 when there are differences. In this run, the version template changed, but nobody ran the generator:

go generate ./... && git diff --exit-code
diff --git a/version/version_gen.go b/version/version_gen.go
index 77decfd..71b11b0 100644
--- a/version/version_gen.go
+++ b/version/version_gen.go
@@ -1,4 +1,4 @@
 package version
 
 // Version is the release version.
-const Version = "1.4.0"
+const Version = "1.5.0"

The generator also needs its tools on the machine, such as protoc for a protobuf generator.

Lint runs as one more step, with a pinned linter release. The guides for golangci-lint and Staticcheck show the commands for each linter.

Where the hand-built setup strains

The commands above work, but each requirement lives in more than one place:

  • The tools have no version file. go.mod sets the Go version, but protoc and C libraries come from each laptop and from the CI job’s install step. Their versions can differ.
  • The project defines its services twice. A docker run command on a laptop and a service container in the workflow describe the same database in two formats.
  • Each Go module needs its own run. The loop or the matrix over modules lives in CI configuration, and each repository writes its own.
  • The drift check reads the working tree. git diff --exit-code reports uncommitted local edits as drift too, so it is reliable only in a clean checkout.
  • A CI failure is hard to repeat. To repeat a failed job, a developer reads the workflow and runs its steps by hand, with the tools of the laptop.

Meet the requirements with the Dagger Go module

The Dagger Go module meets the first four requirements with checks, and a linter module meets the fifth. A Dagger module is a package of functions, checks, and services that a workspace can run. A workspace is the Dagger configuration of a repository, in a file named dagger.toml at its root (dagger.toml). The project must be a Git repository. Create the workspace with the Quickstart, which also recommends modules for the project.

The Dagger team maintains official modules for common tools, and the Go module is one of them. It finds every go.mod in the repository. The workspace installs it under the name go:

dagger module install dagger.io/go

If the Quickstart installed the module already, the command prints Module "go" is already installed and changes nothing. Each install command in this guide does the same. dagger module list shows the modules that the workspace has.

A check is a function that validates the project and passes or fails. List the checks that the Go module adds:

dagger check go -l
MODULE  GO-PACKAGE  GO-TEST  CHECK  DESCRIPTION
go      *                    stale  staleness check: run go generate in this package
go      *           *        test   Run this test.
# Use --all to expand collections and list each item.

Go packages and Go tests are collections. A * in a row stands for each package or each test that the module found. dagger check go -l --all lists one row per package and test. The sections below meet each requirement with these checks.

Use one Go toolchain with the Dagger Go module

The Go module runs each Go module in golang:<version>-alpine, with the version that its go.mod asks for (Go module reference). The same image runs on each laptop and in CI. dagger.lock pins the image, as Run every check locally and with Cloud Checks explains.

To use one Go version for every Go module, set it. A setting changes a default of the module, and the workspace keeps it in dagger.toml:

dagger module settings go version 1.26

The settings apply to each person and each machine that uses the repository.

If the project builds with tags, set them once in goflags, so every Go command uses the same flags. A value that starts with a dash needs -- before it. Without --, the CLI reads the value as its own flags and fails:

invalid argument "extended" for "-s, --silent" flag: strconv.ParseBool: parsing "extended": invalid syntax

Quotes do not help, because the shell removes them before the CLI reads the value. Put -- before the value:

dagger module settings go goflags -- -tags=extended

The command writes goflags = "-tags=extended" under [modules.go.settings] in dagger.toml. The Go module reference describes the settings that this guide does not need, such as build.

Use the same tools with a custom Dagger runtime

If the tests and generators need only Go and the tools in the default image, skip this section. Tests and generators often need more system packages, such as git, a C compiler for cgo, or protoc. The base setting of the Go module takes a container with those packages. A function in a module of your own builds that container.

Write the function in Dang, the native language of Dagger for modules, with no code generation and no build step. If the workspace does not have the Dang SDK, install it. If the project has no dev module, create one:

dagger module install dagger.io/sdk/dang
dagger module init dang

dagger module init creates the module at .dagger/modules/<project>-dev and records it in dagger.toml. It also makes the module the main module of the workspace, so commands accept its functions without the module name. A new module has starter functions, which you can delete.

Add the testRuntime function to .dagger/modules/<project>-dev/main.dang. After you delete the starter functions, the file is:

"""
Project-specific runtime and services for the Go checks.
"""
type ProjectDev {
  """
  A Debian-based Go image with protoc installed, for generators that compile
  protobuf definitions.
  """
  pub testRuntime: Container! {
    container
      .from("golang:1.26-bookworm")
      .withExec(["apt-get", "update"])
      .withExec(["apt-get", "install", "-y", "--no-install-recommends", "protobuf-compiler"])
  }
}

ProjectDev stands in for the type name that dagger module init generated from the module name. Keep the generated type name. Otherwise the module fails to load with no main object.

testRuntime starts from the Debian-based golang:1.26-bookworm image, not the Alpine default. That image includes git and gcc. The function also installs protobuf-compiler, so a //go:generate line that runs protoc works. Keep the function minimal:

  • Only an image and its packages. Do not add a working directory, source files, or cache mounts. The Go module adds its own.
  • The image sets the Go version. When base has a value, each Go module in the workspace uses the Go toolchain of this container.

Run protoc --version in the container that the Go module will get:

dagger shell test-runtime -c 'protoc --version'
libprotoc 3.21.12

Wire the runtime into the Go module

Module wiring puts the value that a function returns into a setting of another module. A setting value of the form dag://<module>/<function> names a function of an installed module. Dagger calls that function when it loads the Go module, and passes the result as the value of the setting. Thus the official module gets a container for this project, and you write no test check of your own.

The base container brings its own Go toolchain, so the Go module ignores the version setting when base has a value. Remove version. Then set base:

dagger module settings -u go version
dagger module settings go base test-runtime

The dev module is the main module of the workspace, so the command accepts test-runtime without the module name. It writes the full address, dag://<project>-dev/test-runtime, to dagger.toml. Run the checks again:

dagger check

Tests and generators now run in the container that testRuntime builds, on each laptop and in CI.

Test every package in every Go module with Dagger

The test check of the Go module runs one go test command for each package, in each Go module that it found. You write no loop over modules. Run the checks:

dagger check go

The report shows each Go test on its own line, so a failure names the test that failed. To leave a Go module out, use the test setting, such as ["**", "!legacy-service"]. A ! pattern excludes a module root (Go module reference).

To run part of the suite, select packages and tests by name. This command runs one test:

dagger check --go-package=store --go-test=TestDatabase

Checking your code shows each way to select packages and tests. A test that skips itself because a service is not available is not a failure. The check passes, but unlike go test ./..., the report adds a skipped count for the package:

== CHECKS ==  ✔ 1 passed
✔ dag://?check=test&go-package=store&go-test=TestDatabase 0.5s OK
  == TESTS ==
    ∅ example.com/greetings/store › TestDatabase SKIP
              store_test.go:15: DATABASE_URL not set

    ∅ 1 skipped

If the report shows a skipped count, the project needs the Postgres service section of this guide. If a test fails because it cannot open a file, see Declare the files a test or generator reads.

Start the services that tests need with Dagger

Give the Go module a test runtime that has the services already. The testRuntime function describes the runtime, so the services go there too. The tests keep the two conventions from the hand-built setup, so the project has no Dagger code in its tests:

  • The tests read the address of the service from an environment variable, such as DATABASE_URL.
  • The tests skip when the variable is unset, so go test ./... still passes on a laptop without the service.

Add a postgres function to .dagger/modules/<project>-dev/main.dang. Bind it in testRuntime. The file is now:

"""
Project-specific runtime and services for the Go checks.
"""
type ProjectDev {
  """
  A Postgres database for development and tests.
  """
  pub postgres: Service! {
    container
      .from("postgres:17-alpine")
      .withEnvVariable("POSTGRES_USER", "test")
      .withEnvVariable("POSTGRES_PASSWORD", "test")
      .withEnvVariable("POSTGRES_DB", "test")
      .withExposedPort(5432)
      .asService
  }

  """
  A Debian-based Go image with protoc installed, Postgres bound as a service,
  and DATABASE_URL pointing at it.
  """
  pub testRuntime: Container! {
    container
      .from("golang:1.26-bookworm")
      .withExec(["apt-get", "update"])
      .withExec(["apt-get", "install", "-y", "--no-install-recommends", "protobuf-compiler"])
      .withServiceBinding("db", postgres)
      .withEnvVariable("DATABASE_URL", "postgres://test:test@db:5432/test?sslmode=disable")
  }
}

postgres describes the database once, in place of the docker run command and the service container. It returns a Service, so dagger start can also start it for local development (Running services).

testRuntime has two new lines. withServiceBinding makes Postgres available at the host name db. Each container that the Go module makes from this base keeps the binding, so Postgres runs when go test runs. DATABASE_URL points to that address, and the tests read it.

List the services of the workspace:

dagger start -l
MODULE         SERVICE   DESCRIPTION
greetings-dev  postgres  A Postgres database for development and tests.

The MODULE column shows the name of the dev module, greetings-dev in this project. If base does not have the runtime yet, create the dev module first, then wire the runtime:

dagger module settings go base test-runtime

Run the tests again:

dagger check --go --check=test
== CHECKS ==  ✔ 2 passed
✔ dag://?check=test&go-package=. 1.4s OK
  == TESTS ==
    ✔ example.com/greetings  1 passed

    ✔ 1 passed
✔ dag://?check=test&go-package=store 1.4s OK
  == TESTS ==
    ✔ example.com/greetings/store  2 passed

    ✔ 2 passed

== SERVICES ==
✔ ku62l0th05qka.4ib43cj4f1spm.dagger.local exec docker-entrypoint.sh postgres 1.0s EXITED span=a7e5e9e140f8d6ff

The test that skipped before now runs, and the SERVICES section shows Postgres. The Go module still owns the test logic. The code of the project is two functions that describe its runtime and its database, and dagger.toml connects them.

Fail on stale generated files with Dagger

The stale check of the Go module runs go generate and fails when it would change a committed file. It reports only the changes that go generate makes, so other local edits do not fail it. The module also has a generator for go generate. A generator is a function that proposes changes to files (Generating code). It runs go generate in each package that has a //go:generate directive:

dagger generate

Dagger shows the result as a changeset, a diff of the proposed changes, for you to review. It writes nothing until you apply the changeset. dagger check includes the stale check, so a pull request with a forgotten go generate fails the checks. The generate setting selects the directories where go generate runs.

Declare the files a test or generator reads

If the tests and generators read only Go source, embedded files, and testdata/ directories, skip this section. Dagger runs each Go command against a directory that it assembles from the repository, not against the checkout itself. Thus Dagger caches the result of a check by exactly the files that the check saw. The Go module reads the Go source to choose those files. In addition to the source files and go.mod, it adds what the code declares:

  • The files that //go:embed names.
  • The Go module that a go -C argument names on a //go:generate line.
  • Local replace targets.

Tests also get each testdata/ directory.

The scan does not see a file that a test opens by a relative path, or a file that a generator reads as input. For these files, the Go module reads two directives, as an alternative to the includeExtraFiles setting:

  • //go:test:include names files to mount when the tests of the module run.
  • //go:generate:include names files to mount when go generate runs.

Each directive takes one or more patterns, with the same syntax as //go:embed. A pattern is relative to the directory of the file that holds the directive. A / at the start makes a pattern relative to the repository root. Put quotes around a path that contains a space. The directive can go in any Go file in the module, such as a _test.go file, next to the code that needs the file.

A test in store/ that reads ../fixtures/users.json declares the file in one line. In this project, the line is in store/fixture_test.go, below the package clause:

//go:test:include ../fixtures/*.json

Without the directive, the test fails as it would on a machine that does not have the file:

dagger check --go-package=store --go-test=TestFixture
== CHECKS ==  ✘ 1 failed
✘ dag://?check=test&go-package=store&go-test=TestFixture 1.6s ERROR
  == TESTS ==
    ✘ example.com/greetings/store FAIL

    ✘ example.com/greetings/store › TestFixture FAIL
              fixture_test.go:12: open ../fixtures/users.json: no such file or directory

    ✘ 1 failed

A generator that reads a template declares it the same way. In version/gen.go, the line goes above the //go:generate line that runs the generator:

//go:generate:include templates/version.txt

Without its input, a generator fails or writes different output, and the stale check reports the difference as drift. Add the directives, and both checks pass.

Choose a directive or includeExtraFiles

Use a directive when one package needs the file. The directive stays with the code, and it applies only to the tests or the generator that need the file. Use includeExtraFiles when each Go module in the workspace needs the file, or when you do not want to change the source. The linter modules read only //go:embed. Thus a file that the linter needs goes in the includeExtraFiles setting of the linter module.

The Dagger repository uses //go:generate:include for its CLI reference. A Go file with no code, generate.go, includes the rendered page. Below it, the //go:generate go -C ../../../../ run ./internal/cmd/dagger/docsgen line renders the page. The stale check compares the new page with the committed page. The go -C argument adds the root Go module of the repository as input to the generator.

Lint with a pinned linter version in the same Dagger run

Lint comes from a separate module, so a project keeps the linter that it uses now. Each linter module pins a release by default, and each one lints the directories that hold its configuration file:

  • golangci-lint, from dagger.io/go/golangci-lint, runs golangci-lint run where the project has a .golangci.yml or another golangci-lint configuration file.
  • Staticcheck, from dagger.io/go/staticcheck, runs staticcheck where the project has a staticcheck.conf.

This guide’s project uses golangci-lint. For Staticcheck, use staticcheck in place of golangci-lint:

dagger module install dagger.io/go/golangci-lint

The check list now has a lint check next to the Go checks:

dagger check -l
MODULE         GO-PACKAGE  GO-TEST  GOLANGCI-LINT-PROJECT  CHECK  DESCRIPTION
dang-sdk                                                   stale  staleness check: re-generate modules and clients managed by the Dang SDK
go             *                                           stale  staleness check: run go generate in this package
go             *           *                               test   Run this test.
golangci-lint                       *                      lint   Run golangci-lint in this project.
# Use --all to expand collections and list each item.

The lint check runs in its own container, so the version and base settings of the Go module do not change it. The linter module has its own settings. The golangci-lint guide and the Staticcheck guide cover the configuration file, the pinned version, and the other lint requirements.

Run every check locally and with Cloud Checks

Run every check in the workspace:

dagger check
== CHECKS ==  ✔ 5 passed
✔ dag://?check=dang-sdk/generate/stale 0.9s OK
✔ dag://?check=go/packages/generate/stale 2.0s OK
✔ dag://?check=golangci-lint/projects/lint 2.0s OK
✔ dag://?check=test&go-package=. 2.0s OK
  == TESTS ==
    ✔ example.com/greetings  1 passed

    ✔ 1 passed
✔ dag://?check=test&go-package=store 2.0s OK
  == TESTS ==
    ✔ example.com/greetings/store  2 passed

    ✔ 2 passed

== SERVICES ==
✔ ku62l0th05qka.utlb5mb6895ge.dagger.local exec docker-entrypoint.sh postgres 1.4s EXITED span=e01d1691a4159a5f

Dagger names each check by its link. A link names packages or tests only when the check does not cover all of them, such as go-package=store. The Dang SDK adds a stale check of its own for the generated files of the dev module.

For a workspace with only the modules from this guide, dagger.toml is now:

[modules.go]
source = "dagger.io/go"

[modules.dang-sdk]
source = "dagger.io/sdk/dang"

[sdks.dang]
module = "dang-sdk"

[modules.<project>-dev]
main = true
source = ".dagger/modules/<project>-dev"

[sdks.dang.scopes.".dagger/modules/<project>-dev"]
is-module = true

[modules.go.settings]
base = "dag://<project>-dev/test-runtime"

[modules.golangci-lint]
source = "dagger.io/go/golangci-lint"

Commit the configuration, the lock file, and the module. Dagger needs all three to load the workspace the same way on each machine:

git add dagger.toml dagger.lock .dagger
git commit -m "Daggerize"

dagger.lock pins each image and Git ref that the workspace resolved. Examples are the golang image tag behind each Go version and the images of the dev module. Any Dagger command can add entries to it. dagger lock update refreshes the pinned versions.

Cloud Checks run the same checks on Dagger Cloud after each push and pull request, with no CI workflow file. The repository must be on GitHub, and its origin remote must point there. From the project root, run:

dagger cloud checks on

In a terminal, the command helps you sign in to Dagger Cloud and connect the GitHub account or organization. Then push the commit. Each push and pull request then runs these checks, with the Postgres service, on Dagger’s Cloud Engines. dagger workspace activity shows the recent runs. To repeat a failed check, run dagger check on the laptop.

Bottom line: To run Go tests, lint, and go generate the same way locally and in CI, pin the toolchain, the tools, and the linter. Test every Go module with its services, and fail when generated files are stale. With Dagger, dagger.io/go and a linter module meet these requirements as checks, and dagger check runs them on a laptop and in Cloud Checks.