# How do I migrate from GitHub Actions to Dagger? URL: https://dagger.io/learn/migrate-from-github-actions You do not have to rewrite your workflows as Dagger code. Dagger's official modules find the tests and linters in your repository and run them as checks, on your machine and in Dagger Cloud. Move in four steps: install the modules, run the checks locally, turn on Cloud Checks, then turn off each workflow that the checks replace. ## Do I have to rewrite my pipelines? No, not for the work that an official module does. A [Dagger module](https://docs.dagger.io/reference/modules/index) adds work to your project, such as tests or lint, and the Dagger team maintains official modules for common tools. Each official module finds the files of its tool in your repository. For example, the [Go module](https://docs.dagger.io/reference/modules/go) finds each `go.mod`, and makes each Go package and test a check. A check is a command that verifies your project and passes or fails, such as a test run or a linter. Today the official modules cover Go, golangci-lint, Staticcheck, Deno, Jest, Vitest, Playwright, ESLint, Prettier, Biome, pytest, ShellCheck, PSScriptAnalyzer, and Helm. The [module reference](https://docs.dagger.io/reference/modules/index) lists them. You write code only for work that no official module does, such as a deploy step. You do not copy the structure of your workflow YAML. ## Install the official modules [Install the Dagger CLI](https://docs.dagger.io/getting-started/install). Then, from the root of your repository, initialize the workspace: ```shell dagger init ``` The workspace is the Dagger configuration of your repository, in a file named `dagger.toml` ([Quickstart](https://docs.dagger.io/getting-started/quickstart)). In a terminal, `dagger init` asks which modules to install. Without a terminal, it prints the next commands: ```text Workspace configuration initialized at ./dagger.toml To continue setup, run these commands in order: #!/bin/sh # 1. Find and install suitable modules dagger module recommend # 2. Enable cloud checks dagger cloud checks on ``` Install the modules that Dagger recommends for your repository: ```shell dagger module recommend ``` In a terminal, select the modules to install. To install all of them with no prompt, add `-y`. In a Go repository, the command installs the Go module and adds it to `dagger.toml`: ```toml [modules.go] source = "dagger.io/go" ``` The command also writes `dagger.lock`. This file pins the versions of the modules and images that the workspace uses. Commit it with `dagger.toml`. If Dagger recommends no module for one of your tools, search for one by the name of the tool, such as `dagger module search eslint`. ## Run the checks on your machine List the checks that the modules found: ```shell dagger check -l ``` ```text MODULE GO-PACKAGE GO-TEST CHECK DESCRIPTION go * stale staleness check: run go generate in this package go * * test Run this test. ``` A `*` stands for each item that the module found, such as each package or each test. Add `--all` to list one row per item. Run all the checks: ```shell dagger check ``` The command exits with a non-zero code when a check fails ([Checking your code](https://docs.dagger.io/using/checking)). To run one test, select it by name: ```shell dagger check --go-test=TestGet ``` Compare the list of checks with the jobs in your workflows. Each job that a check replaces is a job that you can turn off later. ## Move custom steps into your own module Write a Dagger module for each step that no official module covers, such as a release or a deploy. You can write modules in Dang, Dagger's own language, or in Go, Python, TypeScript, Java, PHP, or Elixir ([SDKs](https://docs.dagger.io/reference/sdks/index)). A module can find out if it runs on the main branch. Use that to deploy to production from main, and to a preview environment from a pull request ([branch filtering](https://dagger.io/changelog/branch-filtering)). Keep your secrets where they are. Dagger reads a secret from its provider when a function needs it. Providers include HashiCorp Vault, 1Password, AWS Secrets Manager, and environment variables ([Secrets](https://docs.dagger.io/config/secrets)). A function gives a secret to a container as an environment variable or as a mounted file ([Container API](https://docs.dagger.io/reference/api/container)). Dagger redacts secret values from logs, never writes them to container layers, and leaves them out of cache keys ([Secrets](https://docs.dagger.io/config/secrets#safeguards)). ## Turn on Cloud Checks Cloud Checks run your checks on Dagger Cloud after each push and pull request, with no workflow file ([Set up Cloud Checks](https://docs.dagger.io/cloud-checks)). They use the same checks as `dagger check` on your machine. The repository must be on GitHub, and its `origin` remote must point to it. From the root of your repository, run: ```shell dagger cloud checks on ``` In a terminal, the command helps you sign in to Dagger Cloud and connect your GitHub organization. Then commit `dagger.toml` and `dagger.lock`, and push. The push starts your checks. Each check shows on the pull request as a GitHub check, with a link to its trace ([GitHub Checks integration](https://dagger.io/changelog/more-reliable-github-checks-integration)). ## Move one workflow at a time Keep GitHub Actions on while you set up Dagger. For each workflow, compare its results with the Dagger checks that replace it. Then do these steps: 1. Make the Dagger checks [required status checks](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches#require-status-checks-before-merging) in your branch protection rules. 2. Remove the workflow's checks from the required list. 3. [Disable the workflow](https://docs.github.com/en/actions/how-tos/manage-workflow-runs/disable-and-enable-workflows). During the move, a workflow can also run `dagger check` as one of its steps ([Checking your code in CI](https://docs.dagger.io/using/checking#in-ci)). ## What stays the same GitHub stays your home for code, pull requests, and review. Your tests stay as they are: official modules run them with no change to the test code ([test-native CI](https://dagger.io/changelog/test-native-ci)). Your secret managers stay the source of your secrets. **Bottom line:** You migrate from GitHub Actions to Dagger without a rewrite. Install the official modules that `dagger module recommend` finds, and run the same checks locally. Then turn on Cloud Checks and retire each workflow when its Dagger checks pass. ## Related guides - [What are the alternatives to GitHub Actions, from faster runners to portable pipelines?](/learn/github-actions-alternatives): how Dagger compares with runner vendors and other CI platforms. - [Can Dagger lower CI costs?](/learn/reduce-ci-costs): how Dagger bills compared with runner minutes. - [How do I run integration tests against real databases and Kafka without mocks?](/learn/integration-tests-real-database-containers): tests that need services, run the same way locally and in CI. --- # How do I run Go tests, lint, and go generate the same way locally and in CI? URL: https://dagger.io/learn/daggerize-go-project To run a Go project's tests, lint, and `go generate` the same way on a laptop and in CI, the setup has five requirements. It uses one Go toolchain and the same tools, and it tests every Go module. It starts the services that tests need, fails on stale generated files, and pins the linter. By hand, each laptop and each CI job repeats the `go` commands and installs the tools. Dagger meets the same requirements in one place. The official Go module and a linter module run them as checks, and `dagger check` runs the checks on a laptop and in Cloud Checks. ## What a good Go CI setup needs A Go setup that gives the same result on a laptop and in CI meets five requirements. Each one comes from how the `go` command works: 1. **Use one Go toolchain and the same tools.** The `go` line in `go.mod` ["declares the minimum required Go version"](https://go.dev/doc/toolchain) for the module. A `//go:generate` command must be ["an executable file that can be run locally"](https://pkg.go.dev/cmd/go#hdr-Generate_Go_files_by_processing_source), such as `protoc`. The `go` command turns cgo off when it [cannot find a C compiler](https://pkg.go.dev/cmd/cgo) on the `PATH`. Different tools on two machines give different results. 2. **Test every package in every Go module.** The `./...` pattern does not match directories that hold their own `go.mod`. You reach those packages only when you [change into that module's directory](https://pkg.go.dev/cmd/go#hdr-Package_lists_and_patterns). 3. **Start the services that tests need.** A test that needs a database can [skip at run time](https://pkg.go.dev/testing#hdr-Skipping) when the database is not there. A skipped test does not fail, so CI must start the service, or the test never runs. 4. **Fail on stale generated files.** Go generate ["is never run automatically by go build, go test, and so on"](https://pkg.go.dev/cmd/go#hdr-Generate_Go_files_by_processing_source). A package that others import [must have its generated files in the repository](https://go.dev/blog/generate). Thus CI must check that the committed files match their source. 5. **Lint with a pinned linter version.** The golangci-lint docs recommend ["installing a specific version of golangci-lint"](https://golangci-lint.run/docs/welcome/install/ci/) in CI. The Staticcheck docs give the [same advice](https://staticcheck.dev/docs/running-staticcheck/ci/github-actions/). The examples come from a small project with the Go module `example.com/greetings`. It has a `store` package with tests, and a `version` package with a `//go:generate` line. ## Run the Go toolchain and tests by hand Most Go teams run the `go` commands on a laptop, and repeat them as steps of a CI job. The Go version comes from `go.mod` in both places. Since Go 1.21, the default `GOTOOLCHAIN=auto` setting ["downloads newer toolchains as needed"](https://go.dev/doc/toolchain). In this project, `go.mod` has `go 1.26`, and the laptop has Go 1.24: ```shell go version ``` ```text go version go1.26.0 darwin/arm64 ``` In GitHub Actions, `actions/setup-go` reads the version from `go.mod` through its `go-version-file` input ([setup-go](https://github.com/actions/setup-go)). Tools outside Go, such as `protoc` or a C compiler, come from each laptop's package manager and from an install step in the CI job. Run the tests of the module: ```shell go test ./... ``` ```text ok example.com/greetings 0.005s ok example.com/greetings/store 0.005s ? example.com/greetings/version [no test files] ``` The run passes, but `TestDatabase` did not run. It skips itself when `DATABASE_URL` is not set, and only `-v` shows the skip. On a laptop, start Postgres in a container, then give the test its address: ```shell docker run --rm -d --name greetings-db -p 5432:5432 -e POSTGRES_USER=test -e POSTGRES_PASSWORD=test -e POSTGRES_DB=test postgres:17-alpine DATABASE_URL="postgres://test:test@localhost:5432/test?sslmode=disable" go test -v -run TestDatabase ./store/ ``` ```text === RUN TestDatabase --- PASS: TestDatabase (0.00s) PASS ok example.com/greetings/store 0.009s ``` In GitHub Actions, the job gets the same database from a [service container](https://docs.github.com/en/actions/tutorials/use-containerized-services/create-postgresql-service-containers) in the workflow file. A repository with several Go modules runs `go test ./...` once in each module directory, often in a shell loop or a job matrix. ## Check go generate output and lint by hand The usual drift check runs `go generate`, then fails when Git sees a change. `git diff --exit-code` [exits with status 1](https://git-scm.com/docs/git-diff#Documentation/git-diff.txt---exit-code) when there are differences. In this run, the version template changed, but nobody ran the generator: ```shell go generate ./... && git diff --exit-code ``` ```text diff --git a/version/version_gen.go b/version/version_gen.go index 77decfd..71b11b0 100644 --- a/version/version_gen.go +++ b/version/version_gen.go @@ -1,4 +1,4 @@ package version // Version is the release version. -const Version = "1.4.0" +const Version = "1.5.0" ``` The generator also needs its tools on the machine, such as `protoc` for a protobuf generator. Lint runs as one more step, with a pinned linter release. The guides for [golangci-lint](/learn/golangci-lint-dagger) and [Staticcheck](/learn/staticcheck-dagger) show the commands for each linter. ## Where the hand-built setup strains The commands above work, but each requirement lives in more than one place: - **The tools have no version file.** `go.mod` sets the Go version, but `protoc` and C libraries come from each laptop and from the CI job's install step. Their versions can differ. - **The project defines its services twice.** A `docker run` command on a laptop and a service container in the workflow describe the same database in two formats. - **Each Go module needs its own run.** The loop or the matrix over modules lives in CI configuration, and each repository writes its own. - **The drift check reads the working tree.** `git diff --exit-code` reports uncommitted local edits as drift too, so it is reliable only in a clean checkout. - **A CI failure is hard to repeat.** To repeat a failed job, a developer reads the workflow and runs its steps by hand, with the tools of the laptop. ## Meet the requirements with the Dagger Go module The Dagger Go module meets the first four requirements with checks, and a linter module meets the fifth. A [Dagger module](https://docs.dagger.io/reference/modules/index) is a package of functions, checks, and services that a workspace can run. A workspace is the Dagger configuration of a repository, in a file named `dagger.toml` at its root ([dagger.toml](https://docs.dagger.io/reference/config-files/dagger-toml)). The project must be a Git repository. Create the workspace with the [Quickstart](https://docs.dagger.io/getting-started/quickstart), which also recommends modules for the project. The Dagger team maintains official modules for common tools, and the Go module is one of them. It finds every `go.mod` in the repository. The workspace installs it under the name `go`: ```shell dagger module install dagger.io/go ``` If the Quickstart installed the module already, the command prints `Module "go" is already installed` and changes nothing. Each install command in this guide does the same. `dagger module list` shows the modules that the workspace has. A [check](https://docs.dagger.io/using/checking) is a function that validates the project and passes or fails. List the checks that the Go module adds: ```shell dagger check go -l ``` ```text MODULE GO-PACKAGE GO-TEST CHECK DESCRIPTION go * stale staleness check: run go generate in this package go * * test Run this test. # Use --all to expand collections and list each item. ``` Go packages and Go tests are collections. A `*` in a row stands for each package or each test that the module found. `dagger check go -l --all` lists one row per package and test. The sections below meet each requirement with these checks. ## Use one Go toolchain with the Dagger Go module The Go module runs each Go module in `golang:-alpine`, with the version that its `go.mod` asks for ([Go module reference](https://docs.dagger.io/reference/modules/go)). The same image runs on each laptop and in CI. `dagger.lock` pins the image, as [Run every check locally and with Cloud Checks](#run-every-check-locally-and-with-cloud-checks) explains. To use one Go version for every Go module, set it. A setting changes a default of the module, and the workspace keeps it in `dagger.toml`: ```shell dagger module settings go version 1.26 ``` The settings apply to each person and each machine that uses the repository. If the project builds with tags, set them once in `goflags`, so every Go command uses the same flags. A value that starts with a dash needs `--` before it. Without `--`, the CLI reads the value as its own flags and fails: ```text invalid argument "extended" for "-s, --silent" flag: strconv.ParseBool: parsing "extended": invalid syntax ``` Quotes do not help, because the shell removes them before the CLI reads the value. Put `--` before the value: ```shell dagger module settings go goflags -- -tags=extended ``` The command writes `goflags = "-tags=extended"` under `[modules.go.settings]` in `dagger.toml`. The [Go module reference](https://docs.dagger.io/reference/modules/go) describes the settings that this guide does not need, such as `build`. ## Use the same tools with a custom Dagger runtime If the tests and generators need only Go and the tools in the default image, skip this section. Tests and generators often need more system packages, such as `git`, a C compiler for cgo, or `protoc`. The `base` setting of the Go module takes a container with those packages. A function in a module of your own builds that container. Write the function in [Dang](https://docs.dagger.io/reference/sdks/dang), the native language of Dagger for modules, with no code generation and no build step. If the workspace does not have the Dang SDK, install it. If the project has no dev module, create one: ```shell dagger module install dagger.io/sdk/dang dagger module init dang ``` `dagger module init` creates the module at `.dagger/modules/-dev` and records it in `dagger.toml`. It also makes the module the main module of the workspace, so commands accept its functions without the module name. A new module has starter functions, which you can delete. Add the `testRuntime` function to `.dagger/modules/-dev/main.dang`. After you delete the starter functions, the file is: ```dang """ Project-specific runtime and services for the Go checks. """ type ProjectDev { """ A Debian-based Go image with protoc installed, for generators that compile protobuf definitions. """ pub testRuntime: Container! { container .from("golang:1.26-bookworm") .withExec(["apt-get", "update"]) .withExec(["apt-get", "install", "-y", "--no-install-recommends", "protobuf-compiler"]) } } ``` `ProjectDev` stands in for the type name that `dagger module init` generated from the module name. Keep the generated type name. Otherwise the module fails to load with `no main object`. `testRuntime` starts from the Debian-based `golang:1.26-bookworm` image, not the Alpine default. That image includes `git` and `gcc`. The function also installs `protobuf-compiler`, so a `//go:generate` line that runs `protoc` works. Keep the function minimal: - **Only an image and its packages.** Do not add a working directory, source files, or cache mounts. The Go module adds its own. - **The image sets the Go version.** When `base` has a value, each Go module in the workspace uses the Go toolchain of this container. Run `protoc --version` in the container that the Go module will get: ```shell dagger shell test-runtime -c 'protoc --version' ``` ```text libprotoc 3.21.12 ``` ## Wire the runtime into the Go module [Module wiring](https://docs.dagger.io/config/module-wiring) puts the value that a function returns into a setting of another module. A setting value of the form `dag:///` names a function of an installed module. Dagger calls that function when it loads the Go module, and passes the result as the value of the setting. Thus the official module gets a container for this project, and you write no test check of your own. The `base` container brings its own Go toolchain, so the Go module ignores the `version` setting when `base` has a value. Remove `version`. Then set `base`: ```shell dagger module settings -u go version dagger module settings go base test-runtime ``` The dev module is the main module of the workspace, so the command accepts `test-runtime` without the module name. It writes the full address, `dag://-dev/test-runtime`, to `dagger.toml`. Run the checks again: ```shell dagger check ``` Tests and generators now run in the container that `testRuntime` builds, on each laptop and in CI. ## Test every package in every Go module with Dagger The `test` check of the Go module runs one `go test` command for each package, in each Go module that it found. You write no loop over modules. Run the checks: ```shell dagger check go ``` The report shows each Go test on its own line, so a failure names the test that failed. To leave a Go module out, use the `test` setting, such as `["**", "!legacy-service"]`. A `!` pattern excludes a module root ([Go module reference](https://docs.dagger.io/reference/modules/go)). To run part of the suite, select packages and tests by name. This command runs one test: ```shell dagger check --go-package=store --go-test=TestDatabase ``` [Checking your code](https://docs.dagger.io/using/checking#select-collection-items) shows each way to select packages and tests. A test that skips itself because a service is not available is not a failure. The check passes, but unlike `go test ./...`, the report adds a `skipped` count for the package: ```text == CHECKS == ✔ 1 passed ✔ dag://?check=test&go-package=store&go-test=TestDatabase 0.5s OK == TESTS == ∅ example.com/greetings/store › TestDatabase SKIP store_test.go:15: DATABASE_URL not set ∅ 1 skipped ``` If the report shows a `skipped` count, the project needs the [Postgres service section](#start-the-services-that-tests-need-with-dagger) of this guide. If a test fails because it cannot open a file, see [Declare the files a test or generator reads](#declare-the-files-a-test-or-generator-reads). ## Start the services that tests need with Dagger Give the Go module a test runtime that has the services already. The `testRuntime` function describes the runtime, so the services go there too. The tests keep the two conventions from the hand-built setup, so the project has no Dagger code in its tests: - **The tests read the address of the service from an environment variable**, such as `DATABASE_URL`. - **The tests skip when the variable is unset**, so `go test ./...` still passes on a laptop without the service. Add a `postgres` function to `.dagger/modules/-dev/main.dang`. Bind it in `testRuntime`. The file is now: ```dang """ Project-specific runtime and services for the Go checks. """ type ProjectDev { """ A Postgres database for development and tests. """ pub postgres: Service! { container .from("postgres:17-alpine") .withEnvVariable("POSTGRES_USER", "test") .withEnvVariable("POSTGRES_PASSWORD", "test") .withEnvVariable("POSTGRES_DB", "test") .withExposedPort(5432) .asService } """ A Debian-based Go image with protoc installed, Postgres bound as a service, and DATABASE_URL pointing at it. """ pub testRuntime: Container! { container .from("golang:1.26-bookworm") .withExec(["apt-get", "update"]) .withExec(["apt-get", "install", "-y", "--no-install-recommends", "protobuf-compiler"]) .withServiceBinding("db", postgres) .withEnvVariable("DATABASE_URL", "postgres://test:test@db:5432/test?sslmode=disable") } } ``` `postgres` describes the database once, in place of the `docker run` command and the service container. It returns a `Service`, so `dagger start` can also start it for local development ([Running services](https://docs.dagger.io/using/services)). `testRuntime` has two new lines. `withServiceBinding` makes Postgres available at the host name `db`. Each container that the Go module makes from this base keeps the binding, so Postgres runs when `go test` runs. `DATABASE_URL` points to that address, and the tests read it. List the services of the workspace: ```shell dagger start -l ``` ```text MODULE SERVICE DESCRIPTION greetings-dev postgres A Postgres database for development and tests. ``` The `MODULE` column shows the name of the dev module, `greetings-dev` in this project. If `base` does not have the runtime yet, [create the dev module](#use-the-same-tools-with-a-custom-dagger-runtime) first, then wire the runtime: ```shell dagger module settings go base test-runtime ``` Run the tests again: ```shell dagger check --go --check=test ``` ```text == CHECKS == ✔ 2 passed ✔ dag://?check=test&go-package=. 1.4s OK == TESTS == ✔ example.com/greetings 1 passed ✔ 1 passed ✔ dag://?check=test&go-package=store 1.4s OK == TESTS == ✔ example.com/greetings/store 2 passed ✔ 2 passed == SERVICES == ✔ ku62l0th05qka.4ib43cj4f1spm.dagger.local exec docker-entrypoint.sh postgres 1.0s EXITED span=a7e5e9e140f8d6ff ``` The test that skipped before now runs, and the `SERVICES` section shows Postgres. The Go module still owns the test logic. The code of the project is two functions that describe its runtime and its database, and `dagger.toml` connects them. ## Fail on stale generated files with Dagger The `stale` check of the Go module runs `go generate` and fails when it would change a committed file. It reports only the changes that `go generate` makes, so other local edits do not fail it. The module also has a generator for `go generate`. A generator is a function that proposes changes to files ([Generating code](https://docs.dagger.io/using/generating)). It runs `go generate` in each package that has a `//go:generate` directive: ```shell dagger generate ``` Dagger shows the result as a changeset, a diff of the proposed changes, for you to review. It writes nothing until you apply the changeset. `dagger check` includes the `stale` check, so a pull request with a forgotten `go generate` fails the checks. The `generate` setting selects the directories where `go generate` runs. ## Declare the files a test or generator reads If the tests and generators read only Go source, embedded files, and `testdata/` directories, skip this section. Dagger runs each Go command against a directory that it assembles from the repository, not against the checkout itself. Thus Dagger caches the result of a check by exactly the files that the check saw. The Go module reads the Go source to choose those files. In addition to the source files and `go.mod`, it adds what the code declares: - The files that `//go:embed` names. - The Go module that a `go -C` argument names on a `//go:generate` line. - Local `replace` targets. Tests also get each `testdata/` directory. The scan does not see a file that a test opens by a relative path, or a file that a generator reads as input. For these files, the Go module reads two directives, as an alternative to the `includeExtraFiles` setting: - **`//go:test:include`** names files to mount when the tests of the module run. - **`//go:generate:include`** names files to mount when `go generate` runs. Each directive takes one or more patterns, with the same syntax as [`//go:embed`](https://pkg.go.dev/embed). A pattern is relative to the directory of the file that holds the directive. A `/` at the start makes a pattern relative to the repository root. Put quotes around a path that contains a space. The directive can go in any Go file in the module, such as a `_test.go` file, next to the code that needs the file. A test in `store/` that reads `../fixtures/users.json` declares the file in one line. In this project, the line is in `store/fixture_test.go`, below the `package` clause: ```go //go:test:include ../fixtures/*.json ``` Without the directive, the test fails as it would on a machine that does not have the file: ```shell dagger check --go-package=store --go-test=TestFixture ``` ```text == CHECKS == ✘ 1 failed ✘ dag://?check=test&go-package=store&go-test=TestFixture 1.6s ERROR == TESTS == ✘ example.com/greetings/store FAIL ✘ example.com/greetings/store › TestFixture FAIL fixture_test.go:12: open ../fixtures/users.json: no such file or directory ✘ 1 failed ``` A generator that reads a template declares it the same way. In `version/gen.go`, the line goes above the `//go:generate` line that runs the generator: ```go //go:generate:include templates/version.txt ``` Without its input, a generator fails or writes different output, and the `stale` check reports the difference as drift. Add the directives, and both checks pass. ## Choose a directive or includeExtraFiles Use a directive when one package needs the file. The directive stays with the code, and it applies only to the tests or the generator that need the file. Use `includeExtraFiles` when each Go module in the workspace needs the file, or when you do not want to change the source. The linter modules read only `//go:embed`. Thus a file that the linter needs goes in the `includeExtraFiles` setting of the linter module. The Dagger repository uses `//go:generate:include` for its CLI reference. A Go file with no code, [`generate.go`](https://github.com/dagger/dagger/blob/main/docs/current_docs/reference/cli/generate.go), includes the rendered page. Below it, the `//go:generate go -C ../../../../ run ./internal/cmd/dagger/docsgen` line renders the page. The `stale` check compares the new page with the committed page. The `go -C` argument adds the root Go module of the repository as input to the generator. ## Lint with a pinned linter version in the same Dagger run Lint comes from a separate module, so a project keeps the linter that it uses now. Each linter module pins a release by default, and each one lints the directories that hold its configuration file: - **golangci-lint**, from `dagger.io/go/golangci-lint`, runs `golangci-lint run` where the project has a `.golangci.yml` or another golangci-lint configuration file. - **Staticcheck**, from `dagger.io/go/staticcheck`, runs `staticcheck` where the project has a `staticcheck.conf`. This guide's project uses golangci-lint. For Staticcheck, use `staticcheck` in place of `golangci-lint`: ```shell dagger module install dagger.io/go/golangci-lint ``` The check list now has a lint check next to the Go checks: ```shell dagger check -l ``` ```text MODULE GO-PACKAGE GO-TEST GOLANGCI-LINT-PROJECT CHECK DESCRIPTION dang-sdk stale staleness check: re-generate modules and clients managed by the Dang SDK go * stale staleness check: run go generate in this package go * * test Run this test. golangci-lint * lint Run golangci-lint in this project. # Use --all to expand collections and list each item. ``` The lint check runs in its own container, so the `version` and `base` settings of the Go module do not change it. The linter module has its own settings. The [golangci-lint guide](/learn/golangci-lint-dagger) and the [Staticcheck guide](/learn/staticcheck-dagger) cover the configuration file, the pinned version, and the other lint requirements. ## Run every check locally and with Cloud Checks Run every check in the workspace: ```shell dagger check ``` ```text == CHECKS == ✔ 5 passed ✔ dag://?check=dang-sdk/generate/stale 0.9s OK ✔ dag://?check=go/packages/generate/stale 2.0s OK ✔ dag://?check=golangci-lint/projects/lint 2.0s OK ✔ dag://?check=test&go-package=. 2.0s OK == TESTS == ✔ example.com/greetings 1 passed ✔ 1 passed ✔ dag://?check=test&go-package=store 2.0s OK == TESTS == ✔ example.com/greetings/store 2 passed ✔ 2 passed == SERVICES == ✔ ku62l0th05qka.utlb5mb6895ge.dagger.local exec docker-entrypoint.sh postgres 1.4s EXITED span=e01d1691a4159a5f ``` Dagger names each check by its link. A link names packages or tests only when the check does not cover all of them, such as `go-package=store`. The Dang SDK adds a `stale` check of its own for the generated files of the dev module. For a workspace with only the modules from this guide, `dagger.toml` is now: ```toml [modules.go] source = "dagger.io/go" [modules.dang-sdk] source = "dagger.io/sdk/dang" [sdks.dang] module = "dang-sdk" [modules.-dev] main = true source = ".dagger/modules/-dev" [sdks.dang.scopes.".dagger/modules/-dev"] is-module = true [modules.go.settings] base = "dag://-dev/test-runtime" [modules.golangci-lint] source = "dagger.io/go/golangci-lint" ``` Commit the configuration, the lock file, and the module. Dagger needs all three to load the workspace the same way on each machine: ```shell git add dagger.toml dagger.lock .dagger git commit -m "Daggerize" ``` `dagger.lock` pins each image and Git ref that the workspace resolved. Examples are the `golang` image tag behind each Go version and the images of the dev module. Any Dagger command can add entries to it. `dagger lock update` refreshes the pinned versions. [Cloud Checks](https://docs.dagger.io/cloud-checks) run the same checks on Dagger Cloud after each push and pull request, with no CI workflow file. The repository must be on GitHub, and its `origin` remote must point there. From the project root, run: ```shell dagger cloud checks on ``` In a terminal, the command helps you sign in to Dagger Cloud and connect the GitHub account or organization. Then push the commit. Each push and pull request then runs these checks, with the Postgres service, on Dagger's Cloud Engines. `dagger workspace activity` shows the recent runs. To repeat a failed check, run `dagger check` on the laptop. **Bottom line:** To run Go tests, lint, and `go generate` the same way locally and in CI, pin the toolchain, the tools, and the linter. Test every Go module with its services, and fail when generated files are stale. With Dagger, `dagger.io/go` and a linter module meet these requirements as checks, and `dagger check` runs them on a laptop and in Cloud Checks. ## Related guides - [How do I run golangci-lint the same way locally and in CI?](/learn/golangci-lint-dagger): the golangci-lint requirements and the golangci-lint module. - [How do I run Staticcheck the same way locally and in CI?](/learn/staticcheck-dagger): the Staticcheck requirements and the Staticcheck module. - [How do I run integration tests against real databases and Kafka without mocks?](/learn/integration-tests-real-database-containers): more on test services, in the Compose model and the Testcontainers model. --- # How do I run Playwright tests against a Docker Compose app in CI? URL: https://dagger.io/learn/playwright-tests-docker-compose Playwright tests against a Docker Compose app need five things. Compose starts the app and its database before the tests and waits until they are ready. The tests reach the app through `baseURL`, they run in the Playwright image for your version, and each run controls its test data. Each browser project can also run on its own. Teams meet these with Playwright's `webServer` option, `docker compose`, and a CI job. Dagger meets the same five requirements in one place, with its Docker and Playwright modules, on a laptop and in CI. ## What Playwright tests against a Compose app need Playwright's docs give four requirements for tests against an app that Compose starts. This guide judges each approach by them: 1. **Start the app and its services before the tests.** Playwright's [`webServer`](https://playwright.dev/docs/test-webserver) option can "launch a local dev server before running your tests". It waits until the `url` returns "a 2xx, 3xx, 400, 401, 402, or 403 status code". The app must also wait for its database. Compose does that only with a `healthcheck` and a `service_healthy` condition ([startup order](https://docs.docker.com/compose/how-tos/startup-order/)). 2. **Point the tests at the app with `baseURL`.** Playwright recommends that you set `baseURL` in the `use` block, "so that tests can use relative urls" ([Adding a baseURL](https://playwright.dev/docs/test-webserver#adding-a-baseurl)). The address of the app changes between a laptop and CI, so the config must read it from the environment. 3. **Run the tests in the Playwright image for your exact version.** The [Playwright Docker image](https://playwright.dev/docs/docker) has the browsers and their system dependencies. If its version differs from your `@playwright/test` version, Playwright "will be unable to locate browser executables". The [guide to Playwright in CI with cached browsers](/learn/playwright-ci-docker-browser-caching) covers this requirement in full. 4. **Control the test data.** Playwright's best practices say, "If working with a database then make sure you control the data" ([Testing with a database](https://playwright.dev/docs/best-practices#testing-with-a-database)). A database from an earlier run can hold data that changes the result. 5. **Run each browser project on its own.** A Playwright config can define [projects](https://playwright.dev/docs/test-projects), such as one for each browser. Playwright says, "You can also use projects to run the same tests in different configurations." The `--project` option runs one project. ## Meet the requirements with webServer and Compose Teams start the Compose app from Playwright's own config, and remove it after the run. The examples come from a project with a pnpm workspace. The Playwright suite is in `e2e/`, next to this Compose file, `e2e/compose.yaml`: ```yaml services: web: build: ../app ports: - "8080:8080" environment: DATABASE_URL: postgres://app:app@db:5432/app depends_on: db: condition: service_healthy db: image: postgres:17 environment: POSTGRES_USER: app POSTGRES_PASSWORD: app POSTGRES_DB: app ports: - "5435:5432" volumes: - pgdata:/var/lib/postgresql/data healthcheck: test: ["CMD", "pg_isready", "-U", "app"] interval: 2s volumes: pgdata: ``` **Start the app and its services before the tests.** The `healthcheck` and `condition: service_healthy` make Compose start `web` only after Postgres answers `pg_isready`. In `playwright.config.ts`, a `webServer` runs `docker compose up` and waits for the app: ```typescript webServer: process.env.PLAYWRIGHT_BASE_URL ? undefined : { command: 'docker compose up', url: 'http://localhost:8080', reuseExistingServer: true, }, ``` Do not add `--wait` to this command. With `--wait`, Compose exits after the services start. Playwright then stops the run with `Error: Process from config.webServer exited early.` The condition on `PLAYWRIGHT_BASE_URL` turns `webServer` off when something else starts the app, as in the Dagger sections below. **Point the tests at the app with `baseURL`.** In the `use` block, set `baseURL` from a variable, and keep your local URL as the fallback: ```typescript baseURL: process.env.PLAYWRIGHT_BASE_URL || 'http://localhost:8080', ``` Run the tests from `e2e/`: ```shell npx playwright test ``` Playwright prints the Compose output with a `[WebServer]` prefix. The end of the output in this guide's project: ```text Running 6 tests using 6 workers ✓ 1 [firefox] › tests/db.spec.ts:4:5 › database port is reachable (24ms) ✓ 4 [chromium] › tests/db.spec.ts:4:5 › database port is reachable (7ms) ✓ 5 [chromium] › tests/secure-context.spec.ts:3:5 › page runs in a secure context (389ms) ✓ 6 [chromium] › tests/home.spec.ts:4:5 › home page shows the database version (407ms) ✓ 2 [firefox] › tests/secure-context.spec.ts:3:5 › page runs in a secure context (543ms) ✓ 3 [firefox] › tests/home.spec.ts:4:5 › home page shows the database version (553ms) 6 passed (6.3s) ``` **Control the test data.** After the run, the containers were still up, and the `pgdata` volume kept the database. Remove both, so the next run starts from an empty database: ```shell docker compose down -v ``` ```text Container e2e-web-1 Stopping Container e2e-web-1 Stopped Container e2e-web-1 Removing Container e2e-web-1 Removed Container e2e-db-1 Stopping Container e2e-db-1 Stopped Container e2e-db-1 Removing Container e2e-db-1 Removed Volume e2e_pgdata Removing Network e2e_default Removing Volume e2e_pgdata Removed Network e2e_default Removed ``` The [`-v` flag](https://docs.docker.com/reference/cli/docker/compose/down/) removes "named volumes declared in the "volumes" section of the Compose file". **Run the tests in the Playwright image for your exact version.** Playwright's [CI guide](https://playwright.dev/docs/ci) gives two ways to prepare a CI agent. One is the Playwright image as a container job, such as [`jobs..container`](https://docs.github.com/en/actions/how-tos/write-workflows/choose-where-workflows-run/run-jobs-in-a-container) on GitHub Actions. The other is `npx playwright install --with-deps` on the runner. With Compose, teams often use the second way, because the steps of a container job run in the Playwright image. That image has no Docker CLI. In this guide's project, `command -v docker` in `mcr.microsoft.com/playwright:v1.64.1-noble` found nothing. To run one browser project by hand, pass its name from the config: ```shell npx playwright test --project chromium ``` ```text Running 3 tests using 3 workers ✓ 1 [chromium] › tests/db.spec.ts:4:5 › database port is reachable (7ms) ✓ 3 [chromium] › tests/secure-context.spec.ts:3:5 › page runs in a secure context (171ms) ✓ 2 [chromium] › tests/home.spec.ts:4:5 › home page shows the database version (177ms) 3 passed (4.6s) ``` ## Where the hand-built setup strains The hand-built setup meets the requirements, but each one depends on the machine that runs it: - **Start the app and its services before the tests.** `webServer` starts Compose on a laptop. In CI, the job needs Docker on the runner and free host ports, such as 8080 and 5435. - **Point the tests at the app with `baseURL`.** The address works only where the tests and Compose share a network. In a container job, `localhost` is the job container, not the app. - **Run the tests in the Playwright image for your exact version.** A runner that installs the browsers does not use the image. A container job uses the image, but it cannot run `docker compose`. - **Control the test data.** `docker compose down -v` must run after every run, also when the tests fail. If it does not run, the next run on that machine reuses the old database. - **Run each browser project on its own.** The project name goes on each command line. A CI job for each browser repeats the full job definition with only that name changed. ## Meet the requirements with the Dagger Docker and Playwright modules Connect two official Dagger modules in `dagger.toml`. [Dagger](https://docs.dagger.io/getting-started/introduction) is a CI orchestration engine that runs the same pipelines on a laptop and in CI. A [module](https://docs.dagger.io/reference/modules) is a package of functions, checks, and services that a project installs. The Docker module reads the Compose file and starts the app with its dependencies. The Playwright module runs the tests against that app. You write no Dagger code. Complete the [Quickstart](https://docs.dagger.io/getting-started/quickstart) first. Then install the two modules from the repository root: ```shell dagger module install dagger.io/docker@main dagger module install dagger.io/js/playwright ``` The first command installs the Docker module from its `main` branch. Each install adds the module to `dagger.toml`, the workspace configuration, and the first install creates that file if the project has none. Dagger also writes `dagger.lock`. It pins the Git commit of each module and, after the first run, the images that the checks use, such as `postgres:17`. If a module is already installed, the command prints `Module "playwright" is already installed` and changes nothing. A [check](https://docs.dagger.io/using/checking) is a function that validates the project, such as a test run. List the Playwright checks: ```shell dagger check --playwright -l ``` ```text MODULE PLAYWRIGHT-PROJECT PLAYWRIGHT-TEST-FILE CHECK DESCRIPTION playwright * * test Run this test file. # Use --all to expand collections and list each item. ``` The [Playwright module](https://docs.dagger.io/reference/modules/js/playwright) finds each directory with a `playwright.config.ts` file, or a config file with the `.js`, `.mjs`, `.cjs`, `.mts`, or `.cts` extension. Each of these directories is a Playwright project for the module. A Playwright project is a directory, not a browser project such as `chromium` in the config. The `test` check runs the test files of a project. A `*` stands for every project or test file. ## Start the app and its services before the tests, with Dagger The Docker module starts each Compose service with the services in its `depends_on` list. The [guide to Docker images and Compose services](/learn/docker-compose-dagger) explains how it finds Compose files and which Compose settings it applies. One difference matters here. The current Docker module does not translate Compose health checks. It waits until the ports of the dependencies accept connections ([module README](https://github.com/dagger/docker#running-compose-services)). If the app opens its port before it is ready, add a readiness check to Playwright's [global setup](https://playwright.dev/docs/test-global-setup-teardown). List the services as links: ```shell dagger list services -a -f=link ``` ```text dag+service://?service=batch/up&docker-compose-project=e2e dag+service://?service=docker/cli/engine dag+service://?service=docker/engine dag+service://?service=up&docker-compose-project=e2e&docker-compose-service=db dag+service://?service=up&docker-compose-project=e2e&docker-compose-service=web ``` Copy the link of the service that the browser opens, here `web`. In the link, `docker-compose-project` is the directory of the Compose file, relative to the repository root. It is not the top-level `name` in the file. The value of `docker-compose-service` is the service name in the file. Set the `service` setting of the Playwright module to that link. Put the link in quotes, so that the shell keeps the `&` characters: ```shell dagger module settings playwright service 'dag+service://?service=up&docker-compose-project=e2e&docker-compose-service=web' ``` This is [module wiring](https://docs.dagger.io/config/module-wiring). A setting whose value is the address of a function in another module gets the result of that function. Dagger runs the `up` function of the Docker module and gives the service to the Playwright module. For a workspace with only these two modules, `dagger.toml` is now: ```toml # Dagger workspace configuration # Install modules with: dagger module install # Example: # dagger module install dagger.io/linux/wolfi [modules] [modules.docker] source = "dagger.io/docker@main" [modules.playwright] source = "dagger.io/js/playwright" [modules.playwright.settings] service = "dag+service://?service=up&docker-compose-project=e2e&docker-compose-service=web" ``` Use the link from your own list. Keep the other modules in your `dagger.toml`. Dagger starts the app, so `webServer` must stay off. The config in the previous section turns it off when `PLAYWRIGHT_BASE_URL` has a value, and the Playwright module sets that variable. If `webServer` stays on, Playwright runs its command in the test container, and the run fails: ```text Error: Process from config.webServer was not able to start. Exit code: 127 [WebServer] /bin/sh: 1: docker: not found ``` Some configs read a variable of their own that turns off `webServer`. If yours does, set that variable with the `environment` setting. For a config that reads `PLAYWRIGHT_DISABLE_WEBSERVER`: ```shell dagger module settings playwright environment PLAYWRIGHT_DISABLE_WEBSERVER=1 ``` The command writes `environment = ["PLAYWRIGHT_DISABLE_WEBSERVER=1"]` under `[modules.playwright.settings]`. The command replaces the whole list, so give all the variables in one command. ## Point the tests at the app with baseURL, with Dagger The Playwright module binds the wired service into the test container with the hostname `frontend`. It sets `PLAYWRIGHT_BASE_URL` to the first port of the service, here `http://frontend:8080`. The `baseURL` line from the hand-built setup reads that variable, so tests that open a relative path, such as `page.goto('/')`, open the wired app. Tests and helper clients that make absolute URLs must read the same variable. In the test container, `localhost` is the test container itself, not the app. The Docker module publishes the ports of the selected service first, then the ports of its dependencies. Thus `PLAYWRIGHT_BASE_URL` points at `web`, not `db`. If the app publishes several ports, put its HTTP port first in `ports`. A port that is only in `expose` does not come first. The ports of the dependencies are on `frontend` too. For a database with `5435:5432` in `ports`, a test client connects to `frontend:5435`. Give that hostname to the test helpers with the `environment` setting. Browser features such as service workers and the Web Crypto API work only in a [secure context](https://developer.mozilla.org/en-US/docs/Web/Security/Secure_Contexts). An HTTP page on `frontend` is not a secure context, but a page on `localhost` is. If the tests need a secure context, turn on the localhost proxy of the module: ```shell dagger module settings playwright localhostProxy true ``` The proxy forwards the first port of the wired service from `localhost` in the test container. The module sets `PLAYWRIGHT_LOCALHOST_BASE_URL` to that address, here `http://localhost:8080`. In `playwright.config.ts`, read it first in the `use` block: ```typescript baseURL: process.env.PLAYWRIGHT_LOCALHOST_BASE_URL || process.env.PLAYWRIGHT_BASE_URL || 'http://localhost:8080', ``` In this guide's project, a test of `window.isSecureContext` failed against `frontend` and passed with the proxy. The proxy forwards only the first port, so keep database clients on `frontend`. The module installs the proxy, `socat`, with `apt-get`, which the default Playwright images have. ## Run the tests in the Playwright image for your exact version, with Dagger The Playwright module runs the tests in the `mcr.microsoft.com/playwright` image for your version, and the wired app runs beside it. Commit the lockfile of your package manager. The module reads the `@playwright/test` version from it and picks the image tag for that version. Without a lockfile, pin `@playwright/test` to an exact version in `package.json`. The module reads a range such as `^1.58.2` as its lower bound. The install can then get a newer Playwright than the browsers in the image. The default image has the browsers that Playwright bundles. It does not have the [branded browsers](https://playwright.dev/docs/browsers). Remove projects with `channel: 'chrome'` or `channel: 'msedge'` from the run. To run one browser project at a time, use a Dagger env, as [Run each browser project on its own, with Dagger](#run-each-browser-project-on-its-own-with-dagger) shows. The module also sets `CI=true`, so review the `workers` and `retries` values that your config sets for CI. The module installs the dependencies in the image, but it does not build workspace packages. In a monorepo, the tests can import a workspace package whose entry point a build makes. Without the build, the tests fail at the import: ```text Error: Cannot find module '/app/e2e/node_modules/@app/sdk/dist/index.js'. Please verify that the package.json has a valid "main" entry ``` The `setup` setting runs commands after the install, with the full repository source in place. For a pnpm workspace package named `@app/sdk`, add its build command: ```shell dagger module settings playwright setup 'pnpm --filter @app/sdk build' ``` The command writes `setup = ["pnpm --filter @app/sdk build"]`. Setup runs in the Playwright project directory, before the module binds the wired service. Put preparation that needs the app in Playwright's global setup. ## Control the test data, with Dagger Dagger runs the Compose services in [ephemeral containers](https://docs.dagger.io/using/services). It starts them when the tests need them, and stops them when nothing references them. There is no `down` step to forget. Named volumes are different. The Docker module turns each named volume, such as `pgdata`, into a [cache volume](https://docs.dagger.io/reference/api/cache-volume), a directory whose contents persist across runs ([module README](https://github.com/dagger/docker#running-compose-services)). Thus the database keeps its data between runs, as a Compose volume does without `-v`. If the tests need an empty database, reset the data in Playwright's [global setup](https://playwright.dev/docs/test-global-setup-teardown). ## Run each browser project on its own, with Dagger A Dagger env stores the settings of a named run in `dagger.toml`. Use one env for each browser project, with the shared Compose service and test setup from the base settings. The `--env` option of the [Dagger CLI](https://docs.dagger.io/reference/cli) selects or creates an env. The config of this guide's project defines the projects `chromium` and `firefox`. Create a matching env for `chromium`. The `--` tells `dagger` that the arguments after it are values, not its own flags: ```shell dagger module settings --env chromium playwright args -- --project chromium ``` The command prints `Created env "chromium"` and writes the setting under an env table: ```toml [env.chromium] [env.chromium.modules.playwright.settings] args = ["--project", "chromium"] ``` Run the tests with that env: ```shell dagger check --env chromium --playwright ``` ```text == CHECKS == ✔ 1 passed ✔ dag://?check=test&playwright-project=e2e 6.1s OK == TESTS == ✔ chromium 3 passed ✔ 3 passed ``` The env keeps the base settings, such as the wired Compose service. Its `args` value replaces the base `args` list, so put every Playwright argument that the run needs in it. Without `--env`, `dagger check --playwright` uses the base settings, and both projects ran, with 6 tests. The env name is a label that you choose. The name after `--project` must match a project in your Playwright config, because the env does not create the browser project. Add an env for `firefox` the same way: ```shell dagger module settings --env firefox playwright args -- --project firefox dagger check --env firefox --playwright ``` Commit the env tables with the rest of `dagger.toml`, so that everyone runs the same browser projects. ## Run the same tests locally and with Cloud Checks Run the Playwright checks: ```shell dagger check --playwright ``` Dagger builds or pulls the images of the Compose services, and starts the app and its dependencies. Then it runs the Playwright checks against the app. The `--playwright` flag selects only the checks of the Playwright module. In this guide's project, with three test files and two browser projects, the report shows: ```text == CHECKS == ✔ 1 passed ✔ dag://?check=test&playwright-project=e2e 2.5s OK == TESTS == ✔ chromium 3 passed ✔ firefox 3 passed ✔ 6 passed ``` If the repository has several directories with a Playwright config, select one with `--playwright-project`. The value is the directory of `playwright.config`, which can be different from the directory of the Compose file: ```shell dagger check --playwright-project=e2e ``` Commit `dagger.toml`, `dagger.lock`, and the changes to `playwright.config.ts`. If the lockfile of your package manager is new, commit it too. Before the first push, run every check in the workspace. Cloud Checks run all of them, and these include the lint and build checks of the Docker module: ```shell dagger check ``` ```text == CHECKS == ✔ 5 passed ✔ dag://?check=build-service&docker-compose-project=e2e 0.5s OK ✔ dag://?check=docker/compose/projects/lint 0.5s OK ✔ dag://?check=docker/projects/build 0.4s OK ✔ dag://?check=docker/projects/lint 0.9s OK ✔ dag://?check=test&playwright-project=e2e 2.4s OK == TESTS == ✔ chromium 3 passed ✔ firefox 3 passed ✔ 6 passed ``` In CI, [Cloud Checks](https://docs.dagger.io/cloud-checks) run the checks. Cloud Checks run your checks on Cloud Engines, Dagger's managed engines, after each push and pull request, with no CI workflow file. The repository must be on GitHub, with its `origin` remote set to it. Turn them on: ```shell dagger cloud checks on ``` In an interactive terminal, the command offers to sign you in to Dagger Cloud and to connect GitHub. After that, each push starts the Compose services and the Playwright tests on Cloud Engines, as `dagger check` does on a laptop. To see the runs, use `dagger workspace activity`. **Bottom line:** Start the Compose app before the tests, and point Playwright at it with `baseURL`. Run the tests in the Playwright image for your version, and control the test data. By hand, `webServer` runs `docker compose up` and `docker compose down -v` cleans up. With Dagger, wire the app's Compose service into the Playwright module, and `dagger check --playwright` runs the same tests on a laptop and with Cloud Checks. A Dagger env runs one browser project. ## Related guides - [How do I build Docker images and run Compose services the same way locally and in CI?](/learn/docker-compose-dagger): how the Docker module finds Compose files and starts Compose services. - [How do I run Playwright tests in CI and locally in the same container, with cached browsers?](/learn/playwright-ci-docker-browser-caching): the Playwright image, browser caching, and screenshot baselines, when `webServer` starts the app without Compose. - [How do I run integration tests against real databases and Kafka without mocks?](/learn/integration-tests-real-database-containers): fresh database and Kafka containers for tests, the Compose way or the Testcontainers way. --- # How does Dagger compare to Bazel? URL: https://dagger.io/learn/dagger-vs-bazel Bazel and Dagger work at different layers, so you do not have to choose only one. [Bazel](https://bazel.build/about/intro) is a build and test tool. `BUILD` files declare fine-grained targets, and Bazel caches each action. Bazel can also run actions on remote machines. Dagger is a CI engine. It runs a project's existing tools as checks in containers and caches each operation, the same way on a laptop and in CI. Choose Bazel to build a large polyglot codebase incrementally, one action at a time. Choose Dagger to run the tools you already have the same way everywhere, with no new build files. Many Bazel teams can use both, because Dagger can run `bazel test` as one of its checks. ## The fundamental difference Bazel models the build itself. You declare targets in `BUILD` files with Starlark, and Bazel makes an [action graph](https://bazel.build/about/intro) from them. In this graph, each compile, link, or test step has declared inputs and outputs. Bazel knows every action, so it rebuilds and retests only what a change affects. Its docs say that Bazel handles "builds with 100k+ source files". Dagger models the pipeline around your tools. A Dagger [module](https://docs.dagger.io/reference/modules) is a package of functions that build and test a project. Its checks, such as tests and linters, run with `dagger check`. [Official modules](https://docs.dagger.io/reference/modules) for Go, pytest, Jest, Vitest, ESLint, and other tools read the project's native configuration, so you do not write build files. The Dagger engine runs each step in a container and caches the step by its inputs. | | Bazel | Dagger | | --- | --- | --- | | What you write | `BUILD` files and Starlark rules for each target | Nothing for tools with an official module, or a small module for other tools | | Unit of caching | One build action, such as one compile | One operation, such as one command in a container | | Isolation | A sandbox for each action | A container for each step | | Remote work | Remote cache and remote execution over gRPC, through a service you choose | Whole pipelines on a remote engine or on Dagger Cloud | | CI | Runs inside the CI system you already have | Cloud Checks run `dagger check` on each push | | Cost | Open source. Remote services cost what each vendor charges | Open-source engine. Dagger Cloud has usage pricing | ## Adoption Bazel needs more work at the start than Dagger. Each package needs a `BUILD` file, and each language needs a rule set, such as the [Go, Java, Python, or JavaScript rules](https://bazel.build/rules). The Bazel docs recommend that you [improve hermeticity before you migrate](https://bazel.build/basics/hermeticity). Common problems include system binaries that differ between hosts and build steps that write to the source tree. For a large codebase, the result is correct incremental builds across all its languages. Dagger starts from the files the project already has. Run `dagger init`, then run `dagger module recommend`. Dagger looks for files such as `go.mod` or `pyproject.toml` and recommends the official modules that match them ([Quickstart](https://docs.dagger.io/getting-started/quickstart)). Your tests and build commands stay the same. If a tool has no official module, you write a small module. You can write it in Dang, Dagger's native module language ([Dang SDK](https://docs.dagger.io/reference/sdks/dang)), or in Go, Python, TypeScript, or another SDK language. ## Caching Both tools cache by content, but at different levels of detail. Bazel caches each action. A [remote cache](https://bazel.build/remote/caching) holds an action cache and a content-addressable store, so a team and its CI can share outputs. The `--disk_cache` flag keeps a cache on the local file system. You choose, set up, and maintain the backend for the remote cache yourself, such as bazel-remote, nginx, or Google Cloud Storage. Cache hits across machines depend on hermetic actions. The Bazel docs give a [procedure to debug cache misses](https://bazel.build/remote/cache-remote) with execution logs. Dagger caches each operation in a pipeline by its inputs. An example of an operation is one command in a container. Dagger also has [cache volumes](https://docs.dagger.io/reference/sdks/dang#cache-volumes). A cache volume is a directory that persists across runs, for example for a package manager cache. A Dang module reads project files only when it uses them ([workspace inputs](https://docs.dagger.io/reference/sdks/dang#workspace-inputs)). Two rules decide whether a step runs again: - If the inputs of a step did not change, Dagger skips the step. - If an input of a step changed, the step runs again in full. Incremental work inside a step comes from the tool in that step, for example from Bazel's own cache in a cache volume. ## Remote execution Bazel distributes single actions. [Remote execution](https://bazel.build/remote/rbe) sends build and test actions to many machines over an open gRPC protocol. The Bazel docs say this gives "faster build and test execution through scaling of nodes available for parallel actions". Bazel lists [self-hosted and commercial services](https://bazel.build/community/remote-execution-services), such as Buildbarn, Buildfarm, BuildBuddy, EngFlow, and NativeLink. Your rules must meet the [requirements for remote execution](https://bazel.build/remote/rbe). Dagger runs a whole pipeline on a remote engine. Set `--engine` to an SSH host, a Kubernetes pod, or `cloud` for a managed Cloud Engine ([changelog](https://dagger.io/changelog)). Dagger streams local files, secrets, and your terminal to the engine when it needs them. Dagger also splits large test suites across engines ([test splitting](https://dagger.io/changelog/test-splitting)). ## Local and CI runs Bazel gives the same results on a laptop and in CI when both use the same Bazel version, flags, and environment. The usual tools for this are a `.bazelversion` file, which [Bazelisk](https://bazel.build/install/bazelisk) reads, and a `--config=ci` section in [`.bazelrc`](https://bazel.build/run/bazelrc). Bazel's sandbox limits which inputs an action sees. But the Bazel docs say: ["Sandboxing doesn't hide the host environment in any way."](https://bazel.build/docs/sandboxing) Toolchains that Bazel does not manage can be different on a Mac laptop and on a Linux runner. A CI system must still call Bazel, so part of the pipeline stays in the configuration of that system. With Dagger, the checks run in containers that the module defines, so the environment is part of the pipeline. `dagger check` runs the same checks on a laptop and in CI ([Checking your code](https://docs.dagger.io/using/checking)). [Cloud Checks](https://docs.dagger.io/cloud-checks) connect a GitHub repository to Dagger Cloud. After that, every push and pull request runs your checks on Cloud Engines, with no CI workflow file. When a check fails, `dagger check -i` opens a shell in the state where the command failed ([changelog](https://dagger.io/changelog)). ## Languages Bazel supports many languages through rule sets. The [recommended rules](https://bazel.build/rules) include C/C++, Go, Java, Python, Rust, Scala, Android, and Protocol Buffers. Bazel can also [build for several platforms](https://bazel.build/about/intro) from one project, such as desktop, server, and mobile. Dagger runs any tool that runs in a Linux container. The official modules cover Go, JavaScript and TypeScript (Jest, Vitest, Playwright, ESLint, Prettier, Biome, Deno), Python (pytest), shell scripts, PowerShell, and Helm ([module catalog](https://docs.dagger.io/reference/modules)). For other tools, a module runs the tool's own command in a container. ## Pricing Bazel is [open source](https://bazel.build/about/intro). A remote cache or remote execution service costs what its vendor charges, or what it costs you to run it yourself. Check the pricing page of each vendor. The Dagger Engine is open source, and it costs nothing on your own machine or server. Dagger Cloud bills the CPU, memory, and data that each pipeline uses. The monthly minimum is $20 for individuals and $200 for teams. An operation that could come from the cache costs 5× less ([pricing](https://dagger.io/pricing)). ## Run Bazel inside Dagger You can keep Bazel as the build system and use Dagger to run it. Bazel then builds incrementally inside a container. Dagger supplies the same container, cache, and checks on a laptop and in Cloud Checks. The module below uses the [Bazel container image](https://bazel.build/install/docker-container) that the Bazel project publishes. First, set up the workspace and create the project's dev module. The dev module holds the parts of the pipeline that are specific to the project: ```shell dagger init dagger module install dagger.io/sdk/dang dagger module init dang -y ``` With no `--name`, `dagger module init` creates a module named `-dev` and installs it in `dagger.toml`. This guide's scratch project is named `work`, so its module is `work-dev`. Replace the contents of `.dagger/modules/-dev/main.dang` with this module. The type name `WorkDev` stands for the name that `dagger module init` generated: ```dang """ Run the Bazel build and tests in a container. """ type WorkDev { """ The workspace, without Bazel's output symlinks. """ pub source: Directory! new(ws: Workspace!) { self.source = ws.directory("/", exclude: ["bazel-*", ".dagger", ".git"]) self } """ A container with Bazel, the source, and a cache volume for Bazel's output. """ pub bazel: Container! { container .from("gcr.io/bazel-public/bazel:7.4.1") .withMountedCache( "/home/ubuntu/.cache/bazel", cacheVolume("bazel-output"), sharing: CacheSharingMode.LOCKED, owner: "ubuntu", ) .withDirectory("/src", source, owner: "ubuntu") .withWorkdir("/src") } """ Run every Bazel test target with the ci config from .bazelrc. """ pub test: Void @check { bazel.withExec(["bazel", "test", "--config=ci", "//..."]).sync null } } ``` The cache volume keeps Bazel's output directory between runs, so Bazel's own action cache continues to work. Set the image tag to the version in your `.bazelversion` file. The module copies `.bazelrc` with the source, so `--config=ci` applies your CI flags. The flags for a Bazel remote cache are Bazel flags, so they also go in `.bazelrc`. List the checks: ```shell dagger check -l ``` ```text MODULE CHECK DESCRIPTION dang-sdk stale staleness check: re-generate modules and clients managed by the Dang SDK work-dev test Run every Bazel test target with the ci config from .bazelrc. ``` Run them: ```shell dagger check ``` ```text == CHECKS == ✔ 2 passed ✔ dag://?check=dang-sdk/generate/stale 1.2s OK ✔ dag://?check=test 0.0s OK ``` To run one Bazel target in the same container, use `dagger shell`: ```shell dagger shell bazel -c 'bazel test //greet:greet_test' ``` ## What the two caches did in one run The two caches have different jobs, and one example run on one machine shows each of them. The scratch project had two `sh_test` targets: - **The first run.** `dagger check` took 16.8 seconds, and Bazel ran both tests. - **A run again with no changes.** It took 1.3 seconds. The inputs were the same, so Dagger skipped the Bazel step. - **A one-line change to one test file.** It took 17.7 seconds. An input changed, so Dagger ran the Bazel step again. In that step, Bazel showed `(cached) PASSED` for the test that did not change, and it ran only the other test. ## When to choose Bazel Choose Bazel when the build itself is the slow part. An example is a large monorepo where a change must rebuild only the affected targets. Another example is C++ or Java code with long compile steps. Choose Bazel also when you need to send single actions to a cluster with remote execution. Bazel can find the targets that depend on a change with `rdeps` in [`bazel query`](https://bazel.build/query/guide). Dagger's feature to run only the affected checks is in the [Coming soon section](https://dagger.io/changelog), so it is not released yet. ## When to choose Dagger Choose Dagger when the project builds well with its own tools, such as `go test`, pytest, or Vitest. The work is then in the CI around those tools. Dagger runs those tools as checks with no `BUILD` files, in the same containers on a laptop and in CI. Cloud Checks replace the CI workflow file. Choose Dagger also when the pipeline needs more than a build. Examples are services for integration tests, a shell in the state where a check failed, and remote engines that developers, agents, and CI share. If you already use Bazel, you can use both. Keep `BUILD` files for incremental builds. Use Dagger to run `bazel test` with one container, one cache, and one command everywhere. **Bottom line:** Bazel is a build system. It caches and distributes single build actions that `BUILD` files define, and it is best for large polyglot monorepos. Dagger is a CI engine. It runs your existing tools as cached container checks, the same way on a laptop and in Cloud Checks, with no build files. You can use them together: a Dagger module can run Bazel in a container, with a cache volume for Bazel's output. ## Related guides - [What are the alternatives to GitHub Actions, from faster runners to portable pipelines?](/learn/github-actions-alternatives): where Dagger fits among other CI options. - [How do I migrate from GitHub Actions to Dagger?](/learn/migrate-from-github-actions): how to move a workflow to Dagger checks, including a workflow that calls Bazel. - [Which development tools can Dagger replace or consolidate?](/learn/consolidate-ci-tools): what Dagger can replace and what stays in place. --- # How does Dagger compare to Blacksmith? URL: https://dagger.io/learn/dagger-vs-blacksmith Blacksmith makes your GitHub Actions workflows faster and cheaper with no change to the workflows. You change one line per job, and the job runs on Blacksmith's hardware. Dagger changes what runs: it replaces workflow jobs with checks, which run the same on your machine, on your servers, and in Dagger Cloud. Choose Blacksmith to keep your workflows. Choose Dagger to run the same checks everywhere and see each test. | | Blacksmith | Dagger | | --- | --- | --- | | What changes | The runner of each job | The jobs become Dagger checks | | How to start | Change the `runs-on` label | Install official modules, turn on Cloud Checks | | Where work runs | Blacksmith's machines | Your machine, your servers, or Dagger Cloud | | Cache | Co-located cache for cache actions, Docker layer cache | The engine caches each operation | | Tests | The commands in your workflow | Each test is a check that you can run alone | | Billing | Per runner minute | Used CPU, memory, and data, with repeat work 5× cheaper | ## Adoption Blacksmith needs one change per job. Install its GitHub app for your organization, then change the `runs-on` label of each job, for example from `ubuntu-latest` to `blacksmith-2vcpu-ubuntu-2404` ([Blacksmith quickstart](https://docs.blacksmith.sh/introduction/quickstart)). It works for GitHub organizations, not for personal repositories. Dagger needs more steps. Install the CLI, then run `dagger init` and `dagger module recommend` in your repository. Official modules find your tools and add their checks, such as Go tests or ESLint ([Quickstart](https://docs.dagger.io/getting-started/quickstart)). Then `dagger cloud checks on` runs the checks after each push and pull request, with no workflow file ([Cloud Checks](https://docs.dagger.io/cloud-checks)). A tool with no official module needs a module that you write. ## Speed and hardware Blacksmith runs jobs on bare-metal machines with gaming CPUs, in Firecracker microVMs. It says these are about twice as fast as GitHub's runners for most jobs, and that runners start in under three seconds ([Blacksmith runners](https://docs.blacksmith.sh/blacksmith-runners/overview)). Your job does the same work, on faster hardware. Dagger runs your checks on Cloud Engines, the engines that Dagger Cloud runs for you, with scale-out and a distributed cache ([Cloud Engines](https://dagger.io/changelog/cloud-engines)). It also does not repeat work it already did. The engine caches the result of each operation. When the inputs of an operation do not change, it uses the cached result ([Introduction](https://docs.dagger.io/getting-started/introduction)). ## Cache Blacksmith keeps a cache next to its runners. The usual cache actions, such as `actions/cache`, use it with no change ([Blacksmith runners](https://docs.blacksmith.sh/blacksmith-runners/overview)). For Docker builds, its own build actions keep a Docker layer cache, which is a paid add-on ([Blacksmith pricing](https://www.blacksmith.sh/pricing)). You still choose what to cache and with which key. Dagger caches each operation of a check without cache keys. A cached result is used again when the operation's inputs are the same. ## Tests On Blacksmith, your tests run the way your workflow runs them. A test failure is a failed step in a job log. Dagger sees each test, not only the job that runs it ([test-native CI](https://dagger.io/changelog/test-native-ci)). Official modules report each test separately, with no change to your tests. When a test fails, you see which test, and you run only that test again: ```shell dagger check --go-test=TestGet ``` Dagger also splits large test suites across engines ([test splitting](https://dagger.io/changelog/test-splitting)). ## Local runs and agents Blacksmith runs CI jobs. To reproduce a job on your machine, you run its commands yourself. Dagger runs the same checks on your machine with `dagger check`, and in Dagger Cloud after a push. `dagger check -i` opens a shell in the container where a command failed ([dev shells](https://dagger.io/changelog/dev-shells)). AI agents that write code use the same checks and the same containers. ## Pricing Blacksmith bills per runner minute, at a rate for each OS and architecture ([Blacksmith pricing](https://www.blacksmith.sh/pricing)). It says its per-minute prices are half of GitHub's. Each organization gets 3,000 free minutes each month. Dagger Cloud bills the CPU time, memory, and data that your pipeline uses, and idle capacity is free ([Dagger pricing](https://dagger.io/pricing)). Repeat work costs 5× less: an operation with the same inputs as one in the last 30 days. Each plan has a monthly minimum. The Dagger engine is open source, and runs on your own machines at no cost. ## When to choose Blacksmith - You want to keep your GitHub Actions workflows as they are. - You want a faster CI with a one-line change per job. - You need jobs that run on Windows or macOS machines. Blacksmith offers both runner types. ## When to choose Dagger - You want the same checks on your machine, for your agents, and in CI. - You want to see each test, and run only the test that failed. - You want to pay for the CPU and memory that you use, not for runner minutes. - You want to run on your own servers, on Kubernetes, or in Dagger Cloud with one setup ([remote execution](https://dagger.io/changelog/remote-execution)). You can also use both. A workflow on a Blacksmith runner can run `dagger check` as one of its steps ([Checking your code in CI](https://docs.dagger.io/using/checking#in-ci)). **Bottom line:** Blacksmith runs your GitHub Actions jobs on faster, cheaper runners, with a one-line change per job. Dagger replaces those jobs with checks that run the same everywhere, sees each test, and bills used resources. Choose Blacksmith to keep your workflows, and Dagger to make them portable. ## Related guides - [What are the alternatives to GitHub Actions, from faster runners to portable pipelines?](/learn/github-actions-alternatives): runner vendors, other CI platforms, and Dagger. - [How does Dagger compare to Buildkite?](/learn/dagger-vs-buildkite): a CI platform with self-hosted agents compared with Dagger. - [Can Dagger lower CI costs?](/learn/reduce-ci-costs): how Dagger bills and caches work to lower the cost of CI. --- # How does Dagger compare to Buildkite? URL: https://dagger.io/learn/dagger-vs-buildkite Buildkite is a CI/CD platform for teams that want control of their build machines. Its control plane schedules pipeline steps on agents, which you host on your own infrastructure or let Buildkite host. Dagger is an engine that runs checks, such as tests and linters, the same way on your machine, on your servers, and in Dagger Cloud. Choose Buildkite for large CI fleets on your own infrastructure. Choose Dagger to run the same checks locally, for agents, and in CI. | | Buildkite | Dagger | | --- | --- | --- | | How you define work | Pipeline steps in YAML, plus plugins | Official modules that find your tools, plus your own modules | | Where work runs | Agents on your infrastructure, or hosted agents | Your machine, your servers, or Dagger Cloud | | Tests | Test Engine: analytics, flaky test tools, test splitting | Each test is a check, with test splitting | | Cache | Cache plugin, cache volumes, Docker layer cache | The engine caches each operation | | Local runs | `bk preflight` runs your local changes as a CI build | `dagger check` runs the checks on your machine | | Billing | Per active user, plus hosted agent minutes | Used CPU, memory, and data, with repeat work 5× cheaper | ## Architecture Buildkite splits CI in two parts. Its SaaS control plane coordinates the work and shows the results. Agents run the builds, on your on-premises or cloud machines, so your code and secrets stay in your environment ([Buildkite architecture](https://buildkite.com/docs/pipelines/architecture)). Buildkite hosted agents give you the control plane and the machines together. Dagger runs each check in containers on a Dagger engine. The engine is open source, and runs on your machine or your own server ([dagger.io](https://dagger.io/)). Dagger can also send a pipeline to a remote engine over SSH, on Kubernetes, or on Dagger Cloud ([remote execution](https://dagger.io/changelog/remote-execution)). ## Adoption In Buildkite, you define a pipeline as steps in YAML, in a `pipeline.yml` file in your repository or in the web editor ([steps](https://buildkite.com/docs/pipelines/configure/defining-steps)). Plugins add common tasks to a step ([plugins](https://buildkite.com/docs/pipelines/integrations/plugins/using)). In Dagger, you install official modules, which find your tools and add their checks ([Quickstart](https://docs.dagger.io/getting-started/quickstart)). A module is a package of checks and functions for a tool, such as Go, pytest, or ESLint ([modules](https://docs.dagger.io/reference/modules/index)). Then `dagger cloud checks on` runs the checks after each push and pull request ([Cloud Checks](https://docs.dagger.io/cloud-checks)). Cloud Checks need a GitHub repository. Dagger itself works with any Git host, because any CI system that can run the Dagger CLI can run `dagger check` ([Checking your code in CI](https://docs.dagger.io/using/checking#in-ci)). A tool with no official module needs a module that you write. ## Tests Buildkite Test Engine collects test results from your pipeline jobs, for any test framework ([Test Engine](https://buildkite.com/docs/test-engine)). It tracks slow and unreliable tests, and it can quarantine flaky tests so they do not block builds. Its test splitter, bktec, divides tests evenly across parallel agents. Test Engine also accepts results from other CI systems. Dagger sees each test, not only the job that runs it ([test-native CI](https://dagger.io/changelog/test-native-ci)). Official modules report each test separately, with no change to your tests. When a test fails, you run only that test again: ```shell dagger check --go-test=TestGet ``` Dagger splits large test suites across engines. It balances the parts with the measured resource use, input cost, and cache state of each test ([test splitting](https://dagger.io/changelog/test-splitting)). ## Cache Buildkite gives you several caches ([Buildkite caching](https://buildkite.com/docs/pipelines/best-practices/caching)). The Cache plugin saves directories to object storage, such as Amazon S3, by a key. Hosted agents can attach cache volumes, which Buildkite describes as best-effort. Plugins cache Docker layers, and Buildkite Cache, in public preview, saves and restores by a key that you define. Dagger caches each operation of a check without cache keys. When the inputs of an operation do not change, the engine uses the cached result ([Introduction](https://docs.dagger.io/getting-started/introduction)). Steps pass files and containers to each other in the engine. ## Local runs and agents Buildkite Preflight is an experimental CLI command, `bk preflight`. It pushes your uncommitted changes to a temporary branch, starts a build, and reports failures in your terminal ([preflight](https://buildkite.com/docs/platform/cli/preflight)). The build runs on your Buildkite agents. Dagger runs the checks on your machine with `dagger check`, with no push. `dagger check -i` opens a shell in the container where a command failed ([dev shells](https://dagger.io/changelog/dev-shells)). AI agents that write code use the same checks and containers as your CI. ## Infrastructure control Buildkite is built for control of agent fleets. You run agents on your own machines, put them in clusters and queues, and scale them as you need ([clusters](https://buildkite.com/docs/pipelines/security/clusters)). Dagger gives you control of where the engine runs, with the same checks everywhere. Cloud Checks run on Cloud Engines, which Dagger Cloud runs for you ([Cloud Engines](https://dagger.io/changelog/cloud-engines)). Cloud Checks on your own machines are not released yet ([bring your own compute](https://dagger.io/changelog/byoc)). ## Pricing Buildkite bills per active user each month, and hosted agents per vCPU minute ([Buildkite pricing](https://buildkite.com/pricing/)). Self-hosted agents above an included number cost a monthly fee per agent. Test Engine bills per test execution above an included number. Dagger Cloud bills the CPU time, memory, and data that your pipeline uses ([Dagger pricing](https://dagger.io/pricing)). Repeat work costs 5× less: an operation with the same inputs as one in the last 30 days. Each plan has a monthly minimum. The open-source engine on your own machines costs nothing. ## When to choose Buildkite - You run CI for many teams on your own infrastructure, and you want to manage the agent fleet. - Your code and secrets must stay on your machines, with a hosted control plane. - You want to quarantine flaky tests so they stop blocking builds. ## When to choose Dagger - You want the same checks on your machine, for your agents, and in CI. - You want to see each test, and run only the test that failed, with no change to your tests. - You want a cache for each operation, with no cache keys to maintain. - You want to pay for the CPU and memory that you use. **Bottom line:** Buildkite is a CI/CD platform that runs your pipeline steps on agents that you control, with strong test analytics. Dagger runs the same checks on your machine, on your servers, and in Dagger Cloud, and sees each test. Choose Buildkite to manage a large agent fleet, and Dagger to run one set of checks everywhere. ## Related guides - [How does Dagger compare to Blacksmith?](/learn/dagger-vs-blacksmith): faster GitHub Actions runners compared with Dagger. - [What are the alternatives to GitHub Actions, from faster runners to portable pipelines?](/learn/github-actions-alternatives): runner vendors, other CI platforms, and Dagger. - [Which development tools can Dagger replace or consolidate?](/learn/consolidate-ci-tools): which CI tools Dagger replaces, and which to keep. --- # How do I run pytest the same way locally and in CI? URL: https://dagger.io/learn/pytest-dagger To run pytest the same way on a laptop and in CI, meet the same requirements in both places. A good setup has five requirements: the same pinned environment, a result for each test, the test services, the same test selection, and parallel runs. By hand, teams meet them with uv, a CI job, and CI service containers. Then the Python version and the services live in more than one place. Dagger meets the same requirements in one place. Its official pytest module runs the suite in a container with uv, and Cloud Checks run the same check on each push. ## What a good pytest setup needs A pytest setup that runs the same on a laptop and in CI has five requirements. Each one comes from the docs of pytest or of the tools around it: 1. **The same pinned environment.** Each machine runs the same Python, pytest, and dependency versions, from a lockfile. The pytest docs recommend a virtual environment, because it ["ensures your code and dependencies are isolated from your system Python installation"](https://docs.pytest.org/en/stable/explanation/goodpractices.html#install-package-with-pip). The tests import the project as an installed package, as with the [`src` layout](https://docs.pytest.org/en/stable/explanation/goodpractices.html#tests-outside-application-code). 2. **A result for each test.** CI shows which test failed, not only that the job failed. pytest writes JUnit XML for that purpose, as ["result files which can be read by Jenkins or other Continuous integration servers"](https://docs.pytest.org/en/stable/how-to/output.html#creating-junitxml-format-files). 3. **The test services.** A test that needs a database gets one, at an address that it reads from the environment. Where the database is not available, the test [skips with `skipif`](https://docs.pytest.org/en/stable/how-to/skipping.html#skipif) and does not fail. A skip means that ["you expect your test to pass only if some conditions are met"](https://docs.pytest.org/en/stable/how-to/skipping.html). 4. **The same test selection.** A developer and CI select tests with the same options: a file, a keyword with `-k`, or a marker with `-m`. ["Pytest supports several ways to run and select tests"](https://docs.pytest.org/en/stable/how-to/usage.html#specifying-which-tests-to-run) from the command line. 5. **Parallel runs.** A large suite uses all the CPUs of the machine. With the pytest-xdist plugin and `-n auto`, ["pytest-xdist will use as many processes as your computer has physical CPU cores"](https://pytest-xdist.readthedocs.io/en/stable/distribution.html). ## Meet the requirements by hand with uv On a laptop, uv meets "the same pinned environment". It reads the Python version from a [`.python-version` file](https://docs.astral.sh/uv/concepts/python-versions/#python-version-files) and installs the versions in `uv.lock`. With `--locked`, ["if the lockfile is not up-to-date, uv will raise an error instead of updating the lockfile"](https://docs.astral.sh/uv/concepts/projects/sync/#automatic-lock-and-sync). uv [installs the project itself only when it declares a build system](https://docs.astral.sh/uv/concepts/projects/config/#build-systems), so `pyproject.toml` needs a `[build-system]` table. Install the environment and run the tests: ```shell uv sync --locked uv run pytest -q -rs ``` ```text Resolved 13 packages in 11ms Audited 10 packages in 0.16ms ..s [100%] =========================== short test summary info ============================ SKIPPED [1] tests/test_db.py:8: DATABASE_URL not set 2 passed, 1 skipped in 2.00s ``` `-rs` prints the reason for each skip. `test_database` skips, because `DATABASE_URL` is not set. To give it a database, a developer starts Postgres, for example with Docker Compose, and sets the variable. [How do I run integration tests against real databases and Kafka without mocks?](/learn/integration-tests-real-database-containers) compares Compose, Testcontainers, and CI service containers for that step. Select tests with pytest's own options. This command skips the tests with the `slow` marker: ```shell uv run pytest -q -m "not slow" ``` ```text .s [100%] 1 passed, 1 skipped, 1 deselected in 0.00s ``` For parallel runs, add pytest-xdist to the dev dependencies and give `-n auto`: ```shell uv run pytest -q -n auto ``` ```text bringing up nodes... bringing up nodes... .s. [100%] 2 passed, 1 skipped in 2.62s ``` For a result for each test, add `--junit-xml=report.xml`. pytest then writes a JUnit file that a CI system can read. ## Run pytest in a CI job A CI job runs the same uv commands. On GitHub Actions, the uv docs install uv with the `astral-sh/setup-uv` action, then run `uv sync --locked` and `uv run pytest` ([Using uv in GitHub Actions](https://docs.astral.sh/uv/guides/integration/github/)). The action pins the uv version with its `version` input. For the database, a `services:` block in the job starts Postgres before the steps ([service containers](https://docs.github.com/en/actions/tutorials/use-containerized-services/use-docker-service-containers)). The GitHub guide for Python writes the JUnit file with `--junitxml` and keeps it with `actions/upload-artifact` ([Building and testing Python](https://docs.github.com/en/actions/tutorials/build-and-test-code/python#packaging-workflow-data-as-artifacts)). ## Where the hand-built setup strains The commands work, but the setup grows harder to keep the same in both places: - **The same pinned environment.** The Python version can live in `.python-version` and in the workflow. A matrix version in `setup-uv` ["will override the Python version specification in the `pyproject.toml` or `.python-version` files"](https://docs.astral.sh/uv/guides/integration/github/#multiple-python-versions). Each laptop runs the uv version that its developer installed. - **A result for each test.** The JUnit file reaches the reader as a workflow artifact or through another report step. On a laptop, the result is the terminal output. - **The test services.** CI service containers run only in CI. A laptop needs a second definition of each service, such as a Compose file, and the two can drift apart. - **Parallel runs.** pytest-xdist uses the CPUs of one machine. To use more machines, the workflow needs a matrix and a way to divide the test files between the jobs. ## The same pinned environment, with the Dagger pytest module The [Dagger pytest module](https://docs.dagger.io/reference/modules/python/pytest) runs pytest in a container that has Python and uv. A Dagger [module](https://docs.dagger.io/reference/modules) is a package of functions and checks that Dagger runs in containers. The project must be a Git repository with a [Dagger workspace](https://docs.dagger.io/getting-started/quickstart). A workspace is the `dagger.toml` file that lists the modules a repository uses, and `dagger init` creates it. Install the official module for pytest: ```shell dagger module install dagger.io/python/pytest ``` ```text dagger.io/python/pytest ➡️ https://github.com/dagger/python#v0.2.0:pytest Installed module "pytest" in /work/dagger.toml ``` The command adds a `[modules.pytest]` entry to `dagger.toml`. It also writes `dagger.lock`, which pins the module and the images the workspace resolves to exact versions. The module looks for pytest projects. A project is a directory that holds `pytest.ini`, `pyproject.toml`, `tox.ini`, or `setup.cfg`, and that has test files of its own. A test file is named `test_*.py` or `*_test.py` and defines a `test` function or a `Test` class. The container has Python 3.14 by default. Set the `version` setting to match the project: ```shell dagger module settings pytest version 3.13 ``` A project that does not pin pytest gets pytest 9.1.1, from the `defaultVersion` setting. If the project has a `uv.lock`, the module installs from it with `--locked`. A lockfile that is out of date then fails the environment ([module README](https://github.com/dagger/python)). The module uses uv, so the build system rule from the hand-built setup applies here too. If the tests import the project's own package, as with a `src/` layout, `pyproject.toml` needs a `[build-system]` table. Without one, collection fails: ```text E ModuleNotFoundError: No module named 'shop' ``` ## A result for each test, from the module's trace `dagger check` reports each test by name, on a laptop and in CI. The module adds one check, `test`. A check is a function that passes or fails, and `dagger check` runs every check in the workspace. List the checks: ```shell dagger check -l ``` ```text MODULE PYTEST-PROJECT PYTEST-TEST-FILE CHECK DESCRIPTION pytest * * test Run this test file. # Use --all to expand collections and list each item. ``` Run the tests: ```shell dagger check ``` ```text == CHECKS == ✔ 1 passed ✔ dag://?check=test&pytest-project=. 16.0s OK == TESTS == ∅ tests/test_db.py › test_database SKIP Skipped: DATABASE_URL not set ✔ tests/test_cart.py 2 passed ∅ 1 skipped ✔ 2 passed ``` The module runs one `pytest` command per project. It adds a bundled OpenTelemetry plugin, so each test shows as its own result, with no change to the project. The `tracing` setting loads the plugin, and it is on by default. Dagger Cloud shows the same tests from the trace ([pytest module](https://docs.dagger.io/reference/modules/python/pytest)). A failure names the test and shows the pytest assertion. It also gives a command that runs the same selection again: ```text == CHECKS == ✘ 1 failed ✘ dag://?check=test&pytest-project=. 3.1s ERROR == TESTS == ✘ tests/test_cart.py › test_total FAIL def test_total(): > assert add(2, 2) == 5 E assert 4 == 5 E + where 4 = add(2, 2) tests/test_cart.py:9: AssertionError ∅ tests/test_db.py › test_database SKIP Skipped: DATABASE_URL not set ✘ 1 failed ∅ 1 skipped ✔ 1 passed == RUN LOCALLY == dagger check "dag://?check=test&pytest-project=." ``` ## The test services, with a test runtime If the tests need Postgres, give the module a test container that has the database bound to it. The `base` setting takes that container. A function in a module of your own supplies it, and Dagger gives the function's result to the pytest module. The docs call this [module wiring](https://docs.dagger.io/config/module-wiring). This is the Compose model: one definition of each service, outside the tests. The tests keep the conventions from the hand-built setup, so they have no Dagger code: - The tests read the database address from an environment variable, such as `DATABASE_URL`. - The tests [skip with `skipif`](https://docs.pytest.org/en/stable/how-to/skipping.html#skipif) when the variable is not set, as in the `SKIP` result above. Install the Dang SDK and create the project's dev module. [Dang](https://docs.dagger.io/reference/sdks/dang) is Dagger's own language for modules: ```shell dagger module install dagger.io/sdk/dang dagger module init dang -y ``` `dagger module init` creates the module at `.dagger/modules/-dev` and makes it the workspace main module. Replace the contents of `.dagger/modules/-dev/main.dang` with this code. `ShopDev` stands in for the type name that `dagger module init` generated, so keep your generated name. ```dang type ShopDev { """ A Postgres database for development and tests. """ postgres: Service! { container .from("postgres:17") .withEnvVariable("POSTGRES_PASSWORD", "test") .withExposedPort(5432) .asService } """ The container pytest runs in, with Python, uv, and Postgres. """ testRuntime: Container! { container .from("ghcr.io/astral-sh/uv:python3.13-trixie-slim") .withServiceBinding("db", postgres) .withEnvVariable("DATABASE_URL", "postgresql://postgres:test@db:5432/postgres") } } ``` `testRuntime` starts from an [official uv image with Python](https://docs.astral.sh/uv/guides/integration/docker/#available-images). It binds Postgres under the hostname `db` and sets the variable that the tests read. Each container that the pytest module builds from this base keeps the binding, so Postgres runs whenever pytest runs. `postgres` returns a `Service`, so `dagger start` can also start it for local development: ```shell dagger start -l ``` ```text MODULE SERVICE DESCRIPTION work-dev postgres A Postgres database for development and tests. ``` The `MODULE` column shows the name of your dev module. This guide's scratch project is named `work`, so its module is `work-dev`. The image in `base` sets the Python version, so you cannot set both `version` and `base`. With both set, `dagger check` fails: ```text ! evaluating new() for Pyproject: version and base are mutually exclusive ``` Remove `version` if you set it, then wire the function into `base`: ```shell dagger module settings -u pytest version dagger module settings pytest base test-runtime ``` The command accepts a bare function name from the workspace main module and stores its full address. `dagger.toml` now reads: ```toml [modules.pytest] source = "dagger.io/python/pytest" [modules.pytest.settings] base = "dag://-dev/test-runtime" [modules.dang-sdk] source = "dagger.io/sdk/dang" [sdks.dang] module = "dang-sdk" [modules.-dev] main = true source = ".dagger/modules/-dev" [sdks.dang.scopes.".dagger/modules/-dev"] is-module = true ``` Run the database test again. It passes now, where it skipped before: ```shell dagger check --pytest-test-file=tests/test_db.py ``` ```text == CHECKS == ✔ 1 passed ✔ dag://?check=test&pytest-project=.&pytest-test-file=tests/test_db.py 14.7s OK == TESTS == ✔ tests/test_db.py 1 passed ✔ 1 passed == SERVICES == ✔ q5q0otklmb4uk.8j704dlocoubk.dagger.local exec docker-entrypoint.sh postgres 4.2s EXITED span=5ee7d0524e043670 ``` The `SERVICES` section lists the Postgres service that the check started. Some tests start their own containers, as with Testcontainers. Dagger supports that model too. The tests call the Dagger API, and the `nesting` setting lets them reach a nested Dagger engine. It is on by default. [Start services from test code with a Dagger client](/learn/integration-tests-real-database-containers#start-services-from-test-code-with-a-dagger-client) shows this model. ## The same test selection, with check flags The pytest module turns projects and test files into flags for `dagger check`, and it passes pytest's own options through the `args` setting. Projects and test files are collections. Add `--all` to list one row per test file: ```shell dagger check -l --all ``` ```text MODULE PYTEST-PROJECT PYTEST-TEST-FILE CHECK DESCRIPTION dang-sdk stale staleness check: re-generate modules and clients managed by the Dang SDK pytest . tests/test_cart.py test Run this test file. pytest . tests/test_db.py test Run this test file. ``` The `stale` check comes from the Dang SDK, which you installed for the dev module. Select tests by test file: ```shell dagger check --pytest-test-file=tests/test_db.py ``` ```text == CHECKS == ✔ 1 passed ✔ dag://?check=test&pytest-project=.&pytest-test-file=tests/test_db.py 14.7s OK == TESTS == ✔ tests/test_db.py 1 passed ✔ 1 passed == SERVICES == ✔ q5q0otklmb4uk.8j704dlocoubk.dagger.local exec docker-entrypoint.sh postgres 4.2s EXITED span=5ee7d0524e043670 ``` With every file of a project selected, the module runs `pytest` on the whole project. With some files selected, it gives only those files to `pytest`. Repeat the flag to select more than one file. In a repository with several projects, each project root is a key. List the keys: ```shell dagger list pytest-projects -a ``` ```text MODULE PYTEST-PROJECT DESCRIPTION pytest services/api The pytest project rooted at path. pytest services/worker The pytest project rooted at path. ``` Run one project: ```shell dagger check --pytest-project=services/api ``` ```text == CHECKS == ✔ 1 passed ✔ dag://?check=test&pytest-project=services/api 9.2s OK == TESTS == ✔ tests/test_api.py 2 passed ✔ 2 passed ``` The flags select whole files. To select tests by marker or keyword, give pytest the same options as by hand through `args`. The default is `["-v"]`. The `scope` setting selects project roots, and its default is `["**"]`. A `!` pattern excludes a project, and an exclusion always wins. A list value is one JSON argument. This example skips tests with the `slow` [marker](https://docs.pytest.org/en/stable/how-to/usage.html#specifying-which-tests-to-run) and excludes a `docs` project: ```shell dagger module settings pytest args '["-v", "-m", "not slow"]' dagger module settings pytest scope '["**", "!docs"]' ``` `dagger.toml` then holds the settings, so every machine selects the same tests: ```toml [modules.pytest.settings] args = ["-v", "-m", "not slow"] scope = ["**", "!docs"] ``` [Select collection items](https://docs.dagger.io/using/checking#select-collection-items) covers every selection form. ## Parallel runs, across projects and engines The pytest module tests each selected project in parallel, and it reports every project that fails. Inside a project, the module runs one `pytest` command. If the project depends on [pytest-xdist](https://pytest-xdist.readthedocs.io/en/stable/distribution.html), give `-n auto` through `args`, as by hand. pytest-xdist then starts one worker for each physical CPU core: ```shell dagger module settings pytest args '["-v", "-n", "auto"]' ``` For a large suite, Dagger splits the tests across engines. It balances the parts with the measured resource use, input cost, and cache state of each test ([test splitting](https://dagger.io/changelog/test-splitting)). You write no CI matrix for it. ## Run pytest locally and with Cloud Checks `dagger check` runs the same check on a laptop and in Dagger Cloud. Dagger caches each step by its inputs. In this guide's project, the first `dagger check` took 12.7 seconds. A rerun with no changes took 2.5 seconds. A rerun after an edit to one test file took 6.8 seconds, because the dependency install stayed in the cache. These numbers come from one run on one machine, not from a benchmark. [Cloud Checks](https://docs.dagger.io/cloud-checks) run your checks on Dagger's Cloud Engines for each push and pull request to a GitHub repository, with no CI workflow file. Commit the workspace configuration, the lock file, and the dev module: ```shell git add dagger.toml dagger.lock .dagger git commit -m "Run pytest with Dagger" ``` Turn on Cloud Checks from the project root: ```shell dagger cloud checks on ``` The command finds the repository from its Git `origin` remote. In a terminal, it offers to create a Dagger Cloud account and to connect GitHub if those steps are not done. Push to start the checks. The results show as GitHub check statuses and in Dagger Cloud, and `dagger workspace activity` lists the recent runs. **Bottom line:** To run pytest the same way locally and in CI, pin Python, pytest, and the dependencies, report each test, and give the tests their services. Select tests with the same options everywhere, and run them in parallel. uv, a CI job, and CI service containers do this by hand, with the settings in several files. The Dagger pytest module does it from one `dagger.toml`, and Cloud Checks run the same `dagger check` on each push. ## Related guides - [How do I run integration tests against real databases and Kafka without mocks?](/learn/integration-tests-real-database-containers): the Compose and Testcontainers models for test services, in more depth than the one Postgres container here. - [How do I run Jest in CI and split the tests with --shard?](/learn/jest-dagger): the same steps for JavaScript and TypeScript tests. - [How do I run Go tests, lint, and go generate the same way locally and in CI?](/learn/daggerize-go-project): the same steps for a Go project, with the official Go module. --- # How do I make Go builds faster? URL: https://dagger.io/learn/speed-up-go-builds Most slow Go builds have one cause: the `go` command starts with an empty cache. A fast Go build keeps the build cache (`GOCACHE`) and the module cache (`GOMODCACHE`) between runs. It also keeps the inputs of each cache key stable, lets `go test` reuse passed results, and builds cgo code with a C compiler. On a laptop, Go keeps its caches for you. On a CI runner that starts empty, you must restore both caches yourself. Dagger meets the same requirements in one place. The `dagger.io/go` module mounts both Go caches as persistent volumes and runs the tests of each package as a separate check, in parallel, locally and in CI. ## What makes a Go build slow A fast Go build meets these requirements. The [`go` command documentation](https://pkg.go.dev/cmd/go#hdr-Build_and_test_caching) is the source for each one. 1. **Keep the build cache.** The `go` command caches build outputs in `GOCACHE`, which is `~/.cache/go-build` on Linux. Since [Go 1.20](https://go.dev/doc/go1.20#go-command), Go does not ship precompiled standard library packages. They are "built as needed and cached in the build cache", so an empty `GOCACHE` also compiles `fmt`, `net/http`, and the rest of the standard library again. 2. **Keep the module cache.** The `go` command stores downloaded modules in `GOMODCACHE`, which defaults to `GOPATH/pkg/mod`. An empty module cache downloads every dependency again. 3. **Keep cache keys stable.** The build cache "correctly accounts for changes to Go source files, compilers, compiler options, and so on". A change to the Go version, `CGO_ENABLED`, `-race`, or `-tags` gives new cache keys, so the build compiles again. 4. **Let `go test` reuse results.** `go test` caches passed results only in package list mode, and only with the cacheable flags. 5. **Handle cgo on purpose.** cgo needs a C compiler, and "the build cache does not detect changes to C libraries imported with cgo". ## Measure before you change anything Time the build first, then ask the `go` command why it rebuilt. Use these tools from the [`go help cache`](https://pkg.go.dev/cmd/go#hdr-Build_and_test_caching) and [`go help build`](https://pkg.go.dev/cmd/go#hdr-Compile_packages_and_dependencies) docs: - `time go test ./...` gives the wall time. A second run shows how much the cache saves. - `go build -x` prints each command that runs. A warm build with no change prints no compile commands. - `GODEBUG=gocachehash=1` prints the inputs of each cache key. Compare the output of two runs to find the input that changed. - `GODEBUG=gocachetest=1` prints why `go test` did or did not reuse a cached result. - `GODEBUG=gocacheverify=1` ignores the cache, rebuilds everything, and checks that the results match the cache entries. - `go build -a` rebuilds all packages, also packages that are up to date. This guide used a project with six packages that import the AWS SDK for Go v2 and `k8s.io/client-go`, with 167 lines in `go.sum`. In a `golang:1.26-bookworm` container with empty caches, each command took this long: | Command | Time | | --- | --- | | `go mod download` | 4.9 s | | `go test ./...`, first run | 14.0 s | | `go test ./...`, no change | 0.18 s, every package `(cached)` | | `go test -count=1 ./...` | 0.8 s | | `go test ./...` after a one-line change in one package | 0.8 s | These numbers come from one run on one machine. In this project, the first download and compile took most of the time, so keep the caches first. ## Keep GOCACHE and GOMODCACHE between runs The build cache and the module cache help only if they exist at the next run. On a developer machine, they stay in the home directory. The `go` command "periodically deletes cached data that has not been used recently", so you do not need to clean the cache. Use `go clean -cache` only to debug the cache. On a CI runner that starts empty, save both directories at the end of a job and restore them at the start of the next job. Get the paths with `go env GOCACHE GOMODCACHE`. On GitHub Actions, [`actions/setup-go`](https://github.com/actions/setup-go) caches modules and build outputs by default. By default, it makes the cache key from `go.mod`, and the `cache-dependency-path` input changes the file it uses. For full control, use [`actions/cache`](https://docs.github.com/en/actions/reference/workflows-and-actions/dependency-caching) with your own key. ## Keep cache keys stable Each change to a compiler option gives a new set of cache keys. In this guide's project, a warm build took 0.8 s. The next build, with `CGO_ENABLED=0`, took 8.7 s, because the `go` command compiled each package again for the new keys. Use the same Go version, `GOFLAGS`, build tags, and `CGO_ENABLED` value everywhere that shares a cache. `-race` and `-cover` are build flags too, so their builds use separate cache entries. A binary build has one more step after the compile: the link. In this guide's project, a one-line change to the `main` package rebuilt the binary in 0.4 s. With an empty cache, the same build took 5.8 s. The `go` command compiled only the changed package, then linked the binary again. ## Let go test reuse passed results `go test` caches a passed result only in package list mode. The [`go help test`](https://pkg.go.dev/cmd/go#hdr-Test_packages) docs define the two modes: - **Local directory mode** is `go test` with no package arguments. "In this mode, caching (discussed below) is disabled." - **Package list mode** is `go test` with package arguments, such as `go test ./...` or `go test .`. A passed package prints `(cached)` the next time. `go test` reuses a result only when the flags "come entirely from a restricted set of 'cacheable' test flags": `-benchtime`, `-coverprofile`, `-cpu`, `-failfast`, `-fullpath`, `-list`, `-outputdir`, `-parallel`, `-run`, `-short`, `-skip`, `-timeout`, and `-v`. Any other flag turns off the test cache. "The idiomatic way to disable test caching explicitly is to use -count=1." Search your Makefile and CI scripts for `-count=1`. Keep it only on tests that must run each time, such as tests against a live service. Tests that open files in the module or read environment variables "only match future runs in which the files and environment variables are unchanged". Run `GODEBUG=gocachetest=1 go test ./pkg` to see which input made a test run again. Run `go clean -testcache` to remove the cached test results and keep the build cache. ## Handle cgo on purpose cgo packages need a C compiler, and the build cache does not track the C libraries they use. If the C libraries on the system change, use `go build -a` or clean the cache, as [`go help cache`](https://pkg.go.dev/cmd/go#hdr-Build_and_test_caching) says. Since [Go 1.20](https://go.dev/doc/go1.20#cgo), `CGO_ENABLED` defaults to `0` when the `go` command cannot find a C compiler. The `golang:1.26-alpine` image has no `gcc`, so a file with `import "C"` drops out of the build there. In this guide's run, the error named a missing symbol, not cgo. If the project does not need cgo, set `CGO_ENABLED=0` everywhere, so that the cache keys stay the same. If it needs cgo, build in an image that has a C compiler, such as `golang:1.26-bookworm`. ## Where the hand-built cache strains in CI The requirements stay the same as a project grows, but the CI configuration that meets them gets harder to maintain: - **The cache goes stale.** "You cannot change the contents of an existing cache," as the [GitHub Actions cache reference](https://docs.github.com/en/actions/reference/workflows-and-actions/dependency-caching) says. With a key on `go.mod`, the saved build cache keeps old compiled packages until `go.mod` changes. A key on the source files gives a new cache for each commit, and needs `restore-keys` as a fallback. - **The settings live in many places.** The Go version, `CGO_ENABLED`, and build tags are in the Makefile, the workflow, and the shell of each developer. When one copy changes and the others do not, the cache keys no longer match. ## Keep the Go caches warm with Dagger Dagger runs the same `go` commands in containers on the [Dagger Engine](https://docs.dagger.io/next/reference/glossary), which runs pipelines and stores the cache. A [module](https://docs.dagger.io/reference/modules) is a package of functions that build and test a project. The official `dagger.io/go` module finds each `go.mod`. It runs the tests of each package as a check, a function that passes or fails. Complete the [Quickstart](https://docs.dagger.io/getting-started/quickstart) first, in a Git repository. Install the Go module: ```shell dagger module install dagger.io/go ``` ```text Installed module "go" in /work/dagger.toml ``` The command adds the module to `dagger.toml`. It also writes `dagger.lock`, which pins the module to an exact version. Later Dagger commands add each image that they use to `dagger.lock`. Commit both files. Then run the checks: ```shell dagger check ``` **Keep the build cache and the module cache.** The Go module mounts two [cache volumes](https://docs.dagger.io/reference/api/cache-volume), directories whose contents persist across Dagger sessions. One is at `/go/pkg/mod`, the module cache. The other is at `/root/.cache/go-build`, the build cache. You do not write a save step or a restore step. **Keep cache keys stable.** Each Go module runs in `golang:-alpine`, for the version that its own `go.mod` asks for. The `version`, `goflags`, and `buildFlags` settings in `dagger.toml` apply to every run, on every machine ([Go module reference](https://docs.dagger.io/reference/modules/go)). **Let go test reuse results.** Dagger caches the result of a check by the files that the check saw ([Daggerize a Go Project](https://docs.dagger.io/getting-started/platform-guides/go)). Each package is a separate check that runs `go test` on that package, and `dagger check` runs all checks in parallel ([Checking your code](https://docs.dagger.io/using/checking)). The second run with no changes printed this: ```text == CHECKS == ✔ 7 passed ✔ dag://?check=go/packages/generate/stale 1.1s OK ✔ dag://?check=test&go-package=cart 1.0s OK ✔ dag://?check=test&go-package=inventory 1.0s OK ✔ dag://?check=test&go-package=orders 1.0s OK ✔ dag://?check=test&go-package=payments 1.0s OK ✔ dag://?check=test&go-package=shipping 1.0s OK ✔ dag://?check=test&go-package=users 1.0s OK ``` The same six-package project gave these times on one machine: | `dagger check` run | Time | | --- | --- | | First run, new dependencies | 49.1 s | | No change | 4.7 s | | One-line change in `orders` | 5.6 s | | Change to `README.md` only | 3.8 s | Dagger keeps the caches on each engine, so a CI run that uses the same engine starts warm. To test only the package you changed, select it: ```shell dagger check --go-package=orders ``` ## Run cgo packages in an image with a C compiler If no package uses cgo, skip this section. The default Alpine image has no `gcc`, so a cgo package does not build there. For a test of a cgo function, `dagger check` printed this error: ```text ┃ # example.com/shop/hash [example.com/shop/hash.test] ┃ hash/hash_test.go:6:5: undefined: Abs ┃ FAIL example.com/shop/hash [build failed] ``` Give the Go module a container with a C compiler through its `base` setting. Install the Dang SDK. Then create the project's dev module in [Dang](https://docs.dagger.io/reference/sdks/dang), the native language of Dagger modules: ```shell dagger module install dagger.io/sdk/dang dagger module init dang ``` `dagger module init` creates `.dagger/modules/-dev`. Replace the contents of `.dagger/modules/-dev/main.dang` with this function. `ProjectDev` stands in for the type name that `dagger module init` generated, so keep your own type name: ```dang type ProjectDev { """ The container the Go module runs tests in: Debian, with gcc for cgo. """ testRuntime: Container! { container.from("golang:1.26-bookworm") } } ``` Connect the function to the Go module: ```shell dagger module settings go base test-runtime ``` `dagger.toml` now holds `base = "dag://-dev/test-runtime"` under `[modules.go.settings]`. The image now sets the Go version, so the module ignores its `version` setting. This is the same fix as the hand-built version: an image with a C compiler. Run `dagger check` again, and the cgo test passes. ## Run the same checks locally and in CI [Cloud Checks](https://docs.dagger.io/cloud-checks) run `dagger check` on Dagger's Cloud Engines for each push and pull request to a GitHub repository, with no CI workflow file. Connect the repository: ```shell dagger cloud checks on ``` The checks in CI are the checks that `dagger check` runs on your machine, with the same image, the same `GOFLAGS`, and the same cache keys. **Bottom line:** To make Go builds faster, keep `GOCACHE` and `GOMODCACHE` between runs, and use the same compiler options and Go version everywhere. Run `go test` in package list mode without `-count=1`, and build cgo code in an image with a C compiler. The `dagger.io/go` module keeps both Go caches in persistent volumes and runs the tests of each package in parallel, locally and with Cloud Checks. ## Related guides - [How do I run Go tests, lint, and go generate the same way locally and in CI?](/learn/daggerize-go-project): the full Go module setup, with lint, generators, and test services. - [How do I run golangci-lint the same way locally and in CI?](/learn/golangci-lint-dagger): Go linting as a Dagger check. - [Can Dagger lower CI costs?](/learn/reduce-ci-costs): how cache reuse changes what CI costs. --- # How do I run Vitest in CI, with sharding and merged coverage? URL: https://dagger.io/learn/run-vitest-with-dagger A good Vitest setup for CI runs `vitest run` once, with the project's own Vitest on a supported Node version. It fails when coverage drops below a threshold, and it lets you rerun one test file. For a large suite, it splits the test files into shards and merges their blob reports into one coverage report. You can meet each requirement with the Vitest CLI and a CI job. Dagger's official Vitest module meets the same requirements in one place, and runs the same checks on a laptop and in CI. ## What a good Vitest CI setup needs Vitest's own docs give five requirements. This guide uses them to judge each approach, Dagger included: 1. **Run Vitest once, not in watch mode.** Vitest "starts in watch mode by default in development environment and run mode in CI environment" ([Features](https://vitest.dev/guide/features#watch-mode)). It detects CI from `process.env.CI`. `vitest run` does a single run in every environment. 2. **Use the project's own Vitest on a supported Node version.** The docs say: "It is recommended that you install a copy of vitest in your package.json". The current Vitest requires Node v22.12.0 or newer ([Getting Started](https://vitest.dev/guide/)). 3. **Fail when coverage drops below a threshold.** A positive threshold "will be interpreted as the minimum percentage of coverage required" ([`coverage.thresholds`](https://vitest.dev/config/coverage#coverage-thresholds)). Coverage needs a [coverage provider](https://vitest.dev/guide/coverage), such as `@vitest/coverage-v8`, in the project's `devDependencies`. 4. **Rerun one test file.** With a file name argument, Vitest "will only run test files whose path contains the given string" ([Test Filtering](https://vitest.dev/guide/filtering#filtering-by-file-name)). A developer can then repeat one CI failure without the full suite. 5. **Split a large suite into shards and merge the results.** Vitest runs one part of the suite on each machine with [`--shard` and `--reporter=blob`](https://vitest.dev/guide/improving-performance#sharding). The docs say: ["All test and coverage results can be merged at the end of your CI pipeline using `--merge-reports` command"](https://vitest.dev/guide/features#sharding). ## Run Vitest by hand in a CI job The Vitest CLI meets each requirement. Install `vitest` as a dev dependency, then run the suite one time from the project root: ```shell npx vitest run ``` ```text RUN v5.0.5 /work ✓ src/cart.test.ts (1 test) 2ms ✓ src/login.test.ts (2 tests) 2ms Test Files 2 passed (2) Tests 3 passed (3) Start at 01:29:07 Duration 110ms (transform 68%, import 20%, worker 7%, tests 5%) ``` To rerun one file, give its path: `npx vitest run src/cart.test.ts`. To fail on low coverage, add `--coverage` and a threshold, such as `npx vitest run --coverage --coverage.thresholds.lines=90`, or set `coverage.thresholds` in the Vitest config. In CI, a job checks out the code and installs Node with [`actions/setup-node`](https://github.com/actions/setup-node) or the equivalent of the CI system. Then it installs the dependencies and runs the same command. GitHub Actions sets `CI` to `true` in every job ([variables reference](https://docs.github.com/en/actions/reference/workflows-and-actions/variables#default-environment-variables)), so a plain `vitest` also does one run there. To split the suite, run one shard on each machine with the blob reporter. Then merge the reports in a last step: ```shell vitest run --reporter=blob --shard=1/2 --coverage vitest run --reporter=blob --shard=2/2 --coverage vitest run --merge-reports --coverage ``` Each shard writes its results to `.vitest/blob/`. Before the merge, copy the blob files from each machine into one `.vitest/blob/` directory. Pass `--coverage` to the merge command too. In this guide's project, the first shard reported 33.33% line coverage and the second shard reported 75%. The merged report showed 100%. The report of one shard is not the coverage of the whole suite. The Vitest docs give a [GitHub Actions example](https://vitest.dev/guide/improving-performance#sharding). A [matrix](https://docs.github.com/en/actions/how-tos/write-workflows/choose-what-workflows-do/run-job-variations) runs one job for each shard, and each job uploads `.vitest` as an [artifact](https://docs.github.com/en/actions/tutorials/store-and-share-data). A last job downloads the artifacts and runs `vitest --merge-reports`. ## Where the hand-built setup strains The commands are short, but each requirement lives in a different place: - **The Node version is in two places.** The CI job sets `node-version`, and each laptop has its own Node. They can drift apart, and Vitest then runs on a Node version that the project did not test. - **The shards are a fixed count of files.** Vitest "splits your _test files_, not your test cases, into shards" ([sharding](https://vitest.dev/guide/improving-performance#sharding)). The matrix fixes the shard count, so a few slow files can make one shard much slower than the others. - **The merge is CI plumbing.** Each shard needs an artifact upload, and the merge needs a separate job that downloads them all. A move to another CI system means a rewrite of these jobs. - **A CI failure is hard to repeat.** To rerun a failed file as CI ran it, a developer must match the Node version and the install of the CI job by hand. ## Meet them with the Dagger Vitest module The [Dagger Vitest module](https://docs.dagger.io/reference/modules/js/vitest) meets the same five requirements, with the Node image, the install, and the Vitest command defined in one place. A Dagger [module](https://docs.dagger.io/reference/modules) is a package of functions that test, lint, or build a project. The [Dagger Engine](https://docs.dagger.io/getting-started/introduction) runs those functions in containers, on a laptop or in CI. Complete the [Quickstart](https://docs.dagger.io/getting-started/quickstart) in the repository first. Then install the module from the project root: ```shell dagger module install dagger.io/js/vitest ``` ```text dagger.io/js/vitest ➡️ https://github.com/dagger/vitest#v0.3.0 Installed module "vitest" in /work/dagger.toml ``` The command adds the module to `dagger.toml`, the workspace configuration for everyone who uses the repository: ```toml [modules.vitest] source = "dagger.io/js/vitest" ``` The command also writes `dagger.lock`. This file pins each module, image, and Git source to an exact version. Commit both files. A check is a function that passes or fails, such as a test run. List the checks of the workspace: ```shell dagger check -l ``` ```text MODULE VITEST-PROJECT VITEST-TEST-FILE CHECK DESCRIPTION vitest * * test Run this test file. # Use --all to expand collections and list each item. ``` Each column is a collection: a set of items that the module finds in the project at run time. A `*` stands for every item in the collection. The next sections show how the module meets each requirement. ### Run Vitest once, not in watch mode The module sets `CI` to `true` in its container ([module source](https://github.com/dagger/vitest)), so the project's `vitest` does one run. Run every check in the workspace: ```shell dagger check ``` ```text == CHECKS == ✔ 1 passed ✔ dag://?check=test&vitest-project=. 27.8s OK ``` The module runs Vitest one time per project, over the selected test files. When you select all the files of a project, it runs `vitest` with the project's config, so the config decides what runs. Dagger runs the checks of the workspace in parallel ([Checking your code](https://docs.dagger.io/using/checking)). ### Use the project's own Vitest on a supported Node version The module runs the project's own `node_modules/.bin/vitest`, not a copy of its own ([README](https://github.com/dagger/vitest)). It installs the dependencies with the package manager that the `packageManager` field of `package.json` names. Without that field, it reads the lockfile. If it finds neither, it uses npm. To choose a package manager, set the `packageManager` setting. In a monorepo, the module installs dependencies at the nearest workspace root. That root is a directory with `pnpm-workspace.yaml`, or a `package.json` with `workspaces`. The tests run in `node:25-alpine` by default. If the project uses a different Node release, set the `baseImageAddress` setting to that release: ```shell dagger module settings vitest baseImageAddress node:22-alpine ``` Dagger stores settings in `dagger.toml` under `[modules.vitest.settings]`, so every machine uses the same Node image. A setting applies to every Vitest project in the workspace. The [module reference](https://docs.dagger.io/reference/modules/js/vitest) lists the other install settings, such as `installFlags` and `includeExtraFiles`. ### Fail when coverage drops below a threshold The module runs the project's config, so `coverage.thresholds` in the Vitest config apply. To set coverage for every project instead, use the `flags` setting. It passes extra flags to every `vitest` run. The project must list a coverage provider, such as `@vitest/coverage-v8`, in its `devDependencies`. ```shell dagger module settings vitest flags '["--coverage", "--coverage.thresholds.lines=90"]' ``` The check prints the coverage table. It fails when coverage is below the threshold. In this run, the check ran only `src/cart.test.ts`: ```text ┃ [vitest] % Coverage report from v8 ┃ [vitest] ----------|---------|----------|---------|---------|------------------- ┃ [vitest] File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s ┃ [vitest] ----------|---------|----------|---------|---------|------------------- ┃ [vitest] All files | 40 | 0 | 66.66 | 33.33 | ┃ [vitest] cart.ts | 40 | 0 | 66.66 | 33.33 | 6-7 ┃ [vitest] ----------|---------|----------|---------|---------|------------------- ┃ [vitest] ERROR: Coverage for lines (33.33%) does not meet global threshold (90%) ! Vitest project .: vitest failed (exit 1): ``` To remove the setting, run `dagger module settings -u vitest flags`. ### Rerun one test file Each test file is an item that you can select. Add `--all` to list one row per test file: ```shell dagger check -l --all ``` ```text MODULE VITEST-PROJECT VITEST-TEST-FILE CHECK DESCRIPTION vitest . src/cart.test.ts test Run this test file. vitest . src/login.test.ts test Run this test file. ``` The module treats each directory that holds a `vitest.config.*` or `vite.config.*` file as a Vitest project. The project at the workspace root has the key `.`. Test files are the files that match Vitest's default `include` pattern, such as `*.test.ts` and `*.spec.js`. The module does not read the `include` and `exclude` settings in your config to list files. To run one test file, give its path relative to the project: ```shell dagger check --vitest-test-file=src/cart.test.ts ``` ```text == CHECKS == ✔ 1 passed ✔ dag://?check=test&vitest-project=.&vitest-test-file=src/cart.test.ts 2.2s OK ``` In a monorepo, each app with its own Vitest config is a project. List the projects: ```shell dagger list vitest-projects -a ``` ```text MODULE VITEST-PROJECT DESCRIPTION vitest apps/admin The Vitest project rooted at path. vitest apps/web The Vitest project rooted at path. ``` To run one file in one project, give the project path relative to the workspace root: ```shell dagger check --vitest-project=apps/web --vitest-test-file=src/a.test.ts ``` ```text == CHECKS == ✔ 1 passed ✔ dag://?check=test&vitest-project=apps/web&vitest-test-file=src/a.test.ts 11.2s OK ``` Repeat a flag to select more than one key. To print the tests that Vitest finds in a project, call the project's `list` function: ```shell dagger api call vitest project --path=apps/web list ``` ```text src/a.test.ts > a src/b.test.ts > b ``` When a test fails, the check fails and prints the Vitest output. The report ends with a command that runs the same selection again, in the same container, on any machine: ```text == RUN LOCALLY == dagger check "dag://?check=test&vitest-project=." ``` ### Split a large suite into shards and merge the results The module runs the selected files of each project in one Vitest run. The coverage report of a project then comes from one run and needs no merge. In a monorepo, the module runs each project as a separate check, and Dagger runs the checks in parallel. Do not put `--shard` in `flags`. The module passes `flags` to every run, so each project would run only part of its files. For a large suite, Dagger splits the tests across engines instead. It balances the parts with the measured resource use, input cost, and cache state of each test ([test splitting](https://dagger.io/changelog/test-splitting)). You do not fix a shard count or move blob reports between jobs. ## Run the same checks locally and with Cloud Checks [Cloud Checks](https://docs.dagger.io/cloud-checks) run your checks on Dagger's Cloud Engines on each push and pull request, with no CI workflow file. Cloud Engines are Dagger Engines that Dagger Cloud runs for you. The repository must be on GitHub, and its `origin` remote must point to that repository. Turn on Cloud Checks from the project root: ```shell dagger cloud checks on ``` Commit `dagger.toml` and `dagger.lock`, then push. Each push runs the same checks that `dagger check` runs on your machine, with the same Node image and the same settings. The results show as GitHub check statuses and in Dagger Cloud ([changelog](https://dagger.io/changelog/cloud-checks)). The module reports each test as its own span, so Dagger Cloud shows each test with no change to your Vitest config. To see recent runs from the terminal, run `dagger workspace activity`. If a check fails in the cloud, run `dagger check` locally to get the same failure. **Bottom line:** Run `vitest run` with the project's own Vitest on a supported Node version, and set a coverage threshold. For a large suite, run shards with `--reporter=blob` and merge them with `--merge-reports --coverage`. Dagger's Vitest module meets the same requirements in one place. It pins the Node image in `dagger.toml`, selects single files with `--vitest-test-file`, and runs the same checks on your machine and with Cloud Checks. ## Related guides - [How do I run Jest in CI and split the tests with --shard?](/learn/jest-dagger): the same requirements for a project that tests with Jest. - [How do I run Playwright tests in CI and locally in the same container, with cached browsers?](/learn/playwright-ci-docker-browser-caching): browser tests for the same JavaScript project. - [How do I run ESLint the same way locally and in CI?](/learn/eslint-dagger): lint the same project with its own ESLint. --- # How do I cache Rust builds in CI without rebuilding everything? URL: https://dagger.io/learn/cache-rust-builds-in-ci A Rust CI cache has to keep Cargo's downloaded crates and the compiled dependencies in `target/` between runs. It has to follow changes to `Cargo.lock` and the toolchain, and every machine that builds the project has to reach it. On GitHub Actions, Swatinem/rust-cache meets most of these requirements. On other CI platforms, a platform cache, sccache, or cargo-chef each meets some of them. Dagger meets the same requirements in one place. Its [cache volumes](https://docs.dagger.io/reference/api/cache-volume) keep both directories between runs, and the same checks run on a laptop and in Dagger Cloud. ## What a Rust CI cache has to do A Rust CI cache has four requirements. Each GitHub-hosted runner is a new virtual machine ([GitHub-hosted runners](https://docs.github.com/en/actions/concepts/runners/github-hosted-runners)). A job on a new machine starts without Cargo's files, so Cargo downloads every crate and compiles every dependency again. A cache that meets these requirements lets Cargo skip that work: - **Keep the downloaded crates.** Cargo keeps them in its home directory, `~/.cargo`, or `/usr/local/cargo` in the official `rust` image. Its `registry` directory holds the crates from crates.io, and its `git` directory holds the Git dependencies ([Cargo home](https://doc.rust-lang.org/cargo/guide/cargo-home.html)). - **Keep the compiled dependencies.** Cargo writes the output of a build to the `target/` directory ([build cache](https://doc.rust-lang.org/cargo/reference/build-cache.html)). When the previous `target/` directory is present, Cargo recompiles only the crates that changed ([Cargo source](https://github.com/rust-lang/cargo/blob/master/src/compiler/fingerprint/mod.rs)). - **Follow changes to Cargo.lock and the toolchain.** Cargo's fingerprint of each crate includes the `rustc` version and the fingerprints of its dependencies ([Cargo source](https://github.com/rust-lang/cargo/blob/master/src/compiler/fingerprint/mod.rs)). After a dependency update or a new toolchain, Cargo recompiles the affected crates. The cache then has to store the new output, and an older cache still saves the work for the crates that did not change. - **Share the cache across CI machines and laptops.** A cache helps only the builds that can read it. When jobs run on many ephemeral runners, or on more than one CI platform, each of them needs the same cache. The sections below meet these requirements by hand, then with Dagger. ## Keep Cargo's directories on a long-lived runner On a self-hosted runner that lives a long time, keep both directories on the runner's disk. Set `CARGO_HOME` and `CARGO_TARGET_DIR` to directories on that disk. Nothing moves between runs, and Cargo's fingerprints follow the changes to `Cargo.lock` and the toolchain. The cache serves only that runner. Cargo cleans its global cache in Cargo's home directory automatically, but not build artifacts ([Cargo configuration](https://doc.rust-lang.org/cargo/reference/config.html#cache)). Old artifacts stay in the target directory until you remove them. ## Cache Cargo's home and target/ with Swatinem/rust-cache On GitHub Actions, the [Swatinem/rust-cache](https://github.com/Swatinem/rust-cache) action meets the first three requirements, and the fourth for jobs in GitHub Actions. Install the toolchain before the cache step, because the action puts the `rustc` version in its cache key. In a workflow file such as `.github/workflows/ci.yml`, the steps are: ```yaml - uses: actions/checkout@v6 - run: rustup toolchain install stable --profile minimal - uses: Swatinem/rust-cache@v2 with: shared-key: "ci" save-if: ${{ github.ref == 'refs/heads/main' }} - run: cargo test --locked ``` The [README](https://github.com/Swatinem/rust-cache#cache-details) says how the action meets each requirement: - **Keep the downloaded crates.** It caches `~/.cargo`. - **Keep the compiled dependencies.** It caches `./target`. Before it saves the cache, it removes the workspace crates and the incremental build artifacts. The cache keeps only compiled dependencies. The action also sets `CARGO_INCREMENTAL=0`. - **Follow changes to Cargo.lock and the toolchain.** Its cache key includes the `rustc` version, a hash of every `Cargo.lock` and `Cargo.toml`, the toolchain files, and compiler variables such as `RUSTFLAGS`. When the lockfile changes, it restores the cache of the previous lockfile. Then only the changed dependencies rebuild. - **Share the cache.** `shared-key` gives several jobs one cache. `save-if` limits saves to `main`. The action stores its cache in the GitHub Actions cache, so GitHub's cache limits apply to it. ## Cache Cargo with a platform cache keyed on Cargo.lock On a CI platform without a Rust-specific action, use the platform's own cache with a key from `Cargo.lock`. List Cargo's home and `target/` as the cache paths to keep the downloaded crates and the compiled dependencies. The key from `Cargo.lock` makes a new cache when the dependencies change. GitHub Actions cannot change a cache after it saves it, so a new key is the only way to store new output ([GitHub docs](https://docs.github.com/en/actions/reference/workflows-and-actions/dependency-caching)). On GitHub Actions, the platform cache is `actions/cache` with `hashFiles('**/Cargo.lock')` in the key ([GitHub docs](https://docs.github.com/en/actions/reference/workflows-and-actions/dependency-caching)). On GitLab CI/CD, it is `cache:key:files`, which makes a new key when the listed files change. GitLab cache paths [are relative to the project directory](https://docs.gitlab.com/ci/yaml/#cachepaths), so set `CARGO_HOME` to a directory in the project. In `.gitlab-ci.yml`: ```yaml variables: CARGO_HOME: $CI_PROJECT_DIR/.cargo cache: key: files: - Cargo.lock paths: - .cargo/ - target/ ``` If a `rust-toolchain.toml` file pins the toolchain, add it to the key files. Then a new toolchain also makes a new cache. ## Share compiled crates across machines with sccache [sccache](https://github.com/mozilla/sccache) shares compiled output across machines. Cargo's own docs name it as a way to "share built dependencies across different workspaces" ([build cache](https://doc.rust-lang.org/cargo/reference/build-cache.html)). It caches the output of each `rustc` call, not a whole directory, so it does not keep the downloaded crates. It stores results on local disk or in a remote backend: S3, R2, Google Cloud Storage, Azure, Redis, or the GitHub Actions cache. To turn it on, set `RUSTC_WRAPPER=sccache` in the build environment. For an S3 bucket, also set `SCCACHE_BUCKET` and `SCCACHE_REGION` ([S3 setup](https://github.com/mozilla/sccache/blob/main/docs/S3.md)). Many machines can read and write one bucket, so sccache fits a fleet of ephemeral runners, such as CI agents on Kubernetes. The [Rust notes](https://github.com/mozilla/sccache/blob/main/docs/Rust.md) of sccache list its limits for Rust: - Incremental compilation in `rustc` must be off. - sccache does not cache crates that call the system linker: `bin`, `dylib`, `cdylib`, and `proc-macro` crates. These crates compile on every run. - Procedural macros that read files from the file system may not cache correctly. ## Cache Rust dependencies in Docker builds with cargo-chef When CI builds a container image, [cargo-chef](https://github.com/LukeMathWalker/cargo-chef) keeps the compiled dependencies in a Docker layer of its own. `cargo chef prepare` writes a recipe, `recipe.json`, from the manifests and the lockfile. `cargo chef cook` builds only the dependencies from that recipe. Docker keeps that layer in its cache until the dependencies change. The README warns: "You must use the same Rust version in all stages". The cache reaches the machines that can read the Docker layer cache. ## Rust cache options against the requirements Each option meets a different part of the requirements: | Option | Keep the downloaded crates | Keep the compiled dependencies | Follow changes to Cargo.lock and the toolchain | Share the cache | | --- | --- | --- | --- | --- | | Long-lived runner | Yes, in `CARGO_HOME` on its disk | Yes, in `CARGO_TARGET_DIR` on its disk | Cargo's fingerprints | Only that runner | | Swatinem/rust-cache | Yes, `~/.cargo` | Yes, dependencies only | A key with `rustc`, the lockfiles, and `RUSTFLAGS`, and a fallback to the previous lockfile | GitHub Actions jobs, with `shared-key` | | Platform cache keyed on `Cargo.lock` | The paths you list | The paths you list | A key from the files you list | Jobs on that CI platform | | sccache | No | The output of each `rustc` call, except linked crates | Each `rustc` call caches on its own | Every machine that reads the backend | | cargo-chef | In the dependency layer | Yes, as a Docker layer | A recipe from the manifests and the lockfile | Machines that read the Docker layer cache | ## Limits of these caches as a project grows The hand-built caches strain on the same requirements as the project grows: - **Keep the compiled dependencies.** The target directory is large. An archive takes longer to upload and download as the workspace grows. GitHub gives each repository [10 GB of cache by default](https://docs.github.com/en/actions/reference/workflows-and-actions/dependency-caching). It removes entries that no run accessed in over 7 days. Above the limit, it removes the entries with the oldest last access. Pull request runs can restore caches from the default branch, and their own caches use the same 10 GB. - **Follow changes to Cargo.lock and the toolchain.** Keys miss. A key from `Cargo.lock` misses after each dependency update, unless a fallback key restores an older cache. A new toolchain or a new `RUSTFLAGS` value starts a new cache. - **Share the cache across CI machines and laptops.** Each setup belongs to one platform. rust-cache is a GitHub Action. GitLab, Buildkite, and Jenkins pipelines each need their own cache configuration. The CI platform caches do not apply when a developer runs the same steps on a laptop. sccache reaches every machine, but it does not cache linked crates. Binaries and proc-macro crates compile on every run. ## Cache Rust builds with Dagger cache volumes Dagger runs each Cargo command in a container and keeps Cargo's home and `target/` in cache volumes. A [cache volume](https://docs.dagger.io/reference/api/cache-volume) is a directory whose contents persist across Dagger sessions. It lives on the [Dagger Engine](https://docs.dagger.io/getting-started/introduction), the program that runs the containers. No archive moves between steps. On the next run, Cargo finds its compiled dependencies in the same place and rebuilds only what changed. Dagger also [caches each step by its inputs](https://docs.dagger.io/reference/sdks/dang#expressions-and-chaining), so a rerun skips the steps whose inputs did not change. The [official Dagger modules](https://docs.dagger.io/reference/modules/index) cover tools such as Go, pytest, and ESLint, but not Rust. A search for a Rust module finds nothing: ```shell dagger module search rust ``` ```text No matching modules found. ``` So the Cargo steps go in the project's own [Dagger module](https://docs.dagger.io/reference/sdks/index): a package of functions and checks that build and test the project. Write it in [Dang](https://docs.dagger.io/reference/sdks/dang), Dagger's native language for modules. Run these commands from the root of the Git repository: ```shell dagger init dagger module install dagger.io/sdk/dang dagger module init dang ``` `dagger init` creates `dagger.toml`, the workspace configuration, or uses the file that is there ([Quickstart](https://docs.dagger.io/getting-started/quickstart)). `dagger module init` without a name creates the module at `.dagger/modules/-dev`. It also makes the module the workspace main module. The command returns a [changeset](https://docs.dagger.io/using/generating), a diff of the proposed files, and Dagger shows it for review before it writes any file. Pass `-y` to apply the changeset without the prompt. A headless run, such as CI or an AI agent, needs `-y`. The commands also write `dagger.lock`, which pins the images and Git refs that the workspace uses. After the first `dagger check`, it pins the `rust` image too. Commit `dagger.lock` with `dagger.toml` and `.dagger/`. ## Write the Cargo checks in Dang The module runs `cargo fmt`, `cargo clippy`, and `cargo test` as checks. A [check](https://docs.dagger.io/reference/sdks/index#checks-generators-and-services) is a function that validates the project, such as a test run or a linter. `dagger check` runs every check. Replace the starter code in `.dagger/modules/-dev/main.dang` with the code below. `ProjectDev` stands for the type name that `dagger module init` generated, so keep your own type name. ```dang """ Build, lint and test the Rust workspace with Cargo's caches kept between runs. """ type ProjectDev { let source: Directory! """ Rust version, used for the image tag and the target cache name. """ pub rustVersion: String! new(ws: Workspace!, rustVersion: String! = "1.99") { self.source = ws.directory("/", exclude: ["target", ".git", ".dagger"]) self.rustVersion = rustVersion self } """ A Rust toolchain with the workspace mounted and Cargo's caches attached. """ pub rustEnv: Container! { container .from(`rust:${rustVersion}`) .withExec(["rustup", "component", "add", "clippy", "rustfmt"]) .withMountedCache("/cargo", cacheVolume("cargo-home")) .withEnvVariable("CARGO_HOME", "/cargo") .withMountedDirectory("/src", source) .withMountedCache("/src/target", cacheVolume(`cargo-target-${rustVersion}`)) .withWorkdir("/src") } """ Check formatting with rustfmt. """ pub fmt: Void @check { rustEnv.withExec(["cargo", "fmt", "--all", "--check"]).sync null } """ Lint every crate and target with Clippy, denying warnings. """ pub clippy: Void @check { rustEnv .withExec(["cargo", "clippy", "--workspace", "--all-targets", "--locked", "--", "-D", "warnings"]) .sync null } """ Build and run the workspace's tests. """ pub test: Void @check { rustEnv.withExec(["cargo", "test", "--workspace", "--locked"]).sync null } } ``` `rustEnv` mounts two cache volumes, and all checks share them. The sections below show how each volume meets the requirements. The `rustup component add` step installs Clippy and rustfmt, because the official `rust` image has only `cargo`, `rustc`, and the standard library. Dagger caches this step like any other step. ## Keep the downloaded crates in a Dagger cache volume The `cargo-home` volume keeps the downloaded crates. All checks and all Rust versions share it. `CARGO_HOME` points to the volume, not to `/usr/local/cargo` in the image. This is the directory that rust-cache keeps on GitHub Actions. Cargo's docs advise CI to cache only parts of Cargo's home. An archive of all of it holds the sources of each crate twice ([Cargo home](https://doc.rust-lang.org/cargo/guide/cargo-home.html)). A cache volume is not an archive and does not move between runs, so the module mounts all of Cargo's home. Cargo keeps the lock for its package cache in Cargo's home directory ([Cargo source](https://github.com/rust-lang/cargo/blob/master/src/util/cache_lock.rs)), so that lock is in the volume too. ## Keep the compiled dependencies in a Dagger cache volume The target volume, mounted at `/src/target`, keeps the compiled dependencies. It works like `CARGO_TARGET_DIR` on a long-lived runner. The directory stays where Cargo left it, and Cargo recompiles only the crates that changed. Unlike rust-cache, the module does not remove the workspace crates, so the volume also keeps the project's own compiled crates. The source excludes `target`, `.git`, and `.dagger`. Dagger does not upload your local build output, and changes to it do not invalidate Dagger's cache. The checks share one target directory and one Cargo home. `dagger check` [runs all checks in parallel](https://docs.dagger.io/using/checking), and Cargo's file locks in the two volumes make the checks take turns. When one check waits for another, Cargo prints `Blocking waiting for file lock on package cache` or `Blocking waiting for file lock on build directory`. A target volume that lives a long time grows, as a local `target/` directory does. Cargo cleans its global cache in Cargo's home directory automatically, but not build artifacts ([Cargo configuration](https://doc.rust-lang.org/cargo/reference/config.html#cache)). To start a target directory again on purpose, change the name of the volume. ## Follow changes to Cargo.lock and the toolchain with Dagger Cargo writes to the cache volumes on every run, so no cache key has to change when `Cargo.lock` changes. After a dependency update, Cargo finds the old artifacts in the target volume. It recompiles only the changed dependencies and the crates that depend on them ([Cargo source](https://github.com/rust-lang/cargo/blob/master/src/compiler/fingerprint/mod.rs)). rust-cache gets the same result when it restores the cache of the previous lockfile. The name of the target volume includes the Rust version. Cargo puts a hash that includes the compiler version in the file names of its artifacts ([Cargo source](https://github.com/rust-lang/cargo/blob/master/src/compiler/fingerprint/mod.rs)). Artifacts from two compiler versions thus stay side by side in `target/`. A new name gives a new toolchain an empty target directory, so old artifacts do not collect in one volume. rust-cache also starts a new cache when `rustc` changes. `rustVersion` is a constructor argument, so it is also a module setting. To change it, run this command: ```shell dagger module settings -dev rustVersion 1.98 ``` The command adds the setting to `dagger.toml`. In a project named `greeter`, it adds these lines: ```toml [modules.greeter-dev.settings] rustVersion = "1.98" ``` Every check then uses the new toolchain and a new target volume. ## Share the cache across CI machines and laptops with Dagger Cache volumes live on the Dagger Engine that runs the checks. On a laptop, the engine keeps them between runs, until its garbage collector removes cache to free disk space. By default, the engine prunes its cache when free disk space falls below 20% ([engine.json](https://docs.dagger.io/reference/config-files/engine-json)). The `gc` settings in `engine.json` change these limits. In CI, [Cloud Checks](https://docs.dagger.io/cloud-checks) run the checks on Cloud Engines, the remote engines that Dagger Cloud manages for you. Cloud Engines have scale-out and a distributed cache. Developers, agents, and CI pool one cache, and the `--engine=cloud` flag runs a pipeline from a laptop on Cloud Engines ([Cloud Engines](https://dagger.io/changelog/cloud-engines)). Dagger Cloud also removes stale cache volumes on Cloud Engines automatically ([changelog](https://dagger.io/changelog)). An engine on a laptop keeps its own cache volumes. The module is the same on every engine, and the cache volumes have the same names. One module thus meets the requirements on a laptop and in Dagger Cloud, with no cache configuration for each CI platform. ## Run the same cached checks locally and with Cloud Checks Run `dagger check` on your machine and in CI, and the same checks run in both places. List the checks, then run them: ```shell dagger check -l dagger check ``` In a project named `greeter`, `dagger check -l` lists the module's three checks and a staleness check from the Dang SDK: ```text MODULE CHECK DESCRIPTION greeter-dev clippy Lint every crate and target with Clippy, denying warnings. dang-sdk stale staleness check: re-generate modules and clients managed by the Dang SDK greeter-dev fmt Check formatting with rustfmt. greeter-dev test Build and run the workspace's tests. ``` This guide's `greeter` project has two crates, with `serde`, `serde_json`, and `anyhow` as dependencies. On an engine with an empty cache, the runs took these times: - The first `dagger check` took 28.7 seconds. It pulled the `rust` image, installed Clippy and rustfmt, downloaded the crates, and compiled them. - A rerun with no changes took 1.9 seconds. - After an edit to one file in the `greet` crate, `dagger check` took 1.8 seconds. Cargo compiled only `greet` and `cli`, because `cli` depends on `greet`. These times come from one run on one machine. They are an example, not a benchmark. In CI, [Cloud Checks](https://docs.dagger.io/cloud-checks) run the checks. Cloud Checks run your checks on Cloud Engines, Dagger's managed engines, after each push and pull request, with no CI workflow file. The repository must be on GitHub, with its `origin` remote set to it. Run this command from the project root: ```shell dagger cloud checks on ``` To see the runs, use `dagger workspace activity`. **Bottom line:** A Rust CI cache has to keep Cargo's downloaded crates and the compiled dependencies in `target/`. It has to follow changes to `Cargo.lock` and the toolchain, and reach every machine that builds. On GitHub Actions, Swatinem/rust-cache meets most of them in a few lines of YAML. sccache shares compiled crates across a fleet of machines, and cargo-chef fits Docker builds. Dagger meets the same requirements with cache volumes, and the same `dagger check` runs on a laptop and through Cloud Checks. ## Related guides - [Can Dagger lower CI costs?](/learn/reduce-ci-costs): how cached work and usage-based bills change what CI costs. - [How do I run Playwright tests in CI and locally in the same container, with cached browsers?](/learn/playwright-ci-docker-browser-caching): the same cache approach for Playwright's browsers. - [What are the alternatives to GitHub Actions, from faster runners to portable pipelines?](/learn/github-actions-alternatives): runner vendors, other CI platforms, and Dagger. --- # How do I enforce Prettier formatting in CI? URL: https://dagger.io/learn/prettier-dagger To enforce Prettier formatting in CI, install one exact Prettier version in the project. Run `prettier --check` in CI, so that unformatted files fail the job. Make sure that every run ignores the same files. Format with `prettier --write` before you push, with the same version and config. Teams usually meet these requirements with a CI job and a pre-commit hook. Dagger's official Prettier module meets the same requirements in one place, and `dagger check` runs it the same way on a laptop and in Cloud Checks. ## What an enforced Prettier setup needs Prettier's own docs give four requirements for formatting that CI enforces and every developer can match: 1. **Use one exact Prettier version and the project's config.** Prettier's [install page](https://prettier.io/docs/install) installs an exact version in the project: "This makes sure that everyone in the project gets the exact same version of Prettier." It warns that "even a patch release of Prettier can result in slightly different formatting". Prettier finds the config for each file ["starting from the location of the file being formatted"](https://prettier.io/docs/configuration), and then looks in the directories above it. 2. **Fail CI on unformatted files.** [`prettier --check`](https://prettier.io/docs/cli#--check) lists the files that are not formatted. It exits with code 1 when a file is not formatted, which the docs call "helpful inside the CI pipelines". 3. **Ignore the same files everywhere.** "By default, Prettier looks for `./.gitignore` and `./.prettierignore`" ([`--ignore-path`](https://prettier.io/docs/cli#--ignore-path)). It reads these files only in the directory where it runs. 4. **Write fixes with the same version and config.** [`prettier --write`](https://prettier.io/docs/cli#--write) "rewrites all processed files in place". The install page suggests a pre-commit hook too: "This makes sure all your commits are formatted, without having to wait for your CI build to finish." ## Meet them with prettier --check, a CI job, and a hook By hand, pin Prettier with an exact version in `devDependencies`, as `"prettier": "3.9.10"` with no `^`. Then run the project's Prettier with `npx`. This guide's project has a root `.prettierrc` and a second config in `packages/ui`. Check the formatting from the project root: ```shell npx prettier --check . ``` ```text Checking formatting... [warn] package.json [warn] src/config.js [warn] Code style issues found in 2 files. Run Prettier with --write to fix. ``` The command exited with code 1. Write the fixes: ```shell npx prettier --write . ``` ```text .prettierrc 7ms (unchanged) package-lock.json 2ms (unchanged) package.json 0ms packages/ui/.prettierrc 2ms (unchanged) packages/ui/src/button.js 3ms (unchanged) src/config.js 1ms ``` The ignore files depend on the directory where Prettier runs. The root `.gitignore` of this project ignores `packages/ui/dist`, so the root run did not check it. Run the check from `packages/ui`: ```shell cd packages/ui npx prettier --check . ``` ```text Checking formatting... [warn] dist/app.js [warn] Code style issues found in the above file. Run Prettier with --write to fix. ``` This run read no `.gitignore`, so it checked the build output in `dist`. To skip a directory in every run, add it to a `.prettierignore` in each directory where Prettier runs. In CI, a job installs Node.js, runs [`npm ci`](https://docs.npmjs.com/cli/commands/npm-ci) to install the exact versions in the lockfile, and runs `npx prettier --check .`. The exit code of Prettier fails the job. On GitHub Actions, [`actions/setup-node`](https://github.com/actions/setup-node) installs Node.js. GitLab CI and other systems run the same commands in a Node.js image ([GitLab CI/CD YAML reference](https://docs.gitlab.com/ci/yaml/)). For the pre-commit hook, Prettier's [install page](https://prettier.io/docs/install#git-hooks) sets up husky and lint-staged, which format the staged files. ## Where the hand-built setup strains The hand-built setup meets each requirement in several places, and each place must stay in step: - **Exact version.** A range such as `^3.9.0` lets two machines install different patch releases. The pin must stay exact in every `package.json` that installs Prettier. - **Ignored files.** A run from the root and a run from a package can check different files, as the run above shows. CI checks only what its own command and directory select. - **Fixes.** A pre-commit hook is optional. [`git commit --no-verify`](https://git-scm.com/docs/git-commit) skips it, so CI is the first place that some unformatted files show. - **CI syntax.** Each CI system has its own job syntax. A move to another CI system means a rewrite of the Prettier job. ## Meet them with the Dagger Prettier module The [Dagger Prettier module](https://docs.dagger.io/reference/modules/js/prettier) meets the four requirements with one definition that runs on a laptop and in CI. A Dagger module is a package of functions that build, test, or check a project. If you have not used Dagger before, do the [Quickstart](https://docs.dagger.io/getting-started/quickstart) first. It sets up `dagger.toml`, the workspace configuration that holds your installed modules and their settings. Install the official Prettier module from the project root: ```shell dagger module install dagger.io/js/prettier ``` ```text dagger.io/js/prettier ➡️ https://github.com/dagger/prettier#v0.2.0 Created workspace config in /work Installed module "prettier" in /work/dagger.toml ``` The command creates `dagger.toml` if the file does not exist, and adds the module to it. The command also writes `dagger.lock`, which pins the exact module version. Commit both files. The next four sections show how the module meets each requirement. ### Use one exact Prettier version and the project's config The check runs the Prettier that your project installs. The module does not download a different Prettier version, so the check uses the same version as your editor and `npx prettier`. If the project has a `package.json` without Prettier in it, the check fails with this message: ```text prettier is not installed: add it to the devDependencies of package.json (or of a package.json between it and the project) ``` The module finds projects by the name of the config file. These directories are projects: - A directory that holds `.prettierrc`, or `.prettierrc` with one of these extensions: `.json`, `.json5`, `.yaml`, `.yml`, `.toml`, `.js`, `.mjs`, `.cjs`, `.ts`, `.mts`, `.cts`. - A directory that holds `prettier.config` with one of these extensions: `.js`, `.mjs`, `.cjs`, `.ts`, `.mts`, `.cts`. A config in the `prettier` field of `package.json` does not make a directory a project ([Prettier module reference](https://docs.dagger.io/reference/modules/js/prettier)). If your config is in that field, move it to a `.prettierrc` file. A project covers its directory and every directory below it, except nested projects. Each nested project runs from its own directory, so its own config and `.prettierignore` apply to it ([module source](https://github.com/dagger/prettier)). Install scripts see only the dependency files, not the full source. So a `postinstall` script that needs source files fails ([module source](https://github.com/dagger/prettier)). If you have a script like that, stop the install scripts: ```shell dagger module settings prettier installFlags '["--ignore-scripts"]' ``` The setting goes into `dagger.toml`, so every developer and every CI run uses it. The [module reference](https://docs.dagger.io/reference/modules/js/prettier) covers the Node.js image, the package manager, and environment variables. ### Fail CI on unformatted files A check is a function that passes or fails. `dagger check` runs the checks ([Checking your code](https://docs.dagger.io/using/checking)). List them: ```shell dagger check -l ``` ```text MODULE PRETTIER-PROJECT CHECK DESCRIPTION prettier * format-check Check formatting of this project. # Use --all to expand collections and list each item. ``` The `*` stands for every Prettier project that the module found. To see each project, run `dagger list prettier-projects -a`. Each project's key is its directory, relative to the workspace root: ```text MODULE PRETTIER-PROJECT DESCRIPTION prettier . A Prettier project, keyed by its root. prettier packages/ui A Prettier project, keyed by its root. ``` `format-check` installs dependencies, then runs the project's Prettier with `--check` from the project directory: ```shell dagger check ``` In a project with two unformatted files, the check fails and names each file: ```text == CHECKS == ✘ 1 failed ✘ dag://?check=prettier/projects/format-check 16.5s ERROR ┃ [npm] ┃ [npm] added 1 package, and audited 2 packages in 721ms ┃ [npm] ┃ [npm] 1 package is looking for funding ┃ [npm] run `npm fund` for details ┃ [npm] ┃ [npm] found 0 vulnerabilities ┃ [dagger-prettier] [.] Checking formatting... ┃ [dagger-prettier] [.] [warn] package.json ┃ [dagger-prettier] [.] [warn] src/config.js ┃ [dagger-prettier] [.] [warn] Code style issues found in 2 files. Run Prettier with --write to fix. ┃ [dagger-prettier] [packages/ui] Checking formatting... ┃ [dagger-prettier] [packages/ui] All matched files use Prettier code style! ! Prettier failed in 1 project(s): - .: prettier --check failed (exit 1): [.] [warn] package.json [.] [warn] src/config.js [.] [warn] Code style issues found in 2 files. Run Prettier with --write to fix. ✘ exec.processRun 0.1s ERROR ! /usr/local/bin/crun did not terminate successfully: exit status 1 == RUN LOCALLY == dagger check "dag://?check=prettier/projects/format-check" ``` The `[warn]` lines are the same as the output of `npx prettier --check .`. Prettier's exit code 1 makes the check fail. In a monorepo, the projects run at the same time, and the failure message names every project that failed. To check one project, select it by its key: ```shell dagger check --prettier-project=packages/ui ``` The module installs dependencies again only when a dependency file changes, such as a `package.json` or a lockfile. In this guide's project, the first `dagger check` took 17.0 seconds, and a rerun with no changes took 0.6 seconds. After the guide added a new source file, the next run took 0.8 seconds and did not run npm install. ### Ignore the same files everywhere Before Prettier runs, the module removes the files that any `.gitignore` in the tree ignores ([module source](https://github.com/dagger/prettier)). Prettier also applies the project's `.prettierignore`. Thus, the check skips `packages/ui/dist` for the root project and for the `packages/ui` project, because the root `.gitignore` ignores it. A local `prettier --check .` in `packages/ui` still checks it, as the earlier run shows. That local run can fail when `dagger check` passes. To skip a directory in both, add it to `.prettierignore`. ### Write fixes with the same version and config `format` runs Prettier with `--write`, in the same container and with the same Prettier as the check. It returns the reformatted files as a changeset, a set of file changes that you can apply or reject. `format` is a function, not a check, so `dagger check` does not run it. It is not a generator either, so `dagger generate` does not run it. Call it on the project that contains a path: ```shell dagger api call -y prettier project --path=. format ``` The `-y` flag applies the changeset without a prompt. Without `-y`, the command asks before it writes files. The changes are then in your working tree. This part of the `git diff` output shows the change to `src/config.js`: ```text -const x = {a:1, -b:2} -export default x +const x = { a: 1, b: 2 }; +export default x; ``` To format a different project, give a path inside it, such as `--path=packages/ui`. After you apply the changes, run `dagger check` again: ```text == CHECKS == ✔ 1 passed ✔ dag://?check=prettier/projects/format-check 1.4s OK ``` ## Run the check locally and with Cloud Checks `dagger check` runs the same check on a laptop and in CI. [Cloud Checks](https://docs.dagger.io/cloud-checks) run your checks on Dagger's Cloud Engines after each push and pull request, with no CI workflow file. The repository must be on GitHub, and its Git `origin` remote must point to that repository. From the project root, run this command: ```shell dagger cloud checks on ``` In an interactive terminal, the command offers to complete the prerequisite steps first. These steps are to sign in to Dagger Cloud and to connect your GitHub account. When the command is done, commit the workspace files and push them: ```shell git add dagger.toml dagger.lock git commit -m "Check formatting with Prettier" git push ``` The push starts `format-check` on Dagger Cloud. To see the results, run `dagger workspace activity`. If the check fails, run `dagger api call -y prettier project --path=. format` locally. Then commit and push the fixes. If the project also lints with ESLint, Prettier's docs recommend [eslint-config-prettier](https://prettier.io/docs/install#eslint-and-other-linters), which "turns off all ESLint rules that are unnecessary or might conflict with Prettier". [How do I run ESLint the same way locally and in CI?](/learn/eslint-dagger) covers the ESLint check. Biome is one tool that formats and lints. [How do I run Biome lint and format checks in CI?](/learn/biome-dagger) says when it fits. **Bottom line:** Pin one exact Prettier version, and fail CI with `prettier --check`. Ignore the same files in every run, and write fixes with the same version and config. By hand, that means an exact pin, a CI job, a pre-commit hook, and ignore files in each directory where Prettier runs. The Dagger Prettier module meets these requirements in one place, and `dagger check` runs it the same way locally and in Cloud Checks. ## Related guides - [How do I run ESLint the same way locally and in CI?](/learn/eslint-dagger): lint the same project with its own ESLint, next to the format check. - [How do I run Biome lint and format checks in CI?](/learn/biome-dagger): one tool for format and lint rules, when it can replace Prettier and ESLint. - [How do I run Vitest in CI, with sharding and merged coverage?](/learn/run-vitest-with-dagger): run Vitest tests for each project, with the official Vitest module. --- # How do I run golangci-lint the same way locally and in CI? URL: https://dagger.io/learn/golangci-lint-dagger To run golangci-lint the same way on a laptop and in CI, pin one golangci-lint version and commit a v2 configuration file. Lint every Go module, give the linter code that compiles, and keep its cache between runs. By hand, teams install a pinned binary on each laptop and use golangci-lint-action or a similar step in CI. Dagger meets the same requirements in one place. The official golangci-lint module runs the linter as a check, and `dagger check` runs that check on a laptop and in Cloud Checks. ## What a good golangci-lint setup needs A golangci-lint setup that gives the same result on a laptop and in CI meets five requirements. Each one comes from the golangci-lint docs or from the `go` command: 1. **Pin the golangci-lint version.** The golangci-lint docs recommend ["installing a specific version of golangci-lint"](https://golangci-lint.run/docs/welcome/install/ci/) in CI. A new release can add or change linters, and then all builds fail at the same time. 2. **Commit a v2 configuration file.** golangci-lint looks for `.golangci.yml`, `.golangci.yaml`, `.golangci.toml`, or `.golangci.json`. If it finds none in the project, it [looks in your home directory](https://golangci-lint.run/docs/configuration/file/), which is different on each machine. For the `version` key of a v2 file, ["The only possible value is "2"."](https://golangci-lint.run/docs/configuration/file/) 3. **Lint every Go module.** The `./...` pattern does not match directories that hold their own `go.mod`. You reach those packages only when you [change into that module's directory](https://pkg.go.dev/cmd/go#hdr-Package_lists_and_patterns). 4. **Give the linter code that compiles.** The golangci-lint FAQ says that ["the code to analyze should compile"](https://golangci-lint.run/docs/welcome/faq/#why-do-you-have-typecheck-errors). For cgo, it says ["ensure all required system libraries are installed"](https://golangci-lint.run/docs/welcome/faq/#why-do-you-have-typecheck-errors). golangci-lint also [supports only the Go versions](https://golangci-lint.run/docs/welcome/faq/#which-go-versions-are-supported) up to the one that compiled it. 5. **Keep the cache between runs.** The first run ["caches type information. All subsequent runs will be faster"](https://golangci-lint.run/docs/welcome/faq/), in the words of the FAQ. The examples come from a repository with golangci-lint configuration files at the root, in `services/api`, and in `legacy`. Each of these three directories is a Go module, and a fourth Go module in `tools/migrate` has no configuration file of its own. ## Run golangci-lint by hand On a laptop, most teams install a pinned binary with the [install script](https://golangci-lint.run/docs/welcome/install/local/) and run `golangci-lint run`. The golangci-lint docs recommend the binary and say that `go install` and `tool` directive installs ["aren't guaranteed to work"](https://golangci-lint.run/docs/welcome/install/local/). Install the release that CI uses: ```shell curl -sSfL https://golangci-lint.run/install.sh | sh -s -- -b $(go env GOPATH)/bin v2.11.4 golangci-lint version ``` ```text golangci-lint has version 2.11.4 built with go1.26.1 from 8f3b0c7e on 2026-03-22T17:35:14Z ``` Go 1.26.1 compiled this binary, so it lints Go modules up to Go 1.26.1. Run it from the repository root: ```shell golangci-lint run ./... ``` ```text main.go:10:9: S1003: should use strings.Contains(s, "h") instead (staticcheck) return strings.Index(s, "h") != -1 ^ 1 issues: * staticcheck: 1 ``` The run reports only the root module. `services/api` has its own `go.mod`, so the root run does not lint it. Run golangci-lint in that directory too: ```shell cd services/api && golangci-lint run ./... ``` ```text api.go:11:9: Error return value of `f.Close` is not checked (errcheck) f.Close() ^ 1 issues: * errcheck: 1 ``` Many teams also lint before each commit. The golangci-lint repository [defines hooks](https://github.com/golangci/golangci-lint/blob/main/.pre-commit-hooks.yaml) for the [pre-commit](https://pre-commit.com/) framework. Its `golangci-lint` hook lints only the changes since `HEAD`, and `golangci-lint-full` lints all files. ## Run golangci-lint in a CI job In GitHub Actions, the golangci-lint docs recommend [golangci-lint-action](https://golangci-lint.run/docs/welcome/install/ci/). Its `version` input selects the release, and it saves and restores `~/.cache/golangci-lint` between runs ([golangci-lint-action](https://github.com/golangci/golangci-lint-action)). For a repository with several Go modules, its `working-directory` input is ["useful for monorepos"](https://github.com/golangci/golangci-lint-action). The README also shows a job matrix with one job for each module of a Go workspace. GitLab CI and Buildkite have a component and a plugin, which the [CI installation page](https://golangci-lint.run/docs/welcome/install/ci/) lists. If the code uses cgo, the job installs the C libraries before the lint step. ## Where the hand-built setup strains The steps above meet each requirement, but each one lives in more than one place: - **The version lives in several files.** The install command on each laptop, the action's `version` input, and the pre-commit `rev` must change together. - **Each Go module needs its own run.** In this repository, the root run did not report the problem in `services/api`. A new module needs a new directory in the matrix, or a loop in a script. - **The linter and the Go version must match.** When a `go.mod` moves to a newer Go release, each laptop and each CI job needs a newer golangci-lint binary. - **Each machine installs the C libraries.** Each laptop and each CI job installs the cgo dependencies of the project, and their versions can differ. - **The cache stays on one machine.** The action's cache is for CI only, and each laptop keeps a cache of its own. ## Meet the requirements with the golangci-lint module The Dagger golangci-lint module meets these requirements with one check. A Dagger module is a package of functions that build, test, or check a project. The Dagger team maintains a catalog of [official modules](https://docs.dagger.io/reference/modules) for common tools. The project must be a Git repository with a `dagger.toml` file from the [Quickstart](https://docs.dagger.io/getting-started/quickstart). Install the golangci-lint module from the project root: ```shell dagger module install dagger.io/go/golangci-lint ``` The command adds the module to `dagger.toml`: ```toml [modules.golangci-lint] source = "dagger.io/go/golangci-lint" ``` If you do not know which modules the project needs, run `dagger module recommend`. It suggests this module when it finds a golangci-lint configuration file: ```text ✔ go: found go.mod ┃ ┃ dagger module install dagger.io/go ┃ ┃ golangci-lint: found .golangci.yml ┃ ┃ dagger module install dagger.io/go/golangci-lint ┃ ┃ No modules were installed. Run the commands for the modules you select. ``` A check is a function that passes or fails, such as a test or a linter, and `dagger check` runs it. The module gives each lint project one check, `lint`. A lint project is a directory that holds a golangci-lint configuration file. List the check: ```shell dagger check -l ``` ```text MODULE GOLANGCI-LINT-PROJECT CHECK DESCRIPTION golangci-lint * lint Run golangci-lint in this project. # Use --all to expand collections and list each item. ``` The sections below meet each requirement with this check. ## Pin the golangci-lint version with Dagger The module pins golangci-lint `2.11.4` by digest by default ([golangci-lint module reference](https://docs.dagger.io/reference/modules/go/golangci-lint)). The `version` setting selects the release for every project. To record your choice in `dagger.toml`, set the release without the `v` prefix: ```shell dagger module settings golangci-lint version 2.11.4 ``` List the settings to confirm the value: ```shell dagger module settings golangci-lint ``` ```text MODULE KEY VALUE DESCRIPTION golangci-lint version 2.11.4 golangci-lint release, without the v… golangci-lint goVersion Go version for every module, overrid… golangci-lint base Base image for Go containers. Must i… golangci-lint includeExtraFiles golangci-lint lint Project roots to lint, e.g. ["**", "… ``` Dagger stores settings in `dagger.toml`, so everyone who runs the check uses the same release, on a laptop and in Cloud Checks: ```toml [modules.golangci-lint.settings] version = "2.11.4" ``` The version lives in one file. `dagger.lock` also pins the Git ref of the module and the images that the check uses. ## Commit a v2 configuration file with Dagger The module lints only the directories that hold a golangci-lint configuration file, and it uses the file that the repository has. If a repository has no configuration file, the module has nothing to lint. The smallest valid `.golangci.yml` has one line: ```yaml version: "2" ``` The module uses golangci-lint v2. If a v1 file has no `version` key, golangci-lint stops and the check fails with this error: ```text Error: can't load config: unsupported version of the configuration: "" See https://golangci-lint.run/docs/product/migration-guide for migration instructions ``` The `golangci-lint migrate` command converts a v1 file to v2, as the [golangci-lint migration guide](https://golangci-lint.run/docs/product/migration-guide/) describes. Until you migrate a project, the `lint` setting can skip it. The setting selects lint projects by their path from the repository root. Its default, `["**"]`, selects all of them. A `!` pattern excludes a project and all projects below it, and exclusions always win. Set it in `dagger.toml`: ```toml [modules.golangci-lint.settings] lint = ["**", "!legacy"] ``` An excluded project does not show in `dagger check -l --all`, and golangci-lint does not run for it. In this guide's run, the `legacy` project had a v1 configuration file that made the check fail. After the exclusion and fixes for the other problems, the check passed: ```text == CHECKS == ✔ 1 passed ✔ dag://?check=golangci-lint/projects/lint 1.2s OK ``` ## Lint every Go module with Dagger The module runs `golangci-lint run` one time for each Go module in a lint project. A project covers the Go modules below it, except where a nested configuration file starts a different project. In this repository, the root project also covers `tools/migrate`. You write no loop and no matrix. Add `--all` to list one row for each project: ```shell dagger check golangci-lint -l --all ``` ```text MODULE GOLANGCI-LINT-PROJECT CHECK DESCRIPTION golangci-lint . lint Run golangci-lint in this project. golangci-lint services/api lint Run golangci-lint in this project. ``` Lint every project, or lint one project: ```shell dagger check --golangci-lint dagger check --golangci-lint-project=services/api ``` Diagnostics show the path from the repository root and the linter name. The second command reports the `errcheck` problem in `services/api` that the root run by hand did not report: ```text == CHECKS == ✘ 1 failed ✘ dag://?check=lint&golangci-lint-project=services/api 0.6s ERROR ✘ GomodGoMod.withPackages( ┆ ctr: withWorkdir services/api ┆ command: ["golangci-lint", "run", "--path-prefix", "services/api/", "--output.tab.path=stderr", "--output.tab.print-linter-name=true", "--output.tab.colors=false", "--show-stats=false", "--max-issues-per-linter=0", "--max-same-issues=0"] ┆ under: "services/api" ┆ skip: [] ): Container! 0.1s ERROR ┃ services/api/api.go:11:9 errcheck Error return value of `f.Close` is not checked ! exit code: 1 == RUN LOCALLY == dagger check "dag://?check=lint&golangci-lint-project=services/api" ``` The command under `RUN LOCALLY` runs the same check again for the same project. The linters, rules, and exclusions stay in `.golangci.yml`. The module uses them as golangci-lint finds them. ## Give the linter code that compiles with Dagger The module lints each Go module with the Go toolchain that its `go.mod` asks for. It adds Go and C/C++ sources, embedded assets, and local `replace` dependencies inside the repository to the lint container. Most projects need nothing more. The settings below cover the other cases. **One Go version for every module.** To use one Go version for all Go modules, set `goVersion`: ```shell dagger module settings golangci-lint goVersion 1.26 ``` This setting is separate from the `version` setting of the `dagger.io/go` module, which applies to tests and generators. **Files the linter reads.** If a lint run fails because a file is not there, add a path pattern from the repository root to `includeExtraFiles`: ```toml [modules.golangci-lint.settings] includeExtraFiles = ["shared/config/**"] ``` **C dependencies for cgo.** golangci-lint must load each package before its linters run. A cgo package needs its C headers and tools in the lint container for that step. In this run, a cgo file needs `pkg-config` and zlib, and the default container does not have them: ```text ┃ main.go:1 typecheck : exec: "pkg-config": executable file not found in $PATH ``` The `base` setting takes a container that has a Go toolchain and the C dependencies. The module installs golangci-lint into that container. The container supplies its own Go toolchain, so the module ignores `goVersion` when `base` is set. ## Build a lint container for cgo in Dang Build the `base` container in a small module of your own. Write it in [Dang](https://docs.dagger.io/reference/sdks/dang), Dagger's native language for modules. Install the Dang SDK, then create the project's dev module: ```shell dagger module install dagger.io/sdk/dang dagger module init dang -y ``` `dagger module init` creates the module at `.dagger/modules/-dev` and records it in `dagger.toml`. Replace the starter code in `.dagger/modules/-dev/main.dang` with the function below. `ProjectDev` stands in for the type name that `dagger module init` generated. Keep the generated name: ```dang type ProjectDev { """ The container golangci-lint runs in: Go, a C compiler, and zlib for cgo. """ lintRuntime: Container! { container .from("golang:1.26-bookworm") .withExec(["apt-get", "update"]) .withExec(["apt-get", "install", "-y", "--no-install-recommends", "pkg-config", "zlib1g-dev"]) } } ``` Connect the function to the `base` setting. This is [module wiring](https://docs.dagger.io/config/module-wiring): a setting refers to a function on a different module, and Dagger gives the function's result to the setting. `dagger module init` made the dev module the workspace main module, so the function name is sufficient: ```shell dagger module settings golangci-lint base lint-runtime ``` Dagger stores the full address in `dagger.toml`: ```toml [modules.golangci-lint.settings] base = "dag://-dev/lint-runtime" ``` Run `dagger check` again. The Dang SDK adds its own `stale` check for the dev module: ```text == CHECKS == ✔ 2 passed ✔ dag://?check=dang-sdk/generate/stale 1.1s OK ✔ dag://?check=golangci-lint/projects/lint 2.0s OK ``` The C libraries now come from one function, on each laptop and in Cloud Checks. ## Keep the cache between runs with Dagger The module keeps the Go module, build, and golangci-lint caches between runs ([golangci-lint module reference](https://docs.dagger.io/reference/modules/go/golangci-lint)). It mounts them as [cache volumes](https://docs.dagger.io/reference/api/cache-volume), directories that persist across runs on the Dagger engine. You write no cache key. On a laptop, the cache lives in the local engine. In Cloud Checks, it lives on Dagger's Cloud Engines. [How do I share the Go build cache across CI runs and runners?](/learn/share-go-build-cache-ci) covers how Cloud Engines share their cache. ## Run the lint check locally and with Cloud Checks `dagger check` runs the same lint check on any machine that has the Dagger CLI. Commit the configuration files and the lock file: ```shell git add dagger.toml dagger.lock .golangci.yml git commit -m "Lint with golangci-lint through Dagger" ``` If you created a dev module, add `.dagger` to the commit too. Dagger writes `dagger.lock` when it runs. The file pins the Git ref of each module and the images that the workspace uses, so every run uses the same versions. To run the check in CI, use [Cloud Checks](https://docs.dagger.io/cloud-checks). Cloud Checks run your checks on Dagger's Cloud Engines for every push and pull request to a GitHub repository. You do not need a CI workflow file. The Git `origin` remote of the project must point to the GitHub repository. Enable Cloud Checks from the project root: ```shell dagger cloud checks on ``` In an interactive terminal, the command offers to complete the steps that are not done, such as a Dagger Cloud sign-in and the GitHub connection. `dagger workspace activity` shows the recent runs. If the lint check fails in Cloud Checks, run `dagger check` locally to get the same diagnostics. **Bottom line:** To run golangci-lint the same way locally and in CI, pin its version and commit a v2 configuration file. Lint every Go module with code that compiles, and keep the cache between runs. The `dagger.io/go/golangci-lint` module meets these requirements in one check, and `dagger check` runs it on a laptop and in Cloud Checks. ## Related guides - [How do I run Staticcheck the same way locally and in CI?](/learn/staticcheck-dagger): the same requirements for Staticcheck, which golangci-lint also includes as a linter. - [How do I run Go tests, lint, and go generate the same way locally and in CI?](/learn/daggerize-go-project): the `dagger.io/go` module for Go tests and generators, next to the linter. - [How do I share the Go build cache across CI runs and runners?](/learn/share-go-build-cache-ci): how the Go caches persist on laptops and Cloud Engines. --- # How do I run PSScriptAnalyzer on PowerShell scripts in CI? URL: https://dagger.io/learn/psscriptanalyzer-dagger To run PSScriptAnalyzer in CI, analyze every PowerShell script with one pinned PSScriptAnalyzer version, and fail the build on findings. Keep the rules in a settings file in the repository, and skip the scripts that the repository does not own. By hand, `Install-PSResource` and `Invoke-ScriptAnalyzer -Recurse -EnableExit` meet these requirements on a laptop and in a CI job. Dagger meets the same requirements in one place. Its official PSScriptAnalyzer module runs the same analysis in a container, on a laptop and in Cloud Checks. ## What a good PSScriptAnalyzer setup needs A PSScriptAnalyzer setup for CI has five requirements. Each one comes from Microsoft's PSScriptAnalyzer docs or releases: 1. **Analyze every script.** `Invoke-ScriptAnalyzer` analyzes `.ps1`, `.psm1`, and `.psd1` files. To include the files in subdirectories, use a wildcard or the `-Recurse` parameter ([`Invoke-ScriptAnalyzer`](https://learn.microsoft.com/en-us/powershell/module/psscriptanalyzer/invoke-scriptanalyzer)). 2. **Fail the build on findings.** The cmdlet returns diagnostics as objects. ["For usage in CI systems, the **EnableExit** exits the shell with an exit code equal to the number of error records"](https://learn.microsoft.com/en-us/powershell/module/psscriptanalyzer/invoke-scriptanalyzer). 3. **Pin the PSScriptAnalyzer version.** New releases add rules. For example, [release 1.25.0](https://github.com/PowerShell/PSScriptAnalyzer/releases/tag/1.25.0) added `AvoidReservedWordsAsFunctionNames` and `UseConsistentParameterSetName`. A new rule can fail a build with no change in the repository. 4. **Keep the rules in the repository.** A [`PSScriptAnalyzerSettings.psd1`](https://learn.microsoft.com/en-us/powershell/utility-modules/psscriptanalyzer/using-scriptanalyzer#implicit-search) file selects the rules. PSScriptAnalyzer finds it in the project root when you analyze that root. [`SuppressMessageAttribute`](https://learn.microsoft.com/en-us/powershell/utility-modules/psscriptanalyzer/using-scriptanalyzer#suppressing-rules) in a script turns off a rule for one function. 5. **Skip scripts you do not own.** Test fixtures and vendored modules stay out of the analysis. [`Invoke-ScriptAnalyzer`](https://learn.microsoft.com/en-us/powershell/module/psscriptanalyzer/invoke-scriptanalyzer) has no parameter that excludes a path, so the `-Path` value decides which scripts it analyzes. ## Run PSScriptAnalyzer by hand On a laptop, install a fixed version of PSScriptAnalyzer from the PowerShell Gallery. [`Install-PSResource`](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.psresourceget/install-psresource) takes a `-Version` parameter, and the value "can be an exact version". The same cmdlet reads a `-RequiredResourceFile`, a `.psd1` or `.json` file that lists the modules and versions to install. Commit that file, and each machine installs the same version. Then run `Invoke-ScriptAnalyzer -Path . -Recurse -EnableExit` from the project root. The command analyzes every script below the root and reads `PSScriptAnalyzerSettings.psd1` from the root. To skip a directory, run the command once for each directory that you own, with that directory as `-Path`. In CI, a job runs the same command. On GitHub Actions, a step with `shell: pwsh` runs PowerShell ([workflow syntax](https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax#jobsjob_idstepsshell)). GitHub's Ubuntu and Windows runner images include PowerShell and the PSScriptAnalyzer module. The [Ubuntu 24.04 image readme](https://github.com/actions/runner-images/blob/main/images/ubuntu/Ubuntu2404-Readme.md) lists both versions. To pin the version, the job installs it with `Install-PSResource` before the analysis. ## Where the hand-built setup strains The commands work, but each requirement lives in more than one place: - **Pin the PSScriptAnalyzer version.** The runner image changes its PSScriptAnalyzer version when the image updates. A laptop keeps the version that its developer installed. Only a version in the job's install step and a shared required-resource file keep them the same. - **Analyze every script** and **skip scripts you do not own.** The list of paths lives in the CI step and in each developer's command, so a new directory must go in both. - **Fail the build on findings.** `-EnableExit` must be on every command that CI runs, and on the command that each developer runs before a push. ## Analyze every script with the Dagger PSScriptAnalyzer module The official [PSScriptAnalyzer module](https://docs.dagger.io/reference/modules/dotnet/PsScriptAnalyzer) analyzes every PowerShell script in the project. A [Dagger module](https://docs.dagger.io/reference/modules/index) is a package of functions that build, test, or check a project. The project must be a Git repository with a Dagger workspace. The workspace is the `dagger.toml` file at the project root. If the project does not have this file, follow the [Quickstart](https://docs.dagger.io/getting-started/quickstart). Install the module from the project root: ```shell dagger module install dagger.io/dotnet/PsScriptAnalyzer ``` ```text Installed module "ps-analyzer" in /work/dagger.toml dagger.io/dotnet/PsScriptAnalyzer ➡️ https://github.com/dagger/PsScriptAnalyzer#main ``` The module's name in the workspace is `ps-analyzer`. The command adds these lines to `dagger.toml`: ```toml [modules.ps-analyzer] source = "dagger.io/dotnet/PsScriptAnalyzer" ``` The check copies the `.ps1`, `.psm1`, and `.psd1` files at or below the directory where `dagger` runs into a container ([module README](https://github.com/dagger/PsScriptAnalyzer)). The container runs Alpine Linux with PowerShell and PSScriptAnalyzer. The check runs the same command as by hand: ```powershell Invoke-ScriptAnalyzer -Path . -Recurse -EnableExit ``` If the analysis needs files that are not PowerShell files, such as `data/`, add them with the `includeExtraFiles` setting. ## Fail the build on findings with dagger check The module adds one [check](https://docs.dagger.io/using/checking), `ps-analyzer:check`. A check is a function that passes or fails, and `dagger check` runs it. List the checks in the workspace: ```shell dagger check -l ``` ```text MODULE CHECK DESCRIPTION ps-analyzer check Run PSScriptAnalyzer on all PowerShell scripts at or below the client's ``` Run every check in the workspace: ```shell dagger check ``` To run only the PSScriptAnalyzer check, give its address: ```shell dagger check dag://ps-analyzer/check ``` In this guide's runs, the exit code counted warnings too, so a script with only warnings also makes the check fail. In a test project, `tests/Helper.ps1` used `gci`, `%`, and `Write-Host`. This is the start of the output: ```text == CHECKS == ✘ 1 failed ✘ dag://?check=ps-analyzer/check 13.6s ERROR ✘ withExec pwsh -NoLogo -NoProfile -NonInteractive -Command '$ErrorActionPreference = '\''Stop'\''\nInvoke-ScriptAnalyzer -Path . -Recurse -EnableExit' 0.7s ERROR ┃ ┃ RuleName Severity ScriptName Line Message ┃ -------- -------- ---------- ---- ------- ┃ PSAvoidUsingWriteHost Warning Helper.ps1 1 File 'Helper. ``` Next, the output lists two `PSAvoidUsingCmdletAliases` warnings. It ends with `! exit code: 3`, one for each warning. When every script is clean, the check passes: ```text == CHECKS == ✔ 1 passed ✔ dag://?check=ps-analyzer/check 0.9s OK ``` ## Pin the PSScriptAnalyzer version with the version setting The `version` setting is the PSScriptAnalyzer release that the module installs. The default is `1.22.0`. Set the release that your team uses, so a new release cannot make the check fail without a change in your repository. List the current settings: ```shell dagger module settings ps-analyzer ``` ```text MODULE KEY VALUE DESCRIPTION ps-analyzer version PSScriptAnalyzer version to install ps-analyzer exclude PowerShell script paths to exclude from… ps-analyzer includeExtraFiles Extra non-PowerShell file paths to incl… ``` Set a different release: ```shell dagger module settings ps-analyzer version 1.24.0 ``` The command writes the setting to `dagger.toml`, so it applies to all users of the repository: ```toml [modules.ps-analyzer.settings] version = "1.24.0" ``` The module downloads the `PSScriptAnalyzer..nupkg` file from the [PSScriptAnalyzer GitHub releases](https://github.com/PowerShell/PSScriptAnalyzer/releases). Use a version that has a release on that page. Dagger also writes `dagger.lock`. In this guide's run, the file pinned the module to a Git commit and the `alpine` base image to a digest. Commit it with `dagger.toml`, so every machine runs the same module code on the same image. ## Keep the rules in the repository with PSScriptAnalyzerSettings.psd1 Select the rules in a PSScriptAnalyzer settings file, the same file as by hand. The Dagger module has no rule settings of its own. The check analyzes the project root, so put [`PSScriptAnalyzerSettings.psd1`](https://learn.microsoft.com/en-us/powershell/utility-modules/psscriptanalyzer/using-scriptanalyzer#implicit-search) there. This file removes one rule: ```powershell @{ ExcludeRules = @('PSAvoidUsingWriteHost') } ``` The check copies every `.psd1` file into its container, so it reads this file with no other configuration. In the test project, the `PSAvoidUsingWriteHost` warning stopped. The two alias warnings stayed: ```text ┃ RuleName Severity ScriptName Line Message ┃ PSAvoidUsingCmdletAliases Warning Helper.ps1 2 'gci' is an a ┃ PSAvoidUsingCmdletAliases Warning Helper.ps1 2 '%' is an ali ! exit code: 2 ``` To stop a warning for one function only, add `SuppressMessageAttribute` to that function in the script. Microsoft's [Suppressing Rules](https://learn.microsoft.com/en-us/powershell/utility-modules/psscriptanalyzer/using-scriptanalyzer#suppressing-rules) section shows how. ## Skip scripts you do not own with exclude The `exclude` setting is a list of script paths that the check does not analyze, such as test fixtures or vendored modules. Give the list as one JSON argument: ```shell dagger module settings ps-analyzer exclude '["tests/"]' ``` The command adds the list to the settings in `dagger.toml`: ```toml [modules.ps-analyzer.settings] version = "1.24.0" exclude = ["tests/"] ``` In the test project, this setting removed `tests/Helper.ps1` from the analysis, and the check passed. The paths live in `dagger.toml`, so a laptop and Cloud Checks skip the same scripts. ## Run the check locally and with Cloud Checks The `dagger check` command runs the check on your machine, in the Dagger engine. The engine runs the containers. It [caches each step by its inputs](https://docs.dagger.io/reference/sdks/dang), so a rerun skips the steps that did not change. In this guide's test project: - The first `dagger check` with new content in `PSScriptAnalyzerSettings.psd1` took 1.7 seconds. - A rerun with no changes took 0.6 seconds. - After another change to that file, the check took 1.7 seconds. - The first run after a change to `version` took 3.1 seconds. The engine downloaded the new PSScriptAnalyzer release and used the PowerShell install from its cache. These numbers come from one run on one machine. [Cloud Checks](https://docs.dagger.io/cloud-checks) runs the same checks on Dagger's Cloud Engines for every push and pull request. You do not write a CI workflow file. The repository must be on GitHub, and its `origin` remote must point to it. Enable Cloud Checks from the project root: ```shell dagger cloud checks on ``` If you are not signed in to Dagger Cloud, or GitHub is not connected, the command asks you to do these steps first. Commit the workspace files, then push: ```shell git add dagger.toml dagger.lock git commit -m "Run PSScriptAnalyzer with Dagger" git push -u origin HEAD ``` If you added `PSScriptAnalyzerSettings.psd1`, commit it too. Watch the result with `dagger workspace activity`. If the check fails in Cloud Checks, run `dagger check` locally. It runs the same analyzer version and rules on the same scripts. **Bottom line:** To run PSScriptAnalyzer in CI, analyze every script with a pinned version and `-EnableExit`. Keep the rules in `PSScriptAnalyzerSettings.psd1`, and skip the scripts you do not own. By hand, the version and the paths live in each CI job and on each laptop. The Dagger PSScriptAnalyzer module keeps them in `dagger.toml`, and Cloud Checks run the same `dagger check` on every push. ## Related guides - [How do I lint shell scripts with ShellCheck in CI?](/learn/shellcheck-dagger): the same requirements for shell scripts. - [Which development tools can Dagger replace or consolidate?](/learn/consolidate-ci-tools): how lint, test, and build tools fit in one workspace. - [How do I migrate from GitHub Actions to Dagger?](/learn/migrate-from-github-actions): how to move a lint workflow to Dagger checks. --- # How do I run Staticcheck the same way locally and in CI? URL: https://dagger.io/learn/staticcheck-dagger To run Staticcheck the same way on a laptop and in CI, pin one Staticcheck version and commit your `staticcheck.conf` files. Analyze every Go module with a Go version that the release supports, and keep the caches between runs. By hand, teams run `go install` with a pinned version on each laptop and use the Staticcheck GitHub Action in CI. Dagger meets the same requirements in one place. The official Staticcheck module runs Staticcheck as a check, and `dagger check` runs that check on a laptop and in Cloud Checks. ## What a good Staticcheck setup needs A Staticcheck setup that gives the same result on a laptop and in CI meets five requirements. Each one comes from the Staticcheck docs or from the `go` command: 1. **Pin the Staticcheck version.** The Staticcheck docs tell you to pin it, because ["new versions of Staticcheck introduce new checks that may break your build"](https://staticcheck.dev/docs/running-staticcheck/ci/github-actions/). 2. **Commit the staticcheck.conf files.** Staticcheck configuration files [contain TOML and apply to the subtree of packages below them](https://staticcheck.dev/docs/configuration/). When there are several, Staticcheck merges them, and a file deeper in the tree overrides the files above it. 3. **Analyze every Go module.** `staticcheck ./...` [checks all packages](https://staticcheck.dev/docs/getting-started/) that the pattern matches. The `./...` pattern does not match directories that hold their own `go.mod`. You reach those packages only when you [change into that module's directory](https://pkg.go.dev/cmd/go#hdr-Package_lists_and_patterns). 4. **Use a supported Go version.** ["A given Staticcheck release supports the last two versions of Go"](https://staticcheck.dev/docs/running-staticcheck/ci/github-actions/) at the time of the release. Staticcheck [targets the Go version](https://staticcheck.dev/docs/running-staticcheck/cli/) in the `go` directive of `go.mod`. 5. **Keep the caches between runs.** The Staticcheck GitHub Action ["saves and restores Go's build cache (in addition to Staticcheck's own cache) to speed up future runs"](https://staticcheck.dev/docs/running-staticcheck/ci/github-actions/). The examples come from a repository with one `staticcheck.conf` at the root and a second one in `tools/migrate`. The root file keeps the default checks and turns off the package comment check, ST1000: ```toml checks = ["inherit", "-ST1000"] ``` The `tools/migrate` directory is a separate Go module. Its `staticcheck.conf` also turns off S1003, with `checks = ["inherit", "-S1003"]`. ## Run Staticcheck by hand On a laptop, most teams install Staticcheck with `go install`. The [Staticcheck docs](https://staticcheck.dev/docs/getting-started/) say that you can use a specific version, such as `@2020.2.1`, in place of `@latest`. Install the release that CI uses: ```shell go install honnef.co/go/tools/cmd/staticcheck@2026.2.1 ``` ```text go: downloading honnef.co/go/tools v0.8.1 go: honnef.co/go/tools@v0.8.1 requires go >= 1.26.0; switching to go1.26.10 go: downloading go1.26.10 (darwin/arm64) go: downloading golang.org/x/tools v0.44.1-0.20260420230617-19499e7caabc go: downloading github.com/BurntSushi/toml v1.4.1-0.20240526193622-a339e1f7089c go: downloading golang.org/x/exp/typeparams v0.0.0-20231108232855-2478ac86f678 go: downloading golang.org/x/exp v0.0.0-20231110203233-9a3e6036ecaa go: downloading golang.org/x/sync v0.20.0 go: downloading golang.org/x/mod v0.35.0 ``` This release needs Go 1.26 or newer to build. The laptop had Go 1.24, so the `go` command downloaded a newer toolchain. Each Staticcheck release has two names: `2026.2.1` is also `v0.8.1`. Confirm the version: ```shell staticcheck -version ``` ```text staticcheck 2026.2.1 (0.8.1) ``` Run it from the repository root: ```shell staticcheck ./... ``` ```text main.go:10:9: should use strings.Contains(s, "h") instead (S1003) ``` The run covers only the root Go module. `go list` shows the packages that `./...` matches from the root: ```shell go list ./... ``` ```text example.com/lintdemo ``` To analyze `tools/migrate`, run `staticcheck ./...` again in that directory. There, its own `staticcheck.conf` turns off S1003, and Staticcheck reports nothing. ## Run Staticcheck in a CI job In GitHub Actions, the Staticcheck docs describe [staticcheck-action](https://staticcheck.dev/docs/running-staticcheck/ci/github-actions/). Its `version` input selects the release, and its default is `latest`. Its `install-go` input decides whether the action installs Go. The action also saves and restores the Go build cache and the Staticcheck cache. The docs list no other CI systems, so a job on another system runs the same `go install` and `staticcheck ./...` commands. A repository with several Go modules runs Staticcheck once in each module directory, often in a shell loop or a job matrix. ## Where the hand-built setup strains The steps above meet each requirement, but each one lives in more than one place: - **The version lives in several files.** The `go install` command on each laptop and the action's `version` input must change together. If the action has no `version`, it uses the latest release. - **Each Go module needs its own run.** In this repository, the root run did not analyze `tools/migrate`. A new module needs a new directory in the loop or the matrix. - **The release and the Go version must match.** A release supports two Go versions. When a `go.mod` moves to a newer Go, each laptop and each CI job needs a newer Staticcheck. - **The cache stays on one machine.** The action's cache is for CI only, and each laptop keeps a cache of its own. ## Meet the requirements with the Staticcheck module The Dagger Staticcheck module meets these requirements with one check. A [Dagger module](https://docs.dagger.io/reference/modules/go/staticcheck) is a package of functions that build and test a project. A check is a function that `dagger check` runs and reports as passed or failed. The project needs a Git repository and a Dagger workspace. Run `dagger init` from the [Quickstart](https://docs.dagger.io/getting-started/quickstart) to create `dagger.toml`. This file holds the workspace configuration for everyone who uses the repository. Install the module from the repository root: ```shell dagger module install dagger.io/go/staticcheck ``` The command adds the module to `dagger.toml` under the name `staticcheck`: ```toml [modules.staticcheck] source = "dagger.io/go/staticcheck" ``` The command also creates `dagger.lock`. This file pins the Git ref of the module. The first check run adds the `golang` images that the check uses, so every machine uses the same versions. Commit it with `dagger.toml`. When `dagger module recommend` finds a `staticcheck.conf`, it suggests the module: ```text ┃ staticcheck: found staticcheck.conf ┃ ┃ dagger module install dagger.io/go/staticcheck ┃ ┃ No modules were installed. Run the commands for the modules you select. ``` The module adds one check, `lint`. It runs once for each Staticcheck project. List it: ```shell dagger check -l ``` ```text MODULE STATICCHECK-PROJECT CHECK DESCRIPTION staticcheck * lint Run Staticcheck in this project. # Use --all to expand collections and list each item. ``` The sections below meet each requirement with this check. ## Pin the Staticcheck version with Dagger The `version` setting selects the Staticcheck release for every project, and its default is `v0.8.1` ([Staticcheck module reference](https://docs.dagger.io/reference/modules/go/staticcheck)). The module uses the `v0.x.y` name of a [release](https://github.com/dominikh/go-tools/releases). List the settings: ```shell dagger module settings staticcheck ``` ```text MODULE KEY VALUE DESCRIPTION staticcheck version Staticcheck release, e.g. "v0.8.1". staticcheck goVersion Go version for every module, overriding… staticcheck base Base image for Go containers. Must incl… staticcheck includeExtraFiles Extra workspace files to mount for Go c… staticcheck lint Project roots to lint, e.g. ["**", "!do… ``` Set the release: ```shell dagger module settings staticcheck version v0.8.2 ``` Dagger stores settings in `dagger.toml` under `[modules.staticcheck.settings]`. The pin applies to everyone who uses the repository, and to Cloud Checks. The version lives in one file. ## Commit the staticcheck.conf files with Dagger The module analyzes only the directories that hold a `staticcheck.conf`. A repository with no `staticcheck.conf` has nothing to analyze, so commit one at the repository root first. A Staticcheck project is a directory that holds a `staticcheck.conf`. Staticcheck [merges the configuration files](https://staticcheck.dev/docs/configuration/) itself, and the module keeps their directory structure. Thus `checks = ["inherit", "-S1003"]` in `tools/migrate/staticcheck.conf` has the same effect as with `staticcheck ./...` on your machine. To turn off one Staticcheck check for a directory, use `staticcheck.conf`. To turn off a check for one line, use a [`//lint:ignore` directive](https://staticcheck.dev/docs/configuration/). The `lint` setting of the module decides which projects run. If a directory has its own `staticcheck.conf` but the module must not analyze it yet, exclude it. The setting is a list, so edit it directly in `dagger.toml`: ```toml [modules.staticcheck.settings] version = "v0.8.2" lint = ["**", "!tools/migrate"] ``` Each pattern selects project roots by their path from the repository root. `"**"` matches every project. `!path` excludes that project and the projects below it, and an exclusion always wins. The excluded project does not show in the `dagger check -l --all` list, and Staticcheck does not analyze it. ## Analyze every Go module with Dagger A project covers each Go module below it. It also covers each package below it that a nested `staticcheck.conf` does not claim. You write no loop and no matrix. With `--all`, the list shows one row for each project: ```shell dagger check -l --all ``` ```text MODULE STATICCHECK-PROJECT CHECK DESCRIPTION staticcheck . lint Run Staticcheck in this project. staticcheck tools/migrate lint Run Staticcheck in this project. ``` The module finds the projects at or below the directory where `dagger` runs, plus the nearest project above it. To cover the whole repository, run `dagger` from the root. In this run, Staticcheck found the S1003 problem in the root module: ```shell dagger check --staticcheck ``` ```text == CHECKS == ✘ 1 failed ✘ dag://?check=staticcheck/projects/lint 8.3s ERROR ✘ withExec sh -c 'set -e\npattern=$1\nskips=$2\nshift 2\nIFS='\''\n'\''\nroot=$(pwd -P)\ndirs=$(go list -e -find -f '\''{{.Dir}}'\'' "$pattern")\npkgs=\nfor dir in $dirs; do\n keep=1\n for skip in $skips; do\n case "$dir/" in "$root/$skip"/*) keep= ;; esac\n done\n if [ -n "$keep" ]; then pkgs="$pkgs$dir$IFS"; fi\ndone\nif [ -z "$pkgs" ]; then echo "no packages under $pattern" >&2; exit 0; fi\nexec "$@" $pkgs' go-packages ./... tools/migrate staticcheck 0.6s ERROR ┃ main.go:10:9: should use strings.Contains(s, "h") instead (S1003) ! exit code: 1 == RUN LOCALLY == dagger check "dag://?check=staticcheck/projects/lint" ``` The report shows the diagnostic in Staticcheck's own format, with the check ID. The `RUN LOCALLY` command runs the failed selection again. After the fix, both projects pass: ```text == CHECKS == ✔ 1 passed ✔ dag://?check=staticcheck/projects/lint 0.8s OK ``` To analyze one project, give its directory: ```shell dagger check --staticcheck-project=tools/migrate ``` The module also analyzes test files. It does not run tests or generators. ## Use a supported Go version with Dagger The module analyzes each Go module in `golang:-alpine`, for the version in that module's `go.mod`. Staticcheck then targets that same version. If every Go module must use the same Go toolchain, set `goVersion`. If each `go.mod` already asks for the version you want, skip this setting. ```shell dagger module settings staticcheck goVersion 1.26 ``` This setting is separate from the `version` setting of the [Go module](https://docs.dagger.io/reference/modules/go). That setting applies to tests and generators. Each project runs on a copy of the repository that Dagger assembles. The copy holds the Go sources, embedded assets, the `staticcheck.conf` files, and local `replace` targets. Most projects need nothing more. Two settings cover the other cases: - **`includeExtraFiles`** adds files that the source scan does not find, as patterns from the repository root, such as `["shared-assets/**"]`. The scan follows `//go:embed`, but it does not follow other file reads. - **`base`** gives Staticcheck a container of your own, for example one with the C libraries that cgo packages need. The container must have a Go toolchain, and the module ignores `goVersion` when `base` is set. If the container does not have Staticcheck, the module installs it. A function in your own module can build this container through [module wiring](https://docs.dagger.io/config/module-wiring). The [golangci-lint guide](/learn/golangci-lint-dagger#build-a-lint-container-for-cgo-in-dang) shows this pattern for the same setting. ## Keep the caches between runs with Dagger The module keeps the Go module, build, and Staticcheck caches between runs ([Staticcheck module reference](https://docs.dagger.io/reference/modules/go/staticcheck)). It mounts them as [cache volumes](https://docs.dagger.io/reference/api/cache-volume), directories that persist across runs on the Dagger engine. You write no cache key. On a laptop, the caches live in the local engine. In Cloud Checks, they live on Dagger's Cloud Engines. [How do I share the Go build cache across CI runs and runners?](/learn/share-go-build-cache-ci) covers how Cloud Engines share their cache. ## Run it locally and with Cloud Checks On a laptop, `dagger check` runs the Staticcheck check with the other checks in the workspace. In CI, use Cloud Checks. [Cloud Checks](https://docs.dagger.io/cloud-checks) run your checks on Dagger's Cloud Engines after each push and pull request, with no CI workflow file. Commit the workspace files: ```shell git add dagger.toml dagger.lock staticcheck.conf git commit -m "Run Staticcheck with Dagger" ``` Connect the GitHub repository to Dagger Cloud, then push: ```shell dagger cloud checks on git push -u origin HEAD ``` The command gets the repository from the Git `origin` remote. In an interactive terminal, it offers to sign you in and to connect GitHub if you have not done that yet. Cloud Checks run the same `staticcheck/projects/lint` check, with the same pinned version and the same configuration. `dagger workspace activity` shows the recent runs. **Bottom line:** To run Staticcheck the same way locally and in CI, pin its version and commit your `staticcheck.conf` files. Analyze every Go module with a supported Go version, and keep the caches between runs. The `dagger.io/go/staticcheck` module meets these requirements in one check, and `dagger check` runs it on a laptop and in Cloud Checks. ## Related guides - [How do I run golangci-lint the same way locally and in CI?](/learn/golangci-lint-dagger): the same requirements for golangci-lint, which also runs the Staticcheck checks. - [How do I run Go tests, lint, and go generate the same way locally and in CI?](/learn/daggerize-go-project): the Go module for tests and `go generate`, next to the linter. - [How do I share the Go build cache across CI runs and runners?](/learn/share-go-build-cache-ci): how the Go caches persist on laptops and Cloud Engines. --- # How do I share the Go build cache across CI runs and runners? URL: https://dagger.io/learn/share-go-build-cache-ci To share the Go build cache across CI runs, keep two directories between jobs: `GOMODCACHE` for downloaded modules and `GOCACHE` for compiled packages and test results. Key them so that the build cache gets saved again on new commits, and store them where every runner can reach them. On GitHub Actions, `actions/setup-go` does part of this. On GitLab CI, `cache:` with a `go.sum` key and fallback keys does it. For machines that do not share disks, a `GOCACHEPROG` program can serve the build cache remotely. Dagger meets the same requirements in one place. Its official Go module mounts both caches as cache volumes, and Cloud Checks run it on Cloud Engines that share their cache. ## What a shared Go cache must do A Go cache setup for CI has four requirements. Each one comes from how the `go` command and the CI platforms work: 1. **Keep both caches.** The `go` command keeps downloaded modules in `GOMODCACHE` and build outputs in `GOCACHE`. It also ["caches successful package test results"](https://pkg.go.dev/cmd/go#hdr-Build_and_test_caching) in `GOCACHE`. If you keep only the module cache, every job compiles every package again and runs every test again. 2. **Keep the build cache current.** A build cache that CI saved once and never updated gets older with each commit. Each run then compiles more packages again. 3. **Stay correct.** The `go` command's own cache keys already account for "changes to Go source files, compilers, compiler options, and so on", and the cache is "safe for concurrent invocations of the go command" ([`go help cache`](https://pkg.go.dev/cmd/go#hdr-Build_and_test_caching)). The CI cache key only decides which snapshot a job gets. It does not make the build correct or incorrect. 4. **Reach every runner.** A cache on one machine's disk does not help a job that runs on another machine. ## Cache Go with actions/setup-go on GitHub Actions `actions/setup-go` caches Go modules and build outputs by default (`cache: true`). Since v6, it computes the cache key from `go.mod` in the repository root. To use `go.sum`, or to cover several Go modules, set `cache-dependency-path`, which accepts globs ([setup-go README](https://github.com/actions/setup-go)): ```yaml - uses: actions/setup-go@v7 with: go-version-file: go.mod cache-dependency-path: '**/go.sum' ``` This meets "keep both caches". By default, it does not keep the build cache current. The setup-go docs say that the `GOCACHE` cache ["is not updated based on source changes"](https://github.com/actions/setup-go/blob/main/docs/advanced-usage.md#cache-invalidation-on-source-changes). You can add `**/*.go` to `cache-dependency-path`, but the docs warn that this "can create new caches on many commits". When the key matches, the job restores the files and saves nothing new, because [you cannot change the contents of an existing cache](https://docs.github.com/en/actions/reference/workflows-and-actions/dependency-caching). To save a new build cache on each run, turn off the built-in cache. Then use `actions/cache` with a unique key and a `restore-keys` prefix. setup-go gives the two cache paths as [outputs](https://github.com/actions/setup-go/blob/main/docs/advanced-usage.md#go-environment-outputs): ```yaml - uses: actions/setup-go@v7 id: go with: go-version-file: go.mod cache: false - uses: actions/cache@v5 with: path: | ${{ steps.go.outputs.GOMODCACHE }} ${{ steps.go.outputs.GOCACHE }} key: go-${{ runner.os }}-${{ hashFiles('**/go.sum') }}-${{ github.sha }} restore-keys: | go-${{ runner.os }}-${{ hashFiles('**/go.sum') }}- go-${{ runner.os }}- ``` `restore-keys` restores the most recent cache with the same prefix ([cache key matching](https://docs.github.com/en/actions/reference/workflows-and-actions/dependency-caching#cache-key-matching)). Two rules decide which runs can share it: - **Seed the cache from the default branch.** A cache that a pull request saves is [scoped to its merge ref](https://docs.github.com/en/actions/reference/workflows-and-actions/dependency-caching#restrictions-for-accessing-a-cache), and other pull requests cannot restore it. Run the same job on `push` to `main`. - **Save from one job.** For parallel jobs, setup-go recommends that you ["create the cache in only one build and restore it in the other builds"](https://github.com/actions/setup-go/blob/main/docs/advanced-usage.md#parallel-builds). GitHub stores the caches, so every runner in the repository can restore them. ## Cache Go with cache keys on GitLab CI GitLab caches only paths inside the project directory. Caches ["can't link to files outside it"](https://docs.gitlab.com/ci/caching/). Move both Go caches into `$CI_PROJECT_DIR`, then give each cache its own key: ```yaml variables: GOMODCACHE: $CI_PROJECT_DIR/.go/pkg/mod GOCACHE: $CI_PROJECT_DIR/.cache/go-build test: image: golang:1.26 cache: - key: files: [go.sum] paths: [.go/pkg/mod] - key: go-build-$CI_COMMIT_REF_SLUG fallback_keys: [go-build-$CI_DEFAULT_BRANCH] paths: [.cache/go-build] script: - go test ./... ``` The module cache changes only when `go.sum` changes, so `cache:key:files` fits it. The build cache uses one key for each branch, and each job saves a new copy under that key. A new branch starts from the default branch's cache through `fallback_keys`, which takes [up to five keys](https://docs.gitlab.com/ci/caching/#per-cache-fallback-keys). Three settings decide who shares what: - **Policy.** The default `pull-push` downloads the cache and uploads it again after each job. Set [`policy: pull`](https://docs.gitlab.com/ci/caching/#inherit-global-configuration-but-override-specific-settings-per-job) on jobs that only read. - **Protected branches.** By default, ["protected and non-protected branches do not share the cache"](https://docs.gitlab.com/ci/caching/). GitLab adds a `-protected` or `-non_protected` suffix to each key, fallback keys included. - **Runners.** A cache lives where GitLab Runner is installed. To share it across several runners, [use distributed caching in S3](https://docs.gitlab.com/ci/caching/#good-caching-practices) or a shared network directory. ## Share the build cache remotely with GOCACHEPROG Since Go 1.24, the `GOCACHEPROG` environment variable names a program that [implements the build and test cache](https://go.dev/doc/go1.24#gocacheprog) for the `go` command. The `go` command sends `get` and `put` requests for single cache entries over a [JSON protocol](https://pkg.go.dev/cmd/go/internal/cacheprog). The program can store the entries anywhere, such as in remote storage. Depot's `depot gocache` [is one such program](https://depot.dev/blog/go-remote-cache). This meets "reach every runner" without archive files. Each job gets only the entries its build asks for, and it writes its new entries back, so the cache stays current. `GOCACHEPROG` replaces the build cache only. You still cache `GOMODCACHE` another way, and you run or buy the program and its storage. ## Where the hand-built setup strains The setups above work, but each requirement lives in a different place: - **The keys are a design you maintain.** A `go.sum` key stops build cache updates. A commit key needs `restore-keys` or `fallback_keys`, and one job that writes the cache. - **Every job transfers the whole archive.** GitHub and GitLab restore and save whole directories, so a larger build cache is a larger download and upload in each job. - **The limits are the platform's.** GitHub removes caches that no run used in 7 days and limits a repository to [10 GB by default](https://docs.github.com/en/actions/reference/workflows-and-actions/dependency-caching#usage-limits-and-eviction-policy). GitLab separates protected branches, and it needs S3 or a network directory for several runners. - **The CI caches stay in CI.** The setup-go and GitLab caches do not reach the laptops where developers run the same tests. ## Keep both caches with the Dagger Go module The [Dagger Go module](https://docs.dagger.io/reference/modules/go) runs `go test` and `go generate` for every Go module in the repository. A Dagger module is a package of functions that build and test a project. The Go module mounts `GOMODCACHE` and `GOCACHE` as [cache volumes](https://docs.dagger.io/reference/api/cache-volume), which are directories whose contents persist across runs on the Dagger engine. The engine is the process that runs the module's containers. You write no cache configuration. Complete the [Quickstart](https://docs.dagger.io/quickstart) in the repository first. Then install the module: ```shell dagger module install dagger.io/go ``` ```text dagger.io/go ➡️ https://github.com/dagger/go#v0.3.1 Installed module "go" in /work/dagger.toml ``` Run its checks. A check is a function that passes or fails, such as a test run: ```shell dagger check go ``` ```text == CHECKS == ✔ 2 passed ✔ dag://?check=go/packages/generate/stale 0.5s OK ✔ dag://?check=test&go-package=. 4.8s OK == TESTS == ✔ example.com/inventory 1 passed ✔ 1 passed ``` The module mounts the module cache at `/go/pkg/mod` and the build cache at `/root/.cache/go-build`, and it sets `GOMODCACHE` and `GOCACHE` to those paths. Every Go command that the module runs uses the same two volumes. This guide's project has one package that imports the etcd v3 client. The first `dagger check go` took 6.5 seconds, and its test check took 4.8 seconds. After a one-line change to the package source, `dagger check go` took 1.6 seconds, and the test check took 0.6 seconds. That is one run on one machine, not a benchmark. ## Keep the cache current and correct with Dagger The cache volumes have no key to design. Each run reads from the same volumes and writes its new entries back to them, so the build cache stays current on every commit. The `go` command's own cache keys keep the build correct, the same as on a CI runner. Concurrent checks share the volumes, which the Go cache allows. Dagger also caches each check's result by exactly the files the check saw ([Daggerize a Go Project](https://docs.dagger.io/getting-started/platform-guides/go)). In this guide's project, a rerun with no changes took 1.5 seconds for `dagger check go`, and the test check took 0.4 seconds. ## Share the cache across runners with Cloud Checks [Cloud Checks](https://docs.dagger.io/cloud-checks) run your checks on Cloud Engines on each push and pull request, with no CI workflow file. Cloud Engines are Dagger engines that Dagger Cloud runs for you, with a [distributed cache](https://dagger.io/changelog). Turn on Cloud Checks from the project root: ```shell dagger cloud checks on ``` The command needs a Dagger Cloud account and a connected GitHub repository, and it tells you how to complete what is missing. After that, each push runs the same `dagger check` that you run locally, with the same Go module and the same cache volumes. According to the [changelog](https://dagger.io/changelog), Cloud Engines share their cache through copies of a shared cache disk, and these copies are best effort. A version that shares single results is in the changelog's Coming soon section and is not released. To share one cache between developers and CI, run local commands on the same engines with `dagger check --engine=cloud`. To use your own machine instead, point every runner at one engine, for example with `--engine=ssh://build-host` ([remote execution](https://dagger.io/changelog)). | Requirement | setup-go | GitLab `cache:` | `GOCACHEPROG` | Dagger Go module | | --- | --- | --- | --- | --- | | Keep both caches | Yes, by default | Yes, after you move both into the project directory | Build cache only | Yes, two cache volumes | | Keep the build cache current | Only with a custom key or source globs | With a branch key and fallback keys | Yes, entry by entry | Yes, every run writes back | | Reach every runner | GitHub cache storage | S3 or a network directory for several runners | The program's storage | Cloud Engines, or one shared engine | **Bottom line:** Keep both `GOMODCACHE` and `GOCACHE`, and key the build cache so that it gets saved again on new commits. Store it where every runner can reach it: setup-go with a custom key on GitHub Actions, branch keys with fallbacks and S3 on GitLab CI, or `GOCACHEPROG` for remote entries. The Dagger Go module keeps both caches in cache volumes with no keys to maintain, and Cloud Checks run it on Cloud Engines that share their cache. ## Related guides - [How do I run Go tests, lint, and go generate the same way locally and in CI?](/learn/daggerize-go-project): set up the Go module and its checks for a whole project. - [How do I cache Rust builds in CI without rebuilding everything?](/learn/cache-rust-builds-in-ci): the same question for Cargo builds. - [Can Dagger lower CI costs?](/learn/reduce-ci-costs): how caching and Dagger Cloud pricing affect the CI bill. --- # How do I run Jest in CI and split the tests with --shard? URL: https://dagger.io/learn/jest-dagger A good Jest setup for CI runs the project's own Jest on a supported Node version. It runs Jest in CI mode, so that a new snapshot fails the run. It lets you rerun one test file, and it splits a large suite across machines with `jest --shard`. You can meet each requirement with the Jest CLI and a CI job. Dagger's official Jest module meets the same requirements in one place, and runs the same check on a laptop and in CI. ## What a good Jest CI setup needs Jest's own docs give four requirements. This guide uses them to judge each approach, Dagger included: 1. **Use the project's own Jest on a supported Node version.** The Jest docs install Jest as a dev dependency with `npm install --save-dev jest` ([Getting Started](https://jestjs.io/docs/getting-started)). Jest 30 says: "The minimum supported Node versions are now 18.x" ([Upgrading to Jest 30](https://jestjs.io/docs/upgrading-to-jest30)). 2. **Run Jest in CI mode, so that new snapshots fail.** With [`--ci`](https://jestjs.io/docs/cli#--ci), a new snapshot does not pass. Jest "will fail the test and require Jest to be run with `--updateSnapshot`." The snapshot docs give the reason: ["since new snapshots automatically pass, they should not pass a test run on a CI system"](https://jestjs.io/docs/snapshot-testing#are-snapshots-written-automatically-on-continuous-integration-ci-systems). 3. **Rerun one test file.** `jest path/to/my-test.js` runs only the tests in files that match the argument ([Jest CLI](https://jestjs.io/docs/cli#running-from-the-command-line)). A developer can then repeat one CI failure without the full suite. 4. **Split a large suite across machines.** Jest's [`--shard`](https://jestjs.io/docs/cli#--shard) option runs one part of the suite on each machine. ## Run Jest by hand in a CI job The Jest CLI meets each requirement. Install `jest` as a dev dependency, then run the suite in CI mode from the project root: ```shell npx jest --ci ``` You can usually leave out `--ci` in CI. Jest sets its `ci` option from the environment by default ([Jest source](https://github.com/jestjs/jest/blob/main/packages/jest-config/src/Defaults.ts)). GitHub Actions sets `CI` to `true` in every job ([variables reference](https://docs.github.com/en/actions/reference/workflows-and-actions/variables#default-environment-variables)). To add a snapshot, run `npx jest -u` on your machine and commit the snapshot file. To rerun one file, give its path: `npx jest src/cart.test.js`. In CI, a job checks out the code and installs Node with [`actions/setup-node`](https://github.com/actions/setup-node) or the equivalent of the CI system. Then it installs the dependencies and runs Jest. To split the suite, run one shard on each machine. Both numbers start at 1. This guide's project has three test files, split into two shards: ```shell npx jest --ci --shard=1/2 npx jest --ci --shard=2/2 ``` ```text $ npx jest --ci --shard=1/2 PASS src/sum.test.js PASS src/receipt.test.js Test Suites: 2 passed, 2 total Tests: 2 passed, 2 total Snapshots: 1 passed, 1 total Time: 0.42 s Ran all test suites. $ npx jest --ci --shard=2/2 PASS src/cart.test.js ✓ total of an empty cart is 0 (3 ms) ✓ total multiplies price and quantity Test Suites: 1 passed, 1 total Tests: 2 passed, 2 total Snapshots: 0 total Time: 0.206 s Ran all test suites. ``` In GitHub Actions, a [matrix](https://docs.github.com/en/actions/how-tos/write-workflows/choose-what-workflows-do/run-job-variations) runs one job for each shard index. The Jest docs say: "When `shard` is specified the configured `testSequencer` has to implement a `shard` method." The [default sequencer](https://github.com/jestjs/jest/blob/main/packages/jest-test-sequencer/src/index.ts) sorts test files by a SHA-1 hash of their paths. Then it gives each shard about the same number of files. It does not use test durations, so one shard can get more slow files than the others. To balance shards in a different way, set [`testSequencer`](https://jestjs.io/docs/configuration) to a class that extends `@jest/test-sequencer`. Its `shard(tests, { shardIndex, shardCount })` method returns the files for one shard. The Jest docs say that sharding happens before sorting. ## Where the hand-built setup strains The commands are short, but each requirement lives in a different place: - **The Node version is in two places.** The CI job sets `node-version`, and each laptop has its own Node. They can drift apart, and Jest then runs on a Node version that the project did not test. - **The shards are a fixed count of files.** The matrix fixes the shard count, and the default sequencer splits by file count, not by duration. A balanced split needs a custom sequencer that you write and maintain. - **The matrix is CI plumbing.** A move to another CI system means a rewrite of the shard jobs in the syntax of that system. - **A CI failure is hard to repeat.** To rerun a failed file as CI ran it, a developer must match the Node version and the install of the CI job by hand. ## Meet them with the Dagger Jest module The [Dagger Jest module](https://docs.dagger.io/reference/modules/js/jest) meets the same four requirements, with the Node image, the install, and the Jest command defined in one place. A Dagger [module](https://docs.dagger.io/reference/modules) is a package of functions that test, lint, or build a project. The [Dagger Engine](https://docs.dagger.io/getting-started/introduction) runs those functions in containers, on a laptop or in CI. Complete the [Quickstart](https://docs.dagger.io/getting-started/quickstart) in the repository first. Then install the module from the project root: ```shell dagger module install dagger.io/js/jest ``` ```text dagger.io/js/jest ➡️ https://github.com/dagger/jest#v0.3.0 Installed module "jest" in /work/dagger.toml ``` The command adds the module to `dagger.toml`, the workspace configuration for everyone who uses the repository: ```toml [modules.jest] source = "dagger.io/js/jest" ``` The command also writes `dagger.lock`, which pins the module to an exact Git commit. Commit both files. The module treats each directory with a `jest.config.js`, `.mjs`, `.cjs`, `.ts`, `.mts`, `.cts`, or `.json` file as a Jest project. A `jest` key in `package.json` does not mark a project. The module's [README](https://github.com/dagger/jest) gives the reason: every npm package has a `package.json`. A check is a function that passes or fails, such as a test run. The Jest module has one check, `test`, and runs it once for each Jest project. Add `--all` to the list command to see one row for each test file: ```shell dagger check -l --all ``` ```text MODULE JEST-PROJECT JEST-TEST-FILE CHECK DESCRIPTION jest web src/cart.test.js test Run Jest tests. jest web src/receipt.test.js test Run Jest tests. jest web src/sum.test.js test Run Jest tests. ``` The next sections show how the module meets each requirement. ### Use the project's own Jest on a supported Node version The module installs the project's dependencies and runs the project's own Jest. Run the tests: ```shell dagger check ``` ```text == CHECKS == ✔ 1 passed ✔ dag://?check=test&jest-project=web 10.2s OK == TESTS == ✔ src/cart.test.js 2 passed ✔ src/receipt.test.js 1 passed ✔ src/sum.test.js 1 passed ✔ 4 passed ``` When all the files of a project are selected, the module runs `jest` with the project's config. Jest's own `testMatch` and other options then decide what runs. The Jest config does not change. The module installs dependencies in a separate step. That step reads only the package files and lockfiles. A change to a source file or test file then does not start a new install ([README](https://github.com/dagger/jest)). In this guide's project, the first `dagger check` took 67.7 seconds, with the `npm install`. The output above comes from a later run, with the install already in the cache. A rerun with no changes took 8.8 seconds. A rerun after one new test in a test file took 10.9 seconds. These numbers come from one run on one machine. The tests run in `node:25-alpine` by default. If the project uses a different Node release, set the `baseImageAddress` setting to that release: ```shell dagger module settings jest baseImageAddress node:22-alpine ``` The setting goes into `dagger.toml`, so every machine uses the same Node image: ```toml [modules.jest] source = "dagger.io/js/jest" [modules.jest.settings] baseImageAddress = "node:22-alpine" ``` A setting applies to every Jest project in the workspace. The [module reference](https://docs.dagger.io/reference/modules/js/jest) lists the install settings, such as `packageManager` and `installFlags`. ### Run Jest in CI mode, so that new snapshots fail The module sets `CI` to `true` in its container ([module source](https://github.com/dagger/jest)), so Jest runs in CI mode with no flag. A new snapshot then fails the check on a laptop too. In this run, `src/receipt.test.js` had a snapshot assertion and no snapshot file: ```text ✘ dag://?check=test&jest-project=web 4.2s ERROR == TESTS == ✘ src/receipt.test.js FAIL ✘ src/receipt.test.js › receipt FAIL Error: expect(received).toMatchSnapshot() Snapshot name: `receipt 1` New snapshot was not written. The update flag must be explicitly passed to write a new snapshot. This is likely because this test is run in a continuous integration (CI) environment in which snapshots are not written by default. ``` Write the snapshot with `npx jest -u` on your machine, review it, and commit it, as the Jest docs recommend. ### Rerun one test file Select a project with `--jest-project` and its directory, relative to the workspace root. Select a test file with `--jest-test-file` and its path, relative to the project. Repeat `--jest-test-file` to select more files: ```shell dagger check --jest-project=web --jest-test-file=src/cart.test.js --jest-test-file=src/sum.test.js ``` ```text == CHECKS == ✔ 1 passed ✔ dag://?check=test&jest-project=web&jest-test-file=src/cart.test.js&jest-test-file=src/sum.test.js 1.1s OK == TESTS == ✔ src/cart.test.js 2 passed ✔ src/sum.test.js 1 passed ✔ 3 passed ``` When you select only some files, the module runs `jest --passWithNoTests --runTestsByPath` with those files. To list the keys that you can select, run `dagger list jest-projects -a` or `dagger list jest-test-files -a`. The module finds test files without a run of Jest. It reads `testMatch`, `testRegex`, `testPathIgnorePatterns`, `roots`, and `rootDir` when they are literal values. For other values, it uses Jest's defaults. It skips `node_modules` and the files that `.gitignore` excludes. To see the files that Jest itself finds, call the project's `list` function: ```shell dagger -c 'jest | projects | get web | list' ``` ```text /src/web/src/cart.test.js /src/web/src/receipt.test.js /src/web/src/sum.test.js ``` The directory that you run `dagger` from also selects projects. Dagger selects the projects at or below it, and the nearest project that contains it. When a test fails, the report gives a [DAG link](https://docs.dagger.io/using/checking#select-collection-items). This address selects the same files again, in the same container, on any machine: ```text ✘ 2 failed ✔ 1 passed == RUN LOCALLY == dagger check "dag://?check=test&jest-project=web&jest-test-file=src/cart.test.js&jest-test-file=src/sum.test.js" ``` ### Split a large suite across machines The module runs the selected files of a project in one Jest process. For a large suite, Dagger splits the tests across engines instead. It balances the parts with the measured resource use, input cost, and cache state of each test ([test splitting](https://dagger.io/changelog/test-splitting)). You do not fix a shard count or write a custom sequencer. Do not put `--shard` in the `flags` setting. The module passes `flags` to every `jest` run in the workspace, so every check runs only one shard. In a project with three test files and `flags` set to `["--shard=1/2"]`, the check ran two of the files: ```shell dagger module settings jest flags '["--shard=1/2"]' dagger check ``` ```text == CHECKS == ✔ 1 passed ✔ dag://?check=test&jest-project=web 2.5s OK == TESTS == ✔ src/receipt.test.js 1 passed ✔ src/sum.test.js 1 passed ✔ 2 passed ``` To remove the setting, run `dagger module settings -u jest flags`. ## Run the tests locally and with Cloud Checks [Cloud Checks](https://docs.dagger.io/cloud-checks) run your checks on Dagger's Cloud Engines for each push and pull request, with no CI workflow file. Cloud Engines are Dagger Engines that Dagger Cloud runs for you. The repository must be on GitHub, and its Git `origin` remote must point to it. Turn on Cloud Checks from the project root: ```shell dagger cloud checks on ``` If you do not have a Dagger Cloud account, the command tells you to run `dagger cloud signup` first. Commit `dagger.toml` and `dagger.lock`, then push. Each push then runs the same check, with the same Node image and the same CI mode. The results show as GitHub check statuses and in Dagger Cloud ([changelog](https://dagger.io/changelog/cloud-checks)). The module reports each test as its own result, so Dagger Cloud shows each test. To see recent runs from the terminal, run `dagger workspace activity`. If a check fails, run `dagger check` locally to see the same failure. **Bottom line:** Run the project's own Jest on a supported Node version with `--ci`, so new snapshots fail. Split a large suite with `--shard`, and write a custom `testSequencer` for a split by anything other than file count. Dagger's Jest module meets the same requirements in one place. It pins the Node image in `dagger.toml`, runs in CI mode, and selects single files with `--jest-test-file`. It runs the same check locally and with Cloud Checks. ## Related guides - [How do I run Vitest in CI, with sharding and merged coverage?](/learn/run-vitest-with-dagger): the same requirements for a project that tests with Vitest. - [How do I run Playwright tests in CI and locally in the same container, with cached browsers?](/learn/playwright-ci-docker-browser-caching): browser tests next to Jest unit tests. - [How do I run ESLint the same way locally and in CI?](/learn/eslint-dagger): lint the same project with its own ESLint. --- # How do I run Biome lint and format checks in CI? URL: https://dagger.io/learn/biome-dagger To run Biome lint and format checks in CI, pin the exact Biome version in `package.json`. Run Biome's formatter, linter, and assist actions as one check that fails the CI job. Give each project in a monorepo its root configuration, and apply safe fixes with the same Biome. Teams usually meet these requirements with `biome ci` in a CI job and `biome check --write` in a Git hook. Dagger's official Biome module meets the same requirements in one place, and `dagger check` runs it the same way on a laptop and in Cloud Checks. ## What a reliable Biome CI setup needs Biome's own docs give four requirements for Biome results that are the same on a laptop and in CI: 1. **Pin the exact Biome version.** On Biome's [install page](https://biomejs.dev/guides/manual-installation/), the `-E` flag "pins the exact version in your `package.json`". The page says that this is "so every contributor and your CI run the same version of Biome." The [versioning page](https://biomejs.dev/internals/versioning/) says why: "Even patch releases can result in slightly different formatting or linting results." 2. **Run the formatter, linter, and assist as one check that fails CI.** Biome has two commands that run all checks, `biome check` and `biome ci`. Biome's [CI recipe](https://biomejs.dev/recipes/continuous-integration/) says that `biome ci` "should be used in CI". It has no `--write` option, and on GitHub it prints diagnostics as GitHub annotations. 3. **Use the root configuration for each project.** Since v2, Biome supports monorepos with one root configuration ([big projects](https://biomejs.dev/guides/big-projects/)). A nested configuration sets `"root": false`, or uses `"extends": "//"` to extend from the root configuration. 4. **Apply safe fixes with the same Biome.** `biome check --write` applies formatting and safe fixes. Biome applies unsafe fixes only with [`--unsafe`](https://biomejs.dev/reference/cli/). Biome's [Git hooks recipe](https://biomejs.dev/recipes/git-hooks/) runs `biome check --write --staged` before each commit. ## When Biome fits instead of ESLint and Prettier Biome formats and lints with one tool and one configuration. Biome's [migration guide](https://biomejs.dev/guides/migrate-eslint-prettier/) ports ESLint and Prettier configurations with `biome migrate eslint` and `biome migrate prettier`. The guide also says that "some rule options are not implemented and some rules deliberately differ". The migration cannot load some plugins and shared configurations, and you configure their rules by hand in `biome.json`. If the project depends on ESLint rules or plugins that Biome does not have, keep ESLint and Prettier. [How do I run ESLint the same way locally and in CI?](/learn/eslint-dagger) and [How do I enforce Prettier formatting in CI?](/learn/prettier-dagger) cover those tools. ## Meet them with biome ci, a CI job, and a hook By hand, install Biome with the exact version, as Biome's install page shows: `npm i -D -E @biomejs/biome`. This guide's project then has `"@biomejs/biome": "2.6.2"` in `devDependencies`, with no `^`. Its `src/login.ts` has a lint error and a line that Biome formats differently. Biome reads nested configurations itself, so one run from the root covers each project in a monorepo ([big projects](https://biomejs.dev/guides/big-projects/)). Run the CI command from the project root: ```shell npx biome ci ``` The run reported the `lint/suspicious/noDoubleEquals` rule and a format difference in `src/login.ts`. It ended with this summary and exited with code 1: ```text Checked 3 files in 29ms. No fixes applied. Found 2 errors. ci ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ✖ Some errors were emitted while running checks. ``` Apply the safe fixes: ```shell npx biome check --write ``` ```text Checked 3 files in 2ms. Fixed 1 file. Found 1 error. ``` Biome formatted `src/login.ts`. The `==` lint error remains, because its fix is unsafe. In CI, Biome provides a first-party GitHub Action, [`biomejs/setup-biome`](https://github.com/biomejs/setup-biome). When you give it no `version`, it reads the version of `@biomejs/biome` from the lockfile, and then from `package.json`. The job then runs `biome ci`. Biome's CI recipe also shows a GitLab CI job that runs `biome ci` in the `ghcr.io/biomejs/biome:latest` image. For the hook, the Git hooks recipe shows Lefthook, Husky with lint-staged, pre-commit, and a shell script. ## Where the hand-built setup strains The hand-built setup meets each requirement in more than one place, and each place must stay in step: - **Exact version.** On GitHub Actions, `setup-biome` reads the pinned version from the lockfile. A CI job that uses the Biome image needs an image tag that matches `package.json`. The `latest` tag in the recipe follows the newest release instead. With a pinned tag, each upgrade changes two files. - **Safe fixes.** A Git hook is optional. [`git commit --no-verify`](https://git-scm.com/docs/git-commit) skips it, so CI is the first place that some problems show. - **CI syntax.** Each CI system has its own job syntax: an action on GitHub, an image and a script on GitLab CI. A move to another CI system means a rewrite of the Biome job. ## Meet them with the Dagger Biome module The [Dagger Biome module](https://docs.dagger.io/reference/modules/js/biome) meets the four requirements with one definition that runs on a laptop and in CI. A Dagger module is a package of functions that build, test, or check a project. The Dagger team keeps a catalog of [official modules](https://docs.dagger.io/reference/modules) for common tools. Complete the [Quickstart](https://docs.dagger.io/getting-started/quickstart) first, which creates `dagger.toml`. Install the Biome module from the project root: ```shell dagger module install dagger.io/js/biome ``` ```text dagger.io/js/biome ➡️ https://github.com/dagger/biomejs#v0.1.0 Installed module "biomejs" in /work/dagger.toml ``` The module installs under the name `biomejs`. Its check, its flags, and its settings all use that name. The command adds the module to `dagger.toml`: ```toml [modules.biomejs] source = "dagger.io/js/biome" ``` The project does not need a Dagger module of its own. The Biome module reads what it needs from `biome.json`, `package.json`, and the lockfile. The next four sections show how it meets each requirement. ### Pin the exact Biome version The module runs the Biome version that the project installs, so Dagger and your editor use the same release. If a `package.json` exists but does not list `@biomejs/biome`, the check fails with this message: ```text biome is not installed: add @biomejs/biome to the devDependencies of package.json (or of a package.json between it and the project) ``` If the project has no `package.json`, the [module source](https://github.com/dagger/biomejs) runs Biome through `npx` with no version. Add the dependency to pin the Biome version. The module installs dependencies in a separate step. That step reads only the package manifests, lockfiles, and package manager configuration, as the [module source](https://github.com/dagger/biomejs) shows. A change to a source file does not run the install again. In this guide's single-project repository, the first `dagger check` took 13.8 seconds. A rerun with no changes took 1.0 seconds. Both times include the CLI connection. These numbers come from one machine, so use them only as an example. The [module reference](https://docs.dagger.io/reference/modules/js/biome) covers the install root in a monorepo and the settings for the Node.js image, the package manager, install flags, and environment variables. ### Run the formatter, linter, and assist as one check that fails CI A check is a function that passes or fails, such as a test or a linter. `dagger check` runs the checks. The module gives each Biome project one check, `lint`. List the check: ```shell dagger check -l ``` ```text MODULE BIOMEJS-BIOME-PROJECT CHECK DESCRIPTION biomejs * lint Lint this project. # Use --all to expand collections and list each item. ``` The check has the name `lint`, but it runs `biome check`. So it also fails on files that Biome would format differently. In this excerpt from a run, Biome found a lint error in `src/login.ts`: ```text == CHECKS == ✘ 1 failed ✘ dag://?check=biomejs/projects/lint 12.9s ERROR ┃ [/opt/dagger-biomejs/biome] src/login.ts:2:12 lint/suspicious/noDoubleEquals FIXABLE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ┃ [/opt/dagger-biomejs/biome] ┃ [/opt/dagger-biomejs/biome] × Using == may be unsafe if you are relying on type coercion. ┃ [/opt/dagger-biomejs/biome] Checked 3 files in 3ms. No fixes applied. ┃ [/opt/dagger-biomejs/biome] Found 2 errors. == RUN LOCALLY == dagger check "dag://?check=biomejs/projects/lint" ``` The command under `RUN LOCALLY` runs the same check again. The rules, formatter options, and ignored files stay in `biome.json`, and Biome reads them from there. The module runs `biome check`, not `biome ci`. Both commands apply the same formatter, linter, and assist actions. The module does not print GitHub annotations, which `biome ci` adds on GitHub. ### Use the root configuration for each project A Biome project is a directory with a root `biome.json` or `biome.jsonc`. The [Biome module reference](https://docs.dagger.io/reference/modules/js/biome) gives these rules: - A configuration with `"root": false` belongs to the nearest root configuration above it. - The module skips configuration files in directories that `.gitignore` ignores. In a monorepo, each package can be its own Biome project. This example has two, `apps/web` and `apps/admin`, and each has its own root `biome.json`. Add `--all` to list one row for each project: ```shell dagger check biomejs -l --all ``` ```text MODULE BIOMEJS-BIOME-PROJECT CHECK DESCRIPTION biomejs apps/admin lint Lint this project. biomejs apps/web lint Lint this project. ``` Check every project, or check one project: ```shell dagger check --biomejs dagger check --biomejs-biome-project=apps/admin ``` When a check covers several projects, it reports each project that fails. ### Apply safe fixes with the same Biome Each Biome project has a `fix` function. It runs `biome check --write` with the project's own Biome. It returns the repaired source as a changeset, a set of file changes that Dagger can write to your working tree. `fix` is a regular function, not a check, so `dagger check` does not run it. Fix the root project and write the changes to disk: ```shell dagger -c 'biomejs | projects | get . | fix | export .' ``` In this run, Biome formatted `src/login.ts`. `git diff` shows the change: ```text diff --git a/src/login.ts b/src/login.ts index 3685142..e39dbba 100644 --- a/src/login.ts +++ b/src/login.ts @@ -1,4 +1,6 @@ export function isAdmin(role: string): boolean { - if (role == "admin") { return true } + if (role == "admin") { + return true; + } return false; } ``` This is the same change that `npx biome check --write` made. The `==` stays, because `--write` applies only safe fixes. `fix` returns its changes even when diagnostics remain that Biome cannot fix. Run `dagger check` again to see them. To fix only some files, pass `files`. To limit the changeset to files that match patterns, pass `fixFilter`. Both take paths relative to the project root. ## Run Biome locally and with Cloud Checks `dagger check` runs the same Biome check on any machine that has the Dagger CLI. Commit the configuration and the lock file: ```shell git add dagger.toml dagger.lock biome.json package.json git commit -m "Check code with Biome through Dagger" ``` Dagger writes `dagger.lock` when it runs. The file pins the Git ref of each module and the images that the workspace uses, so every run uses the same versions. To run the check in CI, use [Cloud Checks](https://docs.dagger.io/cloud-checks). Cloud Checks run your checks on Dagger's Cloud Engines for every push and pull request to a GitHub repository. You do not need a CI workflow file. The Git `origin` remote of the project must point to the GitHub repository. Turn on Cloud Checks from the project root: ```shell dagger cloud checks on ``` In an interactive terminal, the command offers to complete the prerequisite steps, such as a Dagger Cloud sign-in and the GitHub connection. To see the results, run `dagger workspace activity`. If the Biome check fails in Cloud Checks, run `dagger check` on your machine to get the same diagnostics. Then run the `fix` function to apply Biome's safe fixes. **Bottom line:** Pin the exact Biome version, and run its formatter, linter, and assist actions as one check that fails CI. Apply safe fixes with the same Biome. By hand, that means an exact pin, `biome ci` in a CI job, and `biome check --write` in a Git hook. The Dagger Biome module meets these requirements in one place, and `dagger check` runs it the same way locally and in Cloud Checks. ## Related guides - [How do I run ESLint the same way locally and in CI?](/learn/eslint-dagger): the ESLint setup, for projects that keep ESLint. - [How do I enforce Prettier formatting in CI?](/learn/prettier-dagger): the Prettier setup, for projects that keep Prettier. - [How do I run Vitest in CI, with sharding and merged coverage?](/learn/run-vitest-with-dagger): the official Vitest module, for the tests next to Biome. --- # How do I run deno test, lint, fmt, and check in CI? URL: https://dagger.io/learn/deno-dagger A good Deno setup for CI runs `deno fmt --check`, `deno lint`, `deno check`, and `deno test` on one pinned Deno release. It grants the tests only the permissions they need, and it caches the dependencies between runs. In a Deno workspace, it runs the tools from the workspace root, so that they cover every member. You can meet each requirement with the Deno CLI and a CI job. Dagger's official Deno module meets the same requirements in one place, and runs the same checks on a laptop and in CI. ## What a good Deno CI setup needs Deno's own docs give five requirements. This guide uses them to judge each approach, Dagger included: 1. **Run the four built-in tools.** Deno's [CI guide](https://docs.deno.com/runtime/reference/continuous_integration/#setting-up-a-basic-pipeline) runs `deno fmt --check`, `deno lint`, and `deno test`. [`deno check`](https://docs.deno.com/runtime/reference/cli/check/) "type-checks your TypeScript (or JavaScript) code without running it", which the docs call "useful in CI pipelines". 2. **Use one Deno release everywhere.** The CI guide's example asks for `v2.x`, the latest stable release. [`denoland/setup-deno`](https://github.com/denoland/setup-deno) also takes an exact version, or reads one from `.dvmrc` or `.tool-versions`. One pinned release gives the same formatter, linter, and type checker on every machine. 3. **Grant the tests only the permissions they need.** Deno code has no access to the file system, the network, or the environment unless you [grant permissions](https://docs.deno.com/runtime/reference/permissions/). The CI guide's example uses `--allow-all`, but adds that "it is recommended to run with the minimal permissions your program needs". 4. **Cache the dependencies between runs.** The CI guide says that a download of the dependencies in every run "can become a time-consuming process". It tells you to [cache them](https://docs.deno.com/runtime/reference/continuous_integration/#caching-dependencies). 5. **Run a Deno workspace from its root.** In a [Deno workspace](https://docs.deno.com/runtime/fundamentals/workspaces/#running-tests), the docs say: "To run tests across all workspace members, simply execute deno test from the workspace root". `deno check`, `deno fmt`, and `deno lint` also cover every member from the root. ## Run the Deno tools by hand in a CI job The Deno CLI meets each requirement. The examples in this guide use a repository with one standalone project and one Deno workspace with two members: ```text apps/api/deno.json standalone project packages/deno.json Deno workspace: { "workspace": ["./core", "./ui"] } packages/core/deno.json member, "name": "@acme/core" packages/ui/deno.json member, imports "@acme/core" ``` In the standalone project, run the four tools. `-P` gives the tests the permissions that `deno.json` declares: ```shell deno fmt --check deno lint deno check deno test -P ``` ```text Checked 3 files Checked 2 files Check main.ts Check main_test.ts Warning Permissions in the config file is an experimental feature and may change in the future. Check main_test.ts running 2 tests from ./main_test.ts greet ... ok (163µs) reads config file ... ok (585µs) ok | 2 passed | 0 failed (5ms) ``` For the Deno workspace, run the same four commands once from `packages`. They cover `core` and `ui`, and the imports between the members resolve as they do in the workspace. In CI, a GitHub Actions job installs Deno with `denoland/setup-deno` and then runs the same commands. Set `deno-version` to an exact release, such as `v2.9.3`, or set `deno-version-file`. Set `cache: true` to save the dependency cache. By default, the cache key includes a hash of the `deno.lock` files ([caching dependencies](https://docs.deno.com/runtime/reference/continuous_integration/#caching-dependencies)). For a repository with several projects, the job runs the commands once in each project directory and once in each workspace root. ## Where the hand-built setup strains The commands are short, but each requirement lives in a different place: - **The Deno release is in two places.** The CI job sets `deno-version`, and each laptop has its own Deno. If they differ, `deno fmt --check` or `deno lint` can pass on a laptop and fail in CI. - **The list of projects is a script.** The CI job must run the tools in each standalone project and each workspace root. A new project means an edit to the job. - **The permissions are in two places.** A test run with `--allow-all` in CI can pass while the same test fails on a laptop that runs it with `-P`. - **The cache stays in CI.** The setup-deno cache does not reach the laptops where developers run the same checks. ## Meet them with the Dagger Deno module The [Dagger Deno module](https://docs.dagger.io/reference/modules/js/deno) meets the same five requirements, with the Deno image, the permissions, and the four tools defined in one place. A Dagger [module](https://docs.dagger.io/reference/modules) is a package of functions that test, lint, or build a project. The [Dagger Engine](https://docs.dagger.io/getting-started/introduction) runs those functions in containers, on a laptop or in CI. Complete the [Quickstart](https://docs.dagger.io/getting-started/quickstart) in the repository first. Then install the module from the repository root: ```shell dagger module install dagger.io/js/deno ``` ```text dagger.io/js/deno ➡️ https://github.com/dagger/deno#v0.3.0 Installed module "deno" in /work/dagger.toml ``` The command adds the module to `dagger.toml`, the workspace configuration for everyone who uses the repository: ```toml [modules.deno] source = "dagger.io/js/deno" ``` The command also writes `dagger.lock`. This file pins the images, Git references, and module addresses that the workspace resolves ([dagger.toml reference](https://docs.dagger.io/reference/config-files/dagger-toml)). Commit both files. A Dagger workspace is the directory that holds `dagger.toml`. It is a different thing from a Deno workspace. The next sections show how the module meets each requirement. ### Run the four built-in tools A check is a function that passes or fails, such as a test run or a linter. The module has a check for each tool. List the checks of the workspace: ```shell dagger check -l ``` ```text MODULE DENO-PROJECT DENO-WORKSPACE CHECK DESCRIPTION deno * format-check Check this project's formatting with `deno fmt --check`. deno * stale staleness check: format this project with `deno fmt` deno * lint Lint this project with `deno lint`. deno * test Run this project's tests with `deno test`. deno * type-check Type-check this project with `deno check`. deno * format-check Check formatting of every member of this workspace with `deno fmt --check`. deno * stale staleness check: format every member of this workspace with `deno fmt` deno * lint Lint every member of this workspace with `deno lint`. deno * test Run tests in every member of this workspace with `deno test`. deno * type-check Type-check every member of this workspace with `deno check`. # Use --all to expand collections and list each item. ``` Each column is a collection: a set of items that the module finds in the repository at run time. A `*` stands for every item in the collection. Run every check in the workspace: ```shell dagger check ``` ```text == CHECKS == ✔ 10 passed ✔ dag://?check=deno/projects/format-check 4.2s OK ✔ dag://?check=deno/projects/format/stale 4.2s OK ✔ dag://?check=deno/projects/lint 4.2s OK ✔ dag://?check=deno/projects/test 4.6s OK ✔ dag://?check=deno/projects/type-check 4.6s OK ✔ dag://?check=deno/workspaces/format-check 4.2s OK ✔ dag://?check=deno/workspaces/format/stale 4.2s OK ✔ dag://?check=deno/workspaces/lint 4.2s OK ✔ dag://?check=deno/workspaces/test 4.6s OK ✔ dag://?check=deno/workspaces/type-check 4.6s OK ``` Dagger runs the checks in parallel, and the command exits with a non-zero code if a check fails ([Checking your code](https://docs.dagger.io/using/checking)). In this guide's example repository, the first `dagger check` after a change to `apps/api/main.ts` took 11.4 seconds. A rerun with no changes took 5.4 seconds. Both times are for the whole command, in one run on one machine. When a check fails, the report shows the Deno output. All checks run, so you see every failure in one run. In this run, `apps/api/main.ts` had a type error and bad indentation: ```text ✘ dag://?check=deno/projects/type-check 0.7s ERROR ✘ withExec deno check . 0.3s ERROR ┃ Check main.ts ┃ Check main_test.ts ┃ TS2322 [ERROR]: Type 'number' is not assignable to type 'string'. ┃ const x: string = 5; ┃ ^ ┃ at file:///src/apps/api/main.ts:2:11 ┃ ┃ error: Type checking failed. ! exit code: 1 == RUN LOCALLY == dagger check "dag://?check=deno/projects/format-check" dagger check "dag://?check=deno/projects/format/stale" dagger check "dag://?check=deno/projects/lint" dagger check "dag://?check=deno/projects/test" dagger check "dag://?check=deno/projects/type-check" == REGENERATE == dagger generate -y "dag://?generator=deno/projects/format" ``` The report gives one command for each failed check. Run a command to run that check again. The `REGENERATE` section gives the command that fixes the formatting. The module has two checks for formatting. `format-check` runs `deno fmt --check`. The `stale` check comes from the `format` generator, a function that changes files and returns the changes as a changeset. The `stale` check fails when `deno fmt` would change a file. To fix the formatting, run the generator. Add `--no-apply` to print the changes and not write them: ```shell dagger generate --no-apply --deno ``` ```text == GENERATORS == ✔ 2 passed ✔ dag://?generator=deno/projects/format 0.1s OK ✔ dag://?generator=deno/workspaces/format 0.1s OK apps/api/main.ts +1 -1 1 file changed, +1 -1 lines Generated changes were not applied (--no-apply). ``` In a terminal, `dagger generate` shows the diff and asks before it writes the files. To write the files with no prompt, add `-y`. `deno fmt` uses the `fmt` options in your `deno.json`, such as `lineWidth` and `semiColons` ([deno.json reference](https://docs.deno.com/runtime/reference/deno_json/)). ### Use one Deno release everywhere The module runs Deno in `denoland/deno:alpine-`, and `version` defaults to `2.9.3`. Set `version` to the Deno release that your team uses. Every project then runs on the same release, on every machine: ```shell dagger module settings deno version 2.9.3 ``` Dagger stores settings in `dagger.toml` under `[modules.deno.settings]`, so they apply to everyone who uses the repository. A setting applies to every project and Deno workspace that the module checks. To list the settings, run `dagger module settings deno`: ```text MODULE KEY VALUE DESCRIPTION deno version 2.9.3 Deno version for the default base image (deno… deno base Base container for Deno commands. Must have `… deno permissions Permissions `test` and `compile` run with: "c… deno testArgs Extra `deno test` flags, e.g. ["--doc", "--pa… deno typeCheckArgs Extra `deno check` flags, e.g. ["--allow-impo… deno typeCheckTargets What `type-check` checks, relative to each pr… ``` To use a different image, set `base` to an image that has `deno` on its `PATH`, such as `docker.io/denoland/deno:debian`. When you set `base`, also set `version` to the release in that image. The module uses `version` to decide how it passes permissions. The [module reference](https://docs.dagger.io/reference/modules/js/deno) describes `testArgs`, `typeCheckArgs`, and `typeCheckTargets`. ### Grant the tests only the permissions they need The `permissions` setting controls the permissions that `deno test` gets: - `config` (the default) runs `deno test -P`. Deno then uses the permissions in your `deno.json`. - `all` runs `deno test -A`, which grants all permissions. - `none` grants no permissions. With a `version` before 2.5.0, `config` makes `test` use `-A`. With the default setting and no permissions in `deno.json`, a test that reads a file fails: ```text ┃ reads config file => ./main_test.ts:7:6 ┃ error: NotCapable: Requires read access to "/src/apps/api/deno.json", run again with the --allow-read flag ``` To fix it, give the tests the permissions they need in `apps/api/deno.json`. Deno [reads permissions from its config file](https://docs.deno.com/runtime/reference/permissions/#configuration-file): ```json { "tasks": {}, "test": { "permissions": { "read": true } } } ``` With this config, the same test passes with the default `permissions` setting. Deno prints a warning that permissions in the config file are an experimental feature. The tests get the same permissions on a laptop and in CI, because both read them from `deno.json`. If you do not want to keep permissions in `deno.json`, grant all permissions to the tests: ```shell dagger module settings deno permissions all ``` To remove a setting and go back to the default, run `dagger module settings -u deno permissions`. ### Cache the dependencies between runs The module sets `DENO_DIR` to a [cache volume](https://docs.dagger.io/reference/api/cache-volume) ([README](https://github.com/dagger/deno)). A cache volume is a directory whose contents persist across runs on the Dagger Engine. Every check of every project uses the same volume, so Deno downloads each dependency once for each engine. There is no cache key to maintain. ### Run a Deno workspace from its root The module puts each directory that has a `deno.json` or `deno.jsonc` in one of two collections: - **Deno workspaces.** A config with a non-empty `workspace` array is a Deno workspace, keyed by its directory. Its checks run from the workspace root and cover every member. The members are not keys of their own. - **Projects.** Every other config is a standalone project, keyed by its directory. A project at the repository root has the key `.`. In the example, `packages/core` and `packages/ui` do not show as projects. The module checks them through `packages`, and the imports between members resolve as they do in the Deno workspace. A new project or workspace gets its checks with no change to `dagger.toml`. Add `--all` to list one row per project and per Deno workspace: ```shell dagger check -l --all ``` ```text MODULE DENO-PROJECT DENO-WORKSPACE CHECK DESCRIPTION deno apps/api format-check Check this project's formatting with `deno fmt --check`. deno packages format-check Check formatting of every member of this workspace with `deno fmt --check`. deno apps/api lint Lint this project with `deno lint`. deno packages lint Lint every member of this workspace with `deno lint`. deno apps/api stale staleness check: format this project with `deno fmt` deno packages stale staleness check: format every member of this workspace with `deno fmt` deno apps/api test Run this project's tests with `deno test`. deno packages test Run tests in every member of this workspace with `deno test`. deno apps/api type-check Type-check this project with `deno check`. deno packages type-check Type-check every member of this workspace with `deno check`. ``` To run every check on one project, give its directory relative to the repository root: ```shell dagger check --deno-project=apps/api ``` ```text == CHECKS == ✔ 5 passed ✔ dag://?check=deno/projects/format-check 0.2s OK ✔ dag://?check=deno/projects/format/stale 0.2s OK ✔ dag://?check=deno/projects/lint 0.2s OK ✔ dag://?check=deno/projects/test 0.2s OK ✔ dag://?check=deno/projects/type-check 0.2s OK ``` To run one check on one Deno workspace, use the two flags together: ```shell dagger check --deno-workspace=packages --check=lint ``` ```text == CHECKS == ✔ 1 passed ✔ dag://?check=deno/workspaces/lint 0.2s OK ``` To run `deno test` on every project and Deno workspace, select the module and the check: ```shell dagger check --deno --check=test ``` ```text == CHECKS == ✔ 2 passed ✔ dag://?check=deno/projects/test 0.1s OK ✔ dag://?check=deno/workspaces/test 0.1s OK ``` To select more than one key, repeat the flag. ## Run the Deno checks locally and with Cloud Checks [Cloud Checks](https://docs.dagger.io/cloud-checks) run your checks on Dagger's Cloud Engines on each push and pull request, with no CI workflow file. Cloud Engines are Dagger Engines that Dagger Cloud runs for you. The repository must be on GitHub, and its `origin` remote must point to that repository. Turn on Cloud Checks from the repository root: ```shell dagger cloud checks on ``` Commit `dagger.toml` and `dagger.lock`, then push. Each push runs the same checks that `dagger check` runs on your machine, with the same Deno release and the same permissions. The results show as GitHub check statuses and in Dagger Cloud ([changelog](https://dagger.io/changelog/cloud-checks)). To see recent runs from the terminal, run `dagger workspace activity`. If a check fails in the cloud, run `dagger check` locally to get the same failure. **Bottom line:** Run `deno fmt --check`, `deno lint`, `deno check`, and `deno test` on one pinned Deno release, with only the permissions the tests need. Cache the dependencies, and run each Deno workspace from its root. Dagger's Deno module meets the same requirements in one place. It finds every project and workspace, pins the release with `version`, and caches `DENO_DIR`. It runs the same checks locally and with Cloud Checks. ## Related guides - [How do I run Vitest in CI, with sharding and merged coverage?](/learn/run-vitest-with-dagger): the same requirements for a Node project that tests with Vitest. - [How do I run Jest in CI and split the tests with --shard?](/learn/jest-dagger): the same requirements for a Node project that tests with Jest. - [How do I enforce Prettier formatting in CI?](/learn/prettier-dagger): format checks for a Node project. --- # How do I run ESLint the same way locally and in CI? URL: https://dagger.io/learn/eslint-dagger To run ESLint the same way locally and in CI, run the project's own ESLint and plugins on a supported Node.js version. Lint each package with its own config. Let lint errors fail the CI job, and apply `--fix` with the same ESLint and config. Teams usually meet these requirements with `npx eslint .` in a CI job and a Git pre-commit hook. Dagger's official ESLint module meets the same requirements in one place, and `dagger check` runs it the same way on a laptop and in Cloud Checks. ## What a consistent ESLint setup needs ESLint's own docs give four requirements for lint results that are the same on a laptop and in CI: 1. **Run the project's own ESLint on a supported Node.js version.** ESLint's [Getting Started page](https://eslint.org/docs/latest/use/getting-started) installs ESLint in the project, not globally. It says that plugins and shareable configs "must still be installed locally" even with a global ESLint. ESLint needs Node.js `^20.19.0`, `^22.13.0`, or `>=24`. A lockfile such as [`package-lock.json`](https://docs.npmjs.com/cli/configuring-npm/package-lock-json) records the exact versions of ESLint and its plugins. 2. **Lint each package with its own config.** ESLint ["determines configuration for each target file by first looking in the directory that contains the file"](https://eslint.org/docs/latest/use/configure/configuration-files), and then looks in the directories above it. The docs say that this lookup supports monorepos, "where subdirectories can have their own configuration files." 3. **Fail the job on lint errors.** ESLint exits with code 1 when it finds at least one error, and with code 2 on a configuration problem ([exit codes](https://eslint.org/docs/latest/use/command-line-interface#exit-codes)). A run with only warnings exits with code 0. The [`--max-warnings`](https://eslint.org/docs/latest/use/command-line-interface#--max-warnings) option sets the number of warnings that makes ESLint exit with an error. 4. **Apply fixes with the same ESLint and config.** The [`--fix`](https://eslint.org/docs/latest/use/command-line-interface#--fix) option writes fixes "to the actual files themselves", and ESLint then reports "only the remaining unfixed issues". Run it with the ESLint version and config that CI uses, so that the fixed files pass the CI check. ## Meet them with npx eslint, a CI job, and a hook By hand, each developer runs the project's ESLint with `npx`, and a CI job runs the same command. This guide's project has a root config and a second config in `packages/ui`. Only the `packages/ui` config turns on the `prefer-const` rule. Run ESLint from the project root: ```shell npx eslint . ``` ```text /work/packages/ui/src/button.js 2:7 error 'prefix' is never reassigned. Use 'const' instead prefer-const ✖ 1 problem (1 error, 0 warnings) 1 error and 0 warnings potentially fixable with the `--fix` option. ``` The command exited with code 1. ESLint used the `packages/ui` config for `button.js`, so one run from the root meets the second requirement. Fix the problem: ```shell npx eslint . --fix ``` The command printed nothing and exited with code 0. It changed `let prefix` to `const prefix` in `packages/ui/src/button.js`. In CI, a job installs Node.js, installs the dependencies, and runs `npx eslint .`. The exit code of ESLint fails the job. On GitHub Actions, [`actions/setup-node`](https://github.com/actions/setup-node) installs the Node.js version from its `node-version` input, or from a file such as `.nvmrc` through `node-version-file`. [`npm ci`](https://docs.npmjs.com/cli/commands/npm-ci) installs the exact versions in `package-lock.json`. GitLab CI and other systems run the same commands in a Node.js image ([GitLab CI/CD YAML reference](https://docs.gitlab.com/ci/yaml/)). Many teams also run ESLint in a Git pre-commit hook, so that a developer sees problems before a push. ESLint's [integrations page](https://eslint.org/docs/latest/use/integrations#source-control) lists hook setups. One of them lints only staged changes. ## Where the hand-built setup strains The hand-built setup meets each requirement in more than one place, and each place must stay in step: - **Node.js version.** The CI job reads the version from its config or a version file. Each developer must switch their own Node.js to the same version. - **The project's own ESLint.** `npm ci` in CI installs from the lockfile. A local `node_modules` that is older than the lockfile runs a different ESLint or plugin version until the developer installs again. - **Fixes.** A pre-commit hook is optional. [`git commit --no-verify`](https://git-scm.com/docs/git-commit) skips it, so CI is the first place that some problems show. - **CI syntax.** Each CI system has its own job syntax. A move to another CI system means a rewrite of the ESLint job. ## Meet them with the Dagger ESLint module The [Dagger ESLint module](https://docs.dagger.io/reference/modules/js/eslint) meets the four requirements with one definition that runs on a laptop and in CI. A [Dagger module](https://docs.dagger.io/reference/modules) is a package of functions that test, lint, or build a project. Official modules install from `dagger.io/`. Complete the [Quickstart](https://docs.dagger.io/getting-started/quickstart) first. It runs `dagger init`, which creates `dagger.toml`. This file holds the workspace configuration for everyone who uses the repository. Install the ESLint module from the project root: ```shell dagger module install dagger.io/js/eslint ``` ```text dagger.io/js/eslint ➡️ https://github.com/dagger/eslint#v0.3.0 Installed module "eslint" in /work/dagger.toml ``` The command adds the module to `dagger.toml`: ```toml [modules.eslint] source = "dagger.io/js/eslint" ``` The command also writes `dagger.lock`, which pins the module to an exact Git commit. Commit both files. The next four sections show how the module meets each requirement. ### Run the project's own ESLint on a supported Node.js version The module installs the project's dependencies and runs the `eslint` binary from the project's own `node_modules`, in the project directory. The result matches what developers see when they run ESLint locally. The module runs ESLint in the `node:25-alpine` image by default. Set `baseImageAddress` to the project's Node.js version. The install step runs without the source files. Thus, a lifecycle script such as `postinstall` or `prepare` fails if it needs source files. The `--ignore-scripts` flag stops these scripts ([README](https://github.com/dagger/eslint)). If an install script needs source files, set `installFlags` to `["--ignore-scripts"]`: ```shell dagger module settings eslint baseImageAddress node:22-alpine dagger module settings eslint installFlags '["--ignore-scripts"]' ``` The settings go into `dagger.toml`, so every developer and every CI run uses the same ones: ```toml [modules.eslint] source = "dagger.io/js/eslint" [modules.eslint.settings] baseImageAddress = "node:22-alpine" installFlags = ["--ignore-scripts"] ``` In a monorepo, the module installs dependencies at the nearest workspace root. A workspace root is a directory with `pnpm-workspace.yaml`, or with a `package.json` that has `workspaces`. If there is no workspace root, the module uses the nearest lockfile, and then the nearest `package.json`. The module mounts that tree, so shared configs and `workspace:` dependencies resolve ([README](https://github.com/dagger/eslint)). The [module reference](https://docs.dagger.io/reference/modules/js/eslint) covers the `packageManager` and `environment` settings. The install step reads only the package files and lockfiles, so a change to a source file does not start a new install ([README](https://github.com/dagger/eslint)). In this guide's project, the first `dagger check` took 12.9 seconds, with the `npm install`. A rerun with no changes took 0.8 seconds. These numbers come from one run on one machine. ### Lint each package with its own config The module treats a directory as an ESLint project if the directory has one of these files: - A flat config: `eslint.config.js`, `.mjs`, `.cjs`, `.ts`, `.mts`, or `.cts`. - A legacy config: `.eslintrc`, `.eslintrc.js`, `.cjs`, `.json`, `.yaml`, or `.yml`. The module does not find a config that is only in the `eslintConfig` field of `package.json`. It also does not find a config that you give with `--config` ([README](https://github.com/dagger/eslint)). A check is a function that passes or fails, such as a lint run. The ESLint module has one check, `lint`, and runs it once for each ESLint project. Add `--all` to list one row for each project: ```shell dagger check -l --all ``` ```text MODULE ESLINT-PROJECT CHECK DESCRIPTION eslint . lint Lint this project. eslint packages/ui lint Lint this project. ``` A nested project gets its own run with its own config, and the project around it skips the nested files. Thus, the module lints no file twice ([README](https://github.com/dagger/eslint)). To lint one project, add `--eslint-project` with its directory, relative to the workspace root. The root project is `.`. Repeat the flag to select more projects: ```shell dagger check --eslint-project=packages/ui ``` ```text == CHECKS == ✔ 1 passed ✔ dag://?check=lint&eslint-project=packages/ui 0.8s OK ``` To list the projects that you can select, run `dagger list eslint-projects -a`: ```text MODULE ESLINT-PROJECT DESCRIPTION eslint . The ESLint project rooted at path. eslint packages/ui The ESLint project rooted at path. ``` ### Fail the job on lint errors `dagger check` runs every check in the workspace. When ESLint finds no errors, the check passes: ```shell dagger check ``` ```text == CHECKS == ✔ 1 passed ✔ dag://?check=eslint/projects/lint 8.4s OK ``` When ESLint reports an error, the check fails. The output shows ESLint's report for each project that failed: ```text ! ESLint failed in 1 of 2 projects: - packages/ui: eslint failed (exit 1): + /src/node_modules/.bin/eslint . /src/packages/ui/src/button.js 2:7 error 'prefix' is never reassigned. Use 'const' instead prefer-const ✖ 1 problem (1 error, 0 warnings) 1 error and 0 warnings potentially fixable with the `--fix` option. == RUN LOCALLY == dagger check "dag://?check=eslint/projects/lint" ``` The module runs `eslint .` with no `--max-warnings` option ([module source](https://github.com/dagger/eslint)). A warning alone does not fail the check. To make a rule fail the check, set it to `"error"` in the ESLint config. ### Apply fixes with the same ESLint and config Each project has a `fix` function that runs ESLint's `--fix` option in the same container as the check. The function returns the changes as a changeset, which is a set of file changes that you apply to your files. `fix` is not a check, so `dagger check` does not run it. Run it from the workspace root: ```shell dagger -c 'eslint | projects | get packages/ui | fix | export .' ``` In this guide's project, the command changed one line in `packages/ui/src/button.js`. This is the change that `git diff` showed: ```text - let prefix = "Button: "; + const prefix = "Button: "; ``` `fix` keeps the fixes even when ESLint cannot fix all problems. The check reports the problems that remain. ## Run ESLint locally and with Cloud Checks `dagger check` runs the same check on a laptop and in CI. [Cloud Checks](https://docs.dagger.io/cloud-checks) run your checks on Dagger's Cloud Engines for each push and pull request, with no CI workflow file. The repository must be on GitHub, and its Git `origin` remote must point to that repository. Turn on Cloud Checks from the project root: ```shell dagger cloud checks on ``` If you do not have a prerequisite, such as a Dagger Cloud account, the command prints the command to run first. Commit `dagger.toml` and `dagger.lock`, then push. Each push then runs the ESLint check. To see the results, run `dagger workspace activity`. If the check fails, run `dagger check` locally to see the same ESLint report. If the project also formats code with Prettier, see [How do I enforce Prettier formatting in CI?](/learn/prettier-dagger). Biome is one tool for both lint and format rules. [How do I run Biome lint and format checks in CI?](/learn/biome-dagger) says when it fits. **Bottom line:** Run the project's own ESLint on a supported Node.js version, and let each package use its own config. Fail CI on errors, and run `--fix` with the same setup. By hand, that means `npx eslint .` in a CI job, a pinned Node.js version, and a pre-commit hook, each kept in step. The Dagger ESLint module meets these requirements in one place, and `dagger check` runs it the same way locally and in Cloud Checks. ## Related guides - [How do I enforce Prettier formatting in CI?](/learn/prettier-dagger): check formatting next to ESLint, with the same project and the official Prettier module. - [How do I run Biome lint and format checks in CI?](/learn/biome-dagger): one tool for lint and format rules, when it can replace ESLint and Prettier. - [How do I run Jest in CI and split the tests with --shard?](/learn/jest-dagger): run JavaScript unit tests with the official Jest module. --- # How do I lint Helm charts and check rendered templates in CI? URL: https://dagger.io/learn/helm-chart-lint-dagger Helm chart checks in CI meet four requirements. They use one Helm version everywhere, run `helm lint` on every chart, and render every chart with each CI values file through `helm template`. They also build chart dependencies before they render. Teams meet these with Helm commands or chart-testing in a CI job. Dagger meets the same four requirements in one place. Its official Helm module finds every chart and runs the same checks on a laptop and in Dagger Cloud Checks. ## What good Helm chart checks need Helm chart checks in CI have four requirements. Each one comes from the docs of Helm and its tools, and this guide judges each approach by them: 1. **Use one Helm version everywhere.** Helm 4 [came out in November 2025](https://helm.sh/blog/helm-v3-end-of-life), and Helm 3 gets security fixes until February 2027. A laptop and a CI runner can have different major versions. The [`azure/setup-helm`](https://github.com/Azure/setup-helm) action for GitHub Actions installs the latest Helm unless you give it a version. Then different Helm releases check the same chart on a laptop and in CI. 2. **Lint every chart.** [`helm lint`](https://helm.sh/docs/helm/helm_lint/) runs "a series of tests to verify that the chart is well-formed". It emits `[ERROR]` messages for problems that "will cause the chart to fail installation". 3. **Render every chart with each CI values file.** Helm's [chart-testing tool](https://github.com/helm/chart-testing/blob/main/doc/ct_lint.md) checks a chart once for each `*-values.yaml` file in its `ci` directory. [`helm template`](https://helm.sh/docs/helm/helm_template/) renders the templates locally. It does no "server-side testing of chart validity (e.g. whether an API is supported)", so it needs no cluster. 4. **Build chart dependencies before you render.** A chart that lists dependencies in `Chart.yaml` needs them in its `charts/` directory. [`helm dependency build`](https://helm.sh/docs/helm/helm_dependency_build/) does this, and it builds "out the charts/ directory from the Chart.lock file". ## Meet the requirements with Helm commands Teams meet the four requirements with the Helm CLI. The examples in this guide use one chart at `charts/api`, with two CI values files: ```text charts/api/ Chart.yaml values.yaml templates/ deployment.yaml ci/ prod-values.yaml minimal-values.yaml ``` **Use one Helm version everywhere.** Check the version on each machine: ```shell helm version --short ``` ```text v3.17.2+gcc0bbbd ``` This guide's laptop had Helm 3.17.2. In CI, give `azure/setup-helm` the same version in its `version` input. **Lint every chart.** Run `helm lint` on each chart: ```shell helm lint charts/api ``` ```text ==> Linting charts/api [INFO] Chart.yaml: icon is recommended 1 chart(s) linted, 0 chart(s) failed ``` **Render every chart with each CI values file.** `helm lint` alone does not prove that each values file renders. In this guide's project, `ci/prod-values.yaml` set `image: null`, and a template reads `.Values.image.repository`. `helm lint charts/api --values charts/api/ci/prod-values.yaml` printed `0 chart(s) failed`, with Helm 3.17.2 and with Helm 4.0.1. `helm template` with the same file failed: ```shell helm template charts/api --values charts/api/ci/prod-values.yaml ``` ```text Error: template: api/templates/deployment.yaml:17:28: executing "api/templates/deployment.yaml" at <.Values.image.repository>: nil pointer evaluating interface {}.repository Use --debug flag to render out invalid YAML ``` Run `helm template` once for each values file. chart-testing's `ct lint` runs `helm lint` with each `ci/*-values.yaml` file. It also checks that the chart version went up, and lints `Chart.yaml` and the values files ([ct lint](https://github.com/helm/chart-testing/blob/main/doc/ct_lint.md)). **Build chart dependencies before you render.** Commit `Chart.lock`. Then fill `charts/` from it before the other commands: ```shell helm dependency build charts/api --skip-refresh ``` ```text Saving 1 charts Downloading redis from repo oci://registry-1.docker.io/bitnamicharts Pulled: registry-1.docker.io/bitnamicharts/redis:25.5.3 Digest: sha256:4987cbf877710e3c083cfbb907a1a4ac4ce389803dbadb7b1f8022d8b3fcb85e Deleting outdated charts ``` The `--skip-refresh` flag tells Helm to "not refresh the local repository cache". This dependency comes from an OCI registry, so Helm needs no repository index. In CI, a GitHub Actions job installs Helm with `azure/setup-helm` and runs the same commands. Helm also has a [`chart-testing-action`](https://github.com/helm/chart-testing-action) that installs `ct`. ## Where the hand-built setup strains Each requirement becomes a step that someone keeps up to date: - **Use one Helm version everywhere.** The version is in each CI workflow, and each laptop runs the Helm that its owner installed. - **Lint every chart.** A job that lists chart paths misses a new chart until someone adds it. `ct` finds charts, but by default it checks only the charts that changed. - **Render every chart with each CI values file.** `helm template` needs a loop over each chart and each values file in a script. `helm lint` alone passed the broken values file in this guide's project. - **Build chart dependencies before you render.** Each job must run `helm dependency build` first, with access to each chart repository. ## Meet the requirements with the Dagger Helm module [Dagger](https://docs.dagger.io/getting-started/introduction) is a CI orchestration engine that runs the same pipelines on a laptop and in CI. A [Dagger module](https://docs.dagger.io/reference/modules) is a package of functions that do work on the project, such as a lint or a test run. The official [Helm module](https://docs.dagger.io/reference/modules/kubernetes/helm) runs `helm lint` and `helm template` on every chart. You write no Dagger code. Complete the [Quickstart](https://docs.dagger.io/getting-started/quickstart) first. It creates `dagger.toml`, the workspace configuration. Then install the Helm module from the project root: ```shell dagger module install dagger.io/kubernetes/helm ``` ```text Installed module "helm" in /work/dagger.toml ``` The command adds the module to `dagger.toml` and writes `dagger.lock`. The lock file pins the image and Git references that the workspace resolved, so every machine loads the same module revision. A [check](https://docs.dagger.io/using/checking) is a function that passes or fails. `dagger check` runs every check in the workspace in parallel and exits non-zero if one fails. List the Helm module's checks: ```shell dagger check -l ``` ```text MODULE CHECK DESCRIPTION helm assert-template Assert every discovered values file renders with helm template --dry-run=client. helm lint Lint every Helm chart visible from the client's working directory. ``` ## Use one Helm version everywhere, with Dagger The Helm module installs Helm at the version in its `version` setting ([Helm module reference](https://docs.dagger.io/reference/modules/kubernetes/helm)). Pin the Helm release that your team uses: ```shell dagger module settings helm version 4.0.1 ``` The setting goes into `dagger.toml`, so local runs and CI use the same Helm release: ```toml [modules.helm] source = "dagger.io/kubernetes/helm" [modules.helm.settings] version = "4.0.1" ``` Every machine that runs `dagger check` gets Helm 4.0.1 in the check's container, whatever Helm the machine has installed. ## Lint every chart with Dagger The `lint` check runs `helm lint` once on each chart with its default values. Then it runs it once more for each CI values file. The module finds every `Chart.yaml` at or below the directory where `dagger` runs. It uses the directory that holds that file as the chart root. Thus templates, subcharts, CRDs, and files that templates read through `.Files` are all available to Helm. The module checks a new chart without a change to any configuration. To see the charts it found, call its `charts` function: ```shell dagger api call helm charts path ``` ```text charts/api ``` A subchart that the repository keeps under a chart's `charts/` directory has its own `Chart.yaml`. The module lists it and checks it as a separate chart too. Run both checks: ```shell dagger check ``` ```text == CHECKS == ✔ 2 passed ✔ dag://?check=helm/assert-template 0.5s OK ✔ dag://?check=helm/lint 0.5s OK ``` To run one check, give its address: `dagger check dag://helm/lint`. In this example, `Chart.yaml` has no `version` field, and `lint` fails with Helm's `[ERROR]` messages: ```text ✘ dag://?check=helm/lint 0.1s ERROR ✘ withExec helm lint . --values ci/prod-values.yaml 0.0s ERROR ┃ Error: 1 chart(s) linted, 1 chart(s) failed ┃ ==> Linting . ┃ [ERROR] Chart.yaml: version is required ┃ [INFO] Chart.yaml: icon is recommended ┃ [WARNING] Chart.yaml: version '' is not a valid SemVerV2 ┃ [ERROR] templates/: validation: chart.metadata.version is required ┃ [ERROR] : unable to load chart ┃ validation: chart.metadata.version is required ! exit code: 1 == RUN LOCALLY == dagger check "dag://?check=helm/lint" ``` ## Render every chart with each CI values file, with Dagger The `assert-template` check renders each chart with each values file through `helm template --dry-run=client`. It fails when a template does not render. It does not render the chart with only its default values, because some charts render only with one of their explicit values files. In this example, `ci/prod-values.yaml` sets `image: null`, and a template reads `.Values.image.repository`: ```text ✘ dag://?check=helm/assert-template 0.6s ERROR ✘ withExec helm template . --values ci/prod-values.yaml --dry-run=client 0.4s ERROR ┃ Error: api/templates/deployment.yaml:17:28 ┃ executing "api/templates/deployment.yaml" at <.Values.image.repository>: ┃ nil pointer evaluating interface {}.repository ┃ ┃ Use --debug flag to render out invalid YAML ! exit code: 1 == RUN LOCALLY == dagger check "dag://?check=helm/assert-template" ``` The values files come from the `valuesGlob` setting. Its default, `ci/*-values.yaml`, is the chart-testing convention. If your charts follow it, add a values file such as `charts/api/ci/staging-values.yaml` to check one more configuration. If your values files are in a different location, change the glob. The glob is relative to each chart root: ```shell dagger module settings helm valuesGlob "values/*.yaml" ``` ```toml [modules.helm] source = "dagger.io/kubernetes/helm" [modules.helm.settings] version = "4.0.1" valuesGlob = "values/*.yaml" ``` The module runs each pair of chart and values file in its own container. Dagger caches the result of each pair separately ([module source](https://github.com/dagger/helm)). ## Build chart dependencies before you render, with Dagger The module runs Helm on the chart directory as it is in the repository, and does not run `helm dependency build` ([module source](https://github.com/dagger/helm/blob/main/helm.dang)). When `Chart.yaml` lists a dependency that the chart's `charts/` directory does not hold, `assert-template` fails: ```text == CHECKS == ✘ 1 failed ✔ 1 passed ✘ dag://?check=helm/assert-template 0.1s ERROR ✘ withExec helm template . --values ci/prod-values.yaml --dry-run=client 0.0s ERROR ┃ Error: an error occurred while checking for chart dependencies. You may need to run `helm dependency build` to fetch missing dependencies: found in Chart.yaml, but missing in charts/ directory: redis ! exit code: 1 ✔ dag://?check=helm/lint 0.1s OK == RUN LOCALLY == dagger check "dag://?check=helm/assert-template" ``` Run `helm dependency build` as in the hand-built setup, and commit the archives that it writes to `charts/`. In this guide's project, both checks passed after the commit of `charts/api/charts/redis-25.5.3.tgz`. The checks then need no access to the chart repository. ## Run the same checks locally and with Cloud Checks Run `dagger check` on your machine before you push. Commit `dagger.toml` and `dagger.lock`. To run the same checks on every push and pull request, use [Cloud Checks](https://docs.dagger.io/cloud-checks). Cloud Checks run your checks on Cloud Engines, Dagger's managed engines, with no CI workflow file. The project must be a GitHub repository, and its Git `origin` remote must point to that repository. Turn them on from the project root: ```shell dagger cloud checks on ``` In an interactive terminal, the command offers to create a Dagger Cloud account and to connect the GitHub organization. Without a terminal, it prints the commands to run first: ```text Error: Cloud checks need a Dagger Cloud account and organization. Complete the prerequisite, then retry: dagger cloud signup dagger cloud checks on ``` Cloud Checks run the same `lint` and `assert-template` checks as `dagger check` on your machine, with the Helm version from `dagger.toml`. To see the runs, use `dagger workspace activity`. **Bottom line:** Pin one Helm version, run `helm lint` on every chart, render each CI values file with `helm template`, and build dependencies first. By hand, that is `azure/setup-helm` with a version, a loop over charts and values files, and `helm dependency build`. Dagger's Helm module finds every chart, pins Helm in `dagger.toml`, and runs both checks on a laptop and with Cloud Checks. ## Related guides - [Which development tools can Dagger replace or consolidate?](/learn/consolidate-ci-tools): how chart checks fit with the linters and test runners already in your CI. - [How do I migrate from GitHub Actions to Dagger?](/learn/migrate-from-github-actions): move a workflow that runs `helm lint` or chart-testing to Dagger, one step at a time. - [How do I build Docker images and run Compose services the same way locally and in CI?](/learn/docker-compose-dagger): lint and build the Dockerfiles of the images that your charts deploy. --- # How do I run Playwright tests in CI and locally in the same container, with cached browsers? URL: https://dagger.io/learn/playwright-ci-docker-browser-caching A reliable Playwright CI setup does four things. It runs the tests in Microsoft's Playwright Docker image, `mcr.microsoft.com/playwright`, at the tag for your exact Playwright version. It does not cache the browsers, because the image already contains them. It updates screenshot baselines in that same image, and it uses one CPU architecture on every machine. You can meet each requirement by hand in each CI config and local script. Dagger's official Playwright module meets all four in one place, and runs the same way on a laptop and in CI. ## What a reliable Playwright CI setup needs Playwright's own docs give four requirements for test runs that behave the same in CI and on a laptop. This guide uses them to judge each approach, Dagger included: 1. **Run the tests in the Playwright image for your exact version.** The Playwright [Docker image](https://playwright.dev/docs/docker) "includes the Playwright browsers and browser system dependencies". It does not include the Playwright package, so the job still installs your dependencies. Pin the image tag to the exact version of `@playwright/test`, such as `v1.64.1-noble` for Ubuntu 24.04. If the two versions differ, Playwright "will be unable to locate browser executables", says its Docker page. 2. **Do not cache the browsers.** Playwright's [CI guide](https://playwright.dev/docs/ci#caching-browsers) says that a restore of the browser cache takes about as long as a fresh download of the browsers. On Linux, a job must still install the operating system dependencies, and Playwright says these are not cacheable. The cache would hold the [browser directory](https://playwright.dev/docs/browsers#managing-browser-binaries), `~/.cache/ms-playwright` on Linux. In the Playwright image, the browsers are part of the image, so there is no browser directory to cache. 3. **Update screenshot baselines in the same image.** Playwright's [visual comparisons page](https://playwright.dev/docs/test-snapshots) warns that "browser rendering can vary based on the host OS, version, settings, hardware, power source (battery vs. power adapter), headless mode, and other factors." It tells you to run the tests in the same environment that generated the baselines. Playwright also puts the browser and the platform in the name of each baseline file, such as `home-chromium-linux.png`. A baseline from a Mac is `home-chromium-darwin.png`, so a Linux CI run does not find it. 4. **Use one CPU architecture.** The same image gives the same fonts, libraries, and browser builds on every machine. It does not give the same CPU architecture. The image has arm64 and amd64 variants, and each machine pulls the variant for its own architecture. In a [Playwright issue about this](https://github.com/microsoft/playwright/issues/13873), a maintainer wrote that "it is expected that arm docker image vs intel docker image produce different screenshots." ## Meet them in your CI config A container job meets the first two requirements. Playwright's [CI guide](https://playwright.dev/docs/ci) shows the image on several CI systems. GitHub Actions uses `jobs..container`, GitLab CI uses a job `image`, Jenkins uses a Docker agent, and CircleCI uses a `docker` entry in the job. The job runs `npm ci` and then `npx playwright test` in the image. It downloads no browsers, so it has nothing to cache. The CI guide also gives a second way to prepare a CI agent: install the browsers with the Playwright CLI in each job. After `npm ci`, one command downloads the browsers and installs the operating system packages that they need: ```shell npx playwright install --with-deps chromium ``` This approach does not cache the browsers, but it does not use the Playwright image, and each job pays for the download again. In this guide's project, the command took 30 seconds in a `node:22-bookworm` container on an arm64 laptop. It put 680 MB in `~/.cache/ms-playwright`: 401 MB for Chromium, 276 MB for the headless shell, and 4 MB for ffmpeg. To meet the third requirement, update the baselines in the image, not on the host. From the project root, run `--update-snapshots` in the image with `docker run`: ```shell docker run --rm --init --ipc=host --platform linux/amd64 -v "$PWD":/work -v /work/node_modules -w /work mcr.microsoft.com/playwright:v1.64.1-noble sh -c "npm ci && npx playwright test --update-snapshots" ``` Playwright's [Docker page](https://playwright.dev/docs/docker) recommends the `--init` and `--ipc=host` flags. Without `--ipc=host`, Chromium can run out of memory and crash. The second `-v` flag gives `node_modules` its own volume, so the Linux packages do not replace the ones in your checkout. In this guide's project, the command wrote `tests/home.spec.ts-snapshots/home-chromium-linux.png`. Review the file and commit it. To meet the fourth requirement, choose one platform and pass it with `--platform` everywhere the image runs. The command above uses `linux/amd64`. If the team uses both arm64 and amd64 machines, run every machine on one platform, or update baselines from one platform only. ## Where the hand-built setup strains The hand-built setup puts the same container settings in several places, and each place must change at the same time: - The image tag is in the CI config, in local `docker run` scripts, and often in a devcontainer. Each copy must change in the commit that upgrades `@playwright/test`. The `--platform` value must match in each copy too. - A developer who skips the `docker run` script updates baselines on the host, not in the image. - A move to another CI system means a rewrite of the container job in the syntax of that system. ## Meet them with the Dagger Playwright module The Dagger Playwright module meets the same four requirements, with the image, the install, and the test command defined in one place. [Dagger](https://docs.dagger.io/getting-started/introduction) is a CI orchestration engine. You define the pipeline once, in code, and it runs the same way on a laptop and in CI. The open-source [Dagger Engine](https://dagger.io/changelog/cloud-engines) runs the pipelines on your machine or on a server. A [module](https://docs.dagger.io/reference/sdks) is a reusable package of functions, checks, and services that a project installs. From the project root, install the official [Playwright module](https://docs.dagger.io/reference/modules/js/playwright): ```shell dagger module install dagger.io/js/playwright ``` The command adds the module to `dagger.toml`, the workspace configuration, and creates that file if the project has none. It also pins the Git ref of the module in `dagger.lock`. Commit both files. The [Quickstart](https://docs.dagger.io/getting-started/quickstart) covers the rest of the Dagger setup for a project. A [check](https://docs.dagger.io/using/checking) is a function that validates the project, such as a test run. List the checks: ```shell dagger check -l ``` ```text MODULE PLAYWRIGHT-PROJECT PLAYWRIGHT-TEST-FILE CHECK DESCRIPTION playwright * * test Run this test file. # Use --all to expand collections and list each item. ``` Each directory with a `playwright.config` is a project, and the check can run each test file on its own. A `*` stands for every item. Run the tests: ```shell dagger check ``` The module installs your dependencies in the Playwright image and runs the project's own `playwright test`. If the config declares a `webServer`, Playwright starts the app inside the container, as it does on your machine. The next four sections show how the module meets each requirement. ### Run the tests in the Playwright image for your exact version The module runs your tests in the same `mcr.microsoft.com/playwright` image as a hand-built container job. It reads your `@playwright/test` version from `package-lock.json`, `pnpm-lock.yaml`, `yarn.lock`, or `bun.lock`, and picks the image tag for that version. Without a lockfile, it reads `package.json`, and it reads a range such as `^1.58.2` as its lower bound. Pin that version exactly, or commit a lockfile. Show the image that the module chose: ```shell dagger api call playwright project --path=. image-address ``` ```text mcr.microsoft.com/playwright:v1.64.1-noble ``` When you upgrade `@playwright/test`, the lockfile changes, and the module picks the new tag. No other file has a copy of the tag. ### Do not cache the browsers With the module, there is no browser directory to cache. The browsers come with the Playwright image, and the Dagger Engine caches the pull of that image. Dagger caches each step by its inputs and runs a step again only when an input changes. A later run does not download the browsers again. The module also mounts a [cache volume](https://docs.dagger.io/reference/api/cache-volume), a directory whose contents persist across runs, for the download cache of the package manager. The step that installs dependencies reads only the package manifests, the lockfiles, and the package manager config. A change to a test or source file does not start a new install. In this guide's project, on an arm64 laptop, the first `dagger check` took 46.6 seconds. That run pulled the Playwright image and ran `npm install`. A rerun with no changes took 2.7 seconds, with every step from the cache: ```text == CHECKS == ✔ 1 passed ✔ dag://?check=test&playwright-project=. 0.3s OK ``` After an edit to the test file, `dagger check` took 4.8 seconds. Dagger ran the tests again and used the cached `npm install`. These numbers come from one run on one machine, so treat them as an example. ### Update screenshot baselines in the same image The module updates baselines in its test container, which is the Playwright image with your dependencies installed. In a project without baselines, the first `dagger check` fails as Playwright does anywhere else: ```text == CHECKS == ✘ 1 failed ✘ dag://?check=test&playwright-project=. 1.5s ERROR == TESTS == ✘ chromium FAIL ✘ chromium::home.spec.ts FAIL ✘ chromium › home.spec.ts › home page FAIL Error: A snapshot doesn't exist at /app/tests/home.spec.ts-snapshots/home-chromium-linux.png, writing actual. at /app/tests/home.spec.ts:5:3 3 | test('home page', async ({ page }) => { 4 | await page.goto('/'); > 5 | await expect(page).toHaveScreenshot('home.png'); | ^ 6 | }); 7 | ✘ 1 failed == RUN LOCALLY == dagger check "dag://?check=test&playwright-project=." ``` Run `--update-snapshots` in the test container, then export the test directory to your checkout: ```shell dagger api call playwright project --path=. base with-exec --args=npx,playwright,test,--update-snapshots directory --path=tests export --path=tests ``` The command writes `tests/home.spec.ts-snapshots/home-chromium-linux.png` from inside the image, as the `docker run` command does. Review the file and commit it. The next `dagger check` compares against it: ```text == CHECKS == ✔ 1 passed ✔ dag://?check=test&playwright-project=. 1.2s OK == TESTS == ✔ chromium 1 passed ✔ 1 passed ``` If your tests are in a directory other than `tests`, change both paths in the command. ### Use one CPU architecture To use one CPU architecture, give the module a base container for one platform. By default, a container uses [the native platform of the engine](https://docs.dagger.io/reference/api/query). An arm64 laptop and an amd64 CI engine then pull different variants of the image. [Cloud Engines](https://dagger.io/changelog/cloud-engines) are the managed Dagger Engines that run [Cloud Checks](https://docs.dagger.io/cloud-checks). They run on amd64 and on arm64 ([arm64 Cloud Engines](https://dagger.io/changelog/cloud-arm64-engines)). Pick one platform for the whole team and pin it. The example below pins amd64. The `baseCtr` setting of the module takes a container from a function in another module. This is [module wiring](https://docs.dagger.io/config/module-wiring). A setting whose value is a `dag:///` address gets the value that the function returns. Write the function in [Dang](https://docs.dagger.io/reference/sdks/dang), the native language of Dagger for modules. Install the Dang SDK, then create the dev module of the project: ```shell dagger module install dagger.io/sdk/dang dagger module init dang ``` `dagger module init` creates the module at `.dagger/modules/-dev`, records it in `dagger.toml`, and shows the changes for you to apply. Replace the contents of `.dagger/modules/-dev/main.dang` with this function. `WebDev` stands in for the type name that `dagger module init` generated, so keep your own: ```dang type WebDev { """ The Playwright image for browser tests, pinned to linux/amd64 so screenshots match on every machine. """ pub playwrightBase: Container! { container(platform: "linux/amd64").from("mcr.microsoft.com/playwright:v1.64.1-noble") } } ``` Wire the function into the Playwright module: ```shell dagger module settings playwright baseCtr playwright-base ``` The command stores the full address in `dagger.toml`: ```toml [modules.playwright.settings] baseCtr = "dag://-dev/playwright-base" ``` Checks and baseline updates now run in the amd64 image on every machine. On an arm64 machine, the amd64 image runs under emulation. In this guide's project on an arm64 laptop, the test container reported `x86_64`. The first check in it pulled the amd64 image and took 43.4 seconds. The Dang SDK also adds its own check, `dang-sdk/generate/stale`, which `dagger check` now runs. With `baseCtr` set, the module no longer takes the image tag from your lockfile. Change the tag in `main.dang` in the commit that upgrades `@playwright/test`. If every machine on the team has the same architecture, skip this section. ## Run the same checks with Cloud Checks In CI, run the same check with Dagger Cloud Checks. [Cloud Checks](https://docs.dagger.io/cloud-checks) run your checks on Dagger's Cloud Engines after each push and pull request, with no CI workflow file. Turn them on from the project root: ```shell dagger cloud checks on ``` The command connects your GitHub repository to Dagger Cloud. The Git `origin` remote of the project must point to that repository. In an interactive terminal, the command offers to sign you in and connect GitHub. [Set up Cloud Checks](https://docs.dagger.io/cloud-checks) covers each step. Cloud Engines are managed Dagger Engines with scale-out and a distributed cache. Dagger Cloud bills the CPU, memory, and data that your pipelines use, as the [pricing page](https://dagger.io/pricing) shows. `dagger.toml` and the module define the image, the install, and the test command once. Your laptop and Cloud Checks run that same definition, so there is no second container job to keep in step. **Bottom line:** Run Playwright in the `mcr.microsoft.com/playwright` image whose tag matches your `@playwright/test` version, and do not cache the browsers. Update screenshot baselines in that image, on one CPU architecture. Dagger's Playwright module meets these requirements in one place and picks the image from your lockfile. It runs the same check on your machine and with Cloud Checks. ## Related guides - [How do I run Playwright tests against a Docker Compose app in CI?](/learn/playwright-tests-docker-compose): wire an app from a Compose file into the Playwright module. - [How do I run integration tests against real databases and Kafka without mocks?](/learn/integration-tests-real-database-containers): start real services for tests, the same way locally and in CI. - [How do I migrate from GitHub Actions to Dagger?](/learn/migrate-from-github-actions): move the checks in GitHub Actions workflows to Dagger. --- # How do I lint shell scripts with ShellCheck in CI? URL: https://dagger.io/learn/shellcheck-dagger To lint shell scripts with ShellCheck in CI, check every script with one pinned ShellCheck version, and fail the build on findings. Keep the configuration in the repository, and skip the scripts that the repository does not own. A `find` command, a pre-commit hook, and a CI job meet these requirements by hand. Dagger meets the same requirements in one place. Its official ShellCheck module checks every `.sh` file in a pinned ShellCheck container, on a laptop and in Cloud Checks. ## What a good ShellCheck setup needs A ShellCheck setup for CI has five requirements. Each one comes from the ShellCheck docs: 1. **Check every script.** ShellCheck checks only the files that it gets. ["By default, `shellcheck` does not support a `-r` option"](https://github.com/koalaman/shellcheck/wiki/Recursiveness), so a command such as `find` gives it the scripts. 2. **Pin the ShellCheck version.** The ShellCheck README recommends that you install a specific version in your build. That ["avoids any surprise build breaks when a new version with new warnings is published"](https://github.com/koalaman/shellcheck#in-your-build-or-test-suites). 3. **Fail on findings.** ShellCheck ["makes canonical use of exit codes"](https://github.com/koalaman/shellcheck#in-your-build-or-test-suites). It exits with code 1 when it finds issues, and the default minimum severity is `style` ([ShellCheck manual](https://github.com/koalaman/shellcheck/blob/master/shellcheck.1.md)). 4. **Keep the configuration in the repository.** [Directives](https://github.com/koalaman/shellcheck/wiki/Directive) in the scripts and a `.shellcheckrc` file hold the configuration. ShellCheck looks for `.shellcheckrc` ["in the script's directory and each parent directory"](https://github.com/koalaman/shellcheck/blob/master/shellcheck.1.md). 5. **Skip scripts you do not own.** Vendored and generated scripts stay out of the check. The [Recursiveness](https://github.com/koalaman/shellcheck/wiki/Recursiveness) page excludes a directory from `find`, and a [`disable=all` directive](https://github.com/koalaman/shellcheck/wiki/Ignore) skips one file. ## Run ShellCheck by hand On a laptop, `find` gives ShellCheck every `.sh` file and skips the `vendor` directory: ```shell find . -type f -name '*.sh' ! -path './vendor/*' -exec shellcheck {} + ``` ```text In ./ci/build.sh line 4: rm -rf $target/build ^-----^ SC2086 (info): Double quote to prevent globbing and word splitting. Did you mean: rm -rf "$target"/build In ./scripts/main.sh line 2: . ./lib.sh ^------^ SC1091 (info): Not following: ./lib.sh was not specified as input (see shellcheck -x). For more information: https://www.shellcheck.net/wiki/SC1091 -- Not following: ./lib.sh was not s... https://www.shellcheck.net/wiki/SC2086 -- Double quote to prevent globbing ... ``` The command exits with code 1, so a CI step that runs it fails. Scripts with no extension, such as `scripts/env`, need a search by shebang, as the [Recursiveness](https://github.com/koalaman/shellcheck/wiki/Recursiveness) page shows. In this guide's project, that search also found Git's sample hooks in `.git/hooks`, so it needs a second exclusion. Both findings above are `info` findings. To fail only on warnings and errors, raise the minimum severity: ```shell shellcheck --severity=warning ci/build.sh; echo "exit code $?" ``` ```text exit code 0 ``` A `.shellcheckrc` file at the project root applies to every script below it. In this guide's project, a root `.shellcheckrc` with `disable=SC2086` stopped the finding in `ci/build.sh`. To check scripts before each commit, the README gives a [pre-commit hook](https://github.com/koalaman/shellcheck#pre-commit) from `koalaman/shellcheck-precommit`, with the ShellCheck version in `rev`. In CI, a job runs the same `find` command. GitHub's Ubuntu 24.04 runner image includes ShellCheck, and [its readme](https://github.com/actions/runner-images/blob/main/images/ubuntu/Ubuntu2404-Readme.md) lists the version. To pin a version, the job installs a file from [ShellCheck releases](https://github.com/koalaman/shellcheck/releases) or runs a [`koalaman/shellcheck` image](https://github.com/koalaman/shellcheck#installing) with a version tag. ## Where the hand-built setup strains The commands work, but each requirement lives in more than one place: - **Pin the ShellCheck version.** The version is in the pre-commit `rev`, in the CI install step, and in each developer's package manager. In this guide's project, Homebrew installed ShellCheck 0.11.0, and the Ubuntu 24.04 runner image lists 0.9.0. A script can then pass in one place and fail in the other. - **Check every script** and **skip scripts you do not own.** The CI job and the pre-commit hook each select the scripts, so a new exclusion must go in both. - **Fail on findings.** The minimum severity is a command-line flag, so each place that runs ShellCheck sets it again. ## Check every script with the Dagger ShellCheck module The official [ShellCheck module](https://docs.dagger.io/reference/modules/shellcheck) finds every `.sh` file in the project and runs ShellCheck on each one. A Dagger module is a package of functions that build, test, or check a project. The Dagger team keeps a catalog of [official modules](https://docs.dagger.io/reference/modules) for common tools. The project needs a Git repository and a `dagger.toml` file from the [Quickstart](https://docs.dagger.io/getting-started/quickstart). Install the ShellCheck module from the project root: ```shell dagger module install dagger.io/shellcheck ``` ```text dagger.io/shellcheck ➡️ https://github.com/dagger/shellcheck#main Installed module "shellcheck" in /work/dagger.toml ``` The module installs under the name `shellcheck`. The command adds it to `dagger.toml`: ```toml [modules.shellcheck] source = "dagger.io/shellcheck" ``` The module finds scripts at or below the directory where `dagger` runs. The `scripts` function returns the scripts that the module found. Each script has a `path`. Print the paths: ```shell dagger -c 'shellcheck | scripts | path' ``` ```text ci/build.sh scripts/compile.sh scripts/lib.sh scripts/main.sh vendor/lib/bad.sh ``` The [module reference](https://docs.dagger.io/reference/modules/shellcheck) says that a script must use the `.sh` extension for the module to find it. In this guide's test, the module skipped a script named `scripts/env` that had a shebang but no extension. To check a script like that, rename it to end in `.sh`. Each script runs in its own container, so Dagger caches the result for each script. The [module README](https://github.com/dagger/shellcheck) states that an edit to one script checks only that script again. In this guide's run, the trace after a change to `scripts/lib.sh` showed one new `shellcheck` run, for that script only. ## Pin the ShellCheck version with dagger.lock The [module source](https://github.com/dagger/shellcheck) runs ShellCheck from the `koalaman/shellcheck-alpine` image with no tag. The [ShellCheck README](https://github.com/koalaman/shellcheck) describes that image as an Alpine Linux image with ShellCheck installed. Dagger pins the image in `dagger.lock`, a file that it writes when it runs. In this guide's run, the file pinned the image to `v0.11.0` by digest, and the module to a Git commit: ```text # Generated by Dagger. Pins the images and git refs this workspace uses. # Any Dagger command may update this file; always commit it with your changes. # Run `dagger lock update` to refresh the pinned versions. [["version","2"]] ["","git-latest",["github.com/dagger/shellcheck"],"refs/heads/main"] ["","git-sha",["github.com/dagger/shellcheck","refs/heads/main"],"756497171e9185db48dc7961649f466b4003cfa9"] ["","oci-latest",["docker.io/koalaman/shellcheck-alpine"],"v0.11.0"] ["","oci-sha",["docker.io/koalaman/shellcheck-alpine:v0.11.0"],"sha256:9955be09ea7f0dbf7ae942ac1f2094355bb30d96fffba0ec09f5432207544002"] ["","vanity-url",["https://dagger.io/shellcheck"],"https://github.com/dagger/shellcheck"] ``` Commit `dagger.lock`, and every laptop and Cloud Checks run the same ShellCheck. To move to a new ShellCheck release, run `dagger lock update` and commit the change. ## Fail on findings with dagger check The module adds one check, `check`, which fails when any script has a finding. A check is a function that passes or fails, such as a test or a linter. `dagger check` runs every check in the project. List the checks: ```shell dagger check -l ``` ```text MODULE CHECK DESCRIPTION shellcheck check Run shellcheck on all shell scripts at or below the client's working ``` Run every check in the project, or run only the ShellCheck check: ```shell dagger check dagger check dag://shellcheck/check ``` The check runs `shellcheck ` on each script, with no severity flag. The module has no severity setting, so the default minimum severity, `style`, applies, and `info` findings also fail the check. In this run, the check failed on `vendor/lib/bad.sh`, a vendored script: ```text == CHECKS == ✘ 1 failed ✘ dag://?check=shellcheck/check 5.8s ERROR ✘ withExec shellcheck vendor/lib/bad.sh 0.0s ERROR ┃ ┃ In vendor/lib/bad.sh line 2: ┃ cd $1 ┃ ^---^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails. ┃ ^-- SC2086 (info): Double quote to prevent globbing and word splitting. ┃ ┃ Did you mean: ┃ cd "$1" || exit ┃ ┃ For more information: ┃ https://www.shellcheck.net/wiki/SC2164 -- Use 'cd ... || exit' or 'cd ... |... ┃ https://www.shellcheck.net/wiki/SC2086 -- Double quote to prevent globbing ... ! exit code: 1 == RUN LOCALLY == dagger check "dag://?check=shellcheck/check" ``` Each finding has a code, such as `SC2086`, and a link to its page on the ShellCheck wiki. The command under `RUN LOCALLY` runs the same check again. To accept a finding, use a directive, as the next section shows. ## Keep the configuration in the repository, in directives Put ShellCheck configuration in the scripts as [directives](https://github.com/koalaman/shellcheck/wiki/Directive). The module mounts only the script file into the container. So ShellCheck does not see a project `.shellcheckrc` or the files that a script sources. In this guide's run, a root `.shellcheckrc` with `disable=SC2086` did not stop the `SC2086` finding above. A directive is a comment that starts with `# shellcheck`. A directive right after the shebang applies to the whole script. Otherwise it applies to the next command. This version of `ci/build.sh` passes the check, because the parameter expansion is in quotes: ```shell #!/bin/sh target=$1 rm -rf "${target:?}/build" ``` To accept a finding on one command, put a `disable` directive above it. The [ShellCheck wiki](https://github.com/koalaman/shellcheck/wiki/Directive) recommends a comment that says why. This `scripts/compile.sh` passes the check: ```shell #!/bin/sh # shellcheck disable=SC2086 # FLAGS must split into separate arguments gcc $FLAGS main.c ``` A script that sources another file gets `SC1091`, because ShellCheck cannot follow the sourced file: ```text ┃ In scripts/main.sh line 2: ┃ . ./lib.sh ┃ ^------^ SC1091 (info): Not following: ./lib.sh was not specified as input (see shellcheck -x). ``` The module still checks `lib.sh` as a separate script. To tell ShellCheck not to follow the sourced file, put a `source=/dev/null` directive above the line. This `scripts/main.sh` passes the check: ```shell #!/bin/sh # shellcheck source=/dev/null . ./lib.sh greet ``` ## Skip scripts you do not own with a directive Skip a script that the repository does not own with a `disable=all` directive at the top of the file. The [ShellCheck wiki](https://github.com/koalaman/shellcheck/wiki/Ignore) documents this directive for ShellCheck 0.8 and newer. This `vendor/lib/bad.sh` passes the check: ```shell #!/bin/sh # shellcheck disable=all cd $1 ``` ## Run ShellCheck locally and with Cloud Checks `dagger check` runs the same ShellCheck check on any machine that has the Dagger CLI. Commit the configuration and the lock file: ```shell git add dagger.toml dagger.lock git commit -m "Check shell scripts with ShellCheck through Dagger" ``` To run the check in CI, use [Cloud Checks](https://docs.dagger.io/cloud-checks). Cloud Checks run your checks on Dagger's Cloud Engines for every push and pull request to a GitHub repository. You do not need a CI workflow file. The Git `origin` remote of the project must point to the GitHub repository. Enable Cloud Checks from the project root: ```shell dagger cloud checks on ``` In an interactive terminal, the command offers to complete the steps that are not done, such as a Dagger Cloud sign-in and the GitHub connection. `dagger workspace activity` lists the recent runs. If the ShellCheck check fails in Cloud Checks, run `dagger check` on your machine to get the same findings. **Bottom line:** To lint shell scripts with ShellCheck in CI, check every script with a pinned version, fail on findings, and keep the configuration in the repository. By hand, a `find` command, a pre-commit hook, and a CI job each repeat the version and the script selection. The Dagger ShellCheck module checks every `.sh` file in a container that `dagger.lock` pins, with configuration in directives. `dagger cloud checks on` runs the same check on every push. ## Related guides - [How do I run PSScriptAnalyzer on PowerShell scripts in CI?](/learn/psscriptanalyzer-dagger): the same requirements for PowerShell scripts. - [Which development tools can Dagger replace or consolidate?](/learn/consolidate-ci-tools): which other development tools Dagger can replace or run. - [How do I run golangci-lint the same way locally and in CI?](/learn/golangci-lint-dagger): another official linter module, for the Go code next to the scripts. --- # How do I build Docker images and run Compose services the same way locally and in CI? URL: https://dagger.io/learn/docker-compose-dagger A Docker setup that works the same on a laptop and in CI meets four requirements. It lints each Dockerfile, builds every image and every test stage, validates each Compose file, and starts each service after the services it depends on. Teams meet them with hadolint, `docker build`, and `docker compose` in a CI job and in local scripts. Dagger meets the same four requirements in one place. Its official Docker module finds every Dockerfile and Compose file, and runs the same checks on a laptop and in Dagger Cloud Checks. ## What a good Docker and Compose setup needs A Docker and Compose setup for CI has four requirements. Each one comes from the docs of Docker and hadolint, and this guide judges each approach by them: 1. **Lint each Dockerfile.** [hadolint](https://github.com/hadolint/hadolint) is a "Dockerfile linter, validate inline bash". It finds problems such as unpinned package versions. Docker's own [build checks](https://docs.docker.com/build/checks/) also lint a Dockerfile. But "check violations for builds are reported as warnings, with exit code 0, by default." A lint that must fail CI needs a tool that exits with an error. 2. **Build every image and every test stage.** Docker recommends that you [build and test your images in CI](https://docs.docker.com/build/building/best-practices/#build-and-test-your-images-in-ci) for each change and pull request. Many Dockerfiles have a `test` or `lint` stage that the final image does not use. BuildKit "only builds the stages that the target stage depends on" ([multi-stage builds](https://docs.docker.com/build/building/multi-stage/#differences-between-legacy-builder-and-buildkit)). Thus a normal build skips those stages. 3. **Validate each Compose file.** The `--quiet` flag of [`docker compose config`](https://docs.docker.com/reference/cli/docker/compose/config/) tells it to "Only validate the configuration, don't print anything". It finds an error in the Compose file before any service starts. 4. **Start each service after the services it depends on.** "On startup, Compose does not wait until a container is "ready", only until it's running" ([startup order](https://docs.docker.com/compose/how-tos/startup-order/)). A `healthcheck` on a service and the `service_healthy` condition in `depends_on` make Compose wait until the dependency is healthy. ## Meet the requirements with Docker commands Teams meet the four requirements with the Docker CLI and hadolint. The examples come from a project with a Dockerfile in `app/` and a Compose file at the root. The Dockerfile has a `test` stage that no other stage uses. The Compose file is `compose.yaml`: ```yaml services: web: build: ./app ports: - "8080:8080" environment: REDIS_URL: redis://redis:6379 depends_on: - redis redis: image: redis:7-alpine ports: - "6379:6379" ``` **Lint each Dockerfile.** Run hadolint from its image, as the [hadolint README](https://github.com/hadolint/hadolint#how-to-use) shows. `NO_COLOR=1` turns off color in the output: ```shell docker run --rm -i -e NO_COLOR=1 hadolint/hadolint:v2.14.0-alpine < app/Dockerfile ``` ```text -:9 DL3018 warning: Pin versions in apk add. Instead of `apk add ` use `apk add =` ``` hadolint exits with code 1, so the CI job fails. To ignore a rule, list it under `ignored` in a [`.hadolint.yaml` file](https://github.com/hadolint/hadolint#configure). **Build every image and every test stage.** Build the final image, then build the `test` stage with `--target`: ```shell docker build -t web app docker build --target test app ``` In this guide's project, the first command built the `deps` stage and the final stage. It did not run the `test` stage. Only the second command ran it. **Validate each Compose file.** Run `docker compose config --quiet` in the directory of the Compose file. It prints nothing and exits with code 0 when the file is valid. **Start each service after the services it depends on.** Start the services, and wait until they run: ```shell docker compose up --wait ``` ```text Network compose_default Creating Network compose_default Created Container compose-redis-1 Creating Container compose-redis-1 Created Container compose-web-1 Creating Container compose-web-1 Created Container compose-redis-1 Starting Container compose-redis-1 Started Container compose-web-1 Starting Container compose-web-1 Started Container compose-redis-1 Waiting Container compose-web-1 Waiting Container compose-redis-1 Healthy Container compose-web-1 Healthy ``` The [`--wait` flag](https://docs.docker.com/reference/cli/docker/compose/up/) waits "for services to be running|healthy". Neither service has a `healthcheck`, so Compose waited only until each container ran. After the run, [`docker compose down`](https://docs.docker.com/reference/cli/docker/compose/down/) stops and removes the containers and the network. In CI, a job runs the same commands on a runner with Docker. On GitHub Actions, Docker's [official actions](https://docs.docker.com/build/ci/github-actions/) set up Buildx and Compose and build images. hadolint has a [GitHub Action and a GitLab CI example](https://github.com/hadolint/hadolint/blob/master/docs/INTEGRATION.md#continuous-integration). ## Where the hand-built setup strains Each requirement becomes one or more steps that someone keeps up to date: - **Lint each Dockerfile.** The hadolint image tag is in each CI job and each local script. If the copies differ, a laptop and CI apply different rules. - **Build every image and every test stage.** Each Dockerfile and each test stage needs its own `docker build` command. A new `test` stage that nobody adds to the CI job never builds, because BuildKit skips it. - **Validate each Compose file.** Each directory with a Compose file needs its own `docker compose config` step. - **Start each service after the services it depends on.** The runner needs a Docker daemon and free host ports, such as 8080 and 6379. The job must also run `docker compose down` when the tests end. ## Meet the requirements with the Dagger Docker module [Dagger](https://docs.dagger.io/getting-started/introduction) is a CI orchestration engine that runs the same pipelines on a laptop and in CI. A [module](https://docs.dagger.io/reference/modules) is a package of checks, services, and functions that a project installs. The official [Docker module](https://github.com/dagger/docker) finds each `Dockerfile` and each Compose file in the repository. It meets each requirement with the same tools as the Docker commands above. You write no Dagger code. Complete the [Quickstart](https://docs.dagger.io/getting-started/quickstart) first. Then, from the repository root, install the Docker module from its `main` branch, because it has no tagged release: ```shell dagger module install dagger.io/docker@main ``` ```text Created workspace config in /work Installed module "docker" in /work/dagger.toml ``` The command adds the module to `dagger.toml`, the workspace configuration, and creates the file if the project has none. It also writes `dagger.lock`, which pins the Git commit of `main` that Dagger used. Commit both files. To move to a newer commit of the module, run `dagger lock update`. A [check](https://docs.dagger.io/using/checking) is a function that validates the project and passes or fails, such as a lint or a build. List the checks of the Docker module: ```shell dagger check --docker -l ``` ```text MODULE DOCKER-COMPOSE-PROJECT DOCKER-COMPOSE-SERVICE DOCKER-PROJECT DOCKER-STAGE CHECK DESCRIPTION docker * lint Validate this project's Compose file with 'docker compose config'. docker * * build-service Build this Compose service's image, if it declares a build. docker * build Build this project's Dockerfile. docker * lint Lint this project's Dockerfile with hadolint. docker * * build-stage Build this Dockerfile stage. # Use --all to expand collections and list each item. ``` A `*` stands for every item that the module found. Add `--all` to list one row for each item: ```shell dagger check --docker -l --all ``` ```text MODULE DOCKER-PROJECT DOCKER-COMPOSE-PROJECT DOCKER-COMPOSE-SERVICE DOCKER-STAGE CHECK DESCRIPTION docker app build Build this project's Dockerfile. docker . redis build-service Build this Compose service's image, if it declares a build. docker . web build-service Build this Compose service's image, if it declares a build. docker app test build-stage Build this Dockerfile stage. docker . lint Validate this project's Compose file with 'docker compose config'. docker app lint Lint this project's Dockerfile with hadolint. ``` The module works on four collections. Each collection adds a column and a flag that selects its items: | Collection | One item is | Flag | | --- | --- | --- | | Docker projects | a directory with a `Dockerfile`, such as `app` | `--docker-project` | | Dockerfile stages | a named stage that the project build does not reach, such as `test` | `--docker-stage` | | Compose projects | a directory with a Compose file, `.` for the root | `--docker-compose-project` | | Compose services | a service name in a Compose file | `--docker-compose-service` | ## Lint each Dockerfile with Dagger The `lint` check of each Docker project runs [hadolint](https://github.com/hadolint/hadolint) on its Dockerfile. The module pins the hadolint image in its `lintImage` setting. The default is `docker.io/hadolint/hadolint:v2.14.0-alpine` ([module README](https://github.com/dagger/docker#settings)). Thus a laptop and CI run the same hadolint release. Run every check in the workspace: ```shell dagger check ``` In this guide's project, the first run failed on the same hadolint rule as the `docker run` command: ```text == CHECKS == ✘ 1 failed ✔ 4 passed ✘ dag://?check=docker/projects/lint 4.3s ERROR ┃ [hadolint] Dockerfile:9 DL3018 warning: Pin versions in apk add. Instead of `apk add ` use `apk add =` ! hadolint failed in 1 of 1 projects: - app: hadolint failed (exit 1): Dockerfile:9 DL3018 warning: Pin versions in apk add. Instead of `apk add ` use `apk add =` ✘ exec.processRun 0.0s ERROR ! /usr/local/bin/crun did not terminate successfully: exit status 1 ✔ dag://?check=build-service&docker-compose-project=. 9.2s OK ✔ dag://?check=build-stage&docker-project=app 2.5s OK ✔ dag://?check=docker/compose/projects/lint 8.6s OK ✔ dag://?check=docker/projects/build 4.0s OK == RUN LOCALLY == dagger check "dag://?check=docker/projects/lint" ``` The `RUN LOCALLY` command runs only the failed check again. To run hadolint on every Dockerfile and nothing else, select the Docker projects and the check name: ```shell dagger check --docker-projects --check lint ``` The module gives hadolint the nearest `.hadolint.yaml` or `.hadolint.yml`, from the directory of the Dockerfile up to the repository root ([module README](https://github.com/dagger/docker#hadolint)). It is the same [hadolint configuration file](https://github.com/hadolint/hadolint#configure) that you use without Dagger. To ignore a rule, create `.hadolint.yaml` at the root: ```yaml ignored: - DL3018 ``` To give hadolint extra arguments, use the `lintArgs` setting. For example, make hadolint fail only on errors, not on warnings: ```shell dagger module settings docker lintArgs '["--failure-threshold", "error"]' ``` The command writes `lintArgs = ["--failure-threshold", "error"]` under `[modules.docker.settings]` in `dagger.toml`. ## Build every image and every test stage with Dagger The `build` check builds the final stage of each Dockerfile. The `build-stage` check builds each named stage that no other stage uses and that is not the final stage ([module README](https://github.com/dagger/docker#stages)). The module reads the stages from the Dockerfile. Thus a new `test` stage gets a check without a change to any CI file. After the `.hadolint.yaml` from the previous section, all five checks passed: ```text == CHECKS == ✔ 5 passed ✔ dag://?check=build-service&docker-compose-project=. 1.1s OK ✔ dag://?check=build-stage&docker-project=app 1.1s OK ✔ dag://?check=docker/compose/projects/lint 0.7s OK ✔ dag://?check=docker/projects/build 1.1s OK ✔ dag://?check=docker/projects/lint 0.6s OK ``` When a `RUN` step fails, the report gives the project and the exit code. For a Dockerfile with `RUN exit 3`, the report showed: ```text ! docker build failed in 1 of 1 projects: - app: docker build failed (exit code 3 in a RUN step; see the trace for its output) ``` The output of the step is in the trace, under the failed `withExec`. Some Dockerfiles end in a development stage, after the production stage. If yours do, set `buildTarget` to the stage that the `build` check must build: ```shell dagger module settings docker buildTarget production ``` The `build` check then builds that stage in each Dockerfile that declares it. A Dockerfile without the stage still builds its final stage. The stage that `buildTarget` names leaves the stages collection, so `build-stage` does not build it a second time. To go back to the default, remove the setting: ```shell dagger module settings -u docker buildTarget ``` ## Validate each Compose file with Dagger The `lint` check of each Compose project runs [`docker compose config --quiet`](https://docs.docker.com/reference/cli/docker/compose/config/) in the directory of the Compose file. The `build-service` check builds each service that has a `build` section, with its build args, target, and Dockerfile. A service that only runs an image, such as `redis`, has nothing to build, so its check passes. To build only the `web` service: ```shell dagger check --docker-compose-service=web ``` The module looks for `compose.yaml`, `compose.yml`, `docker-compose.yml`, and `docker-compose.yaml`, in that order. It uses the first file that it finds in each directory, as [Compose](https://docs.docker.com/compose/intro/compose-application-model/#the-compose-file) does. Compose also merges a standard override file, such as `compose.override.yaml`. The module reads service names from the text of the file. Thus, it does not see services from `include:` or from files that `-f` or `COMPOSE_FILE` names. It also does not see services from YAML merge keys at the `services` level ([module README](https://github.com/dagger/docker#compose-files)). ## Start each service after the services it depends on, with Dagger The module starts Compose services as [Dagger services](https://docs.dagger.io/using/services). A Dagger service is a container that Dagger starts so that other containers or your machine can reach it. List the services: ```shell dagger start -l -a ``` ```text MODULE DOCKER-COMPOSE-PROJECT DOCKER-COMPOSE-SERVICE SERVICE DESCRIPTION docker . batch/up Run the selected services and the services they depend on, behind one docker cli/engine The Docker Engine this client is wired to. docker docker/engine Spawn an ephemeral Docker Engine in a container. docker . redis up Run this service and the services it depends on, behind one proxy that docker . web up Run this service and the services it depends on, behind one proxy that ``` Start the `web` service: ```shell dagger start --docker-compose-service=web ``` Dagger builds `web` from `app/Dockerfile` and starts `redis`, because `web` depends on it. It publishes the ports of both services on your machine. In this guide's project, `curl localhost:8080` returned the page, and Redis answered `PING` on port 6379. To stop the services, press Ctrl+C. To select one Compose project, add `--docker-compose-project`. Always select a project or a service, because a `dagger start` with no filter also starts the Docker engines of the module. Dagger applies the Compose file as far as it can ([module README](https://github.com/dagger/docker#running-compose-services)): - **The current Docker module does not translate Compose health checks.** A service starts after the ports of its dependencies accept connections. The module does not run `healthcheck` tests or the `depends_on` conditions that [Compose uses for startup order](https://docs.docker.com/compose/how-tos/startup-order/). Dagger itself supports Docker health checks on services, with `withDockerHealthcheck` in the [Container API](https://docs.dagger.io/reference/api/container). Until the module translates them, make a dependent service retry its connection if a dependency opens its port before it is ready. - **Services reach each other by name.** `web` connects to `redis:6379`, as on the default Compose network. - **Bind mounts are copies.** Dagger mounts files from the repository as copies, so writes do not change your files. Dagger does not mount a path from outside the repository, such as `/var/run/docker.sock`. - **Named volumes persist.** Each named volume becomes a cache volume that keeps its data between runs. - **Profiles are off.** Dagger does not start services that have a Compose profile. For each setting that Dagger cannot apply, such as `cap_add`, `network_mode`, or a UDP port, it prints a warning in the trace. In a check, Dagger [stops each service](https://docs.dagger.io/using/services) when nothing references it, so there is no `down` step. Other modules can use a Compose service as a setting. The guide to [Playwright tests against a Docker Compose app](/learn/playwright-tests-docker-compose) shows how. ## Run the same checks locally and with Cloud Checks Commit `dagger.toml` and `dagger.lock`. If you created `.hadolint.yaml`, commit it too. The project has no Dagger module code to commit. In CI, [Cloud Checks](https://docs.dagger.io/cloud-checks) run the checks. Cloud Checks run your checks on Cloud Engines, Dagger's managed engines, after each push and pull request. They need no CI workflow file. The repository must be on GitHub, and its `origin` remote must point to it. Turn them on: ```shell dagger cloud checks on ``` In an interactive terminal, the command offers to sign you in to Dagger Cloud and to connect GitHub. After that, each push runs hadolint, the Dockerfile builds, and the Compose checks on Cloud Engines, as `dagger check` does on a laptop. The hadolint release and the list of Dockerfiles, stages, and Compose files come from the same `dagger.toml` and the same repository on both. To see the runs, use `dagger workspace activity`. **Bottom line:** Lint each Dockerfile with hadolint, build every image and every test stage, and validate each Compose file. Start each service after its dependencies. The Docker CLI meets these requirements one command at a time. Dagger's Docker module finds every Dockerfile, stage, and Compose file, and runs the same checks on a laptop and with Cloud Checks. ## Related guides - [How do I run Playwright tests against a Docker Compose app in CI?](/learn/playwright-tests-docker-compose): start an app from its Compose file and run Playwright tests against it. - [How do I run integration tests against real databases and Kafka without mocks?](/learn/integration-tests-real-database-containers): give tests real database and Kafka containers, the Compose way or the Testcontainers way. - [How do I migrate from GitHub Actions to Dagger?](/learn/migrate-from-github-actions): move the build and lint steps of GitHub Actions workflows to Dagger. --- # How do I run integration tests against real databases and Kafka without mocks? URL: https://dagger.io/learn/integration-tests-real-database-containers To replace mocks with real services, start each database and Kafka broker in a container for each test run. A good setup meets four requirements. Each service is fresh for each run and healthy before the tests start. The tests find it at a known address, and the setup is the same on a laptop and in CI. Testcontainers, Docker Compose, and CI service containers each meet some or all of them by hand. Dagger meets the same four requirements in either model. Define the services once outside the tests, as with Compose, or start them from test code, as with Testcontainers. ## What integration tests against real services need Integration tests against real services have to meet four requirements. This guide uses these names for them in each section that follows: - **A fresh service for each run.** Each run starts new containers and removes them after the tests, so no data stays from an earlier run. GitHub Actions "creates a fresh Docker container for each service configured in the workflow, and destroys the service container when the job completes" ([Communicating with Docker service containers](https://docs.github.com/en/actions/tutorials/use-containerized-services/use-docker-service-containers)). - **Healthy before the tests start.** The tests start only after each service accepts connections. Otherwise the first test races the startup of the service. Testcontainers wait strategies "allow your tests to actually wait for the most useful conditions to be met, before continuing with their execution" ([Wait strategies](https://golang.testcontainers.org/features/wait/introduction/)). - **A known address.** The tests find each service at a host and port that they read, such as from an environment variable. Two runs on one machine must not collide on a port. Testcontainers picks random host ports "to avoid port collisions that may arise with locally running software or in between parallel test runs" ([Networking](https://java.testcontainers.org/features/networking/)). - **The same setup locally and in CI.** A laptop and CI use the same images, settings, and health checks. Otherwise a test that passes on a laptop can fail in CI. ## Meet the requirements with Testcontainers Testcontainers meets all four requirements from inside the test process. It is a library that starts containers from test code. Its languages include Java, .NET, Go, Node.js, Python, Rust, and Haskell ([Testcontainers: Getting Started](https://testcontainers.com/getting-started/)). - **A fresh service for each run.** Test code starts a container for each test or test class. A sidecar container, Ryuk, removes the containers after the tests. "Even if you do not call Terminate, Ryuk ensures that the environment will be kept clean" ([Garbage Collector](https://golang.testcontainers.org/features/garbage_collector/)). - **Healthy before the tests start.** Each language has modules for common services, with a wait strategy for each service ([Wait strategies](https://golang.testcontainers.org/features/wait/introduction/)). - **A known address.** Testcontainers maps each container port to a random free port on the host ([Networking](https://java.testcontainers.org/features/networking/)). The module gives the address to the test. In the Go MySQL module, `mysql.Run` takes an image and options for the database name, the credentials, and init scripts. The container that it returns has a `ConnectionString` method ([MySQL module](https://golang.testcontainers.org/modules/mysql/)). - **The same setup locally and in CI.** The same test code runs on a laptop and in CI. Each machine that runs the tests needs a container runtime. Add the modules as test dependencies in each language: ```shell # Go go get github.com/testcontainers/testcontainers-go/modules/mysql # Node.js npm install --save-dev @testcontainers/kafka @testcontainers/mongodb ``` In Java, add the dependency to `pom.xml`: ```xml org.testcontainers testcontainers-kafka 2.0.5 test ``` Testcontainers fits best when a test or a test class controls the lifecycle of its own container from code. ## Meet the requirements with Docker Compose Docker Compose starts the services before the test command, so the tests contain no container code. One Compose file serves every language in the repository. - **A fresh service for each run.** `docker compose down` stops and removes the containers that `up` created ([docker compose down](https://docs.docker.com/reference/cli/docker/compose/down/)). `docker compose up` recreates an old container only when its configuration or image changed ([docker compose up](https://docs.docker.com/reference/cli/docker/compose/up/)). Thus a run that skips `down` can leave old data for the next run. - **Healthy before the tests start.** A health check in the file tells Compose when a service is ready. `docker compose up --wait` waits for the services "to be running|healthy" ([docker compose up](https://docs.docker.com/reference/cli/docker/compose/up/)). - **A known address.** The file maps a host port, such as `5432:5432`, and the tests connect to `localhost:5432`. A fixed host port collides when two runs share a machine. - **The same setup locally and in CI.** The same file runs on each machine that has Docker, a laptop or a CI runner. This `compose.yaml` starts Postgres with a health check: ```yaml services: postgres: image: postgres:17-alpine environment: POSTGRES_PASSWORD: test ports: - "5432:5432" healthcheck: test: ["CMD", "pg_isready", "-U", "postgres"] interval: 2s retries: 15 ``` Start the services, run the tests, then remove the services: ```shell docker compose up --wait go test ./... docker compose down ``` Compose fits a repository where tests in several languages use the same services on one machine. ## Meet the requirements in CI with GitHub Actions service containers CI platforms start service containers before the steps of a job. In GitHub Actions, the `services:` block of a job defines them. - **A fresh service for each run.** GitHub creates a new container for each service and removes it when the job completes ([Communicating with Docker service containers](https://docs.github.com/en/actions/tutorials/use-containerized-services/use-docker-service-containers)). - **Healthy before the tests start.** The `--health-*` options make sure that the service runs before the steps start ([Creating PostgreSQL service containers](https://docs.github.com/en/actions/tutorials/use-containerized-services/create-postgresql-service-containers)). - **A known address.** A job that runs in a container reaches the service by its label, such as `postgres`. A job that runs on the runner machine maps the ports and uses `localhost:` ([Communicating with Docker service containers](https://docs.github.com/en/actions/tutorials/use-containerized-services/use-docker-service-containers)). - **The same setup locally and in CI.** Service containers run only in CI. A laptop needs a second definition, such as a Compose file. This `services:` block starts Postgres with a health check: ```yaml services: postgres: image: postgres env: POSTGRES_PASSWORD: postgres options: >- --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5 ``` GitHub Actions service containers need a Linux runner. The same page says: "If your workflows use Docker container actions, job containers, or service containers, then you must use a Linux runner." Service containers fit tests that run only in GitHub Actions, or a team that also keeps a Compose file for laptops. ## Which requirements each approach meets | Requirement | Testcontainers | Docker Compose | GitHub Actions service containers | | --- | --- | --- | --- | | A fresh service for each run | A container for each test or test class. Ryuk removes it. | `up` creates the containers. `down` removes them. | A new container for each job, removed when the job completes | | Healthy before the tests start | A wait strategy in each module | `healthcheck` with `up --wait` | `--health-*` options | | A known address | A random free host port, from the module | A fixed host port, which collides on a shared machine | The label in a container job, or a mapped `localhost` port | | The same setup locally and in CI | Yes, with a container runtime on each machine | Yes, where CI also runs Compose | CI only, on a Linux runner | | Fits when | A test controls its own container from code | Tests in several languages share services on one machine | Tests run only in GitHub Actions | ## Where the hand-built setup strains as the project grows As a project grows, each requirement gets harder to meet with one definition of each service. **The same setup locally and in CI.** A team that uses Compose on laptops and `services:` in CI has two definitions of Postgres. The image tags, environment variables, and health checks drift apart. Then a test that passes on a laptop can fail in CI. Testcontainers keeps one definition for each language. A repository with a Go service, a Node service, and a Java consumer configures Kafka three times, through three libraries. Each CI job that runs those tests also needs a Docker-compatible runtime. Colima, Podman, and Rancher Desktop each need extra environment variables ([Supported container runtimes](https://java.testcontainers.org/supported_docker_environment/)). For tests inside a container, "the docker socket must be available via a volume mount". The source directory must also have the same path inside the container as on the host. Some CI systems need Docker-in-Docker, which the same docs call "an instrument of last resort" ([Patterns for running tests inside a Docker container](https://java.testcontainers.org/supported_docker_environment/continuous_integration/dind_patterns/)). Images can also disappear from registries. The MinIO community edition became "a source only distribution" in October 2025 ([MinIO issue #21647](https://github.com/minio/minio/issues/21647)), and its repository is now archived. In September 2026, pulls of `minio/minio` from Docker Hub started to fail. A post from StableBuild recommended the same tags on `quay.io/minio/minio` ([StableBuild](https://www.stablebuild.com/blog/minio-images-disappeared-from-docker-hub)). In October 2026, a pull from Quay for this guide also failed: ```shell docker pull quay.io/minio/minio:RELEASE.2025-09-07T16-13-09Z ``` ```text Error response from daemon: unknown: failed to resolve reference "quay.io/minio/minio:RELEASE.2025-09-07T16-13-09Z": unexpected status from HEAD request to https://quay.io/v2/minio/minio/manifests/RELEASE.2025-09-07T16-13-09Z: 401 Unauthorized ``` When an image disappears, each file that names the image must change. **A known address.** Fixed host ports, such as `5432:5432` in the Compose file above, collide when two runs share a machine. **Healthy before the tests start.** Each definition has its own health check. A health check that is too weak lets the tests start before the service accepts connections. ## Two ways to run test services with Dagger Dagger supports both models of the hand-built tools: - **The Compose model.** One Dagger module defines the services, outside the tests. The tests reach each service by hostname and environment variable. The next sections show this model. - **The Testcontainers model.** The test code starts its own services through a Dagger client, and you run the tests with your test command. [Start services from test code with a Dagger client](#start-services-from-test-code-with-a-dagger-client) shows this model. Both models meet the same four requirements, with the same service lifecycle and health checks. ## Define the services once in a Dang module In the Compose model, Dagger meets the four requirements with services that one Dagger module defines, outside the tests. A [service](https://docs.dagger.io/using/services) in Dagger is a container that runs for as long as a client needs it, such as a database. The tests reach each service by a hostname, and they read connection details from environment variables. The test code does not start containers. The services go in a [Dagger module](https://docs.dagger.io/reference/sdks), a package of functions, checks, and services that a project can run. A [check](https://docs.dagger.io/using/checking) is a function that validates the project, and `dagger check` runs every check. The steps below need a Dagger workspace, the `dagger.toml` file that holds the configuration of the project. If the project has no `dagger.toml`, do the [Quickstart](https://docs.dagger.io/getting-started/quickstart) first. The project must be a Git repository. Write the services in the dev module of the project, a module in [Dang](https://docs.dagger.io/reference/sdks/dang). Dang is the native language of Dagger for modules, with no code generation and no build step. Install the Dang SDK and create the module: ```shell dagger module install dagger.io/sdk/dang dagger module init dang ``` `dagger module init` creates the module at `.dagger/modules/-dev` and makes it the [workspace main](https://docs.dagger.io/config/module-wiring#reference-the-workspace-main). The workspace main is the module whose functions you can call without the module name. The command returns a [changeset](https://docs.dagger.io/using/generating), a diff that Dagger shows for review before it writes files. A headless run needs `-y`. Replace `.dagger/modules/-dev/main.dang` with the services that your tests need. `ProjectDev` stands in for the type name that `dagger module init` generated. Keep the type name of your module, or the module fails to load with `no main object`. The output in this guide comes from a project named `orders`, so its dev module is `orders-dev`. ```dang """ Services for integration tests and local development. """ type ProjectDev { pub source: Directory! new(ws: Workspace!) { self.source = ws.directory("/", exclude: ["node_modules", ".git", "dist", "target"]) self } """ A Postgres database. """ pub postgres: Service! { container .from("postgres:17-alpine") .withEnvVariable("POSTGRES_USER", "test") .withEnvVariable("POSTGRES_PASSWORD", "test") .withEnvVariable("POSTGRES_DB", "test") .withExposedPort(5432) .asService } """ A MySQL database. """ pub mysql: Service! { container .from("mysql:8.4") .withEnvVariable("MYSQL_ROOT_PASSWORD", "test") .withEnvVariable("MYSQL_DATABASE", "test") .withEnvVariable("MYSQL_USER", "test") .withEnvVariable("MYSQL_PASSWORD", "test") .withExposedPort(3306) .asService } """ A single-node Kafka broker in KRaft mode, advertised as kafka:9092. """ pub kafka: Service! { container .from("apache/kafka:4.1.0") .withEnvVariable("KAFKA_NODE_ID", "1") .withEnvVariable("KAFKA_PROCESS_ROLES", "broker,controller") .withEnvVariable("KAFKA_LISTENERS", "PLAINTEXT://:9092,CONTROLLER://:9093") .withEnvVariable("KAFKA_ADVERTISED_LISTENERS", "PLAINTEXT://kafka:9092") .withEnvVariable("KAFKA_CONTROLLER_LISTENER_NAMES", "CONTROLLER") .withEnvVariable("KAFKA_LISTENER_SECURITY_PROTOCOL_MAP", "CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT") .withEnvVariable("KAFKA_CONTROLLER_QUORUM_VOTERS", "1@localhost:9093") .withEnvVariable("KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR", "1") .withEnvVariable("KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR", "1") .withEnvVariable("KAFKA_TRANSACTION_STATE_LOG_MIN_ISR", "1") .withEnvVariable("KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS", "0") .withDockerHealthcheck( ["/opt/kafka/bin/kafka-broker-api-versions.sh", "--bootstrap-server", "localhost:9092"], interval: "2s", retries: 30, ) .withExposedPort(9092) .asService } """ A MongoDB server. """ pub mongo: Service! { container .from("mongo:8") .withExposedPort(27017) .asService } } ``` One definition of each service now serves every test in the repository, on a laptop and in CI. ## A fresh service for each run, with the Dagger service lifecycle Dagger runs each service in an ephemeral container for the run that needs it ([Running services](https://docs.dagger.io/using/services)). A service starts when a client first needs it. It stops when no client references it. Thus the tests need no teardown code, as with Ryuk in Testcontainers. The `SERVICES` part of each `dagger check` output below lists each service as `EXITED` after the run. Clients in one run that request the same service share one instance. Testcontainers can give each test class its own container. Here, the Go, Node, and Java tests share one Kafka broker in a run. Give each test its own database, topic, or collection name. Do not expect an empty server. ## Healthy before the tests start, with Dagger health checks Dagger health-checks a service before any client can connect, so the tests cannot start before the service is ready ([Running services](https://docs.dagger.io/using/services)). Each service in the module becomes ready at a different point: - **Postgres, MySQL, and MongoDB** are ready when the Dagger health check on their exposed port passes. The MySQL image runs its init scripts on a temporary server with `--skip-networking`. Thus port 3306 opens only when the real server starts ([MySQL entrypoint](https://github.com/docker-library/mysql/blob/master/8.4/docker-entrypoint.sh)). - **Kafka** has a Docker health check, set with `withDockerHealthcheck` ([Container API](https://docs.dagger.io/reference/api/container)). It is the same kind of check as a `healthcheck` in Compose or the `--health-*` options in GitHub Actions. A Docker health check replaces the default port check of a service ([changelog](https://dagger.io/changelog)). Thus the tests start only after the broker answers Kafka requests. ## A known address for Go tests, with the Go module's test runtime Each Dagger service gets a hostname from its definition, so services have no port conflicts ([Running services](https://docs.dagger.io/using/services)). A test container reaches a service by the name that `withServiceBinding` gives it, such as `db` or `kafka` ([Container API](https://docs.dagger.io/reference/api/container)). The advertised listener of Kafka is `kafka:9092`, so each container that talks to Kafka must bind the service as `kafka`. The module sets every KRaft variable. The `apache/kafka` image says: "if you are overriding any configuration, then none of the default configurations will be used" ([apache/kafka on Docker Hub](https://hub.docker.com/r/apache/kafka/)). The [Dagger module for Go](https://docs.dagger.io/reference/modules/go), `dagger.io/go`, runs the Go tests. Your dev module supplies the container that the tests run in, with the services bound. Install the Go module: ```shell dagger module install dagger.io/go ``` Add a `testRuntime` function to the `ProjectDev` type in `.dagger/modules/-dev/main.dang`. It binds the services and sets the variables that the tests read: ```dang """ The Go test runtime, with Postgres and Kafka bound. """ pub testRuntime: Container! { container .from("golang:1.26-bookworm") .withServiceBinding("db", postgres) .withServiceBinding("kafka", kafka) .withEnvVariable("DATABASE_URL", "postgres://test:test@db:5432/test?sslmode=disable") .withEnvVariable("KAFKA_BROKERS", "kafka:9092") } ``` Connect the function to the `base` setting of the Go module with [module wiring](https://docs.dagger.io/config/module-wiring). Module wiring puts the value that a function returns into the setting of another module. Then run the tests: ```shell dagger module settings go base test-runtime dagger check --go --check=test ``` ```text == CHECKS == ✔ 2 passed ✔ dag://?check=test&go-package=events 1m15s OK == TESTS == ✔ example.com/orders/events 1 passed ✔ 1 passed ✔ dag://?check=test&go-package=store 1m15s OK == TESTS == ✔ example.com/orders/store 1 passed ✔ 1 passed == SERVICES == ✔ ku62l0th05qka.h4eqn5sbc01u2.dagger.local exec docker-entrypoint.sh postgres 51.7s EXITED span=159734715acebc1d ✔ 08husvm992r7i.h4eqn5sbc01u2.dagger.local exec /__cacert_entrypoint.sh /etc/kafka/docker/run 43.4s EXITED span=210cbf590ee489ef ``` The settings command writes `base = "dag://-dev/test-runtime"` to `dagger.toml`. Each container that the Go module makes from this base keeps the service bindings. Thus Postgres and Kafka run whenever `go test` runs. The image supplies the Go toolchain, and the Go module adds its test runner to the image. When `base` is set, the Go module ignores its `version` setting. If you set `version` before, remove it with `dagger module settings -u go version`. [Daggerize a Go Project](https://docs.dagger.io/getting-started/platform-guides/go) shows this wiring step by step. Make each test skip when its variable is not set. Then plain `go test ./...` still passes on a laptop without the services. Dagger reports those tests as skipped, not as failed. ## A known address for Node and Java tests, with a check For Node and Java tests, write a check in the dev module that builds the test container. The tests reach bound services only through the container that they run in. The [Vitest](https://docs.dagger.io/reference/modules/js/vitest) and [Jest](https://docs.dagger.io/reference/modules/js/jest) modules build their Node container from a base image name and environment variables, with no service bindings. Thus your own check binds the services, sets the variables, and runs the test command of the project. Add these functions to the `ProjectDev` type in `.dagger/modules/-dev/main.dang`: ```dang """ Node integration tests against Kafka and MongoDB. """ pub nodeIntegration: Void @check { container .from("node:22-bookworm") .withServiceBinding("kafka", kafka) .withServiceBinding("mongo", mongo) .withEnvVariable("KAFKA_BROKERS", "kafka:9092") .withEnvVariable("MONGO_URL", "mongodb://mongo:27017/test") .withMountedCache("/root/.npm", cacheVolume("npm")) .withDirectory("/app", source) .withWorkdir("/app") .withExec(["npm", "ci"]) .withExec(["npm", "run", "test:integration"]) .sync null } """ Java integration tests against Kafka and MySQL. """ pub javaIntegration: Void @check { container .from("maven:3.9-eclipse-temurin-21") .withServiceBinding("kafka", kafka) .withServiceBinding("mysql", mysql) .withEnvVariable("KAFKA_BOOTSTRAP_SERVERS", "kafka:9092") .withEnvVariable("JDBC_URL", "jdbc:mysql://mysql:3306/test") .withMountedCache("/root/.m2", cacheVolume("maven")) .withDirectory("/app", source) .withWorkdir("/app") .withExec(["mvn", "-B", "verify"]) .sync null } ``` A check passes when it completes without an error. It fails when a `withExec` command exits with a non-zero code. Run one check by its address: ```shell dagger check dag://-dev/node-integration ``` ```text == CHECKS == ✔ 1 passed ✔ dag://?check=node-integration 41.3s OK == SERVICES == ✔ 08husvm992r7i.de48876uvsd0e.dagger.local exec /__cacert_entrypoint.sh /etc/kafka/docker/run 25.0s EXITED span=79cf5a83ea3ddf9d ✔ d0rur6uthqsjm.de48876uvsd0e.dagger.local exec docker-entrypoint.sh mongod 12.8s EXITED span=ba41cd67698295ed ``` ## The same setup locally and in CI, with Cloud Checks `dagger check` runs the tests the same way on a laptop and in CI, because the services, images, and environment variables come from the dev module. Run every check locally: ```shell dagger check ``` ```text == CHECKS == ✔ 6 passed ✔ dag://?check=dang-sdk/generate/stale 1.3s OK ✔ dag://?check=go/packages/generate/stale 10.7s OK ✔ dag://?check=java-integration 10.8s OK ✔ dag://?check=node-integration 10.7s OK ✔ dag://?check=test&go-package=events 10.7s OK == TESTS == ✔ example.com/orders/events 1 passed ✔ 1 passed ✔ dag://?check=test&go-package=store 10.7s OK == TESTS == ✔ example.com/orders/store 1 passed ✔ 1 passed == SERVICES == ✔ d0rur6uthqsjm.tnefdqvqu78tc.dagger.local exec docker-entrypoint.sh mongod 10.5s EXITED span=bb6b3c68fdeb9fee ✔ hde2ut6iqp6ji.tnefdqvqu78tc.dagger.local exec docker-entrypoint.sh mysqld 10.6s EXITED span=3db822a6ad9a76ff ✔ 08husvm992r7i.tnefdqvqu78tc.dagger.local exec /__cacert_entrypoint.sh /etc/kafka/docker/run 10.5s EXITED span=9757895b6ac9879a ✔ ku62l0th05qka.tnefdqvqu78tc.dagger.local exec docker-entrypoint.sh postgres 9.9s EXITED span=3a730798f45179c0 ``` Kafka appears once under `SERVICES`, because the Go, Node, and Java tests share one broker. The Dang SDK adds a drift check for the generated files of the dev module. The Go module adds a drift check for `go generate`. Commit `dagger.toml`, `dagger.lock`, and `.dagger`. `dagger.lock` pins each image that the workspace uses, so CI pulls the same images that you tested with. In CI, [Cloud Checks](https://docs.dagger.io/cloud-checks) run the same checks. Cloud Checks run your checks on Dagger Cloud after each push and pull request, with no CI workflow file. The repository must be on GitHub, and its `origin` remote must point there. Run this command from the project root: ```shell dagger cloud checks on ``` After that, each push and pull request runs these checks, with their services, on Dagger's Cloud Engines. `dagger workspace activity` shows the recent runs. ## Use the same services from the host with dagger start `dagger start` starts the services of the workspace and tunnels their ports to your machine ([Running services](https://docs.dagger.io/using/services)). Tools and tests on the host then use the same definitions as `dagger check`. The functions that return `Service!` in the dev module appear in its list: ```shell dagger start -l ``` ```text MODULE SERVICE DESCRIPTION orders-dev kafka A single-node Kafka broker in KRaft mode, advertised as kafka:9092. orders-dev mongo A MongoDB server. orders-dev mysql A MySQL database. orders-dev postgres A Postgres database. ``` Start the services that you need by name: ```shell dagger start postgres mongo ``` Clients on your machine then connect through `localhost`. Kafka is different, because the broker advertises `kafka:9092`. A client on the host gets that address from the broker and cannot resolve it. A Go test on the host failed with this error: ```text kafka.(*Client).Produce: dial tcp: lookup kafka: no such host ``` To reach Kafka from the host, add a second service function with `localhost:9092` as its advertised listener. ## Start services from test code with a Dagger client In the Testcontainers model, the test starts its own services. A Go test calls the Dagger API through the client library `dagger.io/dagger` ([Client libraries](https://docs.dagger.io/reference/client-libraries)). Add its `dag` package at the version of your Dagger CLI. With the package path, `go get` also adds the modules that `dag` imports: ```shell go get dagger.io/dagger/dag@v1.0.0-beta.20 ``` Put this test in `store/dagger_test.go`. It starts Postgres, opens a tunnel to it from the host, and tests against it: ```go package store import ( "context" "testing" "dagger.io/dagger/dag" "github.com/jackc/pgx/v5" ) func TestOrdersWithDaggerPostgres(t *testing.T) { ctx := context.Background() t.Cleanup(func() { dag.Close() }) postgres := dag.Container(). From("postgres:17-alpine"). WithEnvVariable("POSTGRES_USER", "test"). WithEnvVariable("POSTGRES_PASSWORD", "test"). WithEnvVariable("POSTGRES_DB", "test"). WithExposedPort(5432). AsService() db, err := dag.Host().Tunnel(postgres).Start(ctx) if err != nil { t.Fatal(err) } addr, err := db.Endpoint(ctx) if err != nil { t.Fatal(err) } conn, err := pgx.Connect(ctx, "postgres://test:test@"+addr+"/test") if err != nil { t.Fatal(err) } defer conn.Close(ctx) if _, err := conn.Exec(ctx, "CREATE TABLE orders (id serial PRIMARY KEY, item text)"); err != nil { t.Fatal(err) } var id int err = conn.QueryRow(ctx, "INSERT INTO orders (item) VALUES ('widget') RETURNING id").Scan(&id) if err != nil { t.Fatal(err) } if id != 1 { t.Fatalf("id = %d, want 1", id) } } ``` Run the test with your test command, inside a Dagger session. `dagger api with-session` starts the session and gives it to the client: ```shell dagger api with-session go test ./store/ -run TestOrdersWithDaggerPostgres -v ``` ```text === RUN TestOrdersWithDaggerPostgres --- PASS: TestOrdersWithDaggerPostgres (0.72s) PASS ok example.com/orders/store 0.725s ``` The test meets the four requirements from its own code: - **A fresh service for each run.** The service starts when the test needs it, and stops when the client closes. A second run created the `orders` table again and got `id` 1 again. - **Healthy before the tests start.** `Start` returns after the Dagger health check passes on the exposed port ([Running services](https://docs.dagger.io/using/services)). - **A known address.** The tunnel opens a port on the host, and `Endpoint` returns its address to the test. - **The same setup locally and in CI.** The only external dependency of the test is Dagger ([self-orchestrating tests](https://dagger.io/changelog/self-orchestrating-tests)). ## Choose the Compose model or the Testcontainers model Dagger supports both models, so the choice is where each service definition lives: - **The Compose model** puts one definition in a Dagger module. Choose it when several languages or projects use the same services. The tests read environment variables and have no container code. - **The Testcontainers model** puts the definition in the test. Choose it when a test controls its own service, such as a new database for each test class. The Testcontainers library differs from a Dagger client in two ways. It ships a module for each common service, with a wait strategy already set ([Wait strategies](https://golang.testcontainers.org/features/wait/introduction/)). With a Dagger client, you write the container definition. Testcontainers also runs under the plain test command. The Dagger client library runs inside a Dagger session, such as one from `dagger api with-session` ([Client libraries](https://docs.dagger.io/reference/client-libraries)). **Bottom line:** To test without mocks, give each test run a fresh database or Kafka container. It must be healthy before the tests start, at a known address, and the same locally and in CI. Testcontainers meets these requirements from test code, and Docker Compose or CI service containers meet them from outside the tests. Dagger does both: define the services once in a module, or start them from test code through a Dagger client. ## Related guides - [How do I run Go tests, lint, and go generate the same way locally and in CI?](/learn/daggerize-go-project): the Go module, a custom test runtime, and a Postgres service, step by step. - [How do I run Playwright tests in CI and locally in the same container, with cached browsers?](/learn/playwright-ci-docker-browser-caching): run browser tests in one container on a laptop and in CI. - [How do I migrate from GitHub Actions to Dagger?](/learn/migrate-from-github-actions): move workflows, such as jobs with service containers, to Dagger checks. --- # What are the alternatives to GitHub Actions, from faster runners to portable pipelines? URL: https://dagger.io/learn/github-actions-alternatives The alternatives to GitHub Actions change one of three layers of CI. Runner vendors such as Blacksmith, Depot, Namespace, WarpBuild, and RunsOn keep GitHub Actions and run its jobs on faster or cheaper machines. Other CI platforms, such as Buildkite, CircleCI, and Depot CI, replace GitHub Actions as the system that starts and schedules jobs. Dagger moves the build and test logic out of the CI configuration and into code that runs the same on a laptop and in CI. Its [Cloud Checks](https://docs.dagger.io/cloud-checks) run that code after every push and pull request, with no workflow file and no runners to manage. ## Three layers that the alternatives change Each kind of alternative changes a different layer of your CI, so each one fixes a different problem. - **Runner vendors** keep GitHub Actions as the orchestrator and run your jobs on their machines. Blacksmith, Depot, Namespace, and WarpBuild host the runners. RunsOn starts them in your own AWS account. To move a job, you change its `runs-on:` label. - **Other CI platforms** replace GitHub Actions as the system that triggers and schedules jobs. Buildkite and CircleCI have their own pipeline formats. Depot CI runs GitHub Actions YAML on its own orchestrator. - **A portable pipeline** moves the build and test logic into code that runs the same on a laptop and in CI. Dagger works at this layer. Cloud Checks run that code on Dagger's servers, so Dagger replaces both the workflow files and the runners. ## Criteria to compare GitHub Actions alternatives Compare the alternatives on the problem that you want to fix. These criteria separate them: - **Price per minute, and how each vendor counts a minute.** GitHub rounds each job up to a whole minute ([runner pricing](https://docs.github.com/en/billing/reference/actions-runner-pricing)). Depot tracks usage by the second. Namespace bills in "unit minutes", which it weights by machine size. - **Speed.** Speed comes from the CPU, the speed of cache transfers, and whether a cache stays available between jobs. - **Migration effort.** A label change takes minutes. A move to a new pipeline format takes much longer. - **Lock-in.** Lock-in is the vendor-specific actions, labels, and config files that you must remove to leave. - **Local reproduction.** This is whether a developer or a coding agent can run the same checks before a push. - **Who operates the infrastructure.** The vendor operates the machines, or you operate them in your own cloud account. ## Runner vendors that replace GitHub-hosted runners Blacksmith, Depot, Namespace, WarpBuild, and RunsOn run your current workflow files on their runners. You move a job to one of them with a change to its `runs-on:` label. The baseline is GitHub's own price. A [GitHub-hosted](https://docs.github.com/en/billing/reference/actions-runner-pricing) Linux x64 runner with 2 cores costs $0.006 a minute. The 4-core runner costs $0.012 a minute, and it is a larger runner for organizations on the Team or Enterprise Cloud plan. The table sums up the next two sections. It shows list prices for Linux x64 runners, from each vendor's pricing page. Prices change, so follow the links in those sections before you decide. | Runner | Linux x64 price per minute | Example `runs-on:` label | Where the machines run | | --- | --- | --- | --- | | GitHub-hosted | $0.006 for 2 cores | `ubuntu-latest` | GitHub | | Blacksmith | $0.004 for 2 vCPU | `blacksmith-2vcpu-ubuntu-2404` | Blacksmith | | Depot | $0.006 for 2 vCPU | `depot-ubuntu-24.04` | Depot, or your AWS account on the Business plan | | Namespace | $0.002 prepaid or $0.003 overage, for 2 vCPU and 4 GB | `namespace-profile-` | Namespace's own hardware | | WarpBuild | $0.004 for 2 vCPU and 8 GB, or $0.002 plus your cloud bill on BYOC | `warp-ubuntu-latest-x64-2x` | WarpBuild, or your AWS, GCP, or Azure account on BYOC | | RunsOn | About $0.0009 for 2 vCPU, paid to AWS at spot prices, plus a license | `runs-on=${{ github.run_id }}/family=m8+r8/cpu=4` | Your AWS account | ## Hosted runners: Blacksmith, Depot, Namespace, and WarpBuild Blacksmith, Depot, Namespace, and WarpBuild host the runners, so you have no machines to operate. Each one focuses on a different part of the job. - **Blacksmith** sells faster hardware and a colocated cache. A 2 vCPU Linux runner costs $0.004 a minute, and the first 3,000 minutes each month are free ([pricing](https://www.blacksmith.sh/pricing)). To move a job, replace a label such as `ubuntu-latest` with `blacksmith-2vcpu-ubuntu-2404`, or let the migration wizard do it. On Blacksmith, official and popular third-party cache actions, such as `actions/cache`, use the colocated cache automatically. Blacksmith's own Docker actions add layer caching, which costs $0.50 per GB a month ([Docker builds](https://docs.blacksmith.sh/blacksmith-caching/docker-builds)). Blacksmith works with GitHub organizations, not personal repositories ([quickstart](https://docs.blacksmith.sh/introduction/quickstart)). - **Depot** runners cost $0.006 a minute for 2 vCPU. Plans cost $20 or $200 a month and include 2,000 or 20,000 minutes ([pricing](https://depot.dev/pricing)). Depot tracks usage by the second and bills whole minutes at the end of the month. A job selects a runner with a label such as `depot-ubuntu-24.04`. The runners use Depot's distributed cache, and if you also use Depot container builds, the BuildKit builder runs next to the runner ([docs](https://depot.dev/docs/github-actions/overview)). On the Business plan, Depot can deploy the runners in your own AWS account. - **Namespace** runs on hardware that it deploys and manages itself ([compute](https://namespace.so/docs/architecture/compute)). A Linux runner with 2 vCPU and 4 GB costs $0.002 a minute prepaid, or $0.003 as overage. One unit minute is 1 vCPU and 2 GB of memory for one minute, times a platform multiplier. The Team plan costs $100 a month and includes 100,000 unit minutes ([pricing](https://namespace.so/pricing)). You choose a machine shape in a runner profile, and the workflow selects the profile with a label such as `namespace-profile-`. Namespace also offers Cache Volumes, remote Docker builders, and macOS runners on Apple Silicon ([docs](https://namespace.so/docs/solutions/github-actions)). - **WarpBuild** hosts runners for Linux, Windows, and macOS, and a job selects one with a label such as `warp-ubuntu-latest-x64-2x` ([cloud runners](https://www.warpbuild.com/docs/ci/cloud-runners)). A Linux runner with 2 vCPU and 8 GB costs $0.004 a minute ([pricing](https://www.warpbuild.com/pricing)). On the hosted Ubuntu runners, a job can boot from a VM snapshot that an earlier run saved ([snapshot runners](https://www.warpbuild.com/docs/ci/features/snapshot-runners)). A hosted runner vendor gives the quickest result when your workflows work and the problem is the bill or the wait time. ## Runners in your own cloud account: RunsOn and WarpBuild BYOC RunsOn and the WarpBuild BYOC plan put the runners in a cloud account that you manage. They cost less per minute. In exchange, someone on your team installs them and looks after the cloud side. - **RunsOn** installs as one CloudFormation stack in your AWS account. Jobs run on EC2 spot instances by default, and AWS bills you for the compute directly. Your code stays in your VPC ([RunsOn](https://runs-on.com/)). A job selects a runner with a label such as `runs-on=${{ github.run_id }}/family=m8+r8/cpu=4`. A 2 vCPU runner costs about $0.0009 a minute at spot prices, as a 7-day average in us-east-1. The license starts at €300 a year, and it is free for open-source projects, non-profits, and personal non-commercial use ([pricing](https://runs-on.com/pricing/)). - **WarpBuild BYOC** runs WarpBuild runners in your own AWS, GCP, or Azure account. WarpBuild charges $0.002 per runner minute for Linux, and your cloud provider bills the machines ([BYOC](https://www.warpbuild.com/byoc)). ## Lock-in with a runner vendor Lock-in with a runner vendor comes from vendor-specific actions and settings, not from the label change. If a workflow changes only `runs-on:`, it can move to another vendor, or back to GitHub-hosted runners, with a change to that one line. Some speedups need vendor actions, and you must replace each one to leave: - Blacksmith's Docker layer caching uses `useblacksmith/setup-docker-builder` and `useblacksmith/build-push-action` in place of the Docker actions. - Namespace's Git mirror checkouts and framework caches use `namespacelabs/nscloud-checkout-action` and `namespacelabs/nscloud-cache-action`. - RunsOn labels carry their own settings, such as `family=`, `cpu=`, and `extras=s3-cache`. A runner change can also break required status checks. Branch protection rules require status checks by name. When rules require specific checks, GitHub asks for job names that are unique across workflows ([protected branches](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches)). If the `name:` of a required job contains the runner label, such as through `${{ matrix.os }}`, a new label renames the check. No check reports under the old name, so GitHub blocks the pull request with "Waiting for status to be reported" ([troubleshooting required status checks](https://docs.github.com/en/pull-requests/how-tos/merge-and-close-pull-requests/troubleshooting-required-status-checks)). Keep runner labels out of the names of required jobs. With every runner vendor, the pipeline logic stays in GitHub Actions YAML. ## Other CI platforms that replace GitHub Actions A move to another CI platform replaces the orchestrator. On Buildkite or CircleCI, you rewrite your pipeline in their format. - **Buildkite** runs pipelines on [agents](https://buildkite.com/docs/pipelines/architecture). You can host the agents yourself or use Buildkite's [hosted agents](https://buildkite.com/docs/agent/buildkite-hosted). Its [dynamic pipelines](https://buildkite.com/docs/pipelines/configure/dynamic-pipelines) generate steps at build time, and it includes [test splitting](https://buildkite.com/docs/pipelines/speed-up-builds-with-bktec). The Pro plan costs $30 per active user per month. The Free plan covers up to 5 users ([pricing](https://buildkite.com/pricing)). [How does Dagger compare to Buildkite?](/learn/dagger-vs-buildkite) compares the two. - **CircleCI** bills in credits, which pay for compute, active users, and premium features. The Free plan includes up to 6,000 build minutes and 5 active users. Every plan includes [test splitting](https://circleci.com/docs/guides/optimize/parallelism-faster-jobs/) and [test selection](https://circleci.com/docs/guides/test/smarter-testing-overview/), with a monthly allowance for test selection ([pricing](https://circleci.com/pricing/)). - **Depot CI** runs GitHub Actions YAML on Depot's own orchestrator and compute. Its migrate wizard, `depot ci migrate`, copies your workflows into a `.depot/` directory. Your `.github/` workflows also continue to run on GitHub until you turn them off. You can run a workflow against your local working tree, with uncommitted changes. You can also connect over SSH to the sandbox of a job ([docs](https://depot.dev/docs/ci/overview)). A new platform makes sense when you want a feature that GitHub Actions does not have. Examples are CircleCI's test selection and Depot CI's runs from your working tree. On each of these platforms, a CI configuration format still holds the pipeline. ## Dagger: a portable pipeline that runs locally and in CI Dagger moves the build and test logic out of the CI configuration, so the CI only has to start it. In the words of the [Dagger docs](https://docs.dagger.io/getting-started/introduction), after you move a pipeline to Dagger, "your pipeline logic is decoupled from its environment". You can start it before or after a push, on your laptop or in Dagger's cloud. These parts do the work: - **Modules** supply the checks. A [Dagger module](https://docs.dagger.io/reference/sdks) is a reusable, typed API for automation, such as functions that test or lint a project. [Official modules](https://docs.dagger.io/reference/modules) cover tools such as Go, golangci-lint, Jest, Vitest, Playwright, ESLint, Pytest, and Helm. For anything else, you write a module in [Dang](https://docs.dagger.io/reference/sdks/dang), Dagger's native language for modules, or with an SDK such as Go, TypeScript, or Python. - **Checks** validate the project. A [check](https://docs.dagger.io/reference/sdks) is a module function for a test, a linter, a scan, or a policy. [`dagger check`](https://docs.dagger.io/using/checking) runs every check in the workspace in parallel. The docs say that its results are "identical on your laptop, in CI, and in the cloud". - **The Dagger Engine** runs each pipeline as a graph of steps. It caches each step automatically and traces each run from start to end ([introduction](https://docs.dagger.io/getting-started/introduction)). - **Cloud Checks** run `dagger check` on [Cloud Engines](https://dagger.io/changelog/cloud-engines), the Dagger engines that Dagger Cloud hosts and manages, after each push and pull request. Measured against the criteria above, Dagger compares like this: - **Price per minute.** Dagger Cloud does not bill per runner minute. It bills the CPU, memory, and data that your pipeline uses (see the pricing section below). - **Speed.** The engine caches each step, and runs a step again only when its inputs change. - **Migration effort.** A move to Dagger takes more work than a label change, because your checks run through Dagger modules. Official modules cover common tools. For other tools, find a module in the [module registry](https://docs.dagger.io/reference/cli#dagger-module-search) with `dagger module search`. Or wrap the command you already run in a short module of your own. - **Lock-in.** The checks live in modules in your repository, not in CI configuration. They run on a laptop, on your own engine, or on Dagger Cloud. - **Local reproduction.** Developers and agents run the same `dagger check` before a push that Cloud Checks run after it. When a check fails in CI, you run the same check on your laptop. - **Who operates the infrastructure.** Dagger Cloud operates the Cloud Engines. The open-source engine also runs on your machine or your own server ([dagger.io](https://dagger.io/)). ## Move from GitHub Actions to Dagger To move to Dagger, set up Dagger in the repository, run the checks locally, and then turn on Cloud Checks. 1. [Install the Dagger CLI](https://docs.dagger.io/getting-started/install). 2. From the project root, run these commands: ```shell dagger init dagger module recommend dagger check -l dagger check ``` `dagger init` creates `dagger.toml`, or uses the file if it exists. That file holds the workspace configuration for everyone who uses the repository. `dagger module recommend` finds modules that match the project and asks which ones to install. When Dagger installs a module, it also writes `dagger.lock`, which pins the images and Git refs that the workspace uses. Commit both files. `dagger check -l` lists the checks, and `dagger check` runs them. In a small Go project, `dagger module recommend` installed the official Go module, `dagger.io/go`. Then `dagger check` printed this summary: ```text == CHECKS == ✔ 2 passed ✔ dag://?check=go/packages/generate/stale 0.6s OK ✔ dag://?check=test&go-package=. 0.9s OK == TESTS == ✔ example.com/greet 1 passed ✔ 1 passed ``` The [Quickstart](https://docs.dagger.io/getting-started/quickstart) covers module settings and generators. Then turn on Cloud Checks. The repository must be on GitHub, and its Git `origin` remote must point to it. From the project root, run: ```shell dagger cloud checks on ``` In an interactive terminal, the command offers to sign you in to Dagger Cloud. It also offers to connect the GitHub account or organization that owns the repository. After that, every push and pull request runs the same checks on Cloud Engines, with no workflow file ([Cloud Checks](https://docs.dagger.io/cloud-checks)). `dagger workspace activity` lists the runs. Keep your current workflows until Cloud Checks cover the same checks. Compare the results, and then turn off the workflows. [How do I migrate from GitHub Actions to Dagger?](/learn/migrate-from-github-actions) covers that process. ## Dagger Cloud pricing compared with per-minute runners Dagger Cloud bills for the CPU, memory, and data that your pipeline uses, not for runner minutes. New work costs 0.5¢ per CPU-minute and 0.5¢ per GB-minute of memory. Repeat work is 5× cheaper. An operation is repeat work if it has the same inputs as an operation that you ran in the last 30 days. Plans start at $20 a month, and you pay the plan price or your usage, whichever is higher ([pricing](https://dagger.io/pricing)). A runner vendor bills a machine size for each minute of a job, whatever the job uses of that machine. Dagger Cloud does not bill for idle capacity. Because the units differ, compare the two with your own usage. The pricing page has a calculator that estimates a monthly Dagger Cloud bill from your usage. [Can Dagger lower CI costs?](/learn/reduce-ci-costs) explains how Dagger lowers CI costs. ## How to choose a GitHub Actions alternative Choose by what is wrong with your CI today. - **The bill or the wait time is too high, and the workflows work.** Move to a runner vendor. Choose Blacksmith, Namespace, or WarpBuild for hosted runners with nothing to operate. Choose Depot if container builds take most of the time. Choose RunsOn if you are on AWS and can manage the account. Choose WarpBuild's BYOC plan for AWS, GCP, or Azure. - **You want a feature that a specific platform offers.** Choose Buildkite to manage a fleet of agents on your own infrastructure. Choose CircleCI for its built-in test selection. Choose Depot CI to keep your GitHub Actions YAML on a different orchestrator. - **CI failures do not reproduce locally, or the YAML and the runners take too much work.** Move the pipeline into Dagger and run it with Cloud Checks. **Bottom line:** Runner vendors such as Blacksmith, Depot, Namespace, WarpBuild, and RunsOn make GitHub Actions cheaper or faster, often with a one-line label change. Other CI platforms make sense when you need a feature that they offer, but most of them mean a rewrite of the pipeline. Dagger moves the pipeline into code that runs the same on a laptop and in Cloud Checks, with no workflow files or runners to manage. ## Related guides - [How does Dagger compare to Blacksmith?](/learn/dagger-vs-blacksmith) compares a runner vendor with Dagger directly. - [How do I migrate from GitHub Actions to Dagger?](/learn/migrate-from-github-actions) covers the move from workflow files to Dagger checks, step by step. - [Can Dagger lower CI costs?](/learn/reduce-ci-costs) explains how Dagger usage pricing and caching affect the CI bill. --- # Which development tools can Dagger replace or consolidate? URL: https://dagger.io/learn/consolidate-ci-tools Dagger can replace your CI platform, your hosted runners, and your test splitter for the work that its checks do. It can also replace Testcontainers, because your test code can call Dagger to start its services. It covers part of what CI observability tools and build caches do. Keep your source control, your test frameworks, your secret managers, and your production systems. Dagger can replace several tools because one engine schedules, runs, caches, and traces the work. ## Why one engine can replace several tools A typical CI stack uses a different tool for each part of a run. One tool schedules jobs, another hosts runners, another splits tests, and another stores caches and traces. Each tool knows only its own part. Dagger does these parts in one engine. Its sandbox, scheduler, container runtime, build cache, and tracing share one execution and data model ([dagger.io](https://dagger.io/)). The work is a set of checks. A check is a command that verifies your project and passes or fails, such as a test run or a linter ([Checking your code](https://docs.dagger.io/using/checking)). The same checks run on your machine, on your servers, and in Dagger Cloud. ## What Dagger replaces, and what it covers in part | Category | Example tools | With Dagger | | --- | --- | --- | | CI platform | GitHub Actions, CircleCI, Buildkite | Replace, for the work of your checks | | Hosted runners | Blacksmith, Namespace | Replace, with Cloud Engines | | Test splitting | [Knapsack Pro](https://knapsackpro.com/), [bktec](https://buildkite.com/docs/test-engine/test-splitting) | Replace | | CI observability | [Datadog CI Visibility](https://docs.datadoghq.com/continuous_integration/) | In part, for runs in Dagger | | Build caches | [Depot](https://depot.dev/docs), cache services | In part, for work in Dagger | | Test services | [Testcontainers](https://testcontainers.com/), Docker Compose | Replace | | Registries between jobs | Docker Hub, ECR, GHCR | Not needed between steps | | Remote dev environments | GitHub Codespaces, Gitpod, Coder, Daytona | In part, for shells | Each row is a choice for your team. A tool that does more than the row says, such as dashboards for code that does not run in Dagger, stays. ## CI platform and runners Cloud Checks run your checks on Dagger Cloud after each push and pull request, with no workflow file ([Cloud Checks](https://docs.dagger.io/cloud-checks)). They run on Cloud Engines, which Dagger Cloud runs for you, with scale-out and a distributed cache ([Cloud Engines](https://dagger.io/changelog/cloud-engines)). For the work that your checks do, you need no CI platform and no hosted runners. Cloud Checks need a GitHub repository. ## Test splitting and test results Dagger sees each test, not only the job that runs it ([test-native CI](https://dagger.io/changelog/test-native-ci)). It splits large test suites across engines, and balances the parts with the measured cost of each test ([test splitting](https://dagger.io/changelog/test-splitting)). Dagger Cloud shows each test from the trace, with its status, logs, and duration ([test-aware CI](https://dagger.io/changelog/cloud-tests-ui)). That covers the CI data of the runs in Dagger. An observability tool such as Datadog also covers systems outside Dagger, so keep it where your team uses that data. ## Build caches and artifacts between steps The Dagger engine caches the result of each operation ([Introduction](https://docs.dagger.io/getting-started/introduction)). When the inputs of an operation do not change, the engine uses the cached result. Work that runs in Dagger needs no separate cache service. Build caches for work outside Dagger stay. Steps in Dagger pass containers, files, and services to each other in the engine. A module can take the output of another module as a setting ([module wiring](https://docs.dagger.io/config/module-wiring)). So you do not push an image to a registry only for the next job to pull it. Keep your registry for the images that you publish. ## Test services Dagger replaces Testcontainers. Your test code calls Dagger through a client to start a database or another service, and then tests against it ([self-orchestrating tests](https://dagger.io/changelog/self-orchestrating-tests)). You run the tests with your usual test command, under `dagger api with-session`, and not inside a Dagger module. Dagger publishes a client library for Go, TypeScript, Python, PHP, Java, Elixir, Rust, and .NET ([Client libraries](https://docs.dagger.io/reference/client-libraries)). [How do I run integration tests against real databases and Kafka without mocks?](/learn/integration-tests-real-database-containers#start-services-from-test-code-with-a-dagger-client) shows a Go test that starts its own Postgres this way. The only external dependency of the test is then Dagger, so the test runs the same way on your machine and in CI. Dagger also replaces Docker Compose for services that run next to the tests. `dagger start` starts the services of your workspace and opens their ports on your machine ([Running services](https://docs.dagger.io/using/services)). ## Remote dev environments `dagger shell` opens a shell in a container of your project, and `dagger check -i` opens one where a command failed ([dev shells](https://dagger.io/changelog/dev-shells)). Dagger can also run your work on a remote engine, on your servers or on Dagger Cloud ([remote execution](https://dagger.io/changelog/remote-execution)). That covers a shell in the same environment as CI. It does not replace an editor in the browser or a persistent workspace, so keep a remote dev environment if your team uses those. ## What to keep - **Source control and code review**, such as GitHub. Cloud Checks report to GitHub as checks on each pull request. - **Your test frameworks.** Official modules run them with no change to your tests. - **Your secret managers.** Dagger reads secrets from HashiCorp Vault, 1Password, AWS Secrets Manager, and other providers ([Secrets](https://docs.dagger.io/config/secrets)). - **Your production systems**, such as your deploy targets and your registry for published images. **Bottom line:** Dagger can replace your CI platform, hosted runners, and test splitter for the work that its checks do, and Testcontainers through its clients. It covers part of CI observability, build caches, and remote dev shells. Keep the tools that do more than that, and keep your source control, tests, and secret managers. ## Related guides - [What are the alternatives to GitHub Actions, from faster runners to portable pipelines?](/learn/github-actions-alternatives): runner vendors, other CI platforms, and Dagger. - [How does Dagger compare to Buildkite?](/learn/dagger-vs-buildkite): a CI platform with test analytics compared with Dagger. - [Can Dagger lower CI costs?](/learn/reduce-ci-costs): how fewer tools and less repeated work lower the cost of CI. --- # What is Dagger's role in an agentic software factory? URL: https://dagger.io/learn/agentic-software-factory In an agentic software factory, AI agents write most code changes, and a system verifies each change before it ships. Dagger is that system. It runs each change, from an agent or a person, through the same builds and tests, before or after push ([dagger.io](https://dagger.io/)). Agents use Dagger to run checks, to see which test failed, and to run only that test again. The agent harness plans the work, and Dagger runs and verifies it. ## What an agentic software factory is A software factory is the set of systems that build, test, and ship code. In an agentic factory, AI agents make most of the changes, and many agents work at the same time. Each change needs builds and tests, so the factory runs many more of them than a team of people does. Each result must also be precise, because an agent acts on it with no person to read the log. ## Where Dagger fits Dagger is CI for this load. Dagger describes itself as third-generation CI, after Jenkins and GitHub Actions, built to match the speed and scale of agents that write code ([dagger.io](https://dagger.io/)). The same checks run on a laptop, for an agent, and in Dagger Cloud after a push. A check is a command that verifies your project and passes or fails, such as a test run or a linter ([Checking your code](https://docs.dagger.io/using/checking)). Official modules add checks for common tools, such as Go, pytest, Vitest, and ESLint, with no change to your tests ([modules](https://docs.dagger.io/reference/modules/index)). John McBride, co-founder of Paper Compute, says: "Dagger runs it all! It's what gives us confidence to move fast with AI agents and factories." ([dagger.io](https://dagger.io/)) ## What agents do with Dagger An agent uses the same commands as a developer: - **Run the checks.** `dagger check` runs all checks of the workspace, and exits with a non-zero code when one fails ([Checking your code](https://docs.dagger.io/using/checking)). - **Run one test.** Filters select one test, one file, or one package, such as `dagger check --go-test=TestGet` ([select collection items](https://docs.dagger.io/using/checking#select-collection-items)). - **Start what a test needs.** A test can call Dagger modules to start its own dependencies, such as a database ([self-orchestrating tests](https://dagger.io/changelog/self-orchestrating-tests)). - **Run on a remote engine.** Many agents can run their checks on a shared engine, on your servers or on Dagger Cloud, and not on one machine ([remote execution](https://dagger.io/changelog/remote-execution)). Execution is sandboxed, so developers, agents, and CI can share compute and cache with safety ([dagger.io](https://dagger.io/)). When an agent stops at a failure, `dagger check -i` lets a developer open a shell in the container where the command failed ([dev shells](https://dagger.io/changelog/dev-shells)). ## Why test-native results matter for agents An agent needs to know what failed, not only that a job failed. Dagger sees each test, not only the job that runs it ([test-native CI](https://dagger.io/changelog/test-native-ci)). The trace shows each test with its status, its duration, and its logs. The agent reads which test failed, fixes the code, and runs only that test again. ## Why the cache matters for agents Agents run the same checks many times while they change code. The Dagger engine caches the result of each operation ([Introduction](https://docs.dagger.io/getting-started/introduction)). When the inputs of an operation do not change, the engine uses the cached result and does not run it again. Dagger Cloud bills repeat work at one fifth of the price of new work ([pricing](https://dagger.io/pricing)). ## One engine for the full run Dagger's sandbox, scheduler, container runtime, build cache, and tracing share one execution and data model ([dagger.io](https://dagger.io/)). For example, the scheduler knows what the cache has, and the trace shows each test that the run executed. Dagger says this gives a level of compute and token efficiency that CI built from separate tools cannot reach. ## What Dagger does not do Dagger does not replace the agent harness. The harness reads the task, plans the change, and edits the code. Dagger builds and tests the change, and reports the result. Each system has its own job. **Bottom line:** In an agentic software factory, Dagger runs and verifies the work that agents plan. It runs each change through the same checks for agents, developers, and CI. It shows which test failed, and it reuses cached results across many runs. ## Related guides - [How do I run integration tests against real databases and Kafka without mocks?](/learn/integration-tests-real-database-containers): tests that start their own services, for agents and CI. - [Can Dagger lower CI costs?](/learn/reduce-ci-costs): how the cache and repeat-work pricing lower the cost of many runs. - [How do I migrate from GitHub Actions to Dagger?](/learn/migrate-from-github-actions): move your checks to Dagger one workflow at a time. --- # Can Dagger lower CI costs? URL: https://dagger.io/learn/reduce-ci-costs Yes, for most pipelines, because most CI work repeats. Dagger lowers the cost in two ways. It does less work: it reuses cached results and runs only the tests that you select. It also bills less for the work it does. Dagger Cloud bills the CPU and memory that your pipeline uses, and repeat work costs 5× less. It does not lower costs in every case, and the last section says when. ## Dagger does less work The Dagger engine runs your checks and caches the result of each operation ([Introduction](https://docs.dagger.io/getting-started/introduction)). When the inputs of an operation do not change, the engine uses the cached result and does not run the operation again. A change to one file runs only the operations that read that file. Dagger also sees each test, not only the job that runs it ([test-native CI](https://dagger.io/changelog/test-native-ci)). When a test fails, you run only that test again, not the full suite. For example, with the official Go module: ```shell dagger check --go-test=TestGet ``` The same filters select a package or a test file ([Checking your code](https://docs.dagger.io/using/checking#select-collection-items)). For a large suite, Dagger splits the tests across engines. It balances the parts with the measured resource use, input cost, and cache state of each test ([test splitting](https://dagger.io/changelog/test-splitting)). ## Dagger Cloud bills used resources Most CI services bill the runner for the full time of each job, at the price of its machine size. For example, GitHub bills each GitHub-hosted runner per minute, at a rate for its type and size ([GitHub runner pricing](https://docs.github.com/en/billing/reference/actions-runner-pricing)). A job that waits on the network still pays for all the cores of its runner. Dagger Cloud bills what your pipeline uses ([pricing](https://dagger.io/pricing)): - CPU time, measured from Linux cgroups. - Memory, sampled each second. - Data that your pipeline receives, such as git objects and container images. - Data that your pipeline sends, such as pushed images and network egress. Idle capacity is free. Each plan has a monthly price, and you pay the plan price or your usage, whichever is higher. ## Repeat work costs 5× less An operation is repeat work if it has the same inputs as an operation that you ran in the last 30 days ([pricing](https://dagger.io/pricing)). Dagger Cloud bills repeat work at one fifth of the price of new work. The discount applies when the cache does not have the result and the engine runs the operation again. The pricing page states the reason for this rule. You pay less as your pipeline gets faster, and Dagger earns more when its cache serves more work. Other providers earn less when your pipeline gets faster. ## You can run Dagger on machines you have The Dagger engine is open source. It runs on your machine or on your own server at no cost ([dagger.io](https://dagger.io/)). Dagger can also run a pipeline on a remote engine over SSH, on Kubernetes, or on Dagger Cloud ([remote execution](https://dagger.io/changelog/remote-execution)). Select the engine with `--engine` or the `DAGGER_ENGINE` variable ([engine reference](https://docs.dagger.io/reference/cli#dagger-engine)). ## When Dagger does not lower costs Dagger saves money when work repeats. These cases save less, or nothing: - **Most inputs change on each run.** New inputs get no cache result and no repeat-work price. - **Your CI is already free.** Standard GitHub-hosted runners are free for public repositories. GitHub plans also include 2,000 or 3,000 minutes each month ([GitHub Actions billing](https://docs.github.com/en/billing/concepts/product-billing/github-actions)). - **Your usage is below the plan price.** Dagger Cloud bills at least the monthly price of your plan. - **Setup takes time.** Official modules cover common tools. For other tools, find a module with `dagger module search`, or wrap the command you already run in a short module. That time is a cost too. To compare for your own pipeline, use the bill estimate on the [pricing page](https://dagger.io/pricing#estimate). **Bottom line:** Dagger lowers CI costs when work repeats, which is most CI work. It reuses cached results and runs only the tests that you select. Dagger Cloud bills used CPU and memory, and repeat work costs 5× less. It saves less when inputs change on each run, or when your CI already costs nothing. ## Related guides - [How do I migrate from GitHub Actions to Dagger?](/learn/migrate-from-github-actions): move your checks to Dagger one workflow at a time. - [How does Dagger compare to Blacksmith?](/learn/dagger-vs-blacksmith): faster runners compared with a test-native engine. - [What are the alternatives to GitHub Actions, from faster runners to portable pipelines?](/learn/github-actions-alternatives): runner vendors, other CI platforms, and Dagger.