CI that keeps up
migrationby Kyle PenfoundTested with Dagger v1.0.0-beta.21

How do I migrate from CircleCI to Dagger?

You do not have to rewrite your .circleci/config.yml as Dagger code. Dagger’s official modules find the tests and linters in your repository and run them as checks, on your machine and in CI. Steps that no official module covers, such as a Terraform deploy, become functions in a small module of your own. Move in five steps:

  1. Install the modules.
  2. Run the checks locally.
  3. Move your own steps into a module.
  4. Choose where the checks run. Cloud Checks run them with no CI file, or CircleCI can stay as the trigger.
  5. Turn off each CircleCI job when the checks that replace it pass.

Do I have to rewrite my CircleCI config?

No, not for the work that an official module does. A Dagger module is a package of functions that adds work to your project, such as tests or lint. The Dagger team maintains official modules for common tools, and each one finds the files of its tool in your repository. For example, the Vitest module finds each Vitest config and makes the test files of that project a collection that you can select one by one.

A check is a function that verifies your project and passes or fails, such as a test run or a linter (Checking your code). Today the official modules cover Go, golangci-lint, Staticcheck, Deno, Jest, Vitest, Playwright, ESLint, Prettier, Biome, pytest, ShellCheck, PSScriptAnalyzer, and Helm (module reference).

An orb such as circleci/node packages CircleCI configuration: jobs, commands, and executors (Orbs overview). An official module replaces the orb and the jobs that you wrote around it. You write code only for work that no module does, and you do not copy the job structure of your YAML.

Install the official modules and run the checks locally

Install the Dagger CLI. Then, from the root of your repository, initialize the workspace:

dagger init

The workspace is the Dagger configuration of your repository, in a file named dagger.toml (Quickstart). In a terminal, dagger init asks which modules to install. Without a terminal, it prints the next commands:

Workspace configuration initialized at ./dagger.toml

To continue setup, run these commands in order:

#!/bin/sh

# 1. Find and install suitable modules
dagger module recommend

# 2. Enable cloud checks
dagger cloud checks on

Install the modules that Dagger recommends. The -y flag installs all of them with no prompt. In a repository with a Vitest project in web/, the command installs the Vitest module:

dagger module recommend -y
dagger.io/js/vitest ➡️ https://github.com/dagger/vitest#v0.3.0

The command adds the module to dagger.toml and writes dagger.lock. That file pins the versions of the modules and images that the workspace uses, as a version pins an orb. Commit both files. If Dagger recommends no module for one of your tools, search by the tool’s name, such as dagger module search eslint.

List the checks, then run them:

dagger check -l
dagger check

dagger check runs all checks in parallel and exits with a non-zero code when one fails. To run one test file, select it with the flag that the module adds:

dagger check --vitest-test-file=sum.test.js

Orbs, executors, jobs, and contexts in Dagger

Most CircleCI concepts have a Dagger counterpart. For some, such as executors and cache keys, the modules do the work and you configure nothing.

CircleCI Dagger
Orbs Official modules, pinned in dagger.lock
Executors and images Containers that each module builds
Resource classes Engine size, or usage billing on Cloud Engines
Jobs and workflows Checks, which dagger check runs in parallel
Contexts Secret providers, read when a function needs a secret
save_cache and restore_cache Cache of each operation, with no keys to write
parallelism and test splitting Tests selected one by one, and test splitting across engines
store_test_results Each test reported in the trace, with no JUnit setup

Executors and resource classes. In CircleCI, each job names an executor, such as a Docker image or a Linux VM, and a resource_class for its CPU and RAM (executors, resource classes). In Dagger, each check runs in a container that its module builds. The modules read your native configuration. For example, the Go module reads the Go version from go.mod and builds a container with that version (Workspaces). If a module’s default image does not match your project, change it in dagger.toml, such as dagger module settings vitest baseImageAddress node:22-alpine.

Cloud Engines are engines that Dagger runs for you. They bill the CPU and memory that each pipeline uses, not the full machine for the full job (pricing). On an engine that you host, you choose the machine size (Self-hosting).

Jobs and workflows. A CircleCI workflow lists jobs and orders them with requires (workflows). Dagger has no job list. dagger check runs every check in the workspace in parallel.

Contexts. A CircleCI context injects environment variables into the jobs that name it (contexts). Dagger reads a secret from its provider when a function needs it (Secrets). The section on secrets and deploys below shows both together.

Caches, test splitting, and test results in Dagger

Dagger caches and reports tests with no configuration in your repository. CircleCI caching is manual: save_cache writes paths under a key, restore_cache finds them, and caches stay for up to 15 days (Caching dependencies). The Dagger Engine, the open-source runtime that runs each check, caches all operations by default, keyed by their inputs (Self-hosting, Dang SDK). You write no cache keys.

CircleCI splits tests with parallelism and the CircleCI CLI, which can use timing data from the JUnit files that store_test_results uploads (test splitting, test results). Dagger’s official modules report each test in the trace with its status, duration, and logs, and you do not change the tests (Test-native CI). Dagger splits a large test suite across engines, and balances the work with telemetry from earlier runs (test splitting).

Move custom jobs into a Dang module

Write a module for each step that no official module covers. Write it in Dang, Dagger’s own language for modules (Dang SDK). Install the Dang SDK, then create the project’s dev module:

dagger module install dagger.io/sdk/dang
dagger module init dang -y

With no --name, the command names the module after the project directory, creates it in .dagger/modules/, and installs it as the workspace main module. For a project in a storefront directory, the module is storefront-dev. This example replaces a job that validates and applies Terraform code in infra/. Replace .dagger/modules/storefront-dev/main.dang with this code. The type name StorefrontDev stands in for the one that dagger module init generated:

"""
Project steps that no official module covers.
"""
type StorefrontDev {
  let infra: Directory!

  new(ws: Workspace!) {
    self.infra = ws.directory("/infra")
    self
  }

  let terraform: Container! {
    container
      .from("hashicorp/terraform:1.13")
      .withMountedDirectory("/infra", infra)
      .withWorkdir("/infra")
      .withExec(["terraform", "init", "-backend=false", "-input=false"])
  }

  """
  Check the formatting and the configuration of the Terraform code.
  """
  pub terraformValidate: Void @check {
    terraform
      .withExec(["terraform", "fmt", "-check"])
      .withExec(["terraform", "validate"])
      .sync
    null
  }

  """
  Apply the Terraform configuration with AWS credentials.
  """
  pub deploy(awsAccessKeyId: Secret!, awsSecretAccessKey: Secret!): String! {
    terraform
      .withSecretVariable("AWS_ACCESS_KEY_ID", awsAccessKeyId)
      .withSecretVariable("AWS_SECRET_ACCESS_KEY", awsSecretAccessKey)
      .withExec(["terraform", "apply", "-auto-approve", "-input=false"])
      .stdout
  }
}

The @check function joins the checks of the official modules:

dagger check -l
MODULE          VITEST-PROJECT  VITEST-TEST-FILE  CHECK               DESCRIPTION
dang-sdk                                          stale               staleness check: re-generate modules and clients managed by the Dang SDK
storefront-dev                                    terraform-validate  Check the formatting and the configuration of the Terraform code.
vitest          *               *                 test                Run this test file.
# Use --all to expand collections and list each item.

When a check fails, the report gives the command that runs it again on your machine. Here, a Terraform file needs terraform fmt:

CHECKS  ✘ 1 failed  ✔ 2 passed
✘ dag://?check=terraform-validate 1.3s ERROR
  ∅ withExec terraform fmt -check 0.2s
  ┃ main.tf
  ! exit code: 3
✔ dag://?check=dang-sdk/generate/stale 1.3s OK
✔ dag://?check=test&vitest-project=web 4.3s OK
  TESTS
    ✔ sum.test.js  1 passed

    ✔ 1 passed

RUN LOCALLY
  dagger check "dag://?check=terraform-validate"

Secrets and deploys

The deploy function is not a check, so dagger check does not run it. Call it with dagger api call deploy, and give each secret from a provider. Providers include files, environment variables, HashiCorp Vault, 1Password, and AWS Secrets Manager (Secrets). This call reads the AWS keys from files:

dagger api call deploy --aws-access-key-id=file:./aws-access-key-id --aws-secret-access-key=file:./aws-secret-access-key

Dagger redacts secret values from logs, never writes them to container layers, and leaves them out of cache keys. Your contexts can stay. A CircleCI job that names a context gives its variables to the function, as the deploy job in the CircleCI config below shows.

Run the checks on every push with Cloud Checks

Cloud Checks run your checks on Cloud Engines after each push and pull request, with no CI file (Set up Cloud Checks). They run the same checks as dagger check on your machine. The repository must be on GitHub, and its origin remote must point to it. From the root of the repository, run:

dagger cloud checks on

In a terminal, the command helps you sign in to Dagger Cloud and connect your GitHub organization. Commit dagger.toml, dagger.lock, and .dagger/, then push. The results show as GitHub check statuses on the pull request and in Dagger Cloud (Cloud Checks).

Keep CircleCI as the trigger

A team can keep CircleCI as the system that starts the work, and let Dagger run it. A CircleCI job runs dagger check, and DAGGER_ENGINE selects where the checks run (Self-hosting):

  • cloud selects Cloud Engines. It needs DAGGER_CLOUD_TOKEN.
  • tcp://HOST:PORT selects an engine that your team hosts. This address has no authentication, so keep the port on a private network.

Set DAGGER_ENGINE in every job. Without it, the CLI starts an engine on the runner, and that engine’s cache is lost after the job. Install the CLI at the same version as the engine.

In .circleci/config.yml, replace the jobs that the checks cover:

version: 2.1

commands:
  install-dagger:
    steps:
      - run:
          name: Install the Dagger CLI
          command: |
            curl -fsSL https://dl.dagger.io/dagger/install.sh | BIN_DIR=$HOME/.local/bin sh
            echo 'export PATH="$HOME/.local/bin:$PATH"' >> "$BASH_ENV"

jobs:
  dagger-check:
    docker:
      - image: cimg/base:stable
    environment:
      DAGGER_VERSION: 1.0.0-beta.21
      DAGGER_ENGINE: cloud
    steps:
      - checkout
      - install-dagger
      - run: dagger check
  deploy:
    docker:
      - image: cimg/base:stable
    environment:
      DAGGER_VERSION: 1.0.0-beta.21
      DAGGER_ENGINE: cloud
    steps:
      - checkout
      - install-dagger
      - run:
          name: Deploy with Dagger
          command: |
            dagger api call deploy \
              --aws-access-key-id=env:AWS_ACCESS_KEY_ID \
              --aws-secret-access-key=env:AWS_SECRET_ACCESS_KEY

workflows:
  checks-and-deploy:
    jobs:
      - dagger-check:
          context: dagger-cloud
      - hold:
          type: approval
          requires:
            - dagger-check
          filters:
            branches:
              only: main
      - deploy:
          requires:
            - hold
          context:
            - dagger-cloud
            - aws-production
          filters:
            branches:
              only: main

The dagger-cloud context holds DAGGER_CLOUD_TOKEN, and the aws-production context holds the AWS keys. The env: provider reads each key from the environment variable that the context sets. For a self-hosted engine, set DAGGER_ENGINE to its address and remove the dagger-cloud context.

When to keep CircleCI as the trigger

CircleCI does some jobs that Dagger leaves to the system that starts the checks. Keep CircleCI in front of dagger check in these cases:

  • Deploys wait for an approval job, and restricted contexts control who can release.
  • Jobs get short-lived cloud credentials from CircleCI’s OIDC tokens.
  • The repository is not on GitHub. Cloud Checks start from GitHub events, so a CircleCI job starts the checks instead.
  • Some jobs need CircleCI’s hosted macOS, Windows, or GPU execution environments. Dagger checks run in Linux containers.

Move one job at a time

Keep your CircleCI jobs on while you add Dagger. Run the Dagger checks next to the jobs that they replace, and compare the results on the same commits. When they agree, do these steps:

  1. Make the Dagger checks required status checks in your branch protection rules.
  2. Remove the status of the CircleCI job from the required list.
  3. Delete the job, and the orb if nothing else uses it, from .circleci/config.yml.

Repeat these steps for each job. When no job is left, remove the config, or keep only the jobs that CircleCI still triggers.

What stays the same

Your Git host stays the home for code, pull requests, and review. Your tests stay as they are, because official modules run them with no change to the test code. Your secret managers stay the source of your secrets, and CircleCI contexts can keep them while CircleCI triggers jobs.

Bottom line: You migrate from CircleCI to Dagger without a rewrite. Install the official modules that dagger module recommend finds, and run the same checks locally. Put the steps that no module covers in a small Dang module. Then run the checks with Cloud Checks, or from a thin .circleci/config.yml, and remove each CircleCI job when its checks pass.