CI that keeps up
how-toby Kyle Penfound

How do I run integration tests against real databases and Kafka without mocks?

To replace mocks with real services, start each database and Kafka broker in a container for each test run. A good setup meets four requirements. Each service is fresh for each run and healthy before the tests start. The tests find it at a known address, and the setup is the same on a laptop and in CI.

Testcontainers, Docker Compose, and CI service containers each meet some or all of them by hand. Dagger meets the same four requirements in either model. Define the services once outside the tests, as with Compose, or start them from test code, as with Testcontainers.

What integration tests against real services need

Integration tests against real services have to meet four requirements. This guide uses these names for them in each section that follows:

  • A fresh service for each run. Each run starts new containers and removes them after the tests, so no data stays from an earlier run. GitHub Actions “creates a fresh Docker container for each service configured in the workflow, and destroys the service container when the job completes” (Communicating with Docker service containers).
  • Healthy before the tests start. The tests start only after each service accepts connections. Otherwise the first test races the startup of the service. Testcontainers wait strategies “allow your tests to actually wait for the most useful conditions to be met, before continuing with their execution” (Wait strategies).
  • A known address. The tests find each service at a host and port that they read, such as from an environment variable. Two runs on one machine must not collide on a port. Testcontainers picks random host ports “to avoid port collisions that may arise with locally running software or in between parallel test runs” (Networking).
  • The same setup locally and in CI. A laptop and CI use the same images, settings, and health checks. Otherwise a test that passes on a laptop can fail in CI.

Meet the requirements with Testcontainers

Testcontainers meets all four requirements from inside the test process. It is a library that starts containers from test code. Its languages include Java, .NET, Go, Node.js, Python, Rust, and Haskell (Testcontainers: Getting Started).

  • A fresh service for each run. Test code starts a container for each test or test class. A sidecar container, Ryuk, removes the containers after the tests. “Even if you do not call Terminate, Ryuk ensures that the environment will be kept clean” (Garbage Collector).
  • Healthy before the tests start. Each language has modules for common services, with a wait strategy for each service (Wait strategies).
  • A known address. Testcontainers maps each container port to a random free port on the host (Networking). The module gives the address to the test. In the Go MySQL module, mysql.Run takes an image and options for the database name, the credentials, and init scripts. The container that it returns has a ConnectionString method (MySQL module).
  • The same setup locally and in CI. The same test code runs on a laptop and in CI. Each machine that runs the tests needs a container runtime.

Add the modules as test dependencies in each language:

# Go
go get github.com/testcontainers/testcontainers-go/modules/mysql

# Node.js
npm install --save-dev @testcontainers/kafka @testcontainers/mongodb

In Java, add the dependency to pom.xml:

<dependency>
  <groupId>org.testcontainers</groupId>
  <artifactId>testcontainers-kafka</artifactId>
  <version>2.0.5</version>
  <scope>test</scope>
</dependency>

Testcontainers fits best when a test or a test class controls the lifecycle of its own container from code.

Meet the requirements with Docker Compose

Docker Compose starts the services before the test command, so the tests contain no container code. One Compose file serves every language in the repository.

  • A fresh service for each run. docker compose down stops and removes the containers that up created (docker compose down). docker compose up recreates an old container only when its configuration or image changed (docker compose up). Thus a run that skips down can leave old data for the next run.
  • Healthy before the tests start. A health check in the file tells Compose when a service is ready. docker compose up --wait waits for the services “to be running|healthy” (docker compose up).
  • A known address. The file maps a host port, such as 5432:5432, and the tests connect to localhost:5432. A fixed host port collides when two runs share a machine.
  • The same setup locally and in CI. The same file runs on each machine that has Docker, a laptop or a CI runner.

This compose.yaml starts Postgres with a health check:

services:
  postgres:
    image: postgres:17-alpine
    environment:
      POSTGRES_PASSWORD: test
    ports:
      - "5432:5432"
    healthcheck:
      test: ["CMD", "pg_isready", "-U", "postgres"]
      interval: 2s
      retries: 15

Start the services, run the tests, then remove the services:

docker compose up --wait
go test ./...
docker compose down

Compose fits a repository where tests in several languages use the same services on one machine.

Meet the requirements in CI with GitHub Actions service containers

CI platforms start service containers before the steps of a job. In GitHub Actions, the services: block of a job defines them.

  • A fresh service for each run. GitHub creates a new container for each service and removes it when the job completes (Communicating with Docker service containers).
  • Healthy before the tests start. The --health-* options make sure that the service runs before the steps start (Creating PostgreSQL service containers).
  • A known address. A job that runs in a container reaches the service by its label, such as postgres. A job that runs on the runner machine maps the ports and uses localhost:<port> (Communicating with Docker service containers).
  • The same setup locally and in CI. Service containers run only in CI. A laptop needs a second definition, such as a Compose file.

This services: block starts Postgres with a health check:

services:
  postgres:
    image: postgres
    env:
      POSTGRES_PASSWORD: postgres
    options: >-
      --health-cmd pg_isready
      --health-interval 10s
      --health-timeout 5s
      --health-retries 5

GitHub Actions service containers need a Linux runner. The same page says: “If your workflows use Docker container actions, job containers, or service containers, then you must use a Linux runner.”

Service containers fit tests that run only in GitHub Actions, or a team that also keeps a Compose file for laptops.

Which requirements each approach meets

Requirement Testcontainers Docker Compose GitHub Actions service containers
A fresh service for each run A container for each test or test class. Ryuk removes it. up creates the containers. down removes them. A new container for each job, removed when the job completes
Healthy before the tests start A wait strategy in each module healthcheck with up --wait --health-* options
A known address A random free host port, from the module A fixed host port, which collides on a shared machine The label in a container job, or a mapped localhost port
The same setup locally and in CI Yes, with a container runtime on each machine Yes, where CI also runs Compose CI only, on a Linux runner
Fits when A test controls its own container from code Tests in several languages share services on one machine Tests run only in GitHub Actions

Where the hand-built setup strains as the project grows

As a project grows, each requirement gets harder to meet with one definition of each service.

The same setup locally and in CI. A team that uses Compose on laptops and services: in CI has two definitions of Postgres. The image tags, environment variables, and health checks drift apart. Then a test that passes on a laptop can fail in CI. Testcontainers keeps one definition for each language. A repository with a Go service, a Node service, and a Java consumer configures Kafka three times, through three libraries.

Each CI job that runs those tests also needs a Docker-compatible runtime. Colima, Podman, and Rancher Desktop each need extra environment variables (Supported container runtimes). For tests inside a container, “the docker socket must be available via a volume mount”. The source directory must also have the same path inside the container as on the host. Some CI systems need Docker-in-Docker, which the same docs call “an instrument of last resort” (Patterns for running tests inside a Docker container).

Images can also disappear from registries. The MinIO community edition became “a source only distribution” in October 2025 (MinIO issue #21647), and its repository is now archived. In September 2026, pulls of minio/minio from Docker Hub started to fail. A post from StableBuild recommended the same tags on quay.io/minio/minio (StableBuild). In October 2026, a pull from Quay for this guide also failed:

docker pull quay.io/minio/minio:RELEASE.2025-09-07T16-13-09Z
Error response from daemon: unknown: failed to resolve reference "quay.io/minio/minio:RELEASE.2025-09-07T16-13-09Z": unexpected status from HEAD request to https://quay.io/v2/minio/minio/manifests/RELEASE.2025-09-07T16-13-09Z: 401 Unauthorized

When an image disappears, each file that names the image must change.

A known address. Fixed host ports, such as 5432:5432 in the Compose file above, collide when two runs share a machine.

Healthy before the tests start. Each definition has its own health check. A health check that is too weak lets the tests start before the service accepts connections.

Two ways to run test services with Dagger

Dagger supports both models of the hand-built tools:

  • The Compose model. One Dagger module defines the services, outside the tests. The tests reach each service by hostname and environment variable. The next sections show this model.
  • The Testcontainers model. The test code starts its own services through a Dagger client, and you run the tests with your test command. Start services from test code with a Dagger client shows this model.

Both models meet the same four requirements, with the same service lifecycle and health checks.

Define the services once in a Dang module

In the Compose model, Dagger meets the four requirements with services that one Dagger module defines, outside the tests. A service in Dagger is a container that runs for as long as a client needs it, such as a database. The tests reach each service by a hostname, and they read connection details from environment variables. The test code does not start containers.

The services go in a Dagger module, a package of functions, checks, and services that a project can run. A check is a function that validates the project, and dagger check runs every check. The steps below need a Dagger workspace, the dagger.toml file that holds the configuration of the project. If the project has no dagger.toml, do the Quickstart first. The project must be a Git repository.

Write the services in the dev module of the project, a module in Dang. Dang is the native language of Dagger for modules, with no code generation and no build step. Install the Dang SDK and create the module:

dagger module install dagger.io/sdk/dang
dagger module init dang

dagger module init creates the module at .dagger/modules/<project>-dev and makes it the workspace main. The workspace main is the module whose functions you can call without the module name. The command returns a changeset, a diff that Dagger shows for review before it writes files. A headless run needs -y.

Replace .dagger/modules/<project>-dev/main.dang with the services that your tests need. ProjectDev stands in for the type name that dagger module init generated. Keep the type name of your module, or the module fails to load with no main object. The output in this guide comes from a project named orders, so its dev module is orders-dev.

"""
Services for integration tests and local development.
"""
type ProjectDev {
  pub source: Directory!

  new(ws: Workspace!) {
    self.source = ws.directory("/", exclude: ["node_modules", ".git", "dist", "target"])
    self
  }

  """
  A Postgres database.
  """
  pub postgres: Service! {
    container
      .from("postgres:17-alpine")
      .withEnvVariable("POSTGRES_USER", "test")
      .withEnvVariable("POSTGRES_PASSWORD", "test")
      .withEnvVariable("POSTGRES_DB", "test")
      .withExposedPort(5432)
      .asService
  }

  """
  A MySQL database.
  """
  pub mysql: Service! {
    container
      .from("mysql:8.4")
      .withEnvVariable("MYSQL_ROOT_PASSWORD", "test")
      .withEnvVariable("MYSQL_DATABASE", "test")
      .withEnvVariable("MYSQL_USER", "test")
      .withEnvVariable("MYSQL_PASSWORD", "test")
      .withExposedPort(3306)
      .asService
  }

  """
  A single-node Kafka broker in KRaft mode, advertised as kafka:9092.
  """
  pub kafka: Service! {
    container
      .from("apache/kafka:4.1.0")
      .withEnvVariable("KAFKA_NODE_ID", "1")
      .withEnvVariable("KAFKA_PROCESS_ROLES", "broker,controller")
      .withEnvVariable("KAFKA_LISTENERS", "PLAINTEXT://:9092,CONTROLLER://:9093")
      .withEnvVariable("KAFKA_ADVERTISED_LISTENERS", "PLAINTEXT://kafka:9092")
      .withEnvVariable("KAFKA_CONTROLLER_LISTENER_NAMES", "CONTROLLER")
      .withEnvVariable("KAFKA_LISTENER_SECURITY_PROTOCOL_MAP", "CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT")
      .withEnvVariable("KAFKA_CONTROLLER_QUORUM_VOTERS", "1@localhost:9093")
      .withEnvVariable("KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR", "1")
      .withEnvVariable("KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR", "1")
      .withEnvVariable("KAFKA_TRANSACTION_STATE_LOG_MIN_ISR", "1")
      .withEnvVariable("KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS", "0")
      .withDockerHealthcheck(
        ["/opt/kafka/bin/kafka-broker-api-versions.sh", "--bootstrap-server", "localhost:9092"],
        interval: "2s",
        retries: 30,
      )
      .withExposedPort(9092)
      .asService
  }

  """
  A MongoDB server.
  """
  pub mongo: Service! {
    container
      .from("mongo:8")
      .withExposedPort(27017)
      .asService
  }
}

One definition of each service now serves every test in the repository, on a laptop and in CI.

A fresh service for each run, with the Dagger service lifecycle

Dagger runs each service in an ephemeral container for the run that needs it (Running services). A service starts when a client first needs it. It stops when no client references it. Thus the tests need no teardown code, as with Ryuk in Testcontainers. The SERVICES part of each dagger check output below lists each service as EXITED after the run.

Clients in one run that request the same service share one instance. Testcontainers can give each test class its own container. Here, the Go, Node, and Java tests share one Kafka broker in a run. Give each test its own database, topic, or collection name. Do not expect an empty server.

Healthy before the tests start, with Dagger health checks

Dagger health-checks a service before any client can connect, so the tests cannot start before the service is ready (Running services). Each service in the module becomes ready at a different point:

  • Postgres, MySQL, and MongoDB are ready when the Dagger health check on their exposed port passes. The MySQL image runs its init scripts on a temporary server with --skip-networking. Thus port 3306 opens only when the real server starts (MySQL entrypoint).
  • Kafka has a Docker health check, set with withDockerHealthcheck (Container API). It is the same kind of check as a healthcheck in Compose or the --health-* options in GitHub Actions. A Docker health check replaces the default port check of a service (changelog). Thus the tests start only after the broker answers Kafka requests.

A known address for Go tests, with the Go module’s test runtime

Each Dagger service gets a hostname from its definition, so services have no port conflicts (Running services). A test container reaches a service by the name that withServiceBinding gives it, such as db or kafka (Container API). The advertised listener of Kafka is kafka:9092, so each container that talks to Kafka must bind the service as kafka. The module sets every KRaft variable. The apache/kafka image says: “if you are overriding any configuration, then none of the default configurations will be used” (apache/kafka on Docker Hub).

The Dagger module for Go, dagger.io/go, runs the Go tests. Your dev module supplies the container that the tests run in, with the services bound. Install the Go module:

dagger module install dagger.io/go

Add a testRuntime function to the ProjectDev type in .dagger/modules/<project>-dev/main.dang. It binds the services and sets the variables that the tests read:

  """
  The Go test runtime, with Postgres and Kafka bound.
  """
  pub testRuntime: Container! {
    container
      .from("golang:1.26-bookworm")
      .withServiceBinding("db", postgres)
      .withServiceBinding("kafka", kafka)
      .withEnvVariable("DATABASE_URL", "postgres://test:test@db:5432/test?sslmode=disable")
      .withEnvVariable("KAFKA_BROKERS", "kafka:9092")
  }

Connect the function to the base setting of the Go module with module wiring. Module wiring puts the value that a function returns into the setting of another module. Then run the tests:

dagger module settings go base test-runtime
dagger check --go --check=test
== CHECKS ==  ✔ 2 passed
✔ dag://?check=test&go-package=events 1m15s OK
  == TESTS ==
    ✔ example.com/orders/events  1 passed

    ✔ 1 passed
✔ dag://?check=test&go-package=store 1m15s OK
  == TESTS ==
    ✔ example.com/orders/store  1 passed

    ✔ 1 passed

== SERVICES ==
✔ ku62l0th05qka.h4eqn5sbc01u2.dagger.local exec docker-entrypoint.sh postgres 51.7s EXITED span=159734715acebc1d
✔ 08husvm992r7i.h4eqn5sbc01u2.dagger.local exec /__cacert_entrypoint.sh /etc/kafka/docker/run 43.4s EXITED span=210cbf590ee489ef

The settings command writes base = "dag://<project>-dev/test-runtime" to dagger.toml. Each container that the Go module makes from this base keeps the service bindings. Thus Postgres and Kafka run whenever go test runs.

The image supplies the Go toolchain, and the Go module adds its test runner to the image. When base is set, the Go module ignores its version setting. If you set version before, remove it with dagger module settings -u go version. Daggerize a Go Project shows this wiring step by step.

Make each test skip when its variable is not set. Then plain go test ./... still passes on a laptop without the services. Dagger reports those tests as skipped, not as failed.

A known address for Node and Java tests, with a check

For Node and Java tests, write a check in the dev module that builds the test container. The tests reach bound services only through the container that they run in. The Vitest and Jest modules build their Node container from a base image name and environment variables, with no service bindings. Thus your own check binds the services, sets the variables, and runs the test command of the project.

Add these functions to the ProjectDev type in .dagger/modules/<project>-dev/main.dang:

  """
  Node integration tests against Kafka and MongoDB.
  """
  pub nodeIntegration: Void @check {
    container
      .from("node:22-bookworm")
      .withServiceBinding("kafka", kafka)
      .withServiceBinding("mongo", mongo)
      .withEnvVariable("KAFKA_BROKERS", "kafka:9092")
      .withEnvVariable("MONGO_URL", "mongodb://mongo:27017/test")
      .withMountedCache("/root/.npm", cacheVolume("npm"))
      .withDirectory("/app", source)
      .withWorkdir("/app")
      .withExec(["npm", "ci"])
      .withExec(["npm", "run", "test:integration"])
      .sync
    null
  }

  """
  Java integration tests against Kafka and MySQL.
  """
  pub javaIntegration: Void @check {
    container
      .from("maven:3.9-eclipse-temurin-21")
      .withServiceBinding("kafka", kafka)
      .withServiceBinding("mysql", mysql)
      .withEnvVariable("KAFKA_BOOTSTRAP_SERVERS", "kafka:9092")
      .withEnvVariable("JDBC_URL", "jdbc:mysql://mysql:3306/test")
      .withMountedCache("/root/.m2", cacheVolume("maven"))
      .withDirectory("/app", source)
      .withWorkdir("/app")
      .withExec(["mvn", "-B", "verify"])
      .sync
    null
  }

A check passes when it completes without an error. It fails when a withExec command exits with a non-zero code. Run one check by its address:

dagger check dag://<project>-dev/node-integration
== CHECKS ==  ✔ 1 passed
✔ dag://?check=node-integration 41.3s OK

== SERVICES ==
✔ 08husvm992r7i.de48876uvsd0e.dagger.local exec /__cacert_entrypoint.sh /etc/kafka/docker/run 25.0s EXITED span=79cf5a83ea3ddf9d
✔ d0rur6uthqsjm.de48876uvsd0e.dagger.local exec docker-entrypoint.sh mongod 12.8s EXITED span=ba41cd67698295ed

The same setup locally and in CI, with Cloud Checks

dagger check runs the tests the same way on a laptop and in CI, because the services, images, and environment variables come from the dev module. Run every check locally:

dagger check
== CHECKS ==  ✔ 6 passed
✔ dag://?check=dang-sdk/generate/stale 1.3s OK
✔ dag://?check=go/packages/generate/stale 10.7s OK
✔ dag://?check=java-integration 10.8s OK
✔ dag://?check=node-integration 10.7s OK
✔ dag://?check=test&go-package=events 10.7s OK
  == TESTS ==
    ✔ example.com/orders/events  1 passed

    ✔ 1 passed
✔ dag://?check=test&go-package=store 10.7s OK
  == TESTS ==
    ✔ example.com/orders/store  1 passed

    ✔ 1 passed

== SERVICES ==
✔ d0rur6uthqsjm.tnefdqvqu78tc.dagger.local exec docker-entrypoint.sh mongod 10.5s EXITED span=bb6b3c68fdeb9fee
✔ hde2ut6iqp6ji.tnefdqvqu78tc.dagger.local exec docker-entrypoint.sh mysqld 10.6s EXITED span=3db822a6ad9a76ff
✔ 08husvm992r7i.tnefdqvqu78tc.dagger.local exec /__cacert_entrypoint.sh /etc/kafka/docker/run 10.5s EXITED span=9757895b6ac9879a
✔ ku62l0th05qka.tnefdqvqu78tc.dagger.local exec docker-entrypoint.sh postgres 9.9s EXITED span=3a730798f45179c0

Kafka appears once under SERVICES, because the Go, Node, and Java tests share one broker. The Dang SDK adds a drift check for the generated files of the dev module. The Go module adds a drift check for go generate.

Commit dagger.toml, dagger.lock, and .dagger. dagger.lock pins each image that the workspace uses, so CI pulls the same images that you tested with.

In CI, Cloud Checks run the same checks. Cloud Checks run your checks on Dagger Cloud after each push and pull request, with no CI workflow file. The repository must be on GitHub, and its origin remote must point there. Run this command from the project root:

dagger cloud checks on

After that, each push and pull request runs these checks, with their services, on Dagger’s Cloud Engines. dagger workspace activity shows the recent runs.

Use the same services from the host with dagger start

dagger start starts the services of the workspace and tunnels their ports to your machine (Running services). Tools and tests on the host then use the same definitions as dagger check. The functions that return Service! in the dev module appear in its list:

dagger start -l
MODULE      SERVICE   DESCRIPTION
orders-dev  kafka     A single-node Kafka broker in KRaft mode, advertised as kafka:9092.
orders-dev  mongo     A MongoDB server.
orders-dev  mysql     A MySQL database.
orders-dev  postgres  A Postgres database.

Start the services that you need by name:

dagger start postgres mongo

Clients on your machine then connect through localhost. Kafka is different, because the broker advertises kafka:9092. A client on the host gets that address from the broker and cannot resolve it. A Go test on the host failed with this error:

kafka.(*Client).Produce: dial tcp: lookup kafka: no such host

To reach Kafka from the host, add a second service function with localhost:9092 as its advertised listener.

Start services from test code with a Dagger client

In the Testcontainers model, the test starts its own services. A Go test calls the Dagger API through the client library dagger.io/dagger (Client libraries). Add its dag package at the version of your Dagger CLI. With the package path, go get also adds the modules that dag imports:

go get dagger.io/dagger/dag@v1.0.0-beta.20

Put this test in store/dagger_test.go. It starts Postgres, opens a tunnel to it from the host, and tests against it:

package store

import (
	"context"
	"testing"

	"dagger.io/dagger/dag"
	"github.com/jackc/pgx/v5"
)

func TestOrdersWithDaggerPostgres(t *testing.T) {
	ctx := context.Background()
	t.Cleanup(func() { dag.Close() })

	postgres := dag.Container().
		From("postgres:17-alpine").
		WithEnvVariable("POSTGRES_USER", "test").
		WithEnvVariable("POSTGRES_PASSWORD", "test").
		WithEnvVariable("POSTGRES_DB", "test").
		WithExposedPort(5432).
		AsService()
	db, err := dag.Host().Tunnel(postgres).Start(ctx)
	if err != nil {
		t.Fatal(err)
	}
	addr, err := db.Endpoint(ctx)
	if err != nil {
		t.Fatal(err)
	}

	conn, err := pgx.Connect(ctx, "postgres://test:test@"+addr+"/test")
	if err != nil {
		t.Fatal(err)
	}
	defer conn.Close(ctx)

	if _, err := conn.Exec(ctx, "CREATE TABLE orders (id serial PRIMARY KEY, item text)"); err != nil {
		t.Fatal(err)
	}
	var id int
	err = conn.QueryRow(ctx, "INSERT INTO orders (item) VALUES ('widget') RETURNING id").Scan(&id)
	if err != nil {
		t.Fatal(err)
	}
	if id != 1 {
		t.Fatalf("id = %d, want 1", id)
	}
}

Run the test with your test command, inside a Dagger session. dagger api with-session starts the session and gives it to the client:

dagger api with-session go test ./store/ -run TestOrdersWithDaggerPostgres -v
=== RUN   TestOrdersWithDaggerPostgres
--- PASS: TestOrdersWithDaggerPostgres (0.72s)
PASS
ok  	example.com/orders/store	0.725s

The test meets the four requirements from its own code:

  • A fresh service for each run. The service starts when the test needs it, and stops when the client closes. A second run created the orders table again and got id 1 again.
  • Healthy before the tests start. Start returns after the Dagger health check passes on the exposed port (Running services).
  • A known address. The tunnel opens a port on the host, and Endpoint returns its address to the test.
  • The same setup locally and in CI. The only external dependency of the test is Dagger (self-orchestrating tests).

Choose the Compose model or the Testcontainers model

Dagger supports both models, so the choice is where each service definition lives:

  • The Compose model puts one definition in a Dagger module. Choose it when several languages or projects use the same services. The tests read environment variables and have no container code.
  • The Testcontainers model puts the definition in the test. Choose it when a test controls its own service, such as a new database for each test class.

The Testcontainers library differs from a Dagger client in two ways. It ships a module for each common service, with a wait strategy already set (Wait strategies). With a Dagger client, you write the container definition. Testcontainers also runs under the plain test command. The Dagger client library runs inside a Dagger session, such as one from dagger api with-session (Client libraries).

Bottom line: To test without mocks, give each test run a fresh database or Kafka container. It must be healthy before the tests start, at a known address, and the same locally and in CI. Testcontainers meets these requirements from test code, and Docker Compose or CI service containers meet them from outside the tests. Dagger does both: define the services once in a module, or start them from test code through a Dagger client.