CI that keeps up
how-toby Kyle Penfound

How do I run Playwright tests against a Docker Compose app in CI?

Playwright tests against a Docker Compose app need five things. Compose starts the app and its database before the tests and waits until they are ready. The tests reach the app through baseURL, they run in the Playwright image for your version, and each run controls its test data. Each browser project can also run on its own. Teams meet these with Playwright’s webServer option, docker compose, and a CI job. Dagger meets the same five requirements in one place, with its Docker and Playwright modules, on a laptop and in CI.

What Playwright tests against a Compose app need

Playwright’s docs give four requirements for tests against an app that Compose starts. This guide judges each approach by them:

  1. Start the app and its services before the tests. Playwright’s webServer option can “launch a local dev server before running your tests”. It waits until the url returns “a 2xx, 3xx, 400, 401, 402, or 403 status code”. The app must also wait for its database. Compose does that only with a healthcheck and a service_healthy condition (startup order).
  2. Point the tests at the app with baseURL. Playwright recommends that you set baseURL in the use block, “so that tests can use relative urls” (Adding a baseURL). The address of the app changes between a laptop and CI, so the config must read it from the environment.
  3. Run the tests in the Playwright image for your exact version. The Playwright Docker image has the browsers and their system dependencies. If its version differs from your @playwright/test version, Playwright “will be unable to locate browser executables”. The guide to Playwright in CI with cached browsers covers this requirement in full.
  4. Control the test data. Playwright’s best practices say, “If working with a database then make sure you control the data” (Testing with a database). A database from an earlier run can hold data that changes the result.
  5. Run each browser project on its own. A Playwright config can define projects, such as one for each browser. Playwright says, “You can also use projects to run the same tests in different configurations.” The --project option runs one project.

Meet the requirements with webServer and Compose

Teams start the Compose app from Playwright’s own config, and remove it after the run. The examples come from a project with a pnpm workspace. The Playwright suite is in e2e/, next to this Compose file, e2e/compose.yaml:

services:
  web:
    build: ../app
    ports:
      - "8080:8080"
    environment:
      DATABASE_URL: postgres://app:app@db:5432/app
    depends_on:
      db:
        condition: service_healthy
  db:
    image: postgres:17
    environment:
      POSTGRES_USER: app
      POSTGRES_PASSWORD: app
      POSTGRES_DB: app
    ports:
      - "5435:5432"
    volumes:
      - pgdata:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD", "pg_isready", "-U", "app"]
      interval: 2s

volumes:
  pgdata:

Start the app and its services before the tests. The healthcheck and condition: service_healthy make Compose start web only after Postgres answers pg_isready. In playwright.config.ts, a webServer runs docker compose up and waits for the app:

webServer: process.env.PLAYWRIGHT_BASE_URL
  ? undefined
  : {
      command: 'docker compose up',
      url: 'http://localhost:8080',
      reuseExistingServer: true,
    },

Do not add --wait to this command. With --wait, Compose exits after the services start. Playwright then stops the run with Error: Process from config.webServer exited early. The condition on PLAYWRIGHT_BASE_URL turns webServer off when something else starts the app, as in the Dagger sections below.

Point the tests at the app with baseURL. In the use block, set baseURL from a variable, and keep your local URL as the fallback:

baseURL: process.env.PLAYWRIGHT_BASE_URL || 'http://localhost:8080',

Run the tests from e2e/:

npx playwright test

Playwright prints the Compose output with a [WebServer] prefix. The end of the output in this guide’s project:

Running 6 tests using 6 workers

  ✓  1 [firefox] › tests/db.spec.ts:4:5 › database port is reachable (24ms)
  ✓  4 [chromium] › tests/db.spec.ts:4:5 › database port is reachable (7ms)
  ✓  5 [chromium] › tests/secure-context.spec.ts:3:5 › page runs in a secure context (389ms)
  ✓  6 [chromium] › tests/home.spec.ts:4:5 › home page shows the database version (407ms)
  ✓  2 [firefox] › tests/secure-context.spec.ts:3:5 › page runs in a secure context (543ms)
  ✓  3 [firefox] › tests/home.spec.ts:4:5 › home page shows the database version (553ms)

  6 passed (6.3s)

Control the test data. After the run, the containers were still up, and the pgdata volume kept the database. Remove both, so the next run starts from an empty database:

docker compose down -v
 Container e2e-web-1  Stopping
 Container e2e-web-1  Stopped
 Container e2e-web-1  Removing
 Container e2e-web-1  Removed
 Container e2e-db-1  Stopping
 Container e2e-db-1  Stopped
 Container e2e-db-1  Removing
 Container e2e-db-1  Removed
 Volume e2e_pgdata  Removing
 Network e2e_default  Removing
 Volume e2e_pgdata  Removed
 Network e2e_default  Removed

The -v flag removes “named volumes declared in the “volumes” section of the Compose file“.

Run the tests in the Playwright image for your exact version. Playwright’s CI guide gives two ways to prepare a CI agent. One is the Playwright image as a container job, such as jobs.<job_id>.container on GitHub Actions. The other is npx playwright install --with-deps on the runner. With Compose, teams often use the second way, because the steps of a container job run in the Playwright image. That image has no Docker CLI. In this guide’s project, command -v docker in mcr.microsoft.com/playwright:v1.64.1-noble found nothing.

To run one browser project by hand, pass its name from the config:

npx playwright test --project chromium
Running 3 tests using 3 workers

  ✓  1 [chromium] › tests/db.spec.ts:4:5 › database port is reachable (7ms)
  ✓  3 [chromium] › tests/secure-context.spec.ts:3:5 › page runs in a secure context (171ms)
  ✓  2 [chromium] › tests/home.spec.ts:4:5 › home page shows the database version (177ms)

  3 passed (4.6s)

Where the hand-built setup strains

The hand-built setup meets the requirements, but each one depends on the machine that runs it:

  • Start the app and its services before the tests. webServer starts Compose on a laptop. In CI, the job needs Docker on the runner and free host ports, such as 8080 and 5435.
  • Point the tests at the app with baseURL. The address works only where the tests and Compose share a network. In a container job, localhost is the job container, not the app.
  • Run the tests in the Playwright image for your exact version. A runner that installs the browsers does not use the image. A container job uses the image, but it cannot run docker compose.
  • Control the test data. docker compose down -v must run after every run, also when the tests fail. If it does not run, the next run on that machine reuses the old database.
  • Run each browser project on its own. The project name goes on each command line. A CI job for each browser repeats the full job definition with only that name changed.

Meet the requirements with the Dagger Docker and Playwright modules

Connect two official Dagger modules in dagger.toml. Dagger is a CI orchestration engine that runs the same pipelines on a laptop and in CI. A module is a package of functions, checks, and services that a project installs. The Docker module reads the Compose file and starts the app with its dependencies. The Playwright module runs the tests against that app. You write no Dagger code.

Complete the Quickstart first. Then install the two modules from the repository root:

dagger module install dagger.io/docker@main
dagger module install dagger.io/js/playwright

The first command installs the Docker module from its main branch. Each install adds the module to dagger.toml, the workspace configuration, and the first install creates that file if the project has none. Dagger also writes dagger.lock. It pins the Git commit of each module and, after the first run, the images that the checks use, such as postgres:17. If a module is already installed, the command prints Module "playwright" is already installed and changes nothing.

A check is a function that validates the project, such as a test run. List the Playwright checks:

dagger check --playwright -l
MODULE      PLAYWRIGHT-PROJECT  PLAYWRIGHT-TEST-FILE  CHECK  DESCRIPTION
playwright  *                   *                     test   Run this test file.
# Use --all to expand collections and list each item.

The Playwright module finds each directory with a playwright.config.ts file, or a config file with the .js, .mjs, .cjs, .mts, or .cts extension. Each of these directories is a Playwright project for the module. A Playwright project is a directory, not a browser project such as chromium in the config. The test check runs the test files of a project. A * stands for every project or test file.

Start the app and its services before the tests, with Dagger

The Docker module starts each Compose service with the services in its depends_on list. The guide to Docker images and Compose services explains how it finds Compose files and which Compose settings it applies. One difference matters here. The current Docker module does not translate Compose health checks. It waits until the ports of the dependencies accept connections (module README). If the app opens its port before it is ready, add a readiness check to Playwright’s global setup.

List the services as links:

dagger list services -a -f=link
dag+service://?service=batch/up&docker-compose-project=e2e
dag+service://?service=docker/cli/engine
dag+service://?service=docker/engine
dag+service://?service=up&docker-compose-project=e2e&docker-compose-service=db
dag+service://?service=up&docker-compose-project=e2e&docker-compose-service=web

Copy the link of the service that the browser opens, here web. In the link, docker-compose-project is the directory of the Compose file, relative to the repository root. It is not the top-level name in the file. The value of docker-compose-service is the service name in the file.

Set the service setting of the Playwright module to that link. Put the link in quotes, so that the shell keeps the & characters:

dagger module settings playwright service 'dag+service://?service=up&docker-compose-project=e2e&docker-compose-service=web'

This is module wiring. A setting whose value is the address of a function in another module gets the result of that function. Dagger runs the up function of the Docker module and gives the service to the Playwright module. For a workspace with only these two modules, dagger.toml is now:

# Dagger workspace configuration
# Install modules with: dagger module install <module>
# Example:
#   dagger module install dagger.io/linux/wolfi

[modules]

[modules.docker]
source = "dagger.io/docker@main"

[modules.playwright]
source = "dagger.io/js/playwright"

[modules.playwright.settings]
service = "dag+service://?service=up&docker-compose-project=e2e&docker-compose-service=web"

Use the link from your own list. Keep the other modules in your dagger.toml.

Dagger starts the app, so webServer must stay off. The config in the previous section turns it off when PLAYWRIGHT_BASE_URL has a value, and the Playwright module sets that variable. If webServer stays on, Playwright runs its command in the test container, and the run fails:

Error: Process from config.webServer was not able to start. Exit code: 127
[WebServer] /bin/sh: 1: docker: not found

Some configs read a variable of their own that turns off webServer. If yours does, set that variable with the environment setting. For a config that reads PLAYWRIGHT_DISABLE_WEBSERVER:

dagger module settings playwright environment PLAYWRIGHT_DISABLE_WEBSERVER=1

The command writes environment = ["PLAYWRIGHT_DISABLE_WEBSERVER=1"] under [modules.playwright.settings]. The command replaces the whole list, so give all the variables in one command.

Point the tests at the app with baseURL, with Dagger

The Playwright module binds the wired service into the test container with the hostname frontend. It sets PLAYWRIGHT_BASE_URL to the first port of the service, here http://frontend:8080. The baseURL line from the hand-built setup reads that variable, so tests that open a relative path, such as page.goto('/'), open the wired app. Tests and helper clients that make absolute URLs must read the same variable. In the test container, localhost is the test container itself, not the app.

The Docker module publishes the ports of the selected service first, then the ports of its dependencies. Thus PLAYWRIGHT_BASE_URL points at web, not db. If the app publishes several ports, put its HTTP port first in ports. A port that is only in expose does not come first.

The ports of the dependencies are on frontend too. For a database with 5435:5432 in ports, a test client connects to frontend:5435. Give that hostname to the test helpers with the environment setting.

Browser features such as service workers and the Web Crypto API work only in a secure context. An HTTP page on frontend is not a secure context, but a page on localhost is. If the tests need a secure context, turn on the localhost proxy of the module:

dagger module settings playwright localhostProxy true

The proxy forwards the first port of the wired service from localhost in the test container. The module sets PLAYWRIGHT_LOCALHOST_BASE_URL to that address, here http://localhost:8080. In playwright.config.ts, read it first in the use block:

baseURL: process.env.PLAYWRIGHT_LOCALHOST_BASE_URL || process.env.PLAYWRIGHT_BASE_URL || 'http://localhost:8080',

In this guide’s project, a test of window.isSecureContext failed against frontend and passed with the proxy. The proxy forwards only the first port, so keep database clients on frontend. The module installs the proxy, socat, with apt-get, which the default Playwright images have.

Run the tests in the Playwright image for your exact version, with Dagger

The Playwright module runs the tests in the mcr.microsoft.com/playwright image for your version, and the wired app runs beside it. Commit the lockfile of your package manager. The module reads the @playwright/test version from it and picks the image tag for that version. Without a lockfile, pin @playwright/test to an exact version in package.json. The module reads a range such as ^1.58.2 as its lower bound. The install can then get a newer Playwright than the browsers in the image.

The default image has the browsers that Playwright bundles. It does not have the branded browsers. Remove projects with channel: 'chrome' or channel: 'msedge' from the run. To run one browser project at a time, use a Dagger env, as Run each browser project on its own, with Dagger shows. The module also sets CI=true, so review the workers and retries values that your config sets for CI.

The module installs the dependencies in the image, but it does not build workspace packages. In a monorepo, the tests can import a workspace package whose entry point a build makes. Without the build, the tests fail at the import:

Error: Cannot find module '/app/e2e/node_modules/@app/sdk/dist/index.js'. Please verify that the package.json has a valid "main" entry

The setup setting runs commands after the install, with the full repository source in place. For a pnpm workspace package named @app/sdk, add its build command:

dagger module settings playwright setup 'pnpm --filter @app/sdk build'

The command writes setup = ["pnpm --filter @app/sdk build"]. Setup runs in the Playwright project directory, before the module binds the wired service. Put preparation that needs the app in Playwright’s global setup.

Control the test data, with Dagger

Dagger runs the Compose services in ephemeral containers. It starts them when the tests need them, and stops them when nothing references them. There is no down step to forget.

Named volumes are different. The Docker module turns each named volume, such as pgdata, into a cache volume, a directory whose contents persist across runs (module README). Thus the database keeps its data between runs, as a Compose volume does without -v. If the tests need an empty database, reset the data in Playwright’s global setup.

Run each browser project on its own, with Dagger

A Dagger env stores the settings of a named run in dagger.toml. Use one env for each browser project, with the shared Compose service and test setup from the base settings. The --env option of the Dagger CLI selects or creates an env.

The config of this guide’s project defines the projects chromium and firefox. Create a matching env for chromium. The -- tells dagger that the arguments after it are values, not its own flags:

dagger module settings --env chromium playwright args -- --project chromium

The command prints Created env "chromium" and writes the setting under an env table:

[env.chromium]

[env.chromium.modules.playwright.settings]
args = ["--project", "chromium"]

Run the tests with that env:

dagger check --env chromium --playwright
== CHECKS ==  ✔ 1 passed
✔ dag://?check=test&playwright-project=e2e 6.1s OK
  == TESTS ==
    ✔ chromium  3 passed

    ✔ 3 passed

The env keeps the base settings, such as the wired Compose service. Its args value replaces the base args list, so put every Playwright argument that the run needs in it. Without --env, dagger check --playwright uses the base settings, and both projects ran, with 6 tests.

The env name is a label that you choose. The name after --project must match a project in your Playwright config, because the env does not create the browser project. Add an env for firefox the same way:

dagger module settings --env firefox playwright args -- --project firefox
dagger check --env firefox --playwright

Commit the env tables with the rest of dagger.toml, so that everyone runs the same browser projects.

Run the same tests locally and with Cloud Checks

Run the Playwright checks:

dagger check --playwright

Dagger builds or pulls the images of the Compose services, and starts the app and its dependencies. Then it runs the Playwright checks against the app. The --playwright flag selects only the checks of the Playwright module. In this guide’s project, with three test files and two browser projects, the report shows:

== CHECKS ==  ✔ 1 passed
✔ dag://?check=test&playwright-project=e2e 2.5s OK
  == TESTS ==
    ✔ chromium  3 passed
    ✔ firefox  3 passed

    ✔ 6 passed

If the repository has several directories with a Playwright config, select one with --playwright-project. The value is the directory of playwright.config, which can be different from the directory of the Compose file:

dagger check --playwright-project=e2e

Commit dagger.toml, dagger.lock, and the changes to playwright.config.ts. If the lockfile of your package manager is new, commit it too. Before the first push, run every check in the workspace. Cloud Checks run all of them, and these include the lint and build checks of the Docker module:

dagger check
== CHECKS ==  ✔ 5 passed
✔ dag://?check=build-service&docker-compose-project=e2e 0.5s OK
✔ dag://?check=docker/compose/projects/lint 0.5s OK
✔ dag://?check=docker/projects/build 0.4s OK
✔ dag://?check=docker/projects/lint 0.9s OK
✔ dag://?check=test&playwright-project=e2e 2.4s OK
  == TESTS ==
    ✔ chromium  3 passed
    ✔ firefox  3 passed

    ✔ 6 passed

In CI, Cloud Checks run the checks. Cloud Checks run your checks on Cloud Engines, Dagger’s managed engines, after each push and pull request, with no CI workflow file. The repository must be on GitHub, with its origin remote set to it. Turn them on:

dagger cloud checks on

In an interactive terminal, the command offers to sign you in to Dagger Cloud and to connect GitHub. After that, each push starts the Compose services and the Playwright tests on Cloud Engines, as dagger check does on a laptop. To see the runs, use dagger workspace activity.

Bottom line: Start the Compose app before the tests, and point Playwright at it with baseURL. Run the tests in the Playwright image for your version, and control the test data. By hand, webServer runs docker compose up and docker compose down -v cleans up. With Dagger, wire the app’s Compose service into the Playwright module, and dagger check --playwright runs the same tests on a laptop and with Cloud Checks. A Dagger env runs one browser project.