CI that keeps up
how-toby Kyle Penfound

How do I run Vitest in CI, with sharding and merged coverage?

A good Vitest setup for CI runs vitest run once, with the project’s own Vitest on a supported Node version. It fails when coverage drops below a threshold, and it lets you rerun one test file. For a large suite, it splits the test files into shards and merges their blob reports into one coverage report. You can meet each requirement with the Vitest CLI and a CI job. Dagger’s official Vitest module meets the same requirements in one place, and runs the same checks on a laptop and in CI.

What a good Vitest CI setup needs

Vitest’s own docs give five requirements. This guide uses them to judge each approach, Dagger included:

  1. Run Vitest once, not in watch mode. Vitest “starts in watch mode by default in development environment and run mode in CI environment” (Features). It detects CI from process.env.CI. vitest run does a single run in every environment.
  2. Use the project’s own Vitest on a supported Node version. The docs say: “It is recommended that you install a copy of vitest in your package.json”. The current Vitest requires Node v22.12.0 or newer (Getting Started).
  3. Fail when coverage drops below a threshold. A positive threshold “will be interpreted as the minimum percentage of coverage required” (coverage.thresholds). Coverage needs a coverage provider, such as @vitest/coverage-v8, in the project’s devDependencies.
  4. Rerun one test file. With a file name argument, Vitest “will only run test files whose path contains the given string” (Test Filtering). A developer can then repeat one CI failure without the full suite.
  5. Split a large suite into shards and merge the results. Vitest runs one part of the suite on each machine with --shard and --reporter=blob. The docs say: “All test and coverage results can be merged at the end of your CI pipeline using --merge-reports command”.

Run Vitest by hand in a CI job

The Vitest CLI meets each requirement. Install vitest as a dev dependency, then run the suite one time from the project root:

npx vitest run
 RUN  v5.0.5 /work

 ✓ src/cart.test.ts (1 test) 2ms
 ✓ src/login.test.ts (2 tests) 2ms

 Test Files  2 passed (2)
      Tests  3 passed (3)
   Start at  01:29:07
   Duration  110ms (transform 68%, import 20%, worker 7%, tests 5%)

To rerun one file, give its path: npx vitest run src/cart.test.ts. To fail on low coverage, add --coverage and a threshold, such as npx vitest run --coverage --coverage.thresholds.lines=90, or set coverage.thresholds in the Vitest config.

In CI, a job checks out the code and installs Node with actions/setup-node or the equivalent of the CI system. Then it installs the dependencies and runs the same command. GitHub Actions sets CI to true in every job (variables reference), so a plain vitest also does one run there.

To split the suite, run one shard on each machine with the blob reporter. Then merge the reports in a last step:

vitest run --reporter=blob --shard=1/2 --coverage
vitest run --reporter=blob --shard=2/2 --coverage
vitest run --merge-reports --coverage

Each shard writes its results to .vitest/blob/. Before the merge, copy the blob files from each machine into one .vitest/blob/ directory. Pass --coverage to the merge command too. In this guide’s project, the first shard reported 33.33% line coverage and the second shard reported 75%. The merged report showed 100%. The report of one shard is not the coverage of the whole suite.

The Vitest docs give a GitHub Actions example. A matrix runs one job for each shard, and each job uploads .vitest as an artifact. A last job downloads the artifacts and runs vitest --merge-reports.

Where the hand-built setup strains

The commands are short, but each requirement lives in a different place:

  • The Node version is in two places. The CI job sets node-version, and each laptop has its own Node. They can drift apart, and Vitest then runs on a Node version that the project did not test.
  • The shards are a fixed count of files. Vitest “splits your test files, not your test cases, into shards” (sharding). The matrix fixes the shard count, so a few slow files can make one shard much slower than the others.
  • The merge is CI plumbing. Each shard needs an artifact upload, and the merge needs a separate job that downloads them all. A move to another CI system means a rewrite of these jobs.
  • A CI failure is hard to repeat. To rerun a failed file as CI ran it, a developer must match the Node version and the install of the CI job by hand.

Meet them with the Dagger Vitest module

The Dagger Vitest module meets the same five requirements, with the Node image, the install, and the Vitest command defined in one place. A Dagger module is a package of functions that test, lint, or build a project. The Dagger Engine runs those functions in containers, on a laptop or in CI.

Complete the Quickstart in the repository first. Then install the module from the project root:

dagger module install dagger.io/js/vitest
dagger.io/js/vitest ➡️ https://github.com/dagger/vitest#v0.3.0
Installed module "vitest" in /work/dagger.toml

The command adds the module to dagger.toml, the workspace configuration for everyone who uses the repository:

[modules.vitest]
source = "dagger.io/js/vitest"

The command also writes dagger.lock. This file pins each module, image, and Git source to an exact version. Commit both files.

A check is a function that passes or fails, such as a test run. List the checks of the workspace:

dagger check -l
MODULE  VITEST-PROJECT  VITEST-TEST-FILE  CHECK  DESCRIPTION
vitest  *               *                 test   Run this test file.
# Use --all to expand collections and list each item.

Each column is a collection: a set of items that the module finds in the project at run time. A * stands for every item in the collection. The next sections show how the module meets each requirement.

Run Vitest once, not in watch mode

The module sets CI to true in its container (module source), so the project’s vitest does one run. Run every check in the workspace:

dagger check
== CHECKS ==  ✔ 1 passed
✔ dag://?check=test&vitest-project=. 27.8s OK

The module runs Vitest one time per project, over the selected test files. When you select all the files of a project, it runs vitest with the project’s config, so the config decides what runs. Dagger runs the checks of the workspace in parallel (Checking your code).

Use the project’s own Vitest on a supported Node version

The module runs the project’s own node_modules/.bin/vitest, not a copy of its own (README). It installs the dependencies with the package manager that the packageManager field of package.json names. Without that field, it reads the lockfile. If it finds neither, it uses npm. To choose a package manager, set the packageManager setting.

In a monorepo, the module installs dependencies at the nearest workspace root. That root is a directory with pnpm-workspace.yaml, or a package.json with workspaces.

The tests run in node:25-alpine by default. If the project uses a different Node release, set the baseImageAddress setting to that release:

dagger module settings vitest baseImageAddress node:22-alpine

Dagger stores settings in dagger.toml under [modules.vitest.settings], so every machine uses the same Node image. A setting applies to every Vitest project in the workspace. The module reference lists the other install settings, such as installFlags and includeExtraFiles.

Fail when coverage drops below a threshold

The module runs the project’s config, so coverage.thresholds in the Vitest config apply. To set coverage for every project instead, use the flags setting. It passes extra flags to every vitest run. The project must list a coverage provider, such as @vitest/coverage-v8, in its devDependencies.

dagger module settings vitest flags '["--coverage", "--coverage.thresholds.lines=90"]'

The check prints the coverage table. It fails when coverage is below the threshold. In this run, the check ran only src/cart.test.ts:

  ┃ [vitest]  % Coverage report from v8
  ┃ [vitest] ----------|---------|----------|---------|---------|-------------------
  ┃ [vitest] File      | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s 
  ┃ [vitest] ----------|---------|----------|---------|---------|-------------------
  ┃ [vitest] All files |      40 |        0 |   66.66 |   33.33 |                   
  ┃ [vitest]  cart.ts  |      40 |        0 |   66.66 |   33.33 | 6-7               
  ┃ [vitest] ----------|---------|----------|---------|---------|-------------------
  ┃ [vitest] ERROR: Coverage for lines (33.33%) does not meet global threshold (90%)
  ! Vitest project .: vitest failed (exit 1):

To remove the setting, run dagger module settings -u vitest flags.

Rerun one test file

Each test file is an item that you can select. Add --all to list one row per test file:

dagger check -l --all
MODULE  VITEST-PROJECT  VITEST-TEST-FILE   CHECK  DESCRIPTION
vitest  .               src/cart.test.ts   test   Run this test file.
vitest  .               src/login.test.ts  test   Run this test file.

The module treats each directory that holds a vitest.config.* or vite.config.* file as a Vitest project. The project at the workspace root has the key .. Test files are the files that match Vitest’s default include pattern, such as *.test.ts and *.spec.js. The module does not read the include and exclude settings in your config to list files.

To run one test file, give its path relative to the project:

dagger check --vitest-test-file=src/cart.test.ts
== CHECKS ==  ✔ 1 passed
✔ dag://?check=test&vitest-project=.&vitest-test-file=src/cart.test.ts 2.2s OK

In a monorepo, each app with its own Vitest config is a project. List the projects:

dagger list vitest-projects -a
MODULE  VITEST-PROJECT  DESCRIPTION
vitest  apps/admin      The Vitest project rooted at path.
vitest  apps/web        The Vitest project rooted at path.

To run one file in one project, give the project path relative to the workspace root:

dagger check --vitest-project=apps/web --vitest-test-file=src/a.test.ts
== CHECKS ==  ✔ 1 passed
✔ dag://?check=test&vitest-project=apps/web&vitest-test-file=src/a.test.ts 11.2s OK

Repeat a flag to select more than one key. To print the tests that Vitest finds in a project, call the project’s list function:

dagger api call vitest project --path=apps/web list
src/a.test.ts > a
src/b.test.ts > b

When a test fails, the check fails and prints the Vitest output. The report ends with a command that runs the same selection again, in the same container, on any machine:

== RUN LOCALLY ==
dagger check "dag://?check=test&vitest-project=."

Split a large suite into shards and merge the results

The module runs the selected files of each project in one Vitest run. The coverage report of a project then comes from one run and needs no merge. In a monorepo, the module runs each project as a separate check, and Dagger runs the checks in parallel.

Do not put --shard in flags. The module passes flags to every run, so each project would run only part of its files. For a large suite, Dagger splits the tests across engines instead. It balances the parts with the measured resource use, input cost, and cache state of each test (test splitting). You do not fix a shard count or move blob reports between jobs.

Run the same checks locally and with Cloud Checks

Cloud Checks run your checks on Dagger’s Cloud Engines on each push and pull request, with no CI workflow file. Cloud Engines are Dagger Engines that Dagger Cloud runs for you. The repository must be on GitHub, and its origin remote must point to that repository. Turn on Cloud Checks from the project root:

dagger cloud checks on

Commit dagger.toml and dagger.lock, then push. Each push runs the same checks that dagger check runs on your machine, with the same Node image and the same settings. The results show as GitHub check statuses and in Dagger Cloud (changelog). The module reports each test as its own span, so Dagger Cloud shows each test with no change to your Vitest config. To see recent runs from the terminal, run dagger workspace activity. If a check fails in the cloud, run dagger check locally to get the same failure.

Bottom line: Run vitest run with the project’s own Vitest on a supported Node version, and set a coverage threshold. For a large suite, run shards with --reporter=blob and merge them with --merge-reports --coverage. Dagger’s Vitest module meets the same requirements in one place. It pins the Node image in dagger.toml, selects single files with --vitest-test-file, and runs the same checks on your machine and with Cloud Checks.