CI that keeps up
how-toby Kyle Penfound

How do I run deno test, lint, fmt, and check in CI?

A good Deno setup for CI runs deno fmt --check, deno lint, deno check, and deno test on one pinned Deno release. It grants the tests only the permissions they need, and it caches the dependencies between runs. In a Deno workspace, it runs the tools from the workspace root, so that they cover every member. You can meet each requirement with the Deno CLI and a CI job. Dagger’s official Deno module meets the same requirements in one place, and runs the same checks on a laptop and in CI.

What a good Deno CI setup needs

Deno’s own docs give five requirements. This guide uses them to judge each approach, Dagger included:

  1. Run the four built-in tools. Deno’s CI guide runs deno fmt --check, deno lint, and deno test. deno check “type-checks your TypeScript (or JavaScript) code without running it”, which the docs call “useful in CI pipelines”.
  2. Use one Deno release everywhere. The CI guide’s example asks for v2.x, the latest stable release. denoland/setup-deno also takes an exact version, or reads one from .dvmrc or .tool-versions. One pinned release gives the same formatter, linter, and type checker on every machine.
  3. Grant the tests only the permissions they need. Deno code has no access to the file system, the network, or the environment unless you grant permissions. The CI guide’s example uses --allow-all, but adds that “it is recommended to run with the minimal permissions your program needs”.
  4. Cache the dependencies between runs. The CI guide says that a download of the dependencies in every run “can become a time-consuming process”. It tells you to cache them.
  5. Run a Deno workspace from its root. In a Deno workspace, the docs say: “To run tests across all workspace members, simply execute deno test from the workspace root”. deno check, deno fmt, and deno lint also cover every member from the root.

Run the Deno tools by hand in a CI job

The Deno CLI meets each requirement. The examples in this guide use a repository with one standalone project and one Deno workspace with two members:

apps/api/deno.json          standalone project
packages/deno.json          Deno workspace: { "workspace": ["./core", "./ui"] }
packages/core/deno.json     member, "name": "@acme/core"
packages/ui/deno.json       member, imports "@acme/core"

In the standalone project, run the four tools. -P gives the tests the permissions that deno.json declares:

deno fmt --check
deno lint
deno check
deno test -P
Checked 3 files
Checked 2 files
Check main.ts
Check main_test.ts
Warning Permissions in the config file is an experimental feature and may change in the future.
Check main_test.ts
running 2 tests from ./main_test.ts
greet ... ok (163µs)
reads config file ... ok (585µs)

ok | 2 passed | 0 failed (5ms)

For the Deno workspace, run the same four commands once from packages. They cover core and ui, and the imports between the members resolve as they do in the workspace.

In CI, a GitHub Actions job installs Deno with denoland/setup-deno and then runs the same commands. Set deno-version to an exact release, such as v2.9.3, or set deno-version-file. Set cache: true to save the dependency cache. By default, the cache key includes a hash of the deno.lock files (caching dependencies). For a repository with several projects, the job runs the commands once in each project directory and once in each workspace root.

Where the hand-built setup strains

The commands are short, but each requirement lives in a different place:

  • The Deno release is in two places. The CI job sets deno-version, and each laptop has its own Deno. If they differ, deno fmt --check or deno lint can pass on a laptop and fail in CI.
  • The list of projects is a script. The CI job must run the tools in each standalone project and each workspace root. A new project means an edit to the job.
  • The permissions are in two places. A test run with --allow-all in CI can pass while the same test fails on a laptop that runs it with -P.
  • The cache stays in CI. The setup-deno cache does not reach the laptops where developers run the same checks.

Meet them with the Dagger Deno module

The Dagger Deno module meets the same five requirements, with the Deno image, the permissions, and the four tools defined in one place. A Dagger module is a package of functions that test, lint, or build a project. The Dagger Engine runs those functions in containers, on a laptop or in CI.

Complete the Quickstart in the repository first. Then install the module from the repository root:

dagger module install dagger.io/js/deno
dagger.io/js/deno ➡️ https://github.com/dagger/deno#v0.3.0
Installed module "deno" in /work/dagger.toml

The command adds the module to dagger.toml, the workspace configuration for everyone who uses the repository:

[modules.deno]
source = "dagger.io/js/deno"

The command also writes dagger.lock. This file pins the images, Git references, and module addresses that the workspace resolves (dagger.toml reference). Commit both files. A Dagger workspace is the directory that holds dagger.toml. It is a different thing from a Deno workspace.

The next sections show how the module meets each requirement.

Run the four built-in tools

A check is a function that passes or fails, such as a test run or a linter. The module has a check for each tool. List the checks of the workspace:

dagger check -l
MODULE  DENO-PROJECT  DENO-WORKSPACE  CHECK         DESCRIPTION
deno    *                             format-check  Check this project's formatting with `deno fmt --check`.
deno    *                             stale         staleness check: format this project with `deno fmt`
deno    *                             lint          Lint this project with `deno lint`.
deno    *                             test          Run this project's tests with `deno test`.
deno    *                             type-check    Type-check this project with `deno check`.
deno                  *               format-check  Check formatting of every member of this workspace with `deno fmt --check`.
deno                  *               stale         staleness check: format every member of this workspace with `deno fmt`
deno                  *               lint          Lint every member of this workspace with `deno lint`.
deno                  *               test          Run tests in every member of this workspace with `deno test`.
deno                  *               type-check    Type-check every member of this workspace with `deno check`.
# Use --all to expand collections and list each item.

Each column is a collection: a set of items that the module finds in the repository at run time. A * stands for every item in the collection. Run every check in the workspace:

dagger check
== CHECKS ==  ✔ 10 passed
✔ dag://?check=deno/projects/format-check 4.2s OK
✔ dag://?check=deno/projects/format/stale 4.2s OK
✔ dag://?check=deno/projects/lint 4.2s OK
✔ dag://?check=deno/projects/test 4.6s OK
✔ dag://?check=deno/projects/type-check 4.6s OK
✔ dag://?check=deno/workspaces/format-check 4.2s OK
✔ dag://?check=deno/workspaces/format/stale 4.2s OK
✔ dag://?check=deno/workspaces/lint 4.2s OK
✔ dag://?check=deno/workspaces/test 4.6s OK
✔ dag://?check=deno/workspaces/type-check 4.6s OK

Dagger runs the checks in parallel, and the command exits with a non-zero code if a check fails (Checking your code). In this guide’s example repository, the first dagger check after a change to apps/api/main.ts took 11.4 seconds. A rerun with no changes took 5.4 seconds. Both times are for the whole command, in one run on one machine.

When a check fails, the report shows the Deno output. All checks run, so you see every failure in one run. In this run, apps/api/main.ts had a type error and bad indentation:

✘ dag://?check=deno/projects/type-check 0.7s ERROR
  ✘ withExec deno check . 0.3s ERROR
  ┃ Check main.ts
  ┃ Check main_test.ts
  ┃ TS2322 [ERROR]: Type 'number' is not assignable to type 'string'.
  ┃     const x: string = 5;
  ┃           ^
  ┃     at file:///src/apps/api/main.ts:2:11
  ┃ 
  ┃ error: Type checking failed.
  ! exit code: 1

== RUN LOCALLY ==
dagger check "dag://?check=deno/projects/format-check"
dagger check "dag://?check=deno/projects/format/stale"
dagger check "dag://?check=deno/projects/lint"
dagger check "dag://?check=deno/projects/test"
dagger check "dag://?check=deno/projects/type-check"

== REGENERATE ==
dagger generate -y "dag://?generator=deno/projects/format"

The report gives one command for each failed check. Run a command to run that check again. The REGENERATE section gives the command that fixes the formatting.

The module has two checks for formatting. format-check runs deno fmt --check. The stale check comes from the format generator, a function that changes files and returns the changes as a changeset. The stale check fails when deno fmt would change a file. To fix the formatting, run the generator. Add --no-apply to print the changes and not write them:

dagger generate --no-apply --deno
== GENERATORS ==  ✔ 2 passed
✔ dag://?generator=deno/projects/format 0.1s OK
✔ dag://?generator=deno/workspaces/format 0.1s OK

apps/api/main.ts +1 -1

1 file changed, +1 -1 lines
Generated changes were not applied (--no-apply).

In a terminal, dagger generate shows the diff and asks before it writes the files. To write the files with no prompt, add -y. deno fmt uses the fmt options in your deno.json, such as lineWidth and semiColons (deno.json reference).

Use one Deno release everywhere

The module runs Deno in denoland/deno:alpine-<version>, and version defaults to 2.9.3. Set version to the Deno release that your team uses. Every project then runs on the same release, on every machine:

dagger module settings deno version 2.9.3

Dagger stores settings in dagger.toml under [modules.deno.settings], so they apply to everyone who uses the repository. A setting applies to every project and Deno workspace that the module checks. To list the settings, run dagger module settings deno:

MODULE  KEY               VALUE  DESCRIPTION
deno    version           2.9.3  Deno version for the default base image (deno…
deno    base                     Base container for Deno commands. Must have `…
deno    permissions              Permissions `test` and `compile` run with: "c…
deno    testArgs                 Extra `deno test` flags, e.g. ["--doc", "--pa…
deno    typeCheckArgs            Extra `deno check` flags, e.g. ["--allow-impo…
deno    typeCheckTargets         What `type-check` checks, relative to each pr…

To use a different image, set base to an image that has deno on its PATH, such as docker.io/denoland/deno:debian. When you set base, also set version to the release in that image. The module uses version to decide how it passes permissions. The module reference describes testArgs, typeCheckArgs, and typeCheckTargets.

Grant the tests only the permissions they need

The permissions setting controls the permissions that deno test gets:

  • config (the default) runs deno test -P. Deno then uses the permissions in your deno.json.
  • all runs deno test -A, which grants all permissions.
  • none grants no permissions.

With a version before 2.5.0, config makes test use -A. With the default setting and no permissions in deno.json, a test that reads a file fails:

  ┃ reads config file => ./main_test.ts:7:6
  ┃ error: NotCapable: Requires read access to "/src/apps/api/deno.json", run again with the --allow-read flag

To fix it, give the tests the permissions they need in apps/api/deno.json. Deno reads permissions from its config file:

{
  "tasks": {},
  "test": {
    "permissions": {
      "read": true
    }
  }
}

With this config, the same test passes with the default permissions setting. Deno prints a warning that permissions in the config file are an experimental feature. The tests get the same permissions on a laptop and in CI, because both read them from deno.json. If you do not want to keep permissions in deno.json, grant all permissions to the tests:

dagger module settings deno permissions all

To remove a setting and go back to the default, run dagger module settings -u deno permissions.

Cache the dependencies between runs

The module sets DENO_DIR to a cache volume (README). A cache volume is a directory whose contents persist across runs on the Dagger Engine. Every check of every project uses the same volume, so Deno downloads each dependency once for each engine. There is no cache key to maintain.

Run a Deno workspace from its root

The module puts each directory that has a deno.json or deno.jsonc in one of two collections:

  • Deno workspaces. A config with a non-empty workspace array is a Deno workspace, keyed by its directory. Its checks run from the workspace root and cover every member. The members are not keys of their own.
  • Projects. Every other config is a standalone project, keyed by its directory. A project at the repository root has the key ..

In the example, packages/core and packages/ui do not show as projects. The module checks them through packages, and the imports between members resolve as they do in the Deno workspace. A new project or workspace gets its checks with no change to dagger.toml. Add --all to list one row per project and per Deno workspace:

dagger check -l --all
MODULE  DENO-PROJECT  DENO-WORKSPACE  CHECK         DESCRIPTION
deno    apps/api                      format-check  Check this project's formatting with `deno fmt --check`.
deno                  packages        format-check  Check formatting of every member of this workspace with `deno fmt --check`.
deno    apps/api                      lint          Lint this project with `deno lint`.
deno                  packages        lint          Lint every member of this workspace with `deno lint`.
deno    apps/api                      stale         staleness check: format this project with `deno fmt`
deno                  packages        stale         staleness check: format every member of this workspace with `deno fmt`
deno    apps/api                      test          Run this project's tests with `deno test`.
deno                  packages        test          Run tests in every member of this workspace with `deno test`.
deno    apps/api                      type-check    Type-check this project with `deno check`.
deno                  packages        type-check    Type-check every member of this workspace with `deno check`.

To run every check on one project, give its directory relative to the repository root:

dagger check --deno-project=apps/api
== CHECKS ==  ✔ 5 passed
✔ dag://?check=deno/projects/format-check 0.2s OK
✔ dag://?check=deno/projects/format/stale 0.2s OK
✔ dag://?check=deno/projects/lint 0.2s OK
✔ dag://?check=deno/projects/test 0.2s OK
✔ dag://?check=deno/projects/type-check 0.2s OK

To run one check on one Deno workspace, use the two flags together:

dagger check --deno-workspace=packages --check=lint
== CHECKS ==  ✔ 1 passed
✔ dag://?check=deno/workspaces/lint 0.2s OK

To run deno test on every project and Deno workspace, select the module and the check:

dagger check --deno --check=test
== CHECKS ==  ✔ 2 passed
✔ dag://?check=deno/projects/test 0.1s OK
✔ dag://?check=deno/workspaces/test 0.1s OK

To select more than one key, repeat the flag.

Run the Deno checks locally and with Cloud Checks

Cloud Checks run your checks on Dagger’s Cloud Engines on each push and pull request, with no CI workflow file. Cloud Engines are Dagger Engines that Dagger Cloud runs for you. The repository must be on GitHub, and its origin remote must point to that repository. Turn on Cloud Checks from the repository root:

dagger cloud checks on

Commit dagger.toml and dagger.lock, then push. Each push runs the same checks that dagger check runs on your machine, with the same Deno release and the same permissions. The results show as GitHub check statuses and in Dagger Cloud (changelog). To see recent runs from the terminal, run dagger workspace activity. If a check fails in the cloud, run dagger check locally to get the same failure.

Bottom line: Run deno fmt --check, deno lint, deno check, and deno test on one pinned Deno release, with only the permissions the tests need. Cache the dependencies, and run each Deno workspace from its root. Dagger’s Deno module meets the same requirements in one place. It finds every project and workspace, pins the release with version, and caches DENO_DIR. It runs the same checks locally and with Cloud Checks.