CI that keeps up
how-toby Kyle Penfound

How do I run GitLab CI jobs locally?

To run a GitLab CI job locally, make the pipeline portable. Move the job’s steps out of .gitlab-ci.yml and into something your laptop can also run. Then the pipeline file only calls those steps. A portable pipeline has one definition of its steps, the same image and services everywhere, services that are ready before the tests, local runs on uncommitted files, and a thin CI file. Dagger meets all five requirements in one place. You run dagger check on your laptop, and a GitLab CI job runs the same command, so you do not push to test each change.

The requirements for a portable GitLab pipeline

A job that runs the same way before and after you push meets these requirements:

  1. One definition of the steps. The commands that test the project live in one place. Your laptop and the GitLab job both call it. If the commands live only in script: lines, you can test them only when you push.
  2. The same image and services. The tests use the same Python image and the same Postgres image in both places. In GitLab CI, services start extra containers for the job. The job reaches each service by a hostname that GitLab makes from the image name, such as postgres.
  3. Services ready before the tests. The tests start only after Postgres accepts connections. The GitLab runner waits for the exposed ports of each service before the job script starts.
  4. Uncommitted changes. A local run tests the files in your working tree, including new files that are not in Git yet.
  5. A thin CI file. The job installs a tool and runs one command. A thin file does not change when the tests change.

A typical Python pipeline does not meet the first and fifth requirements. The image, the service, the connection URL, and the commands exist only in .gitlab-ci.yml:

pytest:
  image: python:3.12
  services:
    - postgres:16
  variables:
    POSTGRES_PASSWORD: postgres
    DATABASE_URL: postgresql://postgres:postgres@postgres:5432/postgres
  script:
    - pip install . pytest
    - pytest -v

lint:
  image: python:3.12
  script:
    - pip install ruff
    - ruff check

What happened to gitlab-runner exec, and what replaces it

gitlab-runner exec ran one job from .gitlab-ci.yml on your machine. GitLab deprecated the command and removed it in GitLab Runner 17.0. GitLab first planned the removal for 16.0, then postponed it. The list of GitLab Runner commands has no exec now, and the runner has no command that replaces it.

Other tools run the pipeline YAML on your machine:

  • gitlab-ci-local is an open-source project on GitHub. It runs GitLab pipelines locally “as shell executor or docker executor”. Its README says that “only tracked files are synced” into isolated jobs, so you must run git add on new files first.
  • glci is “an experimental GitLab project”. It starts a mock GitLab API server and the official gitlab/gitlab-runner image, and runs each job in Docker or Podman.
  • A manual docker run copies the job’s image: and script: into one command. You add the services, the network, and the variables yourself.

With each of these tools, you get a result without a push. But the steps still exist in only one form: GitLab CI YAML. You can run them only in GitLab, or in a tool that copies how GitLab reads that YAML.

Where the YAML-only pipeline strains

The pipeline above is clear while it is small. As the project grows, these gaps appear:

  • Each change costs a pipeline run. You can test a new script: line, service, or variable only after a push. You wait for the pipeline to start and finish each time.
  • The services and URLs live in two places. A laptop setup, such as a Compose file and export lines, repeats the images and the DATABASE_URL. The two copies must change together.
  • The laptop environment differs. A local pytest uses your Python and your system libraries. The job uses python:3.12. A test can pass in one place and fail in the other.

Meet the requirements with Dagger

Dagger runs each step in a container that the project defines. The steps and their environment are therefore the same on a laptop and in GitLab CI. The steps run on the Dagger engine, the program that runs the containers and caches their results. On a developer machine, the CLI starts a local engine by default. It starts the engine from an image in a container runtime such as Docker or Podman (Remote execution).

A Dagger module is a package of functions that build and test a project. A check is a function that dagger check runs, and that passes or fails. The official Pytest module finds the Python projects and test files, and runs pytest as a check.

Install the Dagger CLI. Then, from the project root, initialize the workspace:

dagger init

dagger init creates dagger.toml, the workspace configuration for everyone who uses the repository. Install the Pytest module, the Ruff module for the lint job, and the Dang SDK:

dagger module install dagger.io/python/pytest
dagger module install dagger.io/python/ruff
dagger module install dagger.io/sdk/dang

Dagger also writes dagger.lock, which pins each module and image to an exact version. Commit dagger.toml and dagger.lock.

The tests need Postgres. The Pytest module’s base setting takes the container that runs the tests, with Python and uv. A small module supplies that container, with Postgres bound to it. Dang is Dagger’s native language for modules. Create the project’s dev module:

dagger module init dang -y

The command creates .dagger/modules/shop-dev, named after the project directory, and installs it as the workspace main module. In the snippets, shop-dev and ShopDev stand in for the names that dagger module init generated. Replace .dagger/modules/shop-dev/main.dang with this code:

"""
Test services for the shop project: the same Postgres as the GitLab CI job.
"""
type ShopDev {
  """
  Postgres, with the image and password that the GitLab CI job uses.
  """
  pub postgres: Service! {
    container
      .from("postgres:16")
      .withEnvVariable("POSTGRES_PASSWORD", "postgres")
      .withExposedPort(5432)
      .asService
  }

  """
  The pytest container: Python 3.12 and uv, with Postgres bound to it.
  """
  pub testBase: Container! {
    container
      .from("ghcr.io/astral-sh/uv:python3.12-trixie-slim")
      .withServiceBinding("postgres", postgres)
      .withEnvVariable("DATABASE_URL", "postgresql://postgres:postgres@postgres:5432/postgres")
  }
}

Give the Pytest module this container as its base:

dagger module settings pytest base test-base

The command writes a module reference to dagger.toml:

[modules.pytest.settings]
base = "dag://shop-dev/test-base"

One definition of the steps

The test and lint jobs now live in dagger.toml and the dev module. List the checks:

dagger check -l
MODULE    PYTEST-PROJECT  PYTEST-TEST-FILE  RUFF-PROJECT  CHECK  DESCRIPTION
dang-sdk                                                  stale  staleness check: re-generate modules and clients managed by the Dang SDK
pytest    *               *                               test   Run this test file.
ruff                                        *             stale  staleness check: format this project with ruff
ruff                                        *             lint   Lint this project with ruff.
# Use --all to expand collections and list each item.

Run every check on your laptop:

dagger check
CHECKS  ✔ 4 passed
✔ dag://?check=dang-sdk/generate/stale 1.7s OK
✔ dag://?check=ruff/projects/format/stale 2.1s OK
✔ dag://?check=ruff/projects/lint 2.1s OK
✔ dag://?check=test&pytest-project=. 2.0s OK

In this guide’s project, the first dagger check took 8.6 seconds. A rerun with no changes took 3.9 seconds, because Dagger got the results from its cache. These numbers come from one run on one machine, and your project will give different times.

A failure prints the tool’s own output, followed by the command that runs the same check again. For example, a test file that imports sys and does not use it fails the lint check:

CHECKS  ✘ 1 failed  ✔ 1 passed
✘ dag://?check=ruff/projects/lint 1.0s ERROR
  ✘ PyprojectPythonProject.step(
    ┆ ctr: withWorkdir .
    ┆ args: ["ruff", "check"]
    ┆ what: "ruff check"
    ): Container! 0.2s ERROR
  ! ruff check failed (ruff check, exit code 1):
        F401 [*] `sys` imported but unused
         --> tests/test_unused.py:1:8
          |
        1 | import sys
          |        ^^^
        help: Remove unused import: `sys`
          |
          - import sys
        1 |
          |
        Found 1 error.
        [*] 1 fixable with the `--fix` option.
✔ dag://?check=ruff/projects/format/stale 1.1s OK

RUN LOCALLY
  dagger check "dag://?check=ruff/projects/lint"

To open a shell in the exact state where a command failed, run dagger check -i (Dev shells).

The same image and services

The tests run in ghcr.io/astral-sh/uv:python3.12-trixie-slim on every machine, not in the host’s Python. Postgres uses postgres:16, the same image as the GitLab job. withServiceBinding("postgres", ...) makes the service reachable at the hostname postgres. GitLab uses the same hostname for the postgres:16 service, so DATABASE_URL stays the same.

Services ready before the tests

Dagger health-checks a service before any client can connect, so the module needs no wait loop. To use the same Postgres with an app on your laptop, list the services and start one with dagger start postgres. Dagger tunnels the service’s ports to your machine.

dagger start -l
MODULE    SERVICE   DESCRIPTION
shop-dev  postgres  Postgres, with the image and password that the GitLab CI job uses.

Uncommitted changes

Dagger reads the files in your working tree, so a new test file runs before you git add it. To run one test file, select it with --pytest-test-file:

dagger check --pytest-test-file=tests/test_cart.py
CHECKS  ✘ 1 failed
✘ dag://?check=test&pytest-project=.&pytest-test-file=tests/test_cart.py 3.2s ERROR
  TESTS
    ✘ tests/test_cart.py › test_total FAIL
          def test_total():
          >       assert sum([2, 2]) == 5
          E       assert 4 == 5
          E        +  where 4 = sum([2, 2])
          
          tests/test_cart.py:2: AssertionError

    ✘ 1 failed

RUN LOCALLY
  dagger check "dag://?check=test&pytest-project=.&pytest-test-file=tests/test_cart.py"

In this run, tests/test_cart.py was not tracked in Git.

Other jobs: Vitest

A job that runs vitest run in a Node image moves the same way. Install the official Vitest module, and pin the Node image that the job uses:

dagger module install dagger.io/js/vitest
dagger module settings vitest baseImageAddress node:22-alpine

The module finds each vitest.config.* or vite.config.*, installs the dependencies, and runs the project’s own Vitest:

CHECKS  ✔ 1 passed
✔ dag://?check=test&vitest-project=. 17.6s OK
  TESTS
    ✔ src/cart.test.js  1 passed

    ✔ 1 passed

A thin CI file

The GitLab job installs the Dagger CLI and runs dagger check. It holds no images, services, or test commands. Replace the jobs in .gitlab-ci.yml:

check:
  image: alpine:3.22
  variables:
    DAGGER_VERSION: "1.0.0-beta.21"
    DAGGER_ENGINE: cloud
  before_script:
    - apk add --no-cache curl
    - curl -fsSL https://dl.dagger.io/dagger/install.sh | BIN_DIR=/usr/local/bin sh
  script:
    - dagger check

Set DAGGER_VERSION to the version that dagger version prints on your laptop. DAGGER_ENGINE: cloud runs the checks on Cloud Engines, the engines that Dagger Cloud runs for you. This setting also needs DAGGER_CLOUD_TOKEN. Store the token as a masked CI/CD variable in the project settings, not in the file. Cloud Engines bill for the CPU, memory, and data that each pipeline uses, as listed on Dagger pricing.

To keep the checks on your own machines, set DAGGER_ENGINE to a self-hosted engine instead, such as tcp://HOST:PORT or kube-pod://POD?namespace=NAMESPACE. The Dagger Helm chart works best when the CI runners also run in the cluster, for example with the GitLab Runner Kubernetes executor. If you do not set DAGGER_ENGINE, the CLI starts an engine on the runner. An ephemeral runner then loses the cache of that engine after each job.

For a GitLab project, .gitlab-ci.yml runs the checks, as shown above. For a GitHub repository, Cloud Checks runs dagger check after each push with no CI file.

Bottom line: GitLab removed gitlab-runner exec in GitLab Runner 17.0. gitlab-ci-local and glci run your .gitlab-ci.yml on your machine, but the steps still exist only in GitLab YAML. Put the steps, the test image, and the Postgres service in Dagger, and run dagger check on your laptop before you push. A one-job .gitlab-ci.yml then runs the same command in GitLab CI.