Dagger 0.3.10: New pipeline API, support for container metadata, and more

It's the start of a new year and to celebrate, we've been hard at work adding improvements to Dagger. Our latest release of the Dagger is v0.3.10, which incorporates two significant new features for status reporting and container metadata management.

Dagger 0.3.10 Highlights

New Pipeline API

Dagger now provides a Pipeline API, which enables developers and operators to programmatically group related operations together into "pipelines" for status reporting and visualization purposes (e.g. logs). This new feature makes it easier to cut through the "noise" of your CI's daily operations and improves visibility of specific operations or events.

Pipelines are hierarchical and can contain any number of child pipelines. Here's an example of using this new API in Go:

// `c` belongs to the root pipeline at this point
c, _ := dagger.Connect(ctx)

// everything using `c` is nested under the `sdk` pipeline from now on
c = c.Pipeline("sdk")

nodejs :=
c.Pipeline("nodejs").Container().From("node:16").WithExec([]string{"yarn install"}) // pipeline is `sdk/nodejs`
lint := nodejs.Pipeline("lint").WithExec([]string{"yarn lint"}) // 
pipeline is `sdk/nodejs/lint`
test := nodejs.Pipeline("test").WithExec([]string{"yarn test"}) // 
pipeline is `sdk/nodejs/test`

Support for Container Metadata

Dagger now allows developers to add and manage container metadata via support for the Dockerfile LABEL instruction. For any given container, you can now add a label (withLabel), remove a label (withoutLabel), print the value of a label (label) or print all the labels (labels).

Here's an example of using these new methods in Go:

c, _ := dagger.Connect(ctx, dagger.WithLogOutput(os.Stdout))

gitTag := "v0.3.10"
repo :=
c.Git("https://github.com/dagger/dagger.git").Tag(gitTag).Tree()
daggerBinary := c.Container().From("golang:1.19-alpine3.17").
    WithMountedDirectory("/src", repo).
    WithEnvVariable("CGO_ENABLED", "0").
    WithWorkdir("/src").
    WithExec([]string{"go", "build", "-o", "dagger", "./cmd/dagger"}).
    File("dagger")

publishTarget := "fake.registry.invalid/test/publish:latest"
labelName := "dagger.version"
_, err := c.Container().From("gcr.io/distroless/static-debian11")
     .WithFile("/bin/dagger", daggerBinary).
     WithLabel(labelName, gitTag).
     Publish(ctx, publishTarget)
if err != nil {
    return err
}

labelVal, err := c.Container().From(publishTarget).Label(ctx, labelName)
if err != nil {
    return err
}
fmt.Println(labelName, "=", labelVal)

SDK Highlights

We've also released new versions of our SDKs with support for all the new features in Dagger 0.3.10, plus various SDK-specific bug fixes and improvements.

Targeted Builds

All SDKs now have an improved Build() method that allows users to build a particular target in a multi-stage Dockerfile. This was a frequently requested feature from the community. Here's an example using the Nodejs SDK:

import { connect } from "@dagger.io/dagger";

connect(async (client) => {
    const src = client
    .directory()
    .withNewFile("Dockerfile",
      `FROM node:16-slim AS base
       CMD echo "base"
       FROM base AS stage1
       CMD echo "stage1"
       FROM base AS stage2
       CMD echo "stage2"
       `
    )

      const output = await client.container().build(src, { target: 
  "stage1" }).withExec([]).stdout()

    // This will output "stage1"
    console.log(output)
  
  }, { LogOutput: process.stdout })

Improved Error Messages

The Python SDK now provides more verbose error messages whenever a Dagger session connection cannot be established. This makes it quicker and easier to debug failed sessions. Here's an example:

Before

dagger.exceptions.ProvisionError: Dagger engine failed to start

After

dagger.exceptions.SessionError: Dagger engine failed to start:Command 
'dagger/dagger-0.3.9 session' returned non-zero exit status 1.Error: 
failed to run container: docker: Cannot connect to the Docker daemon 
at unix:///var/run/docker.sock.Is the docker daemon running?.

Other Improvements

For a complete list of improvements, refer to the changelog for each SDK:

• Go SDK 0.4.4 changelog and documentation

• Node.js SDK 0.3.2 changelog and documentation

• Python SDK 0.3.1 changelog and documentation

New Documentation

Interested in using Dagger with GitHub? We've added a new end-to-end tutorial that teaches you how to use a Dagger pipeline to continuously build and deploy a Node.js application with GitHub Actions.

Read the tutorial

We look forward to releasing more features and documentation soon! In the meanwhile, if you have feedback or would like to suggest new features or documentation, let us know on Discord or create a GitHub issue.