Dagger 0.4: Service containers, secrets scrubbing, and more

March 8, 2023

Mar 8, 2023

Share
Share
Share
Share

Today we are delighted to introduce Dagger 0.4.0, featuring service containers, secrets scrubbing, and more.

Service containers

Service containers have been one of the most requested features. They are now finally available. This feature enables you to run network services - like databases or webapps - as ephemeral containers, directly inside a Dagger pipeline.

Some common use cases for this new feature are:

  • Run a test database

  • Run end-to-end integration tests

  • Run sidecar services

Service containers come with the following built-in features:

  • Service containers are started just-in-time, de-duplicated, and stopped when no longer needed

  • Service containers are health checked prior to running clients

  • Service containers are given an alias for the client container to use as its hostname

Using service containers

Binding a service to a container expresses a dependency: the service container needs to be running when the client container runs. The bound service container is started automatically whenever its client container runs.

Here's an example of creating an HTTP service and fetching from it:

package main

import (
	"context"
	"fmt"
	"os"

	"dagger.io/dagger"
)

func main() {
	ctx := context.Background()

	// create Dagger client
	client, err := dagger.Connect(ctx, dagger.WithLogOutput(os.Stderr))

	if err != nil {
		panic(err)
	}
	defer client.Close()

	// create HTTP service container with exposed port 8080
	httpSrv := client.Container().
		From("python").
		WithDirectory("/srv", client.Directory().WithNewFile("index.html", "Hello, world!")).
		WithWorkdir("/srv").
		WithExec([]string{"python", "-m", "http.server", "8080"}).
		WithExposedPort(8080)

	// create client container with service binding
	// access HTTP service and print result
	val, err := client.Container().
		From("alpine").
		WithServiceBinding("www", httpSrv).
		WithExec([]string{"wget", "-O-", "http://www:8080"}).
		Stdout(ctx)

	if err != nil {
		panic(err)
	}

	fmt.Println(val)
}

Here, the WithExposedPort() method sets the ports that the container will listen on. Dagger checks the health of each exposed port prior to running any clients that use the service, so that clients don't have to implement their own polling logic.

Want to learn more about how services work in Dagger? Check out our documentation.

Secrets scrubbing

Dagger now automatically scrubs secrets from its various logs and output streams. This ensures that sensitive data does not leak - for example, in the event of a crash. This applies to secrets stored in both environment variables and file mounts.

Here's an example that demonstrates this in action:

package main

import (
	"context"
	"fmt"
	"os"

	"dagger.io/dagger"
)

func main() {
	cleanupEnv := setDemoHostEnv("my_secret_file")
	defer cleanupEnv()

	ctx := context.Background()
	// create Dagger client
	client, err := dagger.Connect(ctx)
	if err != nil {
		panic(err)
	}
	defer client.Close()

	secretEnv := client.Host().EnvVariable("MY_SECRET_ENV").Secret()
	secretFile := client.Host().Directory(".").File("my_secret_file").Secret()

	output, err := client.Container().
		From("alpine@sha256:e2e16842c9b54d985bf1ef9242a313f36b856181f188de21313820e177002501").
		WithSecretVariable("MY_SECRET_ENV", secretEnv).
		WithMountedSecret("/my_secret_file", secretFile).
		WithExec([]string{"sh", "-c", `echo -e "env secret data: $MY_SECRET_ENV || secret file data: "; cat /my_secret_file`}).
		Stdout(ctx)

	if err != nil {
		panic(err)
	}

	fmt.Println(output)
}

func setDemoHostEnv(hostFilePath string) (cleanup func() error) {
	// Setting a test host environment variable
	const envName = "MY_SECRET_ENV"
	err := os.Setenv(envName, "Secret Env Content")
	if err != nil {
		panic(err)
	}

	// Setting a test host file
	const hostFileContent = "Secret File Content"
	err = os.WriteFile(hostFilePath, []byte(hostFileContent), 0o644)
	if err != nil {
		panic(err)
	}

	return func() error {
		return os.Remove(hostFilePath)
	}
}

The output will be:

env secret data: *** || secret file data: 
*

Instead of

env secret data: Secret Env Content || secret file data: 
Secret File Content
SDK Updates

We've also released new versions of our SDKs with support for all the new features in Dagger 0.4.0, plus various SDK-specific bug fixes and improvements.

For a complete list of improvements, refer to the changelog for each SDK:

We look forward to releasing more features soon! Meanwhile, if you have feedback or would like to suggest new features or documentation, let us know on Discord or create a GitHub issue.


Get Involved With the community

Discover what our community is doing, and join the conversation on Discord & GitHub to help shape the evolution of Dagger.

Subscribe to our newsletter

Get Involved With the community

Discover what our community is doing, and join the conversation on Discord & GitHub to help shape the evolution of Dagger.

Subscribe to our newsletter

Get Involved With the community

Discover what our community is doing, and join the conversation on Discord & GitHub to help shape the evolution of Dagger.

Subscribe to our newsletter