Service containers have been available since Dagger 0.4, making it easy to run services - like databases or web applications - as ephemeral containers directly inside Dagger. This first implementation of service containers only supported Container-to-Container (C2C) networking and didn't allow fine-grained control over the service lifecycle.
Today, we are pleased to announce Dagger 0.9, featuring a significantly improved services API that expands Dagger's built-in capabilities and brings us closer to the promise of Application Delivery as Code. This new API adds support for Container-to-Host (C2H) networking, Host-to-Container (H2C) networking, and explicit service lifecycle management.
NOTE: Dagger 0.9 includes a breaking change for binding service containers. The Container.withServiceBinding
API now takes a Service
instead of a Container
, so you must call Container.asService
on its argument. See the section on binding service containers in our documentation for examples.
With the new services API, it is now possible to expose services running inside a Dagger pipeline to the host and allow Dagger pipelines to access host network resources.
We expect this API to unlock a vast number of new use cases, including but not limited to:
Let's take a closer look at the new features. The following code samples are in Go, but you can find code samples for other languages in our documentation.
You can now expose service containers directly to the host so clients on the host can communicate with services running in Dagger. One common use case is application testing, where you need to be able to spin up temporary databases to run tests against.
Here's an example of a host sending requests to a web application running as a container.
package main
import (
"context"
"fmt"
"io"
"net/http"
"os"
"dagger.io/dagger"
)
func main() {
ctx := context.Background()
// create Dagger client
client, err := dagger.Connect(ctx, dagger.WithLogOutput(os.Stderr))
if err != nil {
panic(err)
}
defer client.Close()
// create web service container with exposed port 8080
httpSrv := client.Container().
From("python").
WithDirectory("/srv", client.Directory().WithNewFile("index.html", "Hello, world!")).
WithWorkdir("/srv").
WithExec([]string{"python", "-m", "http.server", "8080"}).
WithExposedPort(8080).
AsService()
// expose web service to host
tunnel, err := client.Host().Tunnel(httpSrv).Start(ctx)
if err != nil {
panic(err)
}
defer tunnel.Stop(ctx)
// get web service address
srvAddr, err := tunnel.Endpoint(ctx)
if err != nil {
panic(err)
}
// access web service from host
res, err := http.Get("http://" + srvAddr)
if err != nil {
panic(err)
}
defer res.Body.Close()
// print response
body, err := io.ReadAll(res.Body)
if err != nil {
panic(err)
}
fmt.Println(string(body))
}
You can also bind containers to host services so that clients in Dagger pipelines can communicate with services running on the host. Of course, this implies that a service is already listening on a port on the host, out-of-band of Dagger.
Here's an example of a container in a Dagger pipeline querying a MariaDB database service on the host:
package main
import (
"context"
"fmt"
"os"
"dagger.io/dagger"
)
func main() {
ctx := context.Background()
// create Dagger client
client, err := dagger.Connect(ctx, dagger.WithLogOutput(os.Stderr))
if err != nil {
panic(err)
}
defer client.Close()
// expose host service on port 3306
hostSrv := client.Host().Service([]dagger.PortForward{
{Frontend: 3306, Backend: 3306},
})
// create MariaDB container
// with host service binding
// execute SQL query on host service
out, err := client.Container().
From("mariadb:10.11.2").
WithServiceBinding("db", hostSrv).
WithExec([]string{"/bin/sh", "-c", "/usr/bin/mysql --user=root --password=secret --host=db -e 'SELECT * FROM mysql.user'"}).
Stdout(ctx)
if err != nil {
panic(err)
}
fmt.Println(out)
}
Services are designed to be expressed as a Directed Acyclic Graph (DAG) with explicit bindings allowing services to be started lazily. But sometimes, you may need bespoke service lifecycle management. Starting with Dagger 0.9, you can explicitly start and stop services as needed.
Here's an example which demonstrates starting a Docker daemon for use in a test suite:
package main_test
import (
"context"
"testing"
"dagger.io/dagger"
"github.com/stretchr/testify/require"
)
func TestFoo(t *testing.T) {
ctx := context.Background()
c, err := dagger.Connect(ctx)
require.NoError(t, err)
dockerd, err := c.Container().From("docker:dind").AsService().Start(ctx)
require.NoError(t, err)
// dockerd is now running, and will stay running
// so you don't have to worry about it restarting after a 10 second gap
// then in all of your tests, continue to use an explicit binding:
_, err = c.Container().From("golang").
WithServiceBinding("docker", dockerd).
WithEnvVariable("DOCKER_HOST", "tcp://docker:2375").
WithExec([]string{"go", "test", "./..."}).
Sync(ctx)
require.NoError(t, err)
// or, if you prefer
// trust `Endpoint()` to construct the address
//
// note that this has the exact same non-cache-busting semantics as WithServiceBinding,
// since hostnames are stable and content-addressed
//
// this could be part of the global test suite setup.
dockerHost, err := dockerd.Endpoint(ctx, dagger.ServiceEndpointOpts{
Scheme: "tcp",
})
require.NoError(t, err)
_, err = c.Container().From("golang").
WithEnvVariable("DOCKER_HOST", dockerHost).
WithExec([]string{"go", "test", "./..."}).
Sync(ctx)
require.NoError(t, err)
// Service.Stop() is available to explicitly stop the service if needed
}
See more examples in our guide on using services in Dagger.
We've also released new versions of our SDKs with support for Dagger 0.9, plus various SDK-specific bug fixes and improvements.
For a complete list of improvements, refer to the changelog for each SDK:
We are excited to hear about the use cases that Dagger’s new Host-to-Container (H2C), Container-to-Host (C2H) and explicit service lifecycle capabilities enable for you. Let us know via Discord, and perhaps come share with the rest of the Dagger Community in our regular fortnightly calls. If you have feedback for us, or an improvement to suggest, join us on Discord or create a GitHub issue.
Looking forward to the automation that you will build with Dagger!