Cache control for modules

TL;DR: Dagger has always cached system functions. Now module functions are cached too. Use cache: "never" for side effects.

Read the changelog

Dagger executes your CI pipelines incrementally: when a pipeline runs twice, it skips work that’s already done and can complete faster. This caching process happens automatically, sparing you the pain of maintaining fragile configuration files.

Until now, only system functions could be cached in this way, and not functions defined in a module.

In practice, system functions do most the heavy lifting; modules are the cheap orchestration glue, so the overhead of re-running them is acceptable - but it can still add up, especially in large pipelines! So, as of Dagger 0.19.4, module functions are cached by default.

Module developers can use a new cache control API to tell the engine which functions have side effects.

Cache control

To cache module functions by default, we needed a way to know whether your function is pure. A function called deploy() looks the same as build() to the engine, and caching the wrong one would break your pipeline. The solution is to annotate deploy() to let us know that it has a side effect. This is the purpose of cache control.

TypeScript

@func()                     // default: cached up to 7 days
async build(): Promise<Directory> { ... }

@func({ cache: "15m" })    // re-fetch periodically
async latestBaseDigest(): Promise<string> { ... }

@func({ cache: "session" }) // this engine session only
async currentUser(): Promise<string> { ... }

@func({ cache: "never" })  // always execute
async deploy(): Promise<string> { ... }

Python

@dagger.function                    # default: cached up to 7 days
def build(self) -> dagger.Directory: ...

@dagger.function(cache="15m")      # re-fetch periodically
def latest_base_digest(self) -> str: ...

@dagger.function(cache="session")  # this session only
def current_user(self) -> str: ...

@dagger.function(cache="never")    # always execute
def deploy(self) -> str: ...

Go

func (m *MyModule) Build() *dagger.Directory { ... }  // default: cached up to 7 days

// +cache="15m"
func (m *MyModule) LatestBaseDigest() string { ... }   // re-fetch periodically

// +cache="session"
func (m *MyModule) CurrentUser() string { ... }        // this session only

// +cache="never"
func (m *MyModule) Deploy() string { ... }             // always execute

The four modes

Mode	Syntax	Use case
Default	(none)	Pure functions. Most of your code.
TTL	"10s", "15m", "2h"	External data that goes stale.
Session	"session"	Consistent within a run, not across runs.
Never	"never"	Deployments, notifications, side effects.

Backwards compatibility

Existing projects get "disableDefaultFunctionCaching": true added to dagger.json, which preserves the old behavior. New projects get caching by default. To opt in, mark your side-effecting functions with cache: "never", then remove the flag.

Learn more in the function caching docs, or see the v0.19.4 changelog.