Introducing Dagger: a new way to create CI/CD pipelines

Hi! We’re the creators of Docker, and today we’re proud to introduce our new project, Dagger. Dagger is a portable devkit for CI/CD pipelines. It allows devops engineers to build powerful CI/CD pipelines quickly, then run them anywhere. Benefits include: unifying dev and CI environments; testing and debugging pipelines locally; and avoiding CI lock-in. Most importantly, it introduces an ecosystem of truly composable software components: a “lego system” which we hope will help the devops community solve its fragmentation problem. Under the hood, Dagger is powered by two of the most exciting technologies in the cloud-native ecosystem: Buildkit and CUE.

We’ve been developing Dagger together with a community of early adopters. Now we are ready to continue developing it in the open. If you’d like to participate, join us on Discord!.

To our alpha and private beta testers: thank you for investing your time and energy to help us test, review and develop Dagger. We dedicate this launch to you, and we cannot wait to grow this community together.

We’re also hiring! If you’d like to help us improve the lives of devops engineers, we’d love to talk to you.

Why Dagger?

A developer’s dream is a devops engineer’s nightmare

There’s never been a better time to be a developer. Thanks to an ever-expanding selection of specialized tools and cloud infrastructure, a dedicated team of developers can ship a competitive product in almost any industry within months. The result is unprecedented growth and innovation at every level of the stack.

This developer’s dream creates a nightmare for devops engineers, whose job is to automate the delivery of increasingly complex applications to increasingly complex cloud infrastructure. In order to continuously build, test and deploy their software, developers need a CI/CD pipeline that glues together an ever-expanding collection of specialized tools. The result is a mess of homemade scripts that are painful to write, difficult to debug and test, and must be rewritten for each new development and CI environment. Component reuse between pipelines is low, which causes fragmentation and a constant reinvention of the wheel.

Modernizing the CI/CD development experience

Dagger solves the devops engineer’s nightmare by telling them: “your pipelines are software. This makes you a developer, and you deserve a proper developer experience”. For example:

• Instead of gluing pipelines together with throwaway scripts, they can “play lego” by composing reusable actions, using an intuitive declarative language and API.
• Actions can safely be shared and reused thanks to a complete package management system, and there is a growing catalog of open-source actions developed by the community.
• Pipelines can be tested and debugged locally, then run on any CI system
• All actions are automatically cached: no manual tweaking necessary

This allows devops engineers to continuously improve their pipelines, with the confidence afforded by modern development practices.

Write your pipeline once, run anywhere

Because Dagger runs equally well anywhere a Docker-compatible runtime is available, it eliminates two major problems in the devops experience: dev/CI drift, and CI lock-in.

• Dev/CI drift: instead of writing the same automation twice - once in a CI-specific configuration, and again in a Docker compose file, shell script or Makefile - simply write a Dagger plan once, then run it in all environments.
• CI lock-in: migrating from one CI system to another, or supporting teams with multiple CI systems, should not require rewriting entire pipelines from scratch! The same Dagger plan can be run without modification on all major CI systems.

What’s next?

Dagger is still in the early stages of development. Thanks to a new round of funding we will continue to expand our team, and continue building the ultimate CI/CD experience: more portable, more productive… and more fun!

If you share our definition of fun, come say hi! Our community is open to all, from beginners to seasoned devops veterans. We look forward to building something great together.

Happy hacking!