We’re always interested in hearing how our community is using Dagger – their use cases, their challenges, and their experiences with deploying Dagger in different environments. Our Discord is a great place to find these stories, and to benefit from the knowledge and experience of the Dagger community.
In this blog post, we’ll share the story of Daggernaut Sean Clayton (aka @clayton9821 on Discord). Sean recently joined a new company, and was surprised to learn he would be the SME for all its CI pipelines. He had never used GitLab's CI/CD feature before, but took initiative to make it better. By switching to Dagger and away from a YAML-based CI workflow, Sean was able to give the development team faster builds and increased reliability, particularly for Maven builds.
"How many times have you pushed your code to the pipeline that works locally and then it fails for some reason in the remote pipeline and you have no idea why? Dagger solves that. By letting you see your pipeline as code, you can do everything that you can in production locally, and then push to the pipeline with so much more confidence." - Sean Clayton
Before Dagger, the existing CI pipeline consisted of 1000+ lines of YAML code, with both actions and templates mixed together. It was so complex that no one in the team understood it completely. Due to the high degree of complexity, the pipeline code was difficult to maintain and update, and CI failures could not be debugged and resolved quickly. These issues were increasing developer friction and frustration, and affecting the team's confidence in the entire CI workflow.
Sean was keen to improve the experience for the development team and started investigating alternatives. During the process, he attended a talk by the Dagger team at KubeHuddle in Toronto and immediately realized that Dagger provided the modern and maintainable approach he was searching for. He started by reading the Dagger documentation, built a proof-of-concept and then began the process of migrating the existing CI pipeline to Dagger.
"When the pipeline failed, people would come to the SRE team and say, it must be the pipeline that's broken. And that was really what drove me to want to improve this, rather than just keep applying band-aids over and over".
With Dagger, Sean has been able to both simplify the CI pipeline and make it more consistent. The Dagger pipeline performs the same steps as the previous YAML pipeline, but using Dagger's Python SDK (for example, replacing a Packer template with Python SDK methods). It operates in a blended environment: QA on Kubernetes and production on bare metal.
"Local testing was the big win because the biggest thing we heard is, the build works locally and it's not working through the pipeline. Having the developers be able to push to the new pre-QA environment and see how the pipeline works was huge."
Some of the key benefits of “Daggerizing” the pipeline are:
"It's definitely faster, and the developer that I'm working with on this is extremely pleased. Giving devs the ability to replicate the pipeline locally increases development speed dramatically. We both know that as soon as we introduce this, it's going to be adopted very quickly."
Sean is working on moving more internal customers over to the new Dagger pipeline, and has various ideas for future improvements:
"The possibility for innovation with Dagger is so huge. Our pipeline is so much shorter, much faster and can be run on any operating system. This was a win directly because we were using Dagger."
Do you have a Dagger story you’d like us to feature? Tell us all about it in Discord!