Operate · Adopt

CI/CD & Canary Releases: Shipping Software Safely to Production

To me, software isn’t finished when it’s committed. A change only delivers value once it runs reliably in production. CI/CD gets it there, and canary releases lower the risk of each deployment even further.

DevOps: shared responsibility for delivery

From 2017 to 2018, I worked as Lead DevOps Engineer at Audi AG. To me, DevOps means one thing above all: development and operations share responsibility for making sure software reaches its users. Handing code over to another team and hoping for the best isn’t enough. Whoever writes a change should also understand how it gets built, tested, deployed and operated.

Shared responsibility needs a technical foundation. Without automated, repeatable processes, collaboration between development and operations is just lip service. This is where continuous integration and continuous delivery come in.

The CI/CD pipeline is a system in its own right

A CI/CD pipeline is often treated as an afterthought: a few scripts someone wrote at some point. I see it differently. The pipeline is a technical system in its own right and needs the same care as the application itself. It has its own requirements, it ages, and it needs clear ownership.

A good pipeline has three qualities. It is fast, so developers get feedback quickly instead of waiting a long time for a result. It is traceable, so it is always clear which version was built and deployed through which steps. And it fails visibly when something is wrong. An error that slips through silently is more dangerous than a red build everyone can see.

Once the pipeline is seen this way, the approach to releases changes too. Instead of rare, large deployments that bundle many changes, small increments ship frequently. Each change stays manageable, is easier to review and can be rolled back faster if something goes wrong. Small, frequent releases reduce risk because any potential error is smaller and easier to find.

Canary releases: the principle

The name comes from mining. Canaries reacted to toxic gases sooner than humans did, so miners used them as an early warning system. In a canary release, a small share of user traffic plays this role.

The principle is simple: the new version of an application runs alongside the old one. Routing initially sends only a small share of requests to the new version. If it behaves as expected, that share is increased step by step until the new version handles all traffic. If problems appear, the release is rolled back and all traffic goes back to the old version. That way, an error affects only a few users rather than everyone at once.

Canary releases therefore complement CI/CD: the pipeline makes sure a change is reliably built and tested, and the canary release keeps the remaining risk in production under control.

Why canary releases require microservices

A canary release only works if individual parts of a system can be deployed independently. With a large monolith, a new version of the entire application would always have to run in parallel, so even the smallest change drags the whole system along with it. Microservices with clearly defined domain boundaries, on the other hand, make it possible to start a new version of exactly the service that has changed, right next to the old one.

This ties architecture and delivery closely together. Cutting services so they can be built, tested and deployed independently lays the groundwork for a separate pipeline per service and for low-risk releases via canary routing. A well-cut service doesn’t just make sense from a domain perspective. It is also a prerequisite for safe delivery.

From theory to practice: articles, talks, a demo platform

In 2016, I published a technical article on canary releases in Java Aktuell magazine. I also presented the topic at conferences including JavaOne, Devoxx Poland, W-JAX and Javaland.

It was important to me not to explain the concept only on slides. With the “Very Awesome Microservices Platform”, I built a platform that shows in practice how microservices, the pipeline and canary releases work together. It lets you see how a new version runs alongside the old one, how traffic is shifted and what a rollback looks like.

A mindset that still shapes my work

The principle behind canary releases is more than a deployment technique. It is a mindset: introduce changes in small, verifiable steps, watch closely from the start and always be able to go back. This mindset also shapes my work on my own websites, such as bernd-zuther.de and ketovavaria.de. The same rule applies there: many small, low-risk changes that can each be checked on their own are better than one big switch where, if something breaks, the cause is hard to pin down. I build the infrastructure for this with Docker and AWS.

Whether it’s an enterprise platform or a personal website, reliable software comes from treating delivery as part of development, not as a tedious final step.