CI/CD at ADP: Going Global with GitOps
By Leo Meirelles, Principal Software Engineer and Principal Architect
As a principal software engineer and principal architect at ADP, I work on multiple projects and provide support as needed. Since ADP has a lot of products, one of our biggest challenges is streamlining our processes. Our engineers work on payroll systems, retirement services, and pension services, to name some, for both small and large companies—and that’s just in the United States. Each country we support has different portfolio options for companies that integrate ADP products. Since we continue to evolve our technology, we’re never short on opportunities.
I worked at ADP for close to seven years the first time. When I returned in 2016, I got involved with a great project to aggregate multi-country payroll; we integrated with in-country providers, with internationalization (i18n) and accessibility (a11y) support. I’d spent a few years at The New York Times and then eighteen months at Google, and I was excited to bring that skillset back to ADP. At ADP, I’d always liked the people I worked alongside, and with ADP’s pivot to a technology-first company, I knew I could have a real impact here. In particular, I had the opportunity to implement continuous integration and continuous deployment (CI/CD), first with one group, then two, and now we’re looking at global implementation throughout the company.
CI/CD combines continuous integration and delivery practices, relying on automation to guarantee that code changes are efficient and that application deployments are reliable. With the project I was initially working on, we had teams in multiple countries and multiple time zones, and when you have such a large amount of people spread out like that, you need to stay efficient.
Before this project, to deploy in a QA environment, we needed a UI Development Lead and a Backend Development Lead to approve a deployment release request since they were the most likely to be aware of any issues that could hinder QA work. They had to give the green light and say, “Hey, this code is good to go.” But when you have 30 developers, things get more complicated since you have to merge multiple pull requests. On top of that, we have eight microservices in the backend and three micro frontends—the login, the old application, and the new application, because we’re migrating a few things toward a new Angular version. This level of complexity underscores why ADP needed a global, automated solution that can work for everyone—and why we started the CI/CD implementation via our GitOps project.
GitOps is an evolution of infrastructure as code, where you build the whole environment from a configuration/definition file, and it stays together with the application source code. Later, it can be run by machines and build new environments without human intervention. The idea of GitOps is to use tools that you already use every day, which makes things easier since you don’t have to add new tools to your stack or change how you get work done. Instead of emailing, messaging, or calling someone to say, “Hey, can you deploy that version for me?” now you do that using Bitbucket and make a pull request, make a commit, and merge your code. After that, everything else happens automatically behind the scenes, and you can skip trying to find people for approvals. Because both the infrastructure and your code go into Bitbucket—in fact, the whole process for a new release—we’re able to have a new deployment in 10 minutes for the local environment using GitOps once the code review is finished, approved, and merged. From that point onward, QA owns the release management for their environment; they control what goes into the environment without involving DevOps and managers, giving more autonomy for the team to manage releases.
We knew that having GitOps would make life easier for DevOps and allow them to focus on other things, like maintaining production and improving monitoring and availability. Also, engineers deserved to oversee their code. But we had to figure out the best way to introduce the new process. So, we started slowly to minimize disruptions with the current software development process and give people time to adjust to a new way of doing things.
We had to get developers over the fear of breaking something. Everybody wanted to be either first or last. If they were first, they wouldn’t have to merge their code, and if they were last, they could make sure everything was working before adding theirs. Until we got into a rhythm, it was a stressful time. But once the teams started adopting the new process, things changed dramatically in a positive way. Instead of making considerable changes in the last few days of a sprint, we slowly transitioned to making several small changes every day. And if something didn’t work, it was a straightforward process to revert changes. Since everything is on Bitbucket, we can see the previous versions, making the long-term management much easier.
I talk to team members often, and we’re in a sweet spot right now. What really helped adoption was implementing a process when everyone was ready and open to trying a new process. I prefer to lead by example rather than trying to force people to do something. But as time passed, we built trust when it became obvious the new system worked. Now people have the flexibility to start working in whatever time zone they’re in and look at our online chats to see what’s changed since they last looked. We even have a bot that judges the quality of a pull request. We have twenty to thirty deployments in our development environment every day, and no one even notices. And if we need to make a production fix, we can do it in 20 minutes using GitOps and our other automation tools.
I’m looking forward to the full adoption of GitOps globally. We have a lot of products on a lot of platforms, so that will take time. But there are a lot of exciting things happening right now at ADP. We’re an evolving tech company and developing a cohesive development engineering team. Since I’ve been back, I’ve seen the environment grow stronger. We communicate and share between teams, do a lot of cross-team collaboration, and help drive innovation and ideas through events like global hackathons. As we globalize and streamline GitOps, we are laying the foundation for our future.
Interested in a tech career at ADP?
Click here to search for your next move and make sure to subscribe to our blog!