Something that I really find useful when working on software with multiple simultaneous collaborators is the concept of “Quality Gates.”
I first heard this term while attending STAREAST 2019, a software testing conference, with my manager and a colleague from another area of the company I worked for at the time. Our organization was slowly trudging towards agile development, DevOps, and modernization, and they sent my boss as vanguard to gather more intel at this conference, with me in tow. We had a mandate to attend as many talks as possible, take a lot of notes, and report back to leadership what we had learned and what we might realistically implement. I heard about quality gates in three different talks over those three days, which tipped me off to the fact that it may be important!
Although I had nominally joined the small “DevOps” team part-time some months before, I had truly only gained, via Googling and poking, a cursory knowledge of what the term entailed outside of the little world of scripts and optimizations we were working in to better our program. This conference was my first introduction into serious thought about DevOps and other related concepts. Little did I know that DevOps would become a main focus and joy of my working life for the next several years!
Donuts …#
Tangentially - our team from my job at the time had a long-running tradition of members bringing in boxes of donuts for the rest of the team to commemorate various momentous occasions or “infractions,” both personal and professional. One of those infractions (perhaps the most frequent) was submitting a build-breaking bug and ruining the next morning for everyone else. Hence, the title of this post. Properly implementing quality gates in your workflow can significantly reduce your donut-bill, as it did for our team, so pay attention.
What are Quality Gates?#
A “Quality Gate” is simply a criterion; a criterion that a change you are making must satisfy before you may apply this change such that it will affect a larger group of people than it already does.
I.e., before you can open the gate and proceed down the road, you must answer these riddles correctly to undo the “lock.”
So, say you are a software developer on a website, for a company that sells stuffed animals. If you are making a change to turn the “Buy Zebra Now” button from “white” to “white with black stripes,” you might be made to answer these basic questions before you can actually “push” the change along the process, so that stuffed animal purchasers will actually see a white-and-black “Buy Now” button when they visit your website.
Examples of Informal Quality Gates#
Here are some very common quality gate-like questions. These questions are part and parcel of almost any software development effort, even if they are not literally called “quality gates.”
“Did you run the website again as a local test from your machine, after making this change?”
“Did you get all the latest changes to the website that your other teammates have made before testing?"
If you are part of a very tightly-run organization or project, you may be asked some questions that are more like these:
“Did all the automated tests associated with your change pass?”
“Did all the automated security scans pass?”
How are Quality Gates Enforced?#
Frequently, in a loosely run or extremely small team, the above checks take the form of here-and-there questioning, perhaps by a colleague or the lead dev. This is not what a true Quality Gate is. A quality gate, should be, ideally, a formal, obvious, unskippable cutscene of a question. There should be evidence that you said you “YES” to the questions asked of you.
For developers, these gates can take the form of a checklist in your bug-tracking, project planning, or source control software. This checklist may be, as a matter of strict protocol, monitored by your lead software dev before merging your changes in with the latest codebase, i.e., opening the gate and letting your change pass through.
Why are Quality Gates Useful?#
Quality gates are useful insofar as they prevent nonfunctional code from slowing everyone else down. “Everyone else” being your fellow team members, your userbase, anyone who will use the results of the changes you are trying to introduce to the system or product. If you are breaking the software by adding a change that doesn’t actually work, a sturdy, well-considered quality gate will reduce the chances that you pass on that breakage to other people, and force you to be aware of it and fix it before proceeding. Without well-formed quality gates, bugs tend to be caught later in the process of developing and releasing products, thus requiring more rework, more spent budget, and more headaches for more people.
You may think to yourself, “Well, aren’t we supposed to be agile software developers, getting rid of cumbersome processes, and putting stuff out there ASAP and making it work later?” To that I would say, yes, as long as it doesn’t make your customer think you’re so bad at your job that they stop using your app/website/game entirely and move to your competitor, or cause your teammates to declare, “Bob is merging his changes today, everyone look out!"
Quality gates, when used correctly, make sure you can move fast without crashing and burning. You wouldn’t want Ford or Toyota to make a change to one of their car models and sell it to you without having even driven it once, only for you to find that this model spontaneously combusts when you make a left turn, right? Nor would you like to deal with ten recalls a year for your car, correct? Quality gates are one aspect of a well-organized development workflow that drastically reduces the chance that our users get burned. This allows us to keep a decent rate of high-quality updates flowing to our product, rather than a mix of high- and low- quality updates that require rework and lose the trust of our customer.
How To Implement Quality Gates in Software Development#
Implementing quality gates is not complicated, particularly if you use a common version control tool or a common bug-tracking tool. These tools typically provide easy-to-use options for quality gates, even if they don’t call it that by name. Get all the stakeholders on your development team together, and consider the questions below.
Question One#
What are the most common types of bugs or other problems that happen on my team over the course of a month of development?
Examples:
• Syntax errors
• Compilation errors
• Runtime errors
• Confusion about what commit is related to what bug
• Post-hoc disagreements about how a bug should have been solved
The answers to this question should inform you in your decision about which quality gates to institute to reduce these errors.
Question Two#
How much new process do I need to add, such that the extra effort that a developer has to make to pass the new gate is worth the time and effort saved from the bugs reduced?
I will explain this by means of an exaggerated example.
If you are building a 40-hour video game adventure, you should probably not add a quality gate that requires every developer to, “play through the entire 40-hour game from start to finish” every time said developer wants to add any sort of change to the codebase for their team to check out. An artist adding a small rock to the side of a road in your game shouldn’t necessitate you retesting the entire 40-hour game at runtime before pushing it to the next stage of development, and it would be foolish of you to add a quality gate that requires this. Playing through the whole 40-hour game over and over should be something that comes either at a regular timed interval, after a very large changeset that impacts the undergirding of the entire game, or before final shipment of an update to a customer.
Furthermore, at a certain size and scope of project, you will (or should) likely have a dedicated “test team” standing between you and the customer, and should negotiate with them on what should and shouldn’t be tested by you as the software developer to be committed to the codebase before it gets into their hands for detailed pre-release testing. This is beyond the scope of this post, but it’s something to keep in mind.
Balancing between “too little control” and “too much control” over changeset flow is a huge part of maintaining high development velocity and high team satisfaction.
Question Three#
What tools do I have at my disposal to add gates in an automatable and strict fashion?
Look at the tool you use to version control your software. Tools like GitHub and GitLab allow you to place quality gates before pushing changes, via their CI/CD pipeline software.
Look at the tool you use to track bugs. Some sophisticated tools, like Jira, allow you as an administrator to not permit the movement of a bug from the “In Progress” state to the “Complete” state unless the user ticks off some custom boxes. Those custom boxes could enumerate quality gates.
Recommended Quality Gates for Team Software Development#
Here are some quality gates I like to ensure are in place to gate an individual developer’s merge request to the wider dev team’s branch. This is a decent set of gates for a team of 4-9 developers, enforcing basic due diligence with low friction for the team size. Feel free to modify it based on team size, project requirements, or workflow design.
• Latest CI/CD pipeline had no build or test failures
• Source branch contains all latest changes from target branch of merge request
• Two code review approvals by any teammate
• One code review approval must be from a teammate listed as an expert in this area of the codebase being changed
• One teammate (not the author of the change) must have seen the affected change work at runtime and signed off
• Git commit description and Git merge request title must contain bug ID
• No new warnings from compiler
• No new security findings detected
Conclusion#
Hopefully this post has been helpful to you in your journey towards high-speed, low-drag software development. And hopefully it will keep you from owing as many boxes of donuts as I have. But, donuts are pretty great, so maybe bring some of your own volition here and there, after you stop breaking the build. :)
Thanks for reading!
Alex