Quality Gates: Why all the fuss?
What is a quality gate?
Let me start this blog post with a question: Is a gate designed to keep things in or let something out? Depending on your mood, you may have different answers to this question which may not be limited to the options I have provided. However, if I am to be diplomatic and keep my emotions out, I would say a gate is designed to control what is allowed to pass through its threshold. For this discussion, what we are looking at controlling, is quality.
Moreover, it is essential to note that a quality gate is only as strong as the will of the gatekeeper.
Role of the gatekeeper
Each stage in the software development lifecycle provides us with several opportunities to implement quality gates. At each quality gate, the gatekeeper could be a person, personnel, or a tool. The main goal of the gatekeeper would be to regulate what goes out and what is restricted. This is easier with tools like Sonar, which provides the facility to configure the quality gates. Such tools can monitor the health of the codebase based on several criteria and indicate where quality aspects such as maintainability and dependability are lacking. The situation gets a bit sticky when people are involved, where added timeline pressures will test the will of the gatekeeper. As I have mentioned earlier, a quality gate is only as good as the will of the gatekeeper. It is a symphony of integrity, accountability, bravery, and negotiation skills of the gatekeeper that provide high-quality products and components to the next set of stakeholders.
Now that we’ve discussed the gatekeeper’s mindset (or ruleset), let’s evaluate which quality gates we should use and when we should use them.
4Ws & H
Why, What, When, Who, and How. These are the questions you should ask yourself, your audience, and the stakeholders to understand the purpose behind introducing quality gates. If we are to search online to find out the best quality gates to use or which are the most attractive quality gates used in the industry, we will end up with a long list of promising options. However, a report or a dashboard should never be ‘everything-but-the-kitchen-sink’—this approach will most definitely confuse, distract and overwhelm your audience. You need to avoid falling into this trap.
Given below is a simple depiction of how your thought process could best work to achieve a carefully curated yet effective report/dashboard containing quality gates.
Why
What
When
Who
How
How effective is the time spent on requirement gathering?
To check the quality of my requirement specifications
Before the subsequent team starts work on the requirements
(E.g., Development Team)
By: Development & Testing Teams
To: Requirement & Management Teams
- Requirement Clarity Index
- Q&A document summary and progress
To check the effectiveness of work done by the development team.
- To see how good the development codebase is
- The amount of defective code introduced?
- The amount of rework required?
Before handing it over to the testing team
By: Development & testing teams
To: Testing & management teams
- Code quality measurement tools (E.g., Sonar)
- Code reviews
- Development unit test pass rate
- First-time test pass rate
- Number and criticality of defects identified
- Defect removal efficiency
- Defect Aging
- Defect severity index
- Defect density
To check the effectiveness of work done by the testing team.
- To see whether test case coverage has been met
- To see if there is no risk to meeting project milestones
- Number of invalid defects recorded
Before handing it over to the client
By: Testing Team
To: Technical, Management & Client Teams
- Test case reviews
- Test execution progress
- Defect to remark ratio
- Test coverage verification
- Test case walkthroughs
Picture mode on!
Now that we have collected all the pieces of the puzzle, it’s time to put them together to create the picture(s). The most exciting part of this stage is creating multiple visuals for different audiences using the same set of quality gates. The mastery is in the selection, benchmarking, representation, and the placement of each KPI related to each quality gate. Also, there is no foolproof way of getting this right the first time. Perfection is a perception that is subjective to the audience.
Below are few examples of quality gates for further clarification:
When
Quality Gate
Benchmark
Accepting a requirement for development
Requirement clarity index
Q&A status
The confidence level should be 4 or more
(in a scale of 1-5, 5 being fully understandable)
Over 90% of the questions should be answered
Accepting a development release for testing
Code review
Development unit test pass rate
Smoke test case pass rate |
Completed
Greater than or equal to 80%
Equal to 80%
Completing testing of a release
Test execution status
Number of defects
Defect severity Index
Completed
No Critical/Major/Minor defects |
Less than or equal to 1.9
(industrial standard)
Best practices
Selection is all about asking yourself tough questions and coming up with decisive answers. Can we justify why we need this quality gate? What do we hope to achieve with this KPI? What visual medium is best suited to reach the end goal? Once you have all the decisive answers, consider your audience and their expectations. They may require information or recommendations and need to make decisions based on them.
Based on what your audience is looking for, consider the most important and the least important. Another consideration is the medium of communication. If this is a report sent via email, place the most critical information needed for decision-making at the top. The low-priority KPIs should be placed at the bottom, where the user can consciously scroll down and read more. If this is a dashboard view, the KPIs should be set from left to right, top to bottom, in the order of importance. This is a standard that UX specialists follow as it is in line with the eye’s natural movement when we look for information. Also, consider different reading practices adopted by audiences in different geographies.
As discussed already, quality gate selection and implementation is a journey rather than a task. This is because we may have to listen and adapt to feedback from the audience to make necessary changes to help them achieve what is expected of them. What may seem perfect now may not be so after six months into the project. The stakeholders will need to bring in more quality gates or take off some that they feel unnecessary to maintain the flow of work. Therefore, we will have to practice a high level of maturity while receiving feedback and negotiations.
4. Personal preference
Go with your gut when it comes down to selecting, implementing, and representing quality gates. There could be situations where a quantitative indicator is better than a qualitative indicator and vice versa. You may have to ask the questions: What is my audience’s confidence level in me? Will my audience need complete transparency of the decision-making process based on the Quality Gate?
My recommendation is to always go with the quantitative approach with numbers and trends as a rule of thumb. It is more of a scientific approach to decision-making, relatable to almost any type of audience. But as the topic says, it is indeed a personal preference.
Is it worth it?
As discussed in the sections above, you may feel quality gates are a lot of work. Hence, let’s weigh in the pros and cons. If I begin with the cons, the only trouble with quality gates is that you will have to spend time working on the 4Ws & H, which would need collective brainpower to set up the framework you plan to follow. But once that is done, the rest of the tasks—adhere, monitor, and adapt—are quite simple.
The main benefit of quality gates is that we take all possible preventive actions to ensure ‘shift-left testing,’ which is all about practicing a quality-focused mindset from the very early stages of the project until the very end. Shift-left testing guarantees less rework effort and ultimately helps in delivering high customer satisfaction.
Another attractive characteristic is that quality gates have no restrictions considering the scale of the project. As a QA professional who has been in the industry for a decade, I have been fortunate enough to work on diverse projects scaling from the small web and mobile applications to large ERP applications. Looking back at projects that adopted quality gates and projects that didn’t, it is a clear win for quality gates. The domain, scale, technology, complexity are no barriers to implementing effective quality gates, which can significantly benefit the quality and ensure the project moves forward according to the timeline.
Conclusion
To conclude this blog post, I will jump back to my title—the ‘fuss’ with quality gates. The main idea I want to convey is if you are serious about the quality of work you produce, invest some quality time to identify quality gates.
Successfully implementing quality gates can deliver many advantages to a software development project. A few are listed below:
- The quality focus throughout the lifecycle
- Improved sense of accountability for teams and individuals
- Proper quality control of work products changing hands (i.e., BA to Dev, Dev to QA, QA to Client)
- Better visibility on project trajectory allows room for necessary actions to control timelines
As discussed throughout this article, not only do quality gates control the flow of quality throughout the project lifecycle, but they can also be converted to KPIs. These KPIs can be a handy tool to monitor the quality, a gamechanger for effective decision-making.
Written by:
Rochana Herath
Senior Technical Lead – QA, Fortude,
Subscribe to our blog to know all the things we do
