The QA Manager’s Keys to Success in a Continuous Testing and DevOps Discipline
The software development landscape is evolving, and the majority of organizations are already practicing Agile, with 50% of them already maturing in DevOps. What does this transformation mean for a QA/QE Manager? Does the traditional methodologies for assuring product still relevant in this new modern world? In this blog, I will try and convey some insights that a QA manager should consider as part of the overall DevOps transformation.
Support the CI, CD, and CT Practices
To enable DevOps, teams are focusing on delivering high value with high quality fast. This translates into efficient Continuous Integration (CI) that builds new software fast and validates its quality in each step of the development process – that’s Continuous Testing (CT). Only by having reliable CI and CT, teams can push software fast and automatically to their end users, as part of their Continuous Delivery/Deployment (CD). The QA manager role in assuring the fluent process of these 3 pillars is fundamental. QA management should have maximum quality visibility in each stage starting from design, implementation, build acceptance, integration testing, and production testing. Such visibility means, that the manager needs actionable test data to base a smart decision on.
Continuous Test Measurement & Improvement
DevOps and software development, in general, is a continuous process. Innovation does not stop at the end of a sprint, nor does the test management and measurement. QA managers need to define metrics to help them measure efficiency from one sprint or release to the next. Such metrics would include things like:
- Test productivity – which is the most efficient test scenarios?
- What is the Pass/Fail rate between cycles?
- Is the defect backlog declining or increasing?
- How is the CI process progressing – is the cycle length increasing? Are builds becoming flaky?
- Does my test coverage and platform coverage align with the market and business requirements? Is it aligned with product analytics/usage?
- What is my overall test automation % vs. manual?
Each QA manager would need to have such metrics with criteria set to each. Such metrics ought to be relevant to his domain. When there is a degradation or miss to some of these metrics, he needs to communicate with his counterparts and build a plan to improve on them.
Team Productivity and Skillset
Continuous improvement in the quality process as mentioned earlier in this post is great, but the process is led by people. The team must continue to sharpen their skillset, to learn new tools, techniques and to measure their efficiency against the overall management objectives. For example, if issues like a test stability problem due to flakiness tests the framework, or if there are too many escaped defects to production appear – it is time for the management and the team to come together and build a plan to evaluate what needs to happen to overcome these issues. The world of testing is evolving fast, with new open-source framework, machine learning, and deep-learning based tools – such innovation opens the door for practitioners and managers to improve and move from a current state to a better mature state.
QA manager can be the steering wheel and the enabler for a mature DevOps practice. It involves leadership, cross-team communication, measurements and improvement based on a metric, and a matching set of technologies to support his agenda. If in the past QA was about how many tests I can develop in a given time, these days the reality is not about the number of tests but rather about
– the reliability, high-value, stability and speed that such tests can run and deliver back to the developer’s feedback. Management is basing today’s decision on a lot of test data and risk-based techniques to be able to move fast, fail fast, and correct.
There is no doubt that DevOps is maturing and is the de-facto method of delivering value to the customer in a fast way. Developers are becoming an integral part of the testing and assuming more responsibility beyond just unit testing. It is on the modern QA manager shoulders (it is a great opportunity as well) to take the lead, and drive the triangle of CI, CT, CD to make the overall software delivery pipeline as smooth, reliable, and efficient as possible.