Modern Testing: Definition, Principles, & FAQ
Modern Testing: Definition, Principles, & FAQ
Modern Testing has taken the software testing world by storm as more organizations embrace and adopt DevOps best practices. But what exactly is Modern Testing?
We recently hosted a webinar with Alan Page, one of the movement’s founders, to discuss Modern Testing in detail. He delved into how Modern Testing has been adopted, what has worked (and what hasn’t), and how that impacts testers everywhere — regardless of where we are on our testing journey.
You can watch the full webinar for Alan’s insights on testers vs. developers, balancing speed and quality, and the Marvel Universe. Below are some of the highlights.
What Is Modern Testing?
Modern Testing adapts Lean, Agile, and DevOps philosophies to emerging business and industry realities. Like shift-left testing, it promotes testing earlier on in the software development lifecycle. Modern Testing emphasizes quality as a shared responsibility between developers and testers, with an emphasis on data.
In the webinar, Alan admitted that Modern Testing isn’t really about testing, and it isn’t really new. Instead, it brings together trends and principles developed by Eric Ries, Gene Kim, and others, and makes them relevant to testers. It’s based on seven core principles.
7 Principles of Modern Testing
Alan Page and co-founder Brent Jensen outline Modern Testing as a set of shared principles, which have proven to be especially suitable for DevOps environments and DevOps culture-building.
- Business First
- Accelerate the Team
- Always Learn and Improve
- Lead a Quality Culture
- Customer Is King
- Data Trumps Intuition
- Embrace the Change
After describing these principles and sharing examples of how testers have adopted them in their own organizations, Page admitted in the webinar that you can’t just take all the principles and run with them. You need to apply your own critical thinking about what works or does not work and prepare to adjust accordingly.
Modern Testing FAQ
After his presentation, Alan fielded questions from the audience. See what he had to say below:
What do you do if developers don’t want to test?
Page: [Laughs] My flippant answer: Fire them!
But that’s not the real answer.
First, figure out if they don’t want to test, don’t know how to test, or don’t feel like it’s their job. You have to unpack the motivation. You have to convince them that quality is a team effort. What can we do to own quality more together? They might say, well I can test this for you, but that will take me away from this and this. Talk about the testing bottlenecks. They might say, well you’re better at this than me, and I’m better at this then you, so we should just focus on what we’re good at.
My answer to that is, “Why don’t we both try to get better at both of these things?” We end up pairing a lot. Let me explain one of the tried-and-true techniques I’ve used.
Developers asks, “Will you test this for me?”
Me: “Sure! What have you tested already?”
Developer: “Nothing! That’s your job.”
Me: “Do you have 15 minutes? Let’s go test this together.”
And they’ll do some pairing on it. The first time, it may just be the tester testing some things and the developer sharing their opinions. But go through that process a few times and then the conversation goes more like this.
Developer: “Will you test this for me?”
Me: “Sure! What have you tested already?”
Developer: “Well, I’ve done A and B.”
Me: “Great. Why don’t we take 10 minutes and do C together?”
Developer: “Oh, OK!”
And you keep doing that. And eventually, you graduate to a time when the developers ask, “Will you test this for me?” and you’ll say “Great, what have you tested already?” and the developer will respond, “I’ve tested A, B, C, and D. And I also tried this other thing I heard about.
Once you get past aligning on why the whole team needs to own quality, going through that gradual pairing process has worked really well for me. That’s my advice in a nutshell.
I’m the product owner of a DevOps team and noticed my team has a difficult time testing their work. The only way they find out if they’ve broken the build is either if the unit testing suite breaks or the acceptance testing suite breaks. Are they right in saying they cannot test their changes beforehand?
Page: No, they’re not right [laughs].
My DevOps team has deployment and repositories especially designed to test our changes against. Going back to the principles, our job is to accelerate the team. And the way I describe our mission in a nutshell in DevOps is we want to make it easy for our team to deploy quickly and safely. We never want to do anything that blocks the team. So, we make sure we have deployment environments and repo environments that allow us to test any changes so that we don’t break anything.
I would ask your team to build those because they never want to be a bottleneck.
What do I do if the developer makes changes for improvement and asks the testers to test them, but doesn’t provide information about the actual changes?
The more isolated the development team is from the testing team, the more difficult the testing. The more collaboration there is, the better.
Generally, on my testing teams, the request for testing comes with a link to the pull request, which usually — even if you can’t read code you can see what part of the code it’s in — has, if it’s a well-written pull request, has some explanation for what the pull change is and why.
But it’s a lack of communication and collaboration that can lead directly to misses and equality issues. The communication and collaboration issue is the first thing to solve because it’s no fun testing while flying blind. It’s inefficient right? I would start by addressing that.
Is it a risk to let developers test what they’ve built/coded?
Page: No. Let me elaborate.
I think there’s this myth that people are unable to test their own code. And I believed that myth until I started teaching developers how to code.
You do have to put a different hat on, but they’re actually very good. I haven’t seen any bias risk in our developers testing their own code.
One thing my boss would ask, “How is the dev team doing in testing?
And I’d say they’re doing 100% of the testing they know how to do.
This was accurate and a little scary.
None of our developers are afraid to test or don’t want to do it. There is a knowledge gap in terms of identifying new or different test ideas. But once we teach them or show them something new, they embrace them and try those as well. They really care about having ownership of their code quality.
I’ve taught hundreds of developers how to test. I’ve seen zero bias risk where someone’s missed something because it was their own code.
How is agile testing different from modern testing?
Page: It’s not really that different. It’s an evolution.
One major difference is that while agile testing does stress whole-team quality, modern testing takes it further. Not only is it whole-team quality, but you take it so far that you don’t need a dedicated testing specialist.
The other difference is that modern testing has an emphasis on using data and learning from that.
Otherwise, there is a lot of overlap; we just wanted to go farther with modern testing.
Is DevOps going to replace testers in the future?
Page: The short answer is no.
Done right, DevOps is an accelerant that can help the team deliver safely and quickly. But there’s no substitute for testing. If, however, you have the right data collection in place and if you have the right quality culture, the testing can be owned by the development team.
Where I am, we’re doing pretty well. We still have some improvements to make. But I still have test experts on the team to help. I have no intention of ever having fewer of them. In fact, I may add more as our organization grows. While the testers work adjacently to DevOps, they often find test ideas or frameworks that can help with testing that incorporate it into our CI/CD system.
They are absolutely necessary. While testers work with DevOps, they don’t replace it. I would always have both developers and testers. It’s a matter of looking at the system of what we’re building and how we’re building it, and then making sure we have the right people to help accelerate that process. So DevOps is going to be tooling for that, but as far as the testing ideas go, my DevOps team don’t have a lot of great testing ideas.
So that’s the longer answer to my short answer of “no.” [laughs]
Is testing important for junior developers to learn early on in their careers?
Page: Yes. I’m seeing more and more universities talking about testing in their courses. They’re also talking about data in their basic engineering course, which is good.
So, yes. We hire college grads at Unity right out of university and, without making a big deal out of it, we tell them that they’re responsible for the quality of their code and that they have to write tests for it. We have experienced developers around, who are pretty good at it, to help them.
It’s a general expectation when they start. So they ramp up pretty quickly.