A Q&A With QA Managers
A Spotlight on Joseph Reid, ecoATM
By Janna Sokolow
Joseph Reid is an expert software QA manager at ecoATM, a company that produces kiosks that allow people to recycle their old devices instantly for cash. Customers get the money they deserve while doing their part to help the environment by keeping electronics out of landfills. The kiosks themselves are much like vending machines from Redbox or Best Buy, and there are thousands scattered around the US, serving hundreds of thousands of people!
I was able to have a fascinating conversation with Joseph over the phone, about both his work with software testing at ecoATM and his unique perspectives on the testing field. Read on to find out about how ecoATM implements Agile, Joseph’s proudest testing moments, and bugs that somehow made their way to release.
The Testing Facts at EcoATM
What are the main challenges in managing your software testing team?
“I’d say the main challenges are the growing complexity of the work that needs to be done, and we’re working on keeping our engineers tooled up. So it’s either a balance of continuing to test in the same way and continuing to do all the same regression testing you need to do to ensure quality or you make the smart choice and move towards automation solutions. Or, you can reduce the amount of regression you do, otherwise, you never get done with testing.”
What percent manual vs. automation testing do you conduct?
“60% automation and 40% manual. Manual testing revolves around new features, and also around items that can’t be tested with automation.”
What testing methods do you generally use?
“Since our kiosks have to look at and interrogate devices, we actually need to look manually at devices. We put the device in the machine, and you’re supposed to have a certain expected result, so that has to be done manually.
“We always have security in mind when we are implementing new features, and security conversations always happen before the implementation of features, and then obviously the QA engineers go in there and they will employ whatever methods they need to – and I have a lot of trust in my team — in order to look for security holes and to try to patch them before they go out.”
Do you run into the issue of having to run different tests on multiple platforms?
“No, we control the hardware so we know exactly the systems and the specifications that we’re running against every time.”
EcoATM’s Current Agile Approach: Working Together & Delivering Software Faster
What’s the connection between QA and developers? Would you consider your company following an Agile approach?
“Like most companies, if you’re going to be producing software in the current age, you’re going to want to move to Agile as soon as possible — And we’re no different! We have Agile processes here, we have stand-ups and Scrum meetings, and we try to follow Agile processes as close as possible because it’s good for the business. It helps the business to know where things are heading and what progress is being done. And it allows you to do your development in smaller bites so you can produce your software faster than if you were using the old waterfall methods.”
“Like most companies, if you’re going to be producing software in the current age, you’re going to want to move to Agile as soon as possible”
Would you say it’s effective? Are there any challenges you find when trying to uphold Agile?
“There were benefits in doing things in the waterfall method, in that there was generally longer wait times between the releases…meaning that you get more time to test and retest. Although most of that time is spent testing code that hasn’t changed looking for faults. In Agile, you’re taking every little component and testing that, and working through that individually. So the burden with Agile is that if you weren’t employing software automation practices, you could have a hard time keeping pace with how things work in Agile.”
So release cycles must be pretty quick?
“They are, but we still have periods of downtime in between sprints. They don’t last very long, but in Agile software development, you have to pace yourself. When you’re developing software and Agile functioning, you want to be really clear about what you’re able to get done. You don’t want to take any heroic efforts.”
“When you’re developing software and Agile functioning, you want to be really clear about what you’re able to get done. You don’t want to take any heroic efforts.”
Can you describe any self-developed tools that are unique to your company?
“We have people from different backgrounds, and we try to keep things as close to industry standards as possible. Now that brings up the question, ‘What’s the industry standard?’ There are tried-and-tested things that have worked in the past: When you’re doing software development it’s always good to have somebody review your code after you’re done, and the process of doing code reviews helps you become a better developer. To get to the point of saying ‘I’m done,’ you yourself actually find more bugs than the person you’re showing.
“Then on the QA side, we also want to do test case reviews with the development team. So QA and dev are related in that we have the QA Engineers and the Software Engineers sitting next to each other, and as the tests get defined, the testers come over and say, ‘This is how we’re going to be testing. What do you think?’ As the QA engineers are sharing the test plan with the people that actually implemented the code, it actually helps the QA Engineers to say ‘Oh! I didn’t think of that!’ or the development engineers to say ‘You know, I didn’t really build it to do that, so maybe I could give you a different build that actually addresses some of this.’ So bugs are hopefully found a little bit earlier in the process.
“Now obviously nothing’s perfect — but when a bug does get out, it has to go through multiple people to get out to the field.”
That may lead to unrealistic expectations for what’s to be shipped off at release date — What’s your threshold for what’s acceptable as a confidence level?
“Well, of course, that all depends on who you ask. Obviously, the people higher in the company want as few bugs as possible – but they also want the software to be released as fast as possible. Other stakeholders in the company might just want you to take your time and wait until the software’s ready, and then there are others who say ‘Look, let’s take some risks, let’s have some bugs in the software. Bugs are going to happen as long as we get the software out there.’ It really depends on who you’re talking to.
Have there ever been any bugs that have slipped through and made their way out there?
“I haven’t worked anywhere that’s released a completely bug-free software. It’s really not avoidable. …In the early days of this company, we had a bug where there was a manufacturing defect on the hardware side. Essentially, when a door was being closed on the kiosk, the latches actually developed a short in them and it knocked down some of our USB devices. That was treated as a software bug! That just goes to the point that we can’t completely automate our [testing] system since I had to really work with the hardware in conjunction with the software in order to determine that this short existed that was knocking out some of our cameras. Only then did we figure out it was a manufacturing problem, not a software problem!”
What are you most proud of in your testing operation?
“The thing that I’m most proud of is the culture that we’ve developed here. We develop a close relationship between software and QA. In some organizations, QA is seen as below software engineering, but we have a very collaborative relationship where the QA and devs are actually working together, sitting together, and when bugs get discovered it’s not an adversarial relationship where a QA [or dev] would point a finger at any individual. …We don’t blame people. We address the problem.”
“The thing that I’m most proud about is the culture that we’ve developed here. We develop a close relationship between software and QA. …We don’t blame people. We address the problem.”
Who would you like to pass the baton to next? Whose interview would you like to read?
“I would say there is a QA manager at Dexcom who I think has done some really good work in a constantly changing environment.”
Special Thank You to Joseph Reid for allowing us to interview you for this article!