Q&A with a Director of Quality
We are pleased to say that The Phoenix, a leading Israeli insurance company, is one of our clients and doing well. We are very proud of the amount of work and effort they have put into their testing. Due to their effort and use of our tool, they have succeeded in running multiple scripts that are connected to their DevOps processes. Chen Halio the Director of Quality at The Phoenix was interviewed recently by Doron Bar, a writer at the Testing World, an Israeli magazine. We have translated the published article into English, for your benefit. Enjoy and congratulations to the QA team at The Phoenix!
The interview was done by Doron Bar, a QA tester of complex and simple systems. He likes to learn about new and interesting technology. His specialty is context driven test methodology. He runs and writes his own blog (since 2008) and has published those articles globally. Doron is also a writer at the Testing World, an Israeli magazine. He is an active member of the QA community in forums and in Meetups and is a testing methodology manager and Verint Systems.
The main challenge was to find a solution with the least dependency on automation engineers for setup and maintenance. We wanted our testers, whom their main strength is their business knowledge to be able to plan and build their test scenarios independently, without the need to code.
I’m Chen Halio, age 42, married, a father of two and live in Ness Ziona, Israel. I’m working at The Phoenix for the last two and a half years. Before I worked for 8 years at Discount Bank, 4 of which was as a manager of the QA department. My QA professional career started in 2002 when I worked for the Tescom company with IDF. That was the first time I worked in the QA field after graduating with a degree in computer science.
Were you thinking of a future in programming?
I believe that everyone who studies computer science is planning to program. I can’t say it didn’t “interest” me as well. When I completed my degree in 2002 the Hi-Tech bubble was booming and many people, like me, chose to enter the testing world, as they thought it could be a good lead to the development world.
In retrospect, I discovered that the QA world is fascinating, with many ways to develop one’s career, both professional and managerial.
How is your QA department built?
My department at The Phoenix has 45 employees, which are divided into two major groups:
The first group is run by Ricky Oren and is in charge of the QA of long-term insurance programs (life insurance, pension, and health insurance).
The second group is run by Nir Ben Shimon and is in charge of the QA of all digital platforms, organizational applications, SAP, and AI systems.
We run several numbers of projects in parallel, some are cross-company projects. The whole idea is to see the overall picture, discover the problematics spots, and take care of them.
Usually, the source of our main projects are our core systems which have an interface with other edge platforms such as web, BI, CRM, SAP, and mobile.
Our testers come from diverse professional knowledge and backgrounds. Many of them started working in different business departments in the company. The testers know our business and our insurance world thoroughly and our challenge is to chase the best of them and get them to know the QA world. We have proved that training employees with business knowledge to testing is shorter than vice versa. As a rule, it is easier to teach QA than to teach insurance and business to an experienced tester.
On the other hand, the difference from the insurance world to the testing world is huge – how do you verify its success?
It is true that the transfer from the business or insurance world into testing is a huge one, therefore our candidates go through a QA role qualification test. Usually, these tests predict the ability of the candidate to fit in testing. In this test, we try to uncover their way of thinking, their sense of curiosity, and the desire to test a platform in different situations. I’m happy to say it works.
We gradually expose the testers to the daily testing tasks. We try to keep the tests simple at first so they will gain confidence. Once they know the systems very well, we can involve them in sanity and regression testing from day one; we quickly got our money’s worth.
What about technical tasks? Debugging, read logs…
As mentioned, they gradually learn and train under an organized training program.
We teach the testers relevant tools and capabilities, according to each tester’s progress and capabilities.
In the digital group, for example, we use a wide variety of technologies, some of which without a user interface. In those cases, technical ability is more important.
Can you share some of your testing groups’ achievements?
Our biggest and most important achievement was the number of projects we’ve managed to release and that hit high customers’ satisfaction rates (our customers are IT and our business customers).
Another achievement – we were looking for solutions to create automation which will increase our test coverage, shorten the test time, and improve the overall quality.
The main challenge was to find a solution with the least dependency on automation engineers for setup and maintenance.
We wanted our testers, whom their main strength is their business knowledge to be able to plan and build their test scenarios independently, without the need to code.
As of today, each group has a solution and the ability to run tests automatically which is tailored to its platforms.
We were looking for solutions and tools for a long period of time.
We went out to visit colleagues in other companies, attended industries’ conferences, and tried several tools in our work environments.
For the company’s core systems (Unix) we found a solution that was developed internally a couple of years ago, for a specific project – we identified its potential and upgraded it, based on our needs.
Today, this product is used for sanity and regression, with a wide spectrum of scenarios on ten thousand different policies.
In one of the conferences we attended, we coincidentally were exposed to a SaaS solution of an Israeli startup called TestCraft. It was a striking discovery for me. After a successful POC on a number of sites, we have implemented the solution in our organization.
As of today, we have hundreds of scripts which run on different browsers and work environments (test and production) on different hours, upon schedule.
We have managed to connect some of the scripts to our DevOps processes and the Jenkins (CI) tools in our organization.
Which work methodology do you apply? Agile/Waterfall?
Most of our projects in our department are ran in a Waterfall methodology.
We have adopted some aspects of Agile in some projects.
These days we are in the process of a consulting process which should result in helping us choose the projects that fit best to Agile, to guide us, and implement the methodology. I believe that after we’ll examine a couple of projects we will be able to know which projects fit and which don’t.
What are the managerial challenges you come across?
As I mentioned we have a number of projects running in parallel.
We have many customers with different needs and requirements. Some are due to regulation and some are business related. The main challenge is to make the right adjustments needed in a speedy and flexible way, to make it on time, without compromising the quality.
Our systems are very complex and influence many satellite systems. We need to uncover the dependencies as early as possible, to communicate it to development, and to engage them in the testing needs. This ensures the stability and efficiency of the testing.
How do you motivate your testers and what else would you like to do?
We shift testers within the departments and teams occasionally.
Our testers like to learn and develop into new worlds. They contribute their knowledge and experience to their new role and team, which helps them inspire each other.
Integrating automation solutions has dramatically improved the desire of our testers to learn and develop.
We conduct organizational training for topics such as automation, accessibility testing, database security, SQL, generic writing, and enriching business courses.
We have developed an internal organization site for two major purposes:
- Preserve and enrich professional and business knowledge.
- Better connection and acquaintance between employees.
Do you have a structured set of Exit Criteria?
Basically, we don’t release with critical or severe bugs.
In instances where there are still severe bugs before the planned launch, there will be a discussion with “The Triangle” representatives: Dev-Test-Business customer to prioritize and decide if there’s a go/no go.
Mostly we launch systems with more than 95% coverage and see above 90% success rate.
The medium and low rated bugs also need to be approved by the business customer before launching.
How do you track the progress and quality of the tests in the different projects?
We have a dashboard that is based on BI Qlikview which shows us the stats and progress at any given moment. Based on this dashboard we report back to all stakeholders regarding the quality status.
We have measurements such as run progress (vs. planned) and bugs status (with different breakdowns) so we can communicate to development the issues and bugs that require a quick fix, which helps us quickly progress in testing each project.
Do you give back to the community?
The IT department in The Phoenix has taken on a yearly volunteering plan. Each week 10 volunteers help Pitchon Lev organization in distributing food supplies straight to the home of those in need.
I personally volunteer every week and go along with one of the testers. On top of the rewarding feeling of performing the good deed, it gives us the opportunity to openly discuss work or other non-related issues.
I think it makes us better people and gives us a different perspective on the world that allows us to acknowledge the good things in life. I believe that every tester that comes back from this volunteering activity receives a life lesson.
What would you recommend to a software tester who is at the beginning of his career?
My recommendation is to keep learning and improving. Don’t be shy to ask your team members, developers, or business customers questions. This is the only way to progress.
If you’re lacking any knowledge, seek for more information internally, on the internet, conferences, or meet-ups.
Some say that testers won’t be needed in the future, how do you think the testing world will look in the future?
The QA world is evolving and progressing constantly. We, as testers, must be updated, and broaden our professional and business knowledge.
We must recognize new tools and work methods that may fit our work environment and apply them. I don’t believe it is possible to give up testing. A tester’s point of view is different than developers and it is important that QA will be done by a qualified, professional entity.
What is your next mission?
We want to dive deeper into API testing.
Another target is to define a set of quality measurements for our team and for us, to track and measure our progress and quality of work within the different teams. (Originally found on Testing World, Interviewed by Doron Bar, translated from Hebrew to English).