Certain Tests You Need to Run with a Touch of AI
There is a constant struggle for test automation engineers and business testers on choosing which tests to automate. I think this question arises from the nature of test automation itself. When you first create a test automation script either in Selenium, Appium, or a commercial tool, the test enters a long cycle of executions, maintenance, debugging and questions around its value to the overall project.
To address the above debates, I would divide the problem into a few areas:
- Test automation complexity and maintainability
- Test persona skill set
- DevOps/CI maturity
Classifying Test Automation Methods: Code vs. Codeless
To address the above struggle let’s first understand when it is best to direct test creation across the 2 methods: Code vs. Codeless.
If we are just looking at web apps, each page might have a varying level of complexity. Some pages will have the common and standard login buttons with static objects, while some will have more dynamic objects with elements that change based context – e.g. location, language selected, user profile (man vs. female), etc.
When dealing with dynamic content, it is quite hard and very costly to create and maintain reliable tests that will run continuously, keeping in mind that web isn’t the only web, and such tests will also need to run on mobile devices as well (responsive web e.g.). In such cases of dynamic content, AI can be a great and recommended solution, since based on output, visuals, and other contextual conditions, the test automation engine can help identify the right element and help continue the test execution properly. In standard cases as mentioned above, using Selenium, Protractor, and other tools will be perfectly fine, and will return the value back to the developers.
Classifying Test Creations based on Skillset
After making the case for AI/ML test creation based on test complexity, it is also important to look at the persona that is tasked with the test creation. In many organizations, there are great QA talents that are mostly focused on either manual and exploratory testing, or in some cases, take part in the Gherkin BDD test creation. Here is an additional opportunity to boost test automation productivity by assigning these personas to start using AI/ML codeless testing to automate their manual and most efficient exploratory tests – such that are historically proven to detect issues. In addition, while fully supportive of BDD, such Gherkin based scenarios are in most cases “Static”. When a UI element or a flow changes, the custom java functions and Gherkin scripts must be modified. The overall manual “payload” BDD/Manual tests carry in organizations other than the fully code-based tests can sometimes be 30% of the overall testing.
DevOps and Continuous Testing/Continuous Integration Process Maturity
Lastly, and very much related to this topic is the level of maturity organizations have in CI/CD/DevOps. As organizations are trying to mature their processes, any flakiness in test automation comes with delays to the release, and de-focus development on noise within the pipeline. False positives, inconsistent test results, complexity in automating important cases that in return are executed manually are a show stopper for maturity. In most cases, they are holding back organizations from moving forward in their DevOps journey.
Organizations should identify the pain points in testing throughout the pipeline. Then assign them to the business testers and developers as candidates to AI-Driven test automation. This will prove itself as a productivity boost to the process, the agility and will allow better synchronization and trust between Dev/Test/Business and Product.
Embedding AI Test Automation
The industry is in its early stages of embedding AI/ML test automation into existing DevOps pipelines. This can be a huge win to the market by increasing test automation coverage, enhancing CI reliability, and delivering better software faster to the market. To succeed in this transformation, organizations need to look at the abovementioned considerations: test type: dynamic vs. static areas, existing skillset, their process maturity and fitness to this technology.