In Blog

Interesting Testing Takeaways

A fascinating survey, “Application Crisis Research” was published by TestPlant that sheds light on the complexities and challenges app developers are facing in today’s high-pressure digital world.

The survey highlights the compromises R&D departments are forced to make in relation to the quality of their apps due to shortening release cycle and market demands. 

These are points that we at TestCraft have been making for some time now – current testing methodologies applied by software and app developers aren’t able to “catch up” with the rapid agile development model used by most companies these days. Whether the majority of testing is done manually on the one hand, or heavy reliance on automation on the other – both create unnecessary pressure on testing teams and the entire development process.

Let’s look at the main findings of the survey, conducted among 750 development team leaders (75% from the US, 25% from the UK) from a wide variety of industries.

The Pressure Cooker Effect

The vast majority of respondents, 91%, agree that user expectations for app quality and innovation have increased in the past year.

These high expectations create pressure – pressure to perform, the pressure to deliver on time, and pressure to wow users. While market expectations are generally a good thing because it drives companies to excel, there are also  “casualties” along the way.

62% say the majority of pressure comes from consumer expectations

74% say they feel pressure because of consumer expectations for app performance

70% say they feel pressure because of consumer expectations to produce innovation quickly

That goes to show that the competition in the field is fierce and that consumers are fickle. There isn’t much loyalty here. If an app fails to impress, users will be quick to abandon it and try out the competitor.

Here’s a stat that’s highly relevant for us in the testing universe:

73% say they feel pressure because of consumer expectations to produce bug-free apps

This one seems almost too obvious, isn’t it? Of course, consumers expect bug-free apps, the same way I expect the food I order in a restaurant not to be overly salty, or the toy I buy for my kid to work properly.

So, the bug-free pressure might be an indication that development team leaders feel that they don’t do all they should, or can, in order to ensure their product is bug-free.

Quality Takes a Hit

The pressure development teams feel takes its toll directly on quality via degrading of the testing processes.

40% put out untested apps

49% put out apps before they go through ideal testing

45% knowingly put out apps that will perform below initial requirements

These stats make our heads spin (and kudos to survey participants for their brutal honesty).

Going back to the restaurant allegory, putting out an untested app is equivalent to serving a dish without tasting it first. It requires a great deal of confidence mixed with arrogance from a chef. With apps, it is a testimony to a failure to prioritize and a very short-sighted agenda, if we might suggest a hint of criticism.

The importance of testing cannot be overstated, especially in light of the intense pressure from consumers for highly polished, bug-free, and wow-inducing apps. With such competition out there and the ability to delete an app in a split second and download the competing one, it is no less than puzzling that 40% are releasing untested apps.

It’s a catch 22 circular disaster waiting to happen. Instead of the pressure from consumers leading to harsher testing criteria, the opposite is happening.

Time Constraints

42% believe that they are expected to design, develop, and test apps in an unrealistic amount of time

36% agree they are not given enough time to ensure apps are properly tested before deployment

Many years ago Tom Waits said, “Time; time is a funny thing.” Well, not anymore. Nothing funny about time these days. Time has become the most coveted commodity for app and software developers, as well as for the public that uses their products.  

This is another point we have been making over and over again – the agile methodology is development-focused; it was designed and executed with the needs and working style of coders, and it completely disregarded the needs and workflow of testers. For years now testing teams are playing catch-up with development and scrambling to find ways to speed up testing.

The main method the testing industry adopted is automation. On the surface it makes sense – automation is quick and can work horizontally, meaning, run numerous tests on multiple platforms simultaneously. So far so good. But when you add maintenance to the equation it all falls apart, so to speak. Automation maintenance is a heavy task that requires not only additional time and resources but also coding skills, a skill traditionally not required from QA testers.

So time is one of the biggest pains of testing teams, a problem that has not been solved so far, at least not by code-based automation solutions.

What about code-free test automation solutions? There is one, it’s called TestCraft.

TestCraft puts to rest the two main obstacles of other test automation solutions: it requires no coding skills (test creation is done on a visual interface of drag-and-drop) and it requires minimal maintenance time (fixing and updating test flows is done on-the-fly during runtime.)


start Free Trial