TestCraft vs. Test Recorders: Which Will Reign Supreme?
Test automation and test recording are two different QA testing methods that are often confused. The source of confusion is that both define themselves as “automated” and “codeless” and in both cases, rightfully so. Let’s take a look under the hood to understand the advantages each tool has when dealing with software development operation.
In this post we’ll set the record straight and highlight the differences between test recording tools and our own test automation tool − TestCraft, broken down to three categories:
- Test creation
- Test execution
- Test maintenance and updating
|Test Creation||Dual creation mechanism:|
- Drag & drop interface
- Live point & click modeling
Create test flows prior to code - “Shift-Left” benefit
|Single recording mechanism |
Record only after the application is working
|Test Execution||On-the-Fly binding (during runtime)|
Run the tests on multiple browsers simultaneously
|Test Maintenance and Updating||Reuse elements and flows across scenarios |
Dual output - data & scenario
Single output - a recording
The Authoring Mechanism
With recorders, the creation process is super simple – capturing user actions in real time. The benefits are clear – ease of use and speed. Tests can be created in the span of time it takes to perform the action. It must be noted though that if a mistake was recorded, there’s no way of fixing it; it’s back to square one, record again.
TestCraft cannot compete with speediness, but can compensate with two test creation mechanisms:
- Create visual flows by dragging and dropping elements, or sequences of elements, onto a virtual canvas.
- Model flows by pointing and clicking on real, online elements in the web application, during runtime.*
*Though number 2 might sound like a recorder, there’s a big difference; we’ll explain this and more in the section about Test Maintenance and Updating.
Test Creation: Before or During Coding
A major implication of the above differences is that with TestCraft, you can create tests even before the actual code has been written.
Working on the virtual canvas, you can design the tests in an abstract way based solely on the product wireframe or product flow (this can be done by the product manager or the tester, in case he’s part of the product design process). Later on, after the development team has coded the feature, you can get the specific details of each flow and bind the actual elements during runtime. This has a strong shift-left benefit for companies that pursue continuous testing to achieve an agile work process.
With recorders, you do not have this advantage, as you have to wait for the feature to be completed in order to create the recording.
Dynamic Data Injection
Another major difference between TestCraft and recorders is that with TestCraft, you can receive data from backend components to be integrated into the test flow and use it for validation purposes.
Let’s look at a real example, used by one of our customers – a bank – for dynamically injecting data from an external source and how this feature can come in handy. The bank runs its testing live, on the actual site, to test a process that involves a confiscated account. But the identity of confiscated accounts changes daily; some accounts are being settled while others are constantly being added.
TestCraft knows how to send a query to the bank’s database before every run, retrieve confiscated account details, and inject this data into the test model.
As far as we can tell, recorders don’t offer this functionality.
With test recorders, the execution stage is very straightforward; whatever was recorded is being “played back” to test the usability of the web application. If the test breaks, you would have to re-record it and let it run all over again.
TestCraft, on the other hand, gives a more hands-on execution of tests; if something breaks, you can fix it right then and there by rebinding the elements.
Our execution mode alerts you when an element breaks so you could fix it on the spot by pointing to the right element. TestCraft will bind the element and continue the test execution. A second execution mode runs completely automatically and will fail immediately when it identifies any breakage.
Another execution implementation of TestCraft is for monitoring, not just testing. High traffic sites, such as e-commerce or online services, often deal with dozens, if not hundreds, of processes simultaneously, and the smallest failure can have direct repercussions on sales or user satisfaction.
With TestCraft, you can create flows to be used for application monitoring. After you created one or more flows, you can schedule them to run on any intervals you choose, ensuring that once it identifies an anomaly in performance, you’ll know about it almost instantly.
TestCraft also provides detailed reports for every execution, from the single element to comprehensive performance monitoring reports.
Reuse Elements and Flows
Test automation delivers speed and ease of use. Recorders definitely offer both; they’re super easy to use and also pretty quick to operate during the creation stage. But what happens afterward, when processes need to be edited or adjusted to accommodate code changes? TestCraft offers the same ease and speed of the creation stage during any following development.
Let’s say I need to test a purchasing process in an online store. There are three main components to this:
- Arriving at the site
- Selecting an item
- Checking out
With a recorder, I’ll go through the entire process as a user. With TestCraft I’ll either drag-and-drop elements onto the canvas, or model through point-and-click on live elements in the shop itself, during runtime.
The result will be similar on the surface – a documented scenario for a purchasing process.
But what happens if now I wish to test the purchase of two products? With a recorder, it is ‘back to the drawing board’ – record from scratch.
With TestCraft, it is much easier – I simply duplicate component number 2 and run a new test for the purchase of two products.
I can reuse components across various flows with higher ease of use.
Dual Output – Data & Scenario
All of the above (and much more) is due to the fact the TestCraft produces two independent outputs – data and scenario, in contrast, to test recorders that have a single output – a recording. The ability to edit, update and implement changes in each of the outputs separately is where TestCraft’s strength shines through.
TestCraft provides fast, easy and complete control over testing and the ability to integrate the QA testing operation deep into the agile development process. Deployment of features and product cycles becomes much smoother and faster – allowing for a more thorough software testing process and improving the overall quality of your web application.
Keeping It Pure
The final element that distinguishes our product from test recorders is that TestCraft is double pure: it is based on pure SaaS – no agents or extensions and is purely codeless – no tech skills required.