In Case Study

1,793

Scheduled monitoring runs per week

156

Monitoring run hours per week

Number of automated tests

5,364

Test steps

Lines of Selenium code

80,500

Lines of code

For online retailers, every minute counts. A bug in their online shop literally costs money every minute it’s not fixed; online users are fickle and impatient – a button doesn’t work, a screen fails to load quickly enough and they close the tab and off to a different site.

This case study will illustrate the importance of Live Monitoring for e-commerce sites through a TestCraft client and will provide real examples of process monitoring.

The client is a leading online retailer with highly successful online shop. The shop has an average of 350k visits per month (This online retailer preferred to stay anonymous due to business strategy considerations).

  • TestCraft enables us to make sure that all our fundamental processes are working properly, constantly, in the easiest, most efficient way.

    CTO, Online Retailer

What Led this Online Retailer to Seek a Live Monitoring Solution

It was crucial for this online retailer to continuously monitor, 24/7, that all the main features and functionalities are working properly at all time.

The goal is to recognize a problem – a bug – before the users stumble upon it, and report it back so, the development team will be able to fix it as soon as possible.

Going for Process Monitoring

Rather than focusing on Basic Monitoring – checking that the site is live like most monitoring solutions do – the online retailer needed Process Monitoring – ensuring all the main process of the e-commerce platform are operating as they should.

The most obvious example of Process Monitoring in e-commerce is the purchase process – selecting an item, adding it to the cart and paying for it.

Setting Up the Purchase Process

The ‘secret’ to live monitoring is not to replicate a process, but to actually conduct the process, live on the site; in case of a purchase process, with a real item that is available on the online shop and a real credit card for going through the actual payment process.

In order not to deplete the inventory of a certain product and distort the overall sale’s stats, a dummy product needs to be created. Since an actual purchase is being made every time the monitoring process runs, it is advisable to price the product as low as possible.

The dummy product is set up on its own item page, but without inner linking in order to hide it from shop visitors.

TestCraft to the Rescue – Continuous Live Monitoring

In order to ensure the online shop is live and fully-functional around the clock, the online retailer decided on a live monitoring run every hour. This was enabled by the TestCraft Scheduler that manages repetitive test actions on a programmed schedule.  

This monitoring scenario runs simultaneously for guest visitor and for a registered user, on mobile and on different desktop browsers, once every hour, 24-7.  

This online retailer conducts an average 1,344 monitoring runs per week (that on top of other test scenarios that are conducted by the R&D team as part of the development and release cycles.)

The Live Monitoring Purchase Process Creation

The creation of the monitoring processes was done using TestCraft’s visual drag-and-drop interface. The process was up and running within 10 minutes, no coding was required.

TestCraft creates a model for each test or monitoring process – not a mere recording. Under the hood, Selenium code is automatically generated, allowing for easier future binding of elements; TestCraft automatically assigns a binding score for each element so if a test breaks, you can easily click-to-rebind the element in real time, allowing the test to resume.

The model enables any tester to maintain the tests and adapt them to any changes that are made to the app.

Here’s how it looks on the TestCraft canvas:

A Successful Bug Detection

Add to Cart is probably the most iconic phrase of the e-commerce era. But what an online shopper to do when she can’t find the cart she had filled with numerous items? Not to mention the online retailer – a full cart gone missing? Nothing can be worse than that.

That’s exactly the bug identified through the process monitoring conducted using TestCraft. A user added to the cart an item only to be unable to find the cart – the cart’s icon was missing from the top navigation bar. The bug was discovered in one of the ongoing live monitoring runs, reported automatically to the development team that fixed it immediately.

Without TestCraft’s live monitoring, this bug could have resulted in numerous cart abandonments and frustrated shoppers that might not have returned to the site in the future. Remember? Online users are fickle and impatient – you only get one chance with them. A lost cart can mean the loss of future potential income from that shopper.

The Lean TestCraft Machine

With TestCraft being a pure SaaS solution, the online retailer didn’t need to:

  • Set up an automation framework (and hire automation specialists)
  • Hire test engineers
  • Integrate the framework with existing tools (and create new work processes)
  • Set up a testing lab or connect with an external one

The live monitoring runs via TestCraft’s remote servers, leaving the online retailer with minimal maintenance to handle, very low operational costs and a highly reliable live monitoring process.

Free Trial

Please register here and start your free trial