A Comprehensive Guide to End to End Testing
Table of Contents
1. What Does End to End Testing Mean?
2. How Does It Work?
3. Types of E2E Tests
4. E2E Testing vs. System Testing
5. End to End Testing Checklist
6. E2E Testing Example
7. E2E Testing Challenges and How to Overcome Them
The importance of testing as part of the software development lifecycle cannot be overemphasized. It is a determining factor of Software Quality Assurance. Testing creates confidence in an application’s build by assuring that every line of code, every feature on the website, and every flow works as expected.
In this comprehensive guide to understanding what end to end (E2E) tests are, we will look at how they work, as well as an example of where you apply end to end tests.
What Does End to End Testing Mean?
End to end testing makes sure a system works as expected for the end user by testing the different flows that a user might take within an application. In fact, the phrase “beginning-to-end” would be more fitting because this testing focuses on testing the flow from the beginning to the end of a given process.
Unit testing focuses on the smallest components of an application — testing an API endpoint, testing a function, etc. While unit testing allows the tester to focus on these components, it is important to ensure all the pieces work together.
Making sure the pieces work together properly is goal of integration testing. While integration testing helps ensure the reliability of an application, it does not cover everything. Because we do not know how the application works for the end user. And this is where end to end tests come in.
E2E testing involves techniques that simulate an actual user of an application. By replicating the actions a user would take, the test helps evaluate whether the results comply with the requirements or expected outcome.
Examples include testing a user experience such as:
- Paying for a product or service on a website.
- Registering on a website.
- Using a feature.
All these would be done in a test environment which would ideally represent the production environment in terms of databases used, API services used, and type of device used.
Illustration of an end to end test of a user logging into a website.
However, there are cases where you cannot perfectly imitate the usage of some services. In such cases, you would need to fake or imitate them (an example is mocking API requests). However, this should be done as infrequently as possible because simulating too much testing reduces confidence.
How Do E2E Tests Work?
E2E tests involve applying tests that follow the flow of an application and help discover a break in the flow. A break in the flow would determine an error in the application.
Here’s the usual process involved in E2E testing:
- Tests are planned and designed for different requirements using testing frameworks or platforms.
- The tests are executed (usually on different devices).
- The breaking tests fail.
- The developer debugs the code that broke.
- Then the tests are re-run.
Types of E2E Tests
There are two main types, manual E2E testing and automated E2E testing.
In manual testing, a business tester pretends to be the user (on as many devices and screen sizes that he has access to) to determine compliance with the application requirements. For example, buying a product on an e-commerce store.
The tester would follow the same process that a normal user would in buying the product — clicking the add-to-cart button, opening the cart, going to the purchase dialogue, and paying.
The tester aims to determine whether any steps in the flow break.
The goal of automated E2E testing is the same. Only that automated testing offers more sophisticated capabilities that a normal tester may not be able to achieve. Different test automation tools offer automated testing services and provide various devices (called “device labs”) that tests can be run on.
Manual Testing vs. Automated Testing
- Manual testing involves a tester implementing tests by hand, as stated above. Human error can contribute to poor testing. For automated tests, test run automatically. So in terms of accuracy, automated tests can be better in many scenarios.
- Manual testing is easier to set up and understand. Most times, automated testing is complex to setup. The learning curve is typically higher for automated testing.
- Manual testing involves less financial and resource investment at the beginning. While automated testing may involve more initial investment, it offers significant ROI as you’re testing process scales and matures. So, this is something to consider when making your choice on automated testing tools.
- In manual testing, it’s easier to simulate a production environment. But some automated tools do not make that process easier. For example, it’s easy to ensure that some conditions are met before a feature works manually, but automated testing tools may be difficult to configure.
E2E Testing vs. System Testing
E2E and system testing are remarkably similar. However, few notable differences can be distinguished between them.
Firstly, let’s look at the flow of system development:
- System testing starts with identifying the requirements (the system should be able to do this or that) based on research carried out.
- The results are then categorized and broken into components that can be worked on individually.
- This is followed by testing — testing the components individually (unit testing)
- Then, integration testing, where two or more components are tested together.
- This is where system testing comes in. It involves testing all components in the system (in no particular order) to ensure it matches the functional and non-functional requirements.
- After this process is E2E testing. It involves testing the actual flow of a system to ensure that the functional requirements are achieved.
Let’s use an example to make the difference between E2E testing and system testing clear: buying a product on an e-commerce website.
System testing involves ensuring the add to cart button works, the product can be searched, the related API for the action for adding to cart is called, and the payment service.
For end to end testing, the following flow is used. A user searches for the product -> clicks add to cart -> cart API is called -> the total number of products in the cart increments -> user clicks on the cart -> and tries to purchase.
System testing finds issues in the components while end to end testing finds issues in the flow.
The key difference is end to end is from one point to another, while System testing is general testing of the applications components.
E2E Testing Checklist
E2E is not solely about testing the user interface because there are many pieces that make up a user interface.
For a blog site, the content may be retrieved from a database through an API. In this case, it’s obvious that the frontend design, the database, and the API must be tested separately and together.
Here’s a checklist that can be followed during E2E tests:
- Database: The database used for your system would need to be tested. You could test to ensure that data are stored properly, organized properly, can be read, and updated correctly.
- Performance: A webpage may navigate correctly but the speed also matters for user experience. Hence, it’s important to test the performance of a page or feature.
- Security: Web application security determines how secure an application is for the user and the organization. Vulnerability testing tools are especially important in this case.
- Functionality: This is the main reason for testing in the first place. All features must function as expected. Unit tests can also be applicable here.
- Usability: As components are functional, they should be usable. Users are as important as the tool, so tests must cover events (like clicks) on elements, proper navigation, and so on.
E2E Testing Example
For our example, we’ll be testing the sign-in process of Twitter using TestCraft, a codeless testing platform that can be used for end to end interface testing.
Before setting up our tests, let’s define a test flow.
- Go to twitter.com.
- Click the login button.
- Ensure the login button does not do anything if the any of inputs are empty.
- Submit the details on clicking the button when inputs are not empty (we’ll test with incorrect information).
- Get an error message (“The username and password you entered did not match our records. Please double-check and try again.”) in red color.
- Save test.
If the credentials are correct, the user will be directed to their account (which can be another test).
Let’s get started with automating that test.
Testing Twitter Sign-in With TestCraft
You will need to create an account to use TestCraft.
After signing into your portal, you are presented with an interface.
Click on New Project, then you’re shown a form for which you provide the project name and the project website.
Click create, and then click on the project to open more details.
Hover on the “Create your first spec” option and choose “on-the-fly modeling.”
Configure the environment (website) and platform (browser) you want, then click execute.
Here’s the dashboard that generates.
The left pane shows the flow , from initializing to navigating to location. As we add more steps, they will be added in the left pane.
The right pane shows the application. On this application, we can target elements, click buttons, check fields, and so on.
Let’s try our flow on this app.
- Step 1 (go to twitter.com) has been achieved already.
- Then click the login button.
- Hover over the login button.
- Select the popup mouse icon to activate a click as the next step.
The action (clicking on login) would be added to the flow.
Next, we want to ensure that the button does nothing when inputs are empty. To do this, we’ll add a click event to the button when both fields are empty and when only one is empty.
Both fields are empty.
One field is empty (the username is 1).
Enter password (1), which should be the wrong credential, and you’ll see an error.
We can further confirm the error by checking if the error is present. To do this, we’ll select the eye icon which has a title “Check element present” like below.
That’s all for our test. With this flow, we automate the process of signing in with the wrong details and ensure the error is shown.
To complete the test, click the last flow shown on the left pane. When you open the test from the dashboard, you’ll have something like below.
This dialogue shows the flow of our application. From entering the website, clicking log in, entering a username and password, and so on.
Click “RUN” (in the top navigation) and the test will be executed. We can quickly determine if something has been broken.
This, of course, is just one way. There are many other capabilities that TestCraft offers when it comes to automated testing.
E2E Testing Challenges and How to Overcome Them
As useful as E2E tests are, they have their challenges and complexities in setting them up or using them optimally. In this section, we’ll look at some of them and the ways to overcome such challenges.
Determining Scenarios to Test
For a sign-in process, it may be easy to determine the steps a user would take, but for other cases, like using a dashboard with many elements, it can become difficult to decide.
To solve this, teams must put heads together to think of all scenarios possible or at least those that would require much attention. User feedback can also help teams consider more scenarios.
Reproducing a Production Environment
A production environment has so many factors that may not be considered in a test environment. Take, for example, the user’s internet connection. In a testing environment, a perfect internet connection may be used, so it’s very hard to evaluate the experience of users with a poor internet connection.
The solution for this is to opt-in for better testing platforms. As these problems (like internet connection) are discovered, many companies make up testing solutions for them. Some of them would provide lower network connection options so that testers can determine the experience.
Choosing a Testing Framework/Service
The decision becomes a challenge as there as many services out there. Not all of them are bad, but to ensure quality testing and access to powerful tools, the right decisions must be made.
When choosing a framework or service, here are few options to consider when making a comparison.
- Which framework is faster.
- Which framework offers more tools.
- Which framework makes creation and modification (management) of tests more seamless.
And many other options.
In this article, we covered the importance of end to end tests and how to go about them.
End to end testing plays a vital role in determining the user experience of an application. More than testing if a button works or if a model displays, testing the user experience from beginning to end brings more assurance regarding the quality of your application.
TestCraft offers a Selenium-based end to end testing tool that allows development teams to expedite their testing of web applications. The codeless Selenium-based automation tool can be easily integrated into your CI/CD workflow within a unified dashboard.
Get in touch with us to learn more about our innovative automation testing solutions.