Compare Test Results with Baseline Recordings
Compare to Recording allows you to compare the current test execution (this run) of the test alongside your authored test case (baseline). This feature serves as a good troubleshooting tool when diagnosing an error that occurs within a test because it enables you to visually compare your executions and provide insights into what was different.
To make a comparison, complete the following steps:
- You can access all the existing test cases & their results by navigating to the Automation tab and clicking on the Runs & Reports tab.
- You can filter out your tests based on if you ran them as individual tests or as schedules. Here, for example, let us see all the lab runs:
- To view the details of a test case, click on the Details icon next to your test case. For example, here we wish to diagnose the test case titled as test1.
- In the new screen which appears, you will see the steps in the form of cards on the screen. In case your test failed, you can click on See Failure Details to view the reasons for test failure:
- You can click on the Compare to Recording option towards the bottom to view the Baseline (original) vs This Run recording:
- In the Compare to Recording modal which now appears, you will see the list of individual steps inside your test. You can navigate to whichever step you think caused the test failure. The Next & Previous arrows on the top enable you to navigate:
- You will also see two screens - the left screen (Baseline) displays the view/status of the screen at a certain step at time of authoring your test, whereas the right screen (This Run) shows the view of the screen during the latest test case execution. In the example shown below, the test case authored (Baseline) moves to the search result screen with a list of hotels available. However, in the current execution (This Run) the screen didn't move on to the search results.
- You will also find the reason of the test failure under the section Failure Reason. In this case, the screen did not transition to the list of hotels as shown on the baseline (left) and hence it failed when the test was run again (this run).
- You can also use the Element Explorer section to select individual elements & view their details such as their class, description, resource ID, and XPath, which can help you better diagnose elements.
