Test Suite
  • StudioPro
  • Orchestrator
  • Testing robots
  • Test Manager
Banner background image
Test Suite User Guide
Last updated Feb 28, 2024

Test Results

Understanding how test results are organized in Test Manager is very important in order to deal with dashboards and reporting.

One fundamental concept in Test Manager is that results need to represent the test how it was designed when it was executed. This needs to hold true even if a test case has been updated or even deleted after it has been executed.

The same applies to test sets. Execution results need to represent the entire set even if case tests have been removed from the set after the initial execution.

When you start executing a test set, Test Manager goes through the following process:

  1. A Test Execution is created in the Test Results section of Test Manager. This serves as the container for the test results.
  2. For every test case within the executed test set, an empty Test Case Log is generated. This log is attached to the Test Execution and will hold its associated test results.
  3. As the execution continues, the Test Manager fills each Test Case Log with results and related log information.
Note: Every time a test execution is created, Test Case Logs for all associated test cases from the original test set are added. This means existing Test Executions remain consistent, regardless of updates or deletions made to the original test set.

The chart below illustrates the process of executing a test set in Test Manager.

As a consequence, after a test set has been executed, any changes to the test set or its associated test cases do not affect the results from previous executions. Even when a test set is deleted, all the Test Executions remain unchanged.

To view test executions, select Test Results from the project navigation. You will see a list of Test Executions each representing the execution of a test set.

Analyzing Test Results

Selecting Test Results from the project navigation enables you to view test results. This is your go-to place for:

  • finding your test results even if they are from manual executions or automatic ones from Orchestrator
  • see progress from running executions
  • analyzing details from results like logs and screenshots
  • creating defects in your defect management system directly from Test Manager

The Overview List

The overview displays all Test Executions available, sorted according to the date on which the execution finished. It also displays executions that are still running, have been canceled, or are pending. Running and pending executions are always on top of the list.

The results column shows the results summary from each execution. Additionally, the results column is color-coded to help you easily figure out what happened with a particular test set execution:

  • Green - test cases that passed;
  • Red - failed test cases;
  • Grey - test cases that don’t have a result, such as test cases that have not been executed yet but are part of a test set that is currently being executed.

Results from Automated Tests

Results from automated tests are always imported from UiPath Orchestrator automatically. To have a particular result from a test case execution imported, the following requirements have to be met:

  • The Orchestrator instance has to be assigned to this Test Manager instance, as described here.
  • The test case has to exist on this Test Manager instance (in any project), as described here.

Results from automated tests are imported into the projects where the corresponding test cases are.

Note: If a Test Execution from Orchestrator holds results for test cases from several projects in Test Manager, the Test Execution gets split in Test Manager. Results then appear in the projects where the test cases are.

Drilling Into Test Execution Results

To get detailed results from a test execution, click its name in the Test Results page. This opens the Test Execution detail view.

In the Overview tab, you find some basic information about the execution such as when it started, the duration, and the accumulated results.

The Reporting Date lets you select a date on which the results from this Test Execution are assigned. This is useful, for example, when dealing with results from nightly runs where some test cases are executed before midnight and others are executed after midnight. In this case, you would get a random split between dates in reports. The reporting date lets you assign all the results to a particular day no matter when the execution technically took place.

At the bottom, a list of all the failed test cases within this Test Execution is displayed, along with additional information.

In the Results tab, the list of results from all the test cases within this Test Execution is displayed.

Each item in the list represents the execution of one test case. Clicking the key of a test case navigates to the test case. Clicking the results label displays the Test Case Log which holds all the log information along with screenshots that were collected during the execution of this particular test case.

Test Case Logs

Clicking a result label opens the Test Case Log of this particular result. This log holds all the data that was collected during execution. It provides a detailed description of all the steps that have been performed during the execution, as well as events that occurred such as error messages that popped up. In addition, all the screenshots that were taken during the execution can be found as part of the Test Case Log.

The Screenshots tab enables you to see all the screenshots that were taken during the execution of a failed test case. Click the preview to get a larger view of a screenshot.

Note: Please note that the Screenshots tab is not displayed if no screenshots were taking during the execution of a test case.

Creating Defects from Results

One of the main purposes of testing is to create a detailed description of the problems that were found during test execution. The quality of information about defects, provided to developers, is critical to a fast-paced development cycle. You can create detailed defect reports directly from Test Manager in your external defect management system if properly configured.

To create a defect in your defect management system, open any Test Case Log and select Create Defect from the Tasks menu. A message is displayed when the defect has been created. In addition, a link to the created defect is displayed in the Test Case Log, under the Defect section.

The defect provides all the information from the Test Case Log along with the screenshots from Orchestrator.

Was this page helpful?

Get The Help You Need
Learning RPA - Automation Courses
UiPath Community Forum
Uipath Logo White
Trust and Security
© 2005-2024 UiPath. All rights reserved.