- StudioPro
- Orchestrator
- Testing robots
- Test Manager
Test Results
Understanding how test results are organized in Test Manager is very important in order to deal with dashboards and reporting.
One fundamental concept in Test Manager is that results need to represent the test how it was designed when it was executed. This needs to hold true even if a test case has been updated or even deleted after it has been executed.
The same applies to test sets. Execution results need to represent the entire set even if case tests have been removed from the set after the initial execution.
When you start executing a test set, Test Manager goes through the following process:
- A Test Execution is created in the Test Results section of Test Manager. This serves as the container for the test results.
- For every test case within the executed test set, an empty Test Case Log is generated. This log is attached to the Test Execution and will hold its associated test results.
- As the execution continues, the Test Manager fills each Test Case Log with results and related log information.
The chart below illustrates the process of executing a test set in Test Manager.
As a consequence, after a test set has been executed, any changes to the test set or its associated test cases do not affect the results from previous executions. Even when a test set is deleted, all the Test Executions remain unchanged.
To view test executions, select Test Results from the project navigation. You will see a list of Test Executions each representing the execution of a test set.
Selecting Test Results from the project navigation enables you to view test results. This is your go-to place for:
- finding your test results even if they are from manual executions or automatic ones from Orchestrator
- see progress from running executions
- analyzing details from results like logs and screenshots
- creating defects in your defect management system directly from Test Manager
The overview displays all Test Executions available, sorted according to the date on which the execution finished. It also displays executions that are still running, have been canceled, or are pending. Running and pending executions are always on top of the list.
The results column shows the results summary from each execution. Additionally, the results column is color-coded to help you easily figure out what happened with a particular test set execution:
- Green - test cases that passed;
- Red - failed test cases;
- Grey - test cases that don’t have a result, such as test cases that have not been executed yet but are part of a test set that is currently being executed.
Results from automated tests are always imported from UiPath Orchestrator automatically. To have a particular result from a test case execution imported, the following requirements have to be met:
- The Orchestrator instance has to be assigned to this Test Manager instance, as described here.
- The test case has to exist on this Test Manager instance (in any project), as described here.
Results from automated tests are imported into the projects where the corresponding test cases are.
To get detailed results from a test execution, click its name in the Test Results page. This opens the Test Execution detail view.
In the Overview tab, you find some basic information about the execution such as when it started, the duration, and the accumulated results.
The Reporting Date lets you select a date on which the results from this Test Execution are assigned. This is useful, for example, when dealing with results from nightly runs where some test cases are executed before midnight and others are executed after midnight. In this case, you would get a random split between dates in reports. The reporting date lets you assign all the results to a particular day no matter when the execution technically took place.
At the bottom, a list of all the failed test cases within this Test Execution is displayed, along with additional information.
In the Results tab, the list of results from all the test cases within this Test Execution is displayed.
Each item in the list represents the execution of one test case. Clicking the key of a test case navigates to the test case. Clicking the results label displays the Test Case Log which holds all the log information along with screenshots that were collected during the execution of this particular test case.
Clicking a result label opens the Test Case Log of this particular result. This log holds all the data that was collected during execution. It provides a detailed description of all the steps that have been performed during the execution, as well as events that occurred such as error messages that popped up. In addition, all the screenshots that were taken during the execution can be found as part of the Test Case Log.
The Screenshots tab enables you to see all the screenshots that were taken during the execution of a failed test case. Click the preview to get a larger view of a screenshot.
One of the main purposes of testing is to create a detailed description of the problems that were found during test execution. The quality of information about defects, provided to developers, is critical to a fast-paced development cycle. You can create detailed defect reports directly from Test Manager in your external defect management system if properly configured.
To create a defect in your defect management system, open any Test Case Log and select Create Defect from the Tasks menu. A message is displayed when the defect has been created. In addition, a link to the created defect is displayed in the Test Case Log, under the Defect section.
The defect provides all the information from the Test Case Log along with the screenshots from Orchestrator.
