Test Executions

Overview

In this topic, you can learn how to manage your test executions. The test executions serve as an immutable record of the execution of any respective test set, at a specific point in time. By comparison, the test set is always subject to change, as it can target a different application version or its test case selection can be modified.

Re-execute Test Set

Important: Known limitation: The results from re-executing a test set in Orchestrator are not synced to Test Manager. We recommend you to execute and re-execute tests directly from Test Manager.

You can re-execute test sets after a previous run. The new test case result will override the previous one shown on the Test Executions page.

Be aware that you cannot re-execute test cases from a Test Set with enabled coverage calculation.

1. Log in to Orchestrator.
2. Open your folder.
3. Navigate to Testing > Test Executions.
4. Select a test run and go to More actions > Details.
5. Select a test case and go to More actions > Re-execute.
6. Configure the execution target:
   • User: Select the user from the dropdown list that is assigned to that specific folder. The default is set to Any User.

   • Machine: Select the machine resource from the dropdown list.

     Note: For the Classic Folder, you need to select your Environment and then choose whether you want to allocate the execution dynamically or to specific robots.
7. (Optional) Define custom Arguments for your test cases to override default values at the test set level. To define arguments, you need to have published a package with arguments.
8. Click Execute to confirm.

Test Execution Coverage

You can view information on the activities that have been covered during a test execution. This option is also available in Studio, as Activity Coverage.

Important: Activity coverage calculator is available when you select this option for existing or new test sets. For more information, see Create test set.

1. Open your folder in Orchestrator.
2. Navigate to Testing > Test Executions.
3. Select a test run and go to More actions > Details.
4. Go to the Coverage tab to view coverage details.

   The coverage files are uploaded to the Attachments tab.

   
   

   

   
   

Analyze Test Executions

You can analyze test executions to understand what happened in instances where the execution failed or got stuck.

Note: You can view screenshots of failed test executions by navigating to Automations > Job > Job Details. To benefit from this feature, you need to enable execution media at the application level. For more information, see Execution Media Permissions.

You can choose from the following actions to examine your test runs.

Actions	Description	Procedure
Execution details	View Robot details, including execution attachments, data variation and screenshots. Make sure that you have Execution Media Permissions.	Open your folder in Orchestrator. Navigate to Testing > Test Executions. Select a test run and go to More Actions > Details Click Details to view the execution details.
View assertions	View failed or passed assertions.	Open your folder in Orchestrator. Navigate to Testing > Test Executions. Select a test run and go to More actions > Details. Select a test case and go to More actions > View assertions.
View logs	View and export a detailed log of execution events.	Open your folder in Orchestrator. Navigate to Testing > Test Executions. Select a test run and go to More actions > Details. Select a test case and go to More details > View logs. Click Export to get the test execution details in a CSV format.

Test Executions API

To query for item related activities, you can use the Background Tasks, internally available through API. For more information on Orchestrator Permissions, see Default Roles. For more information on test automation API, see Test Automation API Permissions.