Analyzing results

Review test execution results, logs, and attachments, and generate reports in Test Manager.

Analyze the results of your test executions in Test Manager, and decide on the actions to take.

Context

Executions in Test Manager provide insights about the execution of test sets.

Test executions are color-coded to indicate their status:

• Green - Test cases that passed.
• Red - Test cases that failed.
• Grey - Test cases without definitive results, such as those not yet executed within an ongoing test set.

By analyzing test executions, you can perform the following actions:

• Identify manual or automated test results.
• Track the progress of ongoing test executions.
• Open the executed test set to review the details.
• Examine logs and attachments associated with test executions.
• Create defects in your defect management system directly from Test Manager if you have an Application Lifecycle Management (ALM) tool integration.
  Note:

  Execution and their associated data remain unaffected by changes or deletion of test sets.

Generate a report with Autopilot that details why your test cases are repeatedly failing. The report can have up to five sections, each consisting of multiple insights. For each insight in the report, select the icon displaying the number of test cases, or Show, to view the impacted related objects, such as test case, or test case logs, and search through them.

Note:

Feature availability depends on the cloud platform you use. For details, refer to the Test Manager feature availability page.

Steps

1. Open Execution in Test Manager to view a list of test executions. Each test execution represents the execution of a test set. The executions are sorted based on the finish date, with ongoing and pending executions displayed at the top.
2. Select a test execution entry to open the detailed view. This view displays information such as execution start time, duration, and execution logs. The Results section provides an overview of all individual test results from test cases, named Test Case Logs. You can take actions on individual test results.
3. Select the test case key within the Execution page to access the execution log.
4. To generate a test report, select Generate Report for the desired execution.
5. Select the results you want included in the report.
6. Select Generate Report again.
7. When the report is ready, select the Execution Insights Report is ready notification, or Open report in the email notification, to access it.

Result

You can analyze the results of your test executions, examine their logs, and create defects from the results.

Information and actions in test results

The test results provide the following information and actions:

• Execution Log: The execution log contains essential information, including assertions, logs, execution details, affected requirements, and attachments. Select the test case key within the Execution page to access the execution log.
• Activity Coverage: You can enable activity coverage for automated test executions. This feature provides information on the activities covered during the test execution.
• Defect Management Integration: If you have an external defect management system integrated with Test Manager, you can create defect reports directly from the execution logs. Open a test case log, select Tasks, and then Create Defect.
• Defect Synchronization: Test Manager allows synchronization of execution results with external tools as part of an ALM tool integration. Execution details, logs, timestamps, and other relevant information are shared with the integrated tool.
• Create and Unlink Defects: Within the Execution page, you can create defects by accessing test case logs, selecting Tasks, and selecting Create Defect. Additionally, you have the option to unlink defects from an external tool, which detaches the test execution result from the external tool's entry.

Related articles

• Test results
• ALM tool integration