AI-Powered insights

Important:

This feature is currently part of an audit process and is not to be considered part of the FedRAMP Authorization until the review is finalized. See here the full list of features currently under review.

Get actionable insights into your test results by generating a report with Autopilot^TM, detailing why your test cases are repeatedly failing. The report can have up to five sections, each consisting of multiple insights. For each insight in the report, select the icon displaying the number of test cases, or Show, to view the impacted related objects, such as test case, or test case logs, and search through them.

Check the details for the identified issues in the following table:

Name	Description
Overview	An overview of the test results that you selected for the report, showing information about the average failure rate, the test set failure rate, and the percentage of errors by severity.
Top Failing Tests	Shows most frequently failed test cases and allows you to directly access them.
Common Errors	Highlights the most common errors encountered during test executions.
Error Patterns	Categorizes errors and allows you to identify failure patterns based on them. This helps in easier troubleshooting and resolution.
Recommendations	Offers recommendations to prevent the errors encountered in the chosen test executions. Designed to guide your next steps in optimizing the stability of your test execution.

Generating a test insights report

Prerequisites: To generate a test insights report, your Test Manager role must have the following permissions: Test Execution - Read.

Open a project in Test Manager.
Go to Execution and select Generate Insights.
In the Results table, select the test results to be included in the report.
Optionally, you can filter the test results by:
1. Keywords - use the Search bar.
2. Execution Type - the type of execution.
3. Execution Finished - the execution completion time.
4. Status - the execution status.
Select Generate Insights.
When the generation process finishes, you receive an in-app notification and email indicating the report status: Ready or Failed.
If the report is ready, select the Execution Insights Report is ready notification, or Open report in the email notification, to access the report. If the report failed, you may regenerate it.
Select Show for each section of the report to view the test cases impacted by a certain error or recommendation.

Downloading a test insights report

Navigate to the Insights tab inside Execution.
1. Optionally, to rename the insights report, select More Options, and then Rename.
Select the Download button.
The test insights report is downloaded as a DOCX file.

User access management for AI-powered testing

The AI Trust Layer governance policy allows you to manage the use of AI-powered features within your organization. Although all members have default access to these features, you can use this policy to restrict access as needed. The AI Trust Layer governance policy empowers you to limit a user's access to certain AI-powered features or all of them, at a user, group, or tenant level. Additionally, it gives you the ability to decide which AI products users can access. You can create, modify, and implement this governance policy in AutomationOps.

If you want to deploy an AI Trust Layer governance policy and still use the AI-powered testing capabilities, ensure that, within the policy's Features Toggle, you select Yes for Enabling Test Manager features.

Check the following resources to learn how to create, configure, and deploy a governance policy for your organization.

On this page