Autonomous Test Execution for Test Cloud

Autonomous Test Execution in Test Manager for running manual test cases automatically with Autopilot, without building or maintaining scripted automation.

Autonomous Test Execution is a capability in Test Manager that runs pre-defined manual test cases automatically using Autopilot. A human tester has already written the test steps; Autopilot executes them without further intervention, replacing the effort of converting a manual test into a fully scripted Studio automation.

Important :

Autonomous Test Execution is available in Test Manager (Commercial delivery) only when delivered via Test Cloud.

Fonctionnalités clés

• Unattended execution only: runs test cases without attended-mode execution, including from a pending execution.
• Web-based scope: covers web-based test cases.
• Credential and parameter support: test steps reference dynamic values through the {{parameterName}} syntax, and Orchestrator credential assets through the {{credential:credentialName}} syntax.
• Model choice: model options include artificial intelligence (AI) models from Gemini and Anthropic, filtered to those allowed by the organization's Automation Ops policy.
• Detailed execution results: a full trace of actions, screenshots, and the agent's reasoning on failures, instead of a bare pass or fail indicator. Robot Logs are also available.
• Live streaming, remote control, and cancellation: a run in progress can be watched, taken over through remote control if needed, and canceled at any time.

When to use Autonomous Test Execution

• Regression testing: existing manual test cases for a stable feature are run repeatedly without human effort.
• Credential-dependent testing: credentials are stored securely in Orchestrator and injected at runtime, useful across environments where credentials must not be hard-coded or exposed.
• One-time or rarely executed tests: avoids the overhead of building persistent automation for temporary validations or infrequent checks.
• High-change or unstable interfaces: Autopilot adapts at runtime instead of requiring continuous script maintenance in applications where the interface changes frequently.
• Business-user or non-technical testing: business analysts or subject matter experts can execute tests from manual instructions, without learning automation tooling or maintaining scripts.

Gestion des informations d'identification

Instead of plain text, Autonomous Test Execution test steps can reference an Orchestrator Credential-type asset using the {{credential:credentialName}} syntax. At runtime, an existing Get Credential activity fetches the value.

Test Manager itself never sees or stores the username or password, only the asset name. The value is carried through the workflow as a secure string: the AI model sees only a secure placeholder, never the plaintext, until it is entered into the target field.

• Credentials are fetched only from the project's default Orchestrator folder, configured in Project Settings. No per-run folder override is available.
• Both the test case author and the test case executor need the AssetsView permission on that folder; otherwise the run fails at login.
• Credential selection is dropdown-only, so an invalid or mistyped asset name cannot be entered. Missing permissions still cause a failure.
• Credential resolution applies only during Autonomous Test Execution. A person running the same steps manually must look up the credential separately.
• Project and folder settings, including the default folder used for credential resolution, are not carried over when exporting or importing test cases between tenants or projects.

Live streaming, remote control, and cancellation

A running Autonomous Test Execution job can be watched live, and remote control of the session can be taken if needed. Both capabilities reuse the existing automated-execution live-streaming infrastructure.

Unlike Autonomous Exploration's persistent side panel, Autonomous Test Execution's live view is surfaced per test case run through a Video column on the Execution results table, which supports triggering several runs in parallel rather than one at a time.

A running unattended job can also be canceled at any time. Pausing a run is not supported.

Limites connues

• There is no dedicated execution-type label to distinguish Autonomous Test Execution runs from regular manual runs in Test Set Execution; the execution type shows generically as "Manual."
• Autonomous Test Execution always runs, and fetches credentials, from the project's default Orchestrator folder. No folder override is available yet, though one is planned before or at general availability.
• To change how Autopilot approaches a test, the manual steps must be edited directly; there is no Prompt Library support at this time.
• There is no test set-level execution and no way to distribute unattended runs across a pool of robots. Autonomous Test Execution runs per test case, including from a pending execution, not across a whole test set with mixed execution types.
• Dynamic Test Data Management and Data Fabric integration are not available.
• Full human-in-the-loop control is not implemented. If Autopilot cannot identify an interface element or hits a blocker, execution halts and shows the specific issue, which must be resolved manually. A pause mechanism for this scenario is planned, though no timeline has been confirmed.

For step-by-step instructions, see Running Autonomous Test Execution. For details on reviewing a run's results, see Reviewing Autonomous Test Execution results.