test-suite

latest

false

Test Suite User Guide

DELIVERY:

Automation Cloud Automation Cloud Public Sector Automation Suite Standalone

Last updated Nov 15, 2024

Quickstart guide

This quickstart guide provides you with a comprehensive introduction to Test Manager, empowering you to efficiently manage and execute test cases for your Studio projects. Test Manager offers a robust test management solution with support for multiple test frameworks, integration with various source control systems, and advanced reporting capabilities.

Follow these steps to get started with Test Manager:

Create a project in Test Manager, or import an existing project.
Visit Import project to learn how to import testing projects from other tools.
Create the requirements for your test cases.
Create test cases either in Studio, or directly in Test Manager.
Create test sets and assign test cases to them.
Execute the tests.
Analyze the test results.

Creating requirements

It is very common to use specialized requirements management tools to manage requirements like user stories, epics, or other artifacts. Test Manager supports the synchronization of requirements from external tools as well as the creation of requirements from within Test Manager. In a Test Manager project, there can be requirements that are synced from external tools as well as requirements that were created in Test Manager.

In the Test Design menu, navigate to the Requirements page.
Click Create Requirement. The Create Requirement window is displayed.
Fill in the fields accordingly:
- Name - The name for your requirement.
- Description - (Optional) Add any description to the requirement. In order to add formatted text, use Markdown syntax.
- Optionally, add any number object labels.
Click Create. Your new requirement is displayed in the Requirements page.

Tip: Upload documents

Quality-check requirements

Autopilot^TM assists you in evaluating requirements for quality aspects such as clarity, completeness, and consistency in Test Manager.

For an existent requirement, select Evaluate quality and evaluate the requirement using Autopilot^TM, to generate a list of suggestions that you can directly implement. Provide supporting documents to Autopilot^TM and additional guidance through a prompt that you choose from your library or that you type in yourself.

After you trigger the evaluation, expand each suggestion to update its status based on your implementation progress. You can choose to add the suggestion to your requirements, marking its status as In Work or Done. The option to remove suggestions is also available.

Generate more suggestions or regenerate them with different supporting documents or additional guidance using Suggest More and Regenerate. You can also keep suggestions for future reference by exporting them to Word.

Visit Quality-check requirements -Best practices to understand how to efficiently evaluate your requirements using Autopilot^TM.

Cloning requirements

To expand your testing portfolio, you can clone one requirement at a time. To clone a requirement:

Select More Options for the requirement that you want to duplicate, then Clone.
Choose the information that you want to clone, and then select Confirm.

Creating test cases

Test Manager provides a set of features for creating, managing, and executing test cases for your projects. This guide walks you through the process of creating test cases, assigning automations to them, adding manual steps, and linking requirements.

Creating test cases

Build your testing portfolio, and create test cases within Test Manager:

Open your project in Test Manager and navigate to the Test Cases section.
Select Create Test Case and configure the test case details such as name, version, description, and labels.
Select Confirm, and the test case will be displayed under Test Cases.

Generating tests for requirements

Use Autopilot^TM to generate a list of potential test cases.

You can use two types of test case generation:

Generate tests from requirement: You can generate test cases from the requirement details, followed by uploaded documents, and additional instructions.
Open a requirement and select Generate tests to generate tests for a requirement.
Generate tests for SAP transactions: You can generate test cases for SAP transactions from Heatmap and gaps discovered in Change impact analysis, using uploaded documents and additional instructions.

Refine the generation process with documents and instructions for Autopilot^TM to use in generating the test cases. After you select Generate tests, review the generated test cases and create tests if satisfied, or refine them with more details if otherwise.

Visit Generate tests for requirements: Best practices to check how to use the test generation feature at its full potential.

Linking test cases to requirements

To establish traceability between test cases and requirements, you can assign test cases to requirements in Test Manager.

Open the desired test case.
Go to Tasks, and select Assign to Requirement.
Choose the requirements you want to link the test case to.
Select Assign to save the changes.

Assigning Studio automation to test cases

Test cases in Test Manager can include both manual steps and a Studio automation. The automations are created in Studio as test cases, using the available activities. You need to assign Studio automations to test cases from Test Manager, to use automated executions.

To link an automation created in Studio to a test case in Test Manager:

Integrate Studio with Test Manger, and configure the Test Manager Settings from the Studio Design ribbon.
Select Link to Test Manager.
Configure the test case information, such as:
- Project: Select your project. If you did not set a default project for Test Manager, see Integrating with Studio.
- Test: Select a test case from the dropdown list.
- Name: Enter a name only if you create a new test case in Test Manager.
- Requirement: Leave this field blank if you don't want to link the test case to an existing requirement. The name is read-only in Test Manager. Alternatively, you can select a Requirement from the dropdown list if you want to assign the test case to a specific requirement in Test Manager.

Adding manual steps to test cases

In addition to the automation, you can also include manual steps within test cases. These manual steps provide instructions for executing specific actions during the testing process. When you create or edit a test case, you can add manual steps, describe the action, expected behavior, and define clipboard values, so you don't have to manually type into fields.

Visit Manual test cases to learn how to add manual steps to test cases inside Test Manager.

Uploading documents

In Test Manager, you have the option to upload relevant documents as attachments to test cases.

To upload documents, choose from the following options:

Option	Description
Manual upload	Follow the steps below: Open your project in Test Manager. Navigate to one of the following artefacts: Requirements Test Case Test Set Click an artefact key in the list to open it. Go to the Documents tab. Click the Add document button and choose a file or drag & drop it directly in the page. The file is added to your test.
Attach Document activity	Add the Attach Document activity in your Studio automation. The attachments will appear inside the Attachments tab on the Execution page after execution.

Cloning test cases

To easily create multiple similar test sets, you can also clone individual test cases. To clone a test case:

Select More Options for the test case that you want to duplicate, and then Clone.
Choose if you want to also clone related information, such as documents, labels, or assignments to requirements.
Select Clone to save your changes.

Managing test cases

Test Manager provides a range of actions that can be performed on individual test cases:

View an overview of a test case when you select its key. This takes you to the Overview page.
Access the log of the latest result of the test case. Select Open execution log for a test case and you can see details about the latest test execution.
Delete test cases.
Unlink automation.
Clone test cases.

Related articles

Creating test sets

Test Sets in Test Manager are used to group test cases and define which tests should be executed together in a single run. They provide a way to organize tests based on their purpose and execution requirements. To create a test set, follow the steps below:

Creating test sets

To create a test set:

Open your project in Test Manager, and select Test Sets.
Select Create Test Set.
Fill in the information about the test set, such as name, description, if it should display the activity coverage(Enable RPA activity coverage calculation), and labels.
Select Create to save your changes.

Assigning test cases to test sets

After you create a test set, you need to assign test cases to it. To assign test cases to a test set, open the desired test set, and in the Tasks menu select Assign Test Cases.

You can simultaneously assign test cases to a test set in two different ways:

Using Static assignment - manually select the test cases that you want to link.
Using Dynamic assignment - input the label of the test cases that you want to assign. Whenever an execution is started, all test cases which have at least one of the selected labels assigned will be added to the execution.

The animated image below shows how to create a test set and assign both static and dynamic test cases to it.

Configuring test sets to run from a specified execution folder

Test sets can be set to run from either a default or a specified execution folder. This allows you to exclusively execute test cases that are part of that particular folder. Moreover, you also have the option to specify particular packages and their versions from which the test cases will be exclusively run. To use this capability, follow these steps:

Check the tenant level settings:
1. Visit Test Manager, select Settings from the header, and then go to the General tab.
2. Check the Allow legacy execution context setting:
  - If the setting is enabled: Test cases run without a specific execution folder.
  - If the setting is disabled: Test cases can run only from a specific Orchestrator execution folder, and from a specified package and version.
Set a default execution folder in the project settings:
1. Go to Project Settings and select the Automation tab.
2. Set your default execution folder.
  This default folder applies to all test sets unless a unique execution folder is specified.
Select an execution folder for a test set:
1. Navigate to Test Sets and open a test set.
2. In the Test casestab, select Select Automation.
3. Choose a preferred folder, from the Execution folder dropdown list.
4. Select Save to apply the change.
Select a particular package and version:
1. In the Test Cases tab of a test set, select Select Automation again.
2. You can use the following options:
  - Autoselect packages: Selects all available packages in the folder with their latest versions, based on the test cases assigned to the test set.
  - Add automation: Allows you to manually choose a package and its version from the corresponding dropdown lists. Use Select to confirm each package-version pairing.

Linking Orchestrator test sets

You can link your test sets from Orchestrator and execute them in Test Manager. To achieve this, you first need to create the test set in Orchestrator, and assign only automated test cases to it.

Note: This feature will be deprecated starting with April 2025. Visit Orchestrator's upcoming deprecations to learn more information about the deprecation.

After you create the test set in Orchestrator, follow the steps below:

Open your project in Test Manager.
Navigate to Test Sets and then Link.
Select a test set from the list and select Link.

Cloning test sets

In order to speed up your testing management, after you cloned requirements and test cases, you can also clone test sets, so you get to rapidly expand your testing projects. To clone a test set:

Select More Options for the test set that you want to duplicate, and select Clone.
Choose if you want to also clone related information, such as Documents and Labels..
Select Clone.

Test sets

Executing tests

In Test Manager, you have the option to execute tests manually or using automations. Manual test executions are performed directly in Test Manager, while automated executions are initiated through Test Manager and carried out via Orchestrator. Before executing test cases, it is necessary to create a test set, which will be used for the execution.

Automated execution

Automated execution refers to test sets created in Test Manager with linked test cases from Studio. Before executing automated tests in Test Manager, make sure you have created the test sets in Orchestrator.

You can execute all test sets using the Execute Automated option. Orchestrator is invoked for test cases linked to Test Manager. To execute test sets linked from Orchestrator, ensure that you create the test sets in Orchestrator.

Executing tests using automations

To execute test sets using automations, perform the steps below:

Open your project in Test Manager and then navigate to Test Sets.
Select a test set and click the vertical ellipsis.
Select Execute Automated.

Manual execution

Manual execution applies to test sets created in Test Manager, including test cases with manual steps. Manual execution can be performed for both Test Manager test sets and linked test sets from Orchestrator.

Manually executing tests

To execute test sets manually, perform the steps below:

Open your project in Test Manager and then navigate to Test Sets.
Select a test set and click the vertical ellipsis.
Select Execute Manually. The Manual execution assistant opens in a new window. Make sure your popup blocker allows the window to be opened.

Executing tests

Analyzing results

Execution in Test Manager provide insights about the execution of test sets. Below you have a breakdown of how you can analyze and work with test results.

Accessing test results

Open Execution in Test Manager to view a list of test executions. Each test execution represents the execution of a test set. The executions are sorted based on the finish date, with ongoing and pending executions displayed at the top.

Execution details

When you select a test execution entry, a detailed view opens. This view displays information such as execution start time, duration, and execution logs. The Results section provides an overview of all individual test results from test cases, named Test Case Logs. This allows you to take actions on individual test results.

Understanding test execution status

Test executions are color-coded to indicate their status:

Green - Test cases that passed.
Red - Test cases that failed.
Grey - Test cases without definitive results, such as those not yet executed within an ongoing test set.

Exploring test results

This section describes the information that you can find as part of the test results, and the further actions that you can take after obtaining the results.

Result	Description
Execution Log	The execution log contains essential information, including assertions, logs, execution details, affected requirements, and attachments. Select the test case key within the Execution page to access the execution log.
Activity Coverage	You can enable activity coverage for automated test executions. This feature provides information on the activities covered during the test execution.
Defect Management Integration	If you have an external defect management system integrated with Test Manager, you can create defect reports directly from the execution logs. Open a test case log, select Tasks, and then Create Defect.
Defect Synchronization	Test Manager allows synchronization of execution results with external tools as part of an ALM tool integration. This synchronization ensures that execution details, logs, timestamps, and other relevant information are shared with the integrated tool.
Create and Unlink Defects	Within the Execution page, you can create defects by accessing test case logs, selecting Tasks, and selecting Create Defect. Additionally, you have the option to unlink defects from an external tool, which detaches the test execution result from the external tool's entry.

Generating test insights report

Get actionable insights into your test results by generating a report with Autopilot^TM, detailing why your test cases are repeatedly failing. The report can have up to five sections, each consisting of multiple insights. For each insight in the report, select the icon displaying the number of test cases, or Show, to view the impacted related objects, such as test case, or test case logs, and search through them.

To generate a test insights report for a test execution, select Generate Insights for the desired execution, select the results you want included in the report, and finally, select Generate insights again.

If the report is ready, select the Execution Insights Report is ready notification, or Open report in the email notification, to access the report.

Working with test results

By analyzing test executions, you can perform the following actions:

Identify manual or automated test results.
Track the progress of ongoing test executions.
Open the executed test set to review the details.
Examine logs and attachments associated with test executions.
Create defects in your defect management system directly from Test Manager if you have an ALM tool integration.

Note: Execution and their associated data remain unaffected by changes or deletion of test sets.