Test sets

Test Sets are logical groups of test cases. The purpose of test sets is to define groups of tests that should be executed together within one run. For instance, a smoke test is a group of tests that only check for top-critical capabilities. Whenever any test case from a smoke test fails, there is a critical problem. Executing a smoke test should not take too long. They are executed for instance whenever a developer commits changes to the source code. By comparison, a regression test is a comprehensive test which should provide a detailed overview over the system under test. Execution often takes hours or even days. To define the set of tests to be executed for those purposes, test sets are created. A test case can be assigned to several test sets.

Conditions

• Test sets in Test Manager can include both manual and automated test cases, including automated test cases from multiple Studio projects. For more information, see Automated executions conditions.
• Test sets linked from Orchestrator to Test Manager include only the test cases from Orchestrator.

Creating Test Sets

To create a test set:

1. In the Planning section, click Test Sets. The Test Sets page is displayed.
2. Click Create Test Set. The Create Test Set window is displayed.
   
   

   

   
   
3. Fill in the form as follows:
   • Name - Enter a name for the test set.

   • Description - (Optional) Enter a description for the test case.

     In order to add formatted text, you can use Markdown syntax.

   • Labels - Add any number of labels which makes it easy to find test set with joint labels.
4. Click Create. The test set is created and displayed in the Test Sets page.

Assigning test cases to a test set

You can simultaneously assign test cases to a test set in two different ways:

• Using Static Assignment - select the test cases manually.
• Using Dynamic Assignment - input the label of the test cases that you want to assign.

Statically assigning test cases to test set

1. Open the test set you want to assign test cases to by clicking it.
2. In the Tasks menu, select Assign Test Cases.
   The Assign Test Cases window is displayed.
3. Select the test cases you want to add. You can search for test cases by entering the name, the key, or a label.
   
   

   

   
   
4. Select Assign. The test cases are assigned to the test set and are displayed on the Test Cases tab.
   Tip:

   • Assign test cases to test sets
   • Upload documents
   • Execute tests

Dynamically assigning test cases to test set

You can choose to assign all test cases to a test set, based on their labels.

1. Select the test set that you want to assign test cases to.
2. In the Tasks menu, select Assign Test Cases.
   The Assign Test Cases window is displayed.
3. Input the labels of the test cases that you want to assign.
   For the example in the below illustration, all test cases with the label ASP:9 will be assigned to the current test set.
   
   

   

   
   
4. Select Assign. The test cases are assigned to the test set and are displayed on the Test Cases tab.

On the Test Cases tab you can see and edit test cases that are assigned both statically and dynamically in separate sub-tabs.

Navigation and search

To quickly find your test sets, use the search function and the filters. Navigate within the page using and configuring the paginator. Alternatively, you can use the breadcrumb to navigate between the pages.

1. Filter - You can use the filter to narrow your search. For example, you can search for the test sets that have been updated by your user, by specific labels, or the source (e.g., Orchestrator). The filters are automatically saved and kept active until you clear them.
2. Search - Use the search bar to find test results by their key, name or labels (requires full search term match)
   You can use the search bar at the top of the page to look for test results.

Linking Orchestrator test sets

You can link your test sets from Orchestrator and execute them in Test Manager.

Prerequisites: Before you link a test set from Orchestrator to Test Manager, ensure you first link the test cases from this Orchestrator test set to Test Manager in Studio. For detailed steps on how to link test cases from Studio to Test Manager, visit Linking test cases in Studio to Test Manager.

1. Open your project in Test Manager.
2. Navigate to Test Sets > Link.
3. Select a test set from the list and click Link. The Test Sets page now shows your linked test set.
   Important:

   • To link a test set to Test Manager, you need to create it in Orchestrator.
   • You can only link individual test sets that contain automated test cases. Note that each of these automated test cases must first be linked to Test Manager from Studio.
   • Test set linking is available for connected Orchestrator instances version 2020.10 or higher.
   • Test sets linked from Orchestrator are read-only. This means that you cannot edit the name, description, or version. Any changes need to be made in Orchestrator.
   • The displayed number of test cases might differ between Test Manager and Orchestrator. Test Manager only shows test cases that have been created on its platform.