September 2024

September 18, 2024

What's new

Orchestrator folder and package version selection for executing test sets

We're excited to announce an update that improves the management and execution of specific System Under Test (SUT) versions, especially for patch testing. You now have the ability to select test cases to be executed from a particular Orchestrator package version when running a test set.

Key updates include the following:

Default folder association: Each Test Manager project can now be linked to an Orchestrator folder as the default folder. This is the folder context for any automated execution. This helps when a certain folder is used most of the time for a particular Test Manager project. For more flexibility, you can override this default folder on each test set.
Package and version selection: On a test set, you can now select which packages and versions are used for execution. This allows testing of different versions of a system under test since it allows to select compatible automation versions for each run.
Autoselecting packages: The Autoselect option automatically selects all packages to be added to the test set, chooses the latest version of each package, and skips any test cases that aren't available in the selected package or the chosen version. Autoselection happens based on the test cases assigned to the test set.
Execution indication: Each test case from the test set now displays whether it can be executed automatically based on the currently selected folder/package/version combination.

Tenants created before this update have the Allow legacy execution context setting enabled, allowing you to execute tests with or without execution folder contexts. Tenants created after this update have this setting disabled, allowing you to execute sets, only with execution folder contexts configured.

Visit Tenant level toggles, Automation project configuration and Configuring test sets for specific execution folders for more information.

Concurrently executing manual test cases

Multiple users can now execute the same manual test case concurrently, to decrease the total runtime of a test execution. This is accessible from the Execution section.

This capability applies specifically to manual test cases. The final result of a manual test case is determined by the most recent user interaction, such as when you trigger an execution, you initiate a pending test execution, or when you re-execute a test case. The ExecutedBy column, for your test cases and executions, displays the latest updates on the test case results.

If you want to see previous execution results, initiated by all users, navigate to the Logs of a test case.

Visit Executing test cases simultaneously for more information.

Assigning manual executions to users

You can now assign manual test executions to users that are collaborating on the same testing project. For enhanced productivity, you can also set a Due Date for when the manual test execution is planned.

When you have been assigned or unassigned a test case, you can receive either an in-app or an email notification for it, by subscribing to the following notification events for Test Manager:

Test case assigned
Test case unassigned

Visit Assigning manual executions to users and Scheduling a due date for manual executions for more information.

Improvements

In all Test Manager projects, we have renamed the Test Results section to Execution for a more accurate representation of its functionality. Besides results, the Execution section serves as the starting point for planning and reporting, consolidating everything related to test execution. This change is also mirrored in the breadcrumb navigation within your project when accessing test objects.
We have improved the Excel report that you can download for a test execution with additional details. The enhanced Excel report now contains four sheets detailing the following aspects:
- Overview: Provides an overview of the test execution.
- Test case logs: Displays information about the executing user, the type of execution, and the source package.
- Assertions: Offers details on each test case along with their successful or unsuccessful assertions.
  - For manual test cases: Each test step is a verification and the tester's comment is the message.
  - For automated test cases: The name and message from the automation are displayed, along with a hyperlink to a screenshot, if available.
- Requirements: Indicates the number of test cases assigned to a requirement and the results of those test cases.