Best practices and limitations

Guidance and limitations for Autopilot for Testers across requirement evaluation, test generation, text-to-code conversion, test data, and result analysis.

Autopilot helps automate and enhance tasks across your testing lifecycle. The following best practices and limitations help you improve the accuracy and usability of Autopilot for Testers, organized by capability.

注:

Requirement evaluation, Manual test generation, and Analyzing test results apply to Test Manager — for the full delivery-model breakdown, see the Test Manager feature availability page. Converting text into code, Automating manual tests, and Generating test data apply to Studio, subject to their own version requirements below.

要件の評価

Autopilot evaluates the quality of a requirement and suggests enhancements. The following guidelines improve the relevance and accuracy of these evaluations:

• Clear, complete requirements with measurable acceptance criteria produce more accurate evaluations.
• Focused evaluations, such as security or performance checks, return more relevant suggestions.
• Supporting files, such as guidelines or specifications, give Autopilot additional context.

追加の指示

You can guide the evaluation by providing additional instructions in the Provide Additional Guidance screen.

The Prompt Library provides out-of-the-box prompts that analyze requirements for performance-related aspects, for security-related aspects, and from perspectives such as consistency, clarity, and completeness. You can also add your own custom prompts to the Prompt Library for software quality attributes you evaluate frequently, which standardizes evaluations across your team.

To control how many enhancement suggestions Autopilot returns, include the count in your instruction, for example: "Generate the top 20 enhancement suggestions for this requirement and rank them by priority."

サポート ドキュメント

Supporting documents complement the requirement description with additional information. Autopilot considers both the requirement description and its supporting documents when it shapes and evaluates the requirement, and it also suggests enhancements to the documents themselves.

• Security guidelines — Security guidelines, policy documents, or security audit findings. Autopilot checks them for inconsistencies within the documents or against the security aspects in the requirement description.
• Accessibility guidelines — Accessibility guidelines, audit reports, or user accessibility specifications. Autopilot cross-checks them against the accessibility criteria in the requirement description.
• Compliance checklists — Compliance checklists, regulatory summaries, or industry compliance requirements. Autopilot checks them against the compliance criteria in the requirement description to identify gaps in regulatory adherence.

機能制限

サポートされているファイルの種類

You can upload files in the following formats:

• Processes only text content – TXT, BPMN, CSV
• Processes both text and image content – DOCX, XLSX, PNG, JPG, PDF

入力トークンの容量

The input token capacity of Autopilot is 128,000 tokens, which is approximately 96,000 words or 512,000 characters. Keep your requirement description and supporting documents within this limit. This is a product-level limit, separate from the context-window and output-token limits of the underlying model — for those, see Supported models and limits.

ヒント:

As an approximation, 100 tokens correspond to about 75 words or 400 characters.

To estimate the token count of a document, save it as a TXT file and paste the content into the OpenAI Tokenizer tool. The result is approximate, and the actual count can be higher.

提案の生成

Autopilot generates a maximum of 50 suggestions at a time. If you do not specify a number, Autopilot generates the top 15 suggestions.

手動テストの生成

Autopilot generates manual test cases and test steps from requirements and supporting documents. Test quality depends heavily on the quality of the requirement you provide.

要件の説明

A well-formed requirement describes its purpose, the application logic, and clear acceptance criteria. Requirements often relate to quality aspects such as functionality, performance, usability, and security.

Requirement purpose

Autopilot relies on specificity. Broad or vague descriptions can lead to irrelevant or incorrect test cases. Start with a concise, user-focused statement of the requirement's purpose that emphasizes the benefit to the user.

Example: "As a potential policyholder, I want to calculate my insurance premiums so that I can understand my potential costs."

Application logic

The accuracy of the generated test steps depends on how well Autopilot understands the user journey and the application sequence. Describe the specific interactions the user has with the application, and the application's responses, from the start of the flow to the final test action.

Example: "The user starts on the main screen and navigates to the 'Get a quote' screen through the main menu. They enter their personal data, including age and gender, then select the desired insurance coverage and policy term. When they select 'Calculate Premium', the application calculates and displays the premium on the next screen."

承認基準

Clear, measurable acceptance criteria guide Autopilot to verify specific outcomes. Cover both positive and negative scenarios, including invalid input and error states, and non-functional factors such as security, usability, and scalability. Without acceptance criteria, Autopilot might generate inadequate test cases.

例:

• "The system calculates the premium based on the user's age. For every year above 25, it adds $5 to the base premium of $100."
• "The system increases the premium by $50 for smokers because of the higher associated health risk."
• "If the user enters an age below 18, the system displays an error message."
• "The premium calculation completes within 3 seconds when the number of concurrent users is 1,000 or fewer."

追加の指示

You can focus the generation by providing additional instructions in the Provide Additional Guidance screen.

The Prompt Library provides out-of-the-box prompts that generate end-to-end test cases from flow diagrams, generate tests for valid end-to-end scenarios, and generate tests that surface elusive issues. You can also add your own custom prompts for scenarios you generate frequently.

By default, Autopilot generates test cases with the following behavior:

• Expected results are generated only for the final test step in each test case. To generate expected results for every step, include an instruction such as "Generate expected results for each test step."
• Preconditions and postconditions are generated only on request; specify what to include, since neither is generated by default.
• The number of test cases is not fixed — to control it, include the count in your instruction, for example: "Generate the top 20 test cases for this requirement."

サポート ドキュメント

Supporting documents complement the requirement description and help Autopilot generate more accurate test cases.

• Process diagrams — Use case diagrams, flowcharts, and process diagrams as images, or BPMN files. They help Autopilot follow the sequential and logical flow of user activities.
• Mockups and wireframes — Visual diagrams of your UI/UX requirements. They clarify layout, user interactions, and elements to test, which is useful when testing new front-end functionality.
• Compliance documents — Compliance and regulatory documents for regulated industries such as healthcare, finance, or telecommunications. Autopilot integrates these standards into every generated test case, not only those linked to specific compliance criteria.
• Discussion transcripts — Transcripts of requirement discussions among developers, product owners, and testers. They convey how the team interprets or plans to implement a requirement.

機能制限

サポートされているファイルの種類

You can upload files in the following formats:

• Processes only text content – TXT, BPMN, CSV
• Processes both text and image content – DOCX, XLSX, PNG, JPG, PDF

入力トークンの容量

The input token capacity of Autopilot is 128,000 tokens, which is approximately 96,000 words or 512,000 characters. Keep your requirement description and supporting documents within this limit. This is a product-level limit, separate from the context-window and output-token limits of the underlying model — for those, refer to Supported models and limits. To estimate token counts, use the tokenizer guidance described under Requirement evaluation.

テスト ケースの生成

Autopilot generates a maximum of 50 test cases at a time. If you do not specify a number, Autopilot generates 10 test cases.

テキストをコードに変換する

Autopilot converts natural language prompts into code within Studio coded automations. Use the Generate Code action from the context menu of a coded automation to convert a prompt into code. Autopilot can generate C# code, refactor existing code, and generate a UiPath automation. For the available methods, refer to Generating code.

注:

テキストをコードに変換するには、v2024.10.1 以降の Studio を使用します。

For better results:

• Specify the language and the goal, for example "Refactor this C# method."
• 短く直接的なプロンプトにする

プロンプトの例

• 以下のメソッドをリファクタリングして、パフォーマンスを上げてください。
• "Create a method to compare two strings for similarity, with the input parameters 'string1' and 'string2'."
• "Select on 'Products'."
• "Run job 'myDemoJob' on Orchestrator."

手動テストを自動化する

After you link Studio to Test Manager, the Test Explorer in Studio Desktop shows all test cases in your Test Manager project, both automated and manual. Autopilot converts manual test cases into automated ones. To learn how, see Creating a coded test case from a manual test case and Creating a low-code test case from a manual test case.

注:

手動テストを自動化するには、v2024.10.1 以降のデスクトップ版の Studio を使用します。

Prepare a consistent object repository

Autopilot uses UI Automation to convert manual test steps into automated steps. To reference the UI elements under test, Autopilot needs an object repository reference for each of them. Use identical control names in your manual test steps and in your object repository.

For example, for the manual step "Select on 'Submit loan application'", Autopilot generates a Select activity and looks for an object repository element named Submit loan application. Consistent control names make the conversion smoother.

Use common activity names

A typical manual test step, such as "Type '[e-mail]' into the 'Email' field", contains an action ("Type", recognized as the Type Into activity), data ('[e-mail]'), and a target ('Email', recognized as the object repository element).

The following table lists the manual-step verbs to use and the Studio activities and coded automation APIs they map to.

手動テスト ステップ	UiPath のアクティビティ	UiPath coded automation API
[マイ ボタン] を選択する	選択	選択
[マイ テキストフィールド] に「ハロー・ワールド」と入力する	文字を入力	文字を入力
[マイ ラベル] のテキストを取得する	テキストを取得	GetText
[マイ チェックボックス] のチェックをオンにする	チェック	チェック
[ローン期間] で「3 年」の項目を選択する	項目を選択	項目を選択

Automate form filling

For a form that requires data input, use the 'Fill Form' keyword to have Autopilot automate it.

注:

この機能は、コード化されたテスト ケースを生成する場合にのみ使用できます。

For example, for the step "Fill the form on the screen named 'myFormScreen' with the following values: Email: '[e-mail]', Loan Amount: '10000', Loan Term: '3'", Autopilot runs the 'Fill Form' command in your coded automation and populates the values across the identified controls in the form.

テスト データを生成する

Autopilot generates synthetic test data for your test cases. Because test data management can consume up to 50% of testing effort, this can save significant time.

注:

合成テスト データを生成するには、v2024.10.1 以降のデスクトップ版の Studio を使用します。

Create arguments for your data fields

When generating test data, Autopilot considers the existing arguments in your workflow together with the instructions in your prompt, and aims for high coverage within the test case. Use arguments wherever feasible, instead of local variables or fixed string values, so that the generated data aligns with the dynamic requirements of your workflow.

Choose a data combination method

By default, Autopilot generates approximately 10 data records, aiming for high code or activity coverage. If your test strategy requires a specific algorithm, such as pairwise or all-combinations, include it in your prompt, for example: "Generate test data for every argument and combine the data fields pairwise."

Customize your dataset

You can instruct Autopilot to perform other tasks, such as generating a specific number of records or including additional fields. For example:

• 最低 25 件のデータ レコードを生成してください。
• データセットに「Country」フィールドを含め、各レコードに個別の国の値を割り当ててください。

テスト結果を分析する

Autopilot provides insights into failed test cases and recommendations for reducing the failure rate across your test portfolio. The more test results you include when generating the report, especially failed test cases, the more effective the insights. The goal is to understand the main reasons your tests are failing.

Each report section covers one area of your test results:

• Common Errors — Groups similar error messages semantically to highlight the most frequent issues.
• Error Patterns — Categorizes failed test cases into broader themes to reveal recurring and systemic problems.
• Recommendations — Provides actionable recommendations to guide your next steps in stabilizing test execution.