Best practices and limitations

Guidance and limitations for Autopilot for Testers across requirement evaluation, test generation, text-to-code conversion, test data, and result analysis.

Autopilot helps automate and enhance tasks across your testing lifecycle. The following best practices and limitations help you improve the accuracy and usability of Autopilot for Testers, organized by capability.

Observação:

Requirement evaluation, Manual test generation, and Analyzing test results apply to Test Manager — for the full delivery-model breakdown, see the Test Manager feature availability page. Converting text into code, Automating manual tests, and Generating test data apply to Studio, subject to their own version requirements below.

Avaliação de requisitos

Autopilot evaluates the quality of a requirement and suggests enhancements. The following guidelines improve the relevance and accuracy of these evaluations:

• Clear, complete requirements with measurable acceptance criteria produce more accurate evaluations.
• Focused evaluations, such as security or performance checks, return more relevant suggestions.
• Supporting files, such as guidelines or specifications, give Autopilot additional context.

Instruções adicionais

You can guide the evaluation by providing additional instructions in the Provide Additional Guidance screen.

The Prompt Library provides out-of-the-box prompts that analyze requirements for performance-related aspects, for security-related aspects, and from perspectives such as consistency, clarity, and completeness. You can also add your own custom prompts to the Prompt Library for software quality attributes you evaluate frequently, which standardizes evaluations across your team.

To control how many enhancement suggestions Autopilot returns, include the count in your instruction, for example: "Generate the top 20 enhancement suggestions for this requirement and rank them by priority."

Documentos de suporte

Supporting documents complement the requirement description with additional information. Autopilot considers both the requirement description and its supporting documents when it shapes and evaluates the requirement, and it also suggests enhancements to the documents themselves.

• Security guidelines — Security guidelines, policy documents, or security audit findings. Autopilot checks them for inconsistencies within the documents or against the security aspects in the requirement description.
• Accessibility guidelines — Accessibility guidelines, audit reports, or user accessibility specifications. Autopilot cross-checks them against the accessibility criteria in the requirement description.
• Compliance checklists — Compliance checklists, regulatory summaries, or industry compliance requirements. Autopilot checks them against the compliance criteria in the requirement description to identify gaps in regulatory adherence.

Limitações funcionais

Tipos de arquivos compatíveis

You can upload files in the following formats:

• Processes only text content – TXT, BPMN, CSV
• Processes both text and image content – DOCX, XLSX, PNG, JPG, PDF

Capacidade de token de entrada

The input token capacity of Autopilot is 128,000 tokens, which is approximately 96,000 words or 512,000 characters. Keep your requirement description and supporting documents within this limit. This is a product-level limit, separate from the context-window and output-token limits of the underlying model — for those, see Supported models and limits.

Dica:

As an approximation, 100 tokens correspond to about 75 words or 400 characters.

To estimate the token count of a document, save it as a TXT file and paste the content into the OpenAI Tokenizer tool. The result is approximate, and the actual count can be higher.

Geração de sugestões

Autopilot generates a maximum of 50 suggestions at a time. If you do not specify a number, Autopilot generates the top 15 suggestions.

Geração de testes manuais

Autopilot generates manual test cases and test steps from requirements and supporting documents. Test quality depends heavily on the quality of the requirement you provide.

Descrição do requisito

A well-formed requirement describes its purpose, the application logic, and clear acceptance criteria. Requirements often relate to quality aspects such as functionality, performance, usability, and security.

Requirement purpose

Autopilot relies on specificity. Broad or vague descriptions can lead to irrelevant or incorrect test cases. Start with a concise, user-focused statement of the requirement's purpose that emphasizes the benefit to the user.

Example: "As a potential policyholder, I want to calculate my insurance premiums so that I can understand my potential costs."

Application logic

The accuracy of the generated test steps depends on how well Autopilot understands the user journey and the application sequence. Describe the specific interactions the user has with the application, and the application's responses, from the start of the flow to the final test action.

Example: "The user starts on the main screen and navigates to the 'Get a quote' screen through the main menu. They enter their personal data, including age and gender, then select the desired insurance coverage and policy term. When they select 'Calculate Premium', the application calculates and displays the premium on the next screen."

Critérios de aceitação

Clear, measurable acceptance criteria guide Autopilot to verify specific outcomes. Cover both positive and negative scenarios, including invalid input and error states, and non-functional factors such as security, usability, and scalability. Without acceptance criteria, Autopilot might generate inadequate test cases.

Exemplos:

• "The system calculates the premium based on the user's age. For every year above 25, it adds $5 to the base premium of $100."
• "The system increases the premium by $50 for smokers because of the higher associated health risk."
• "If the user enters an age below 18, the system displays an error message."
• "The premium calculation completes within 3 seconds when the number of concurrent users is 1,000 or fewer."

Instruções adicionais

You can focus the generation by providing additional instructions in the Provide Additional Guidance screen.

The Prompt Library provides out-of-the-box prompts that generate end-to-end test cases from flow diagrams, generate tests for valid end-to-end scenarios, and generate tests that surface elusive issues. You can also add your own custom prompts for scenarios you generate frequently.

By default, Autopilot generates test cases with the following behavior:

• Expected results are generated only for the final test step in each test case. To generate expected results for every step, include an instruction such as "Generate expected results for each test step."
• Preconditions and postconditions are generated only on request; specify what to include, since neither is generated by default.
• The number of test cases is not fixed — to control it, include the count in your instruction, for example: "Generate the top 20 test cases for this requirement."

Documentos de suporte

Supporting documents complement the requirement description and help Autopilot generate more accurate test cases.

• Process diagrams — Use case diagrams, flowcharts, and process diagrams as images, or BPMN files. They help Autopilot follow the sequential and logical flow of user activities.
• Mockups and wireframes — Visual diagrams of your UI/UX requirements. They clarify layout, user interactions, and elements to test, which is useful when testing new front-end functionality.
• Compliance documents — Compliance and regulatory documents for regulated industries such as healthcare, finance, or telecommunications. Autopilot integrates these standards into every generated test case, not only those linked to specific compliance criteria.
• Discussion transcripts — Transcripts of requirement discussions among developers, product owners, and testers. They convey how the team interprets or plans to implement a requirement.

Limitações funcionais

Tipos de arquivos compatíveis

You can upload files in the following formats:

• Processes only text content – TXT, BPMN, CSV
• Processes both text and image content – DOCX, XLSX, PNG, JPG, PDF

Capacidade de token de entrada

The input token capacity of Autopilot is 128,000 tokens, which is approximately 96,000 words or 512,000 characters. Keep your requirement description and supporting documents within this limit. This is a product-level limit, separate from the context-window and output-token limits of the underlying model — for those, refer to Supported models and limits. To estimate token counts, use the tokenizer guidance described under Requirement evaluation.

Geração de caso de teste

Autopilot generates a maximum of 50 test cases at a time. If you do not specify a number, Autopilot generates 10 test cases.

Conversão de texto em código

Autopilot converts natural language prompts into code within Studio coded automations. Use the Generate Code action from the context menu of a coded automation to convert a prompt into code. Autopilot can generate C# code, refactor existing code, and generate a UiPath automation. For the available methods, refer to Generating code.

Observação:

Para converter texto em código, use a versão do Studio 2024.10.1 ou superior.

For better results:

• Specify the language and the goal, for example "Refactor this C# method."
• Mantenha os prompts curtos e diretos.

Exemplos de prompts

• "Refatore o método abaixo e torne-o mais eficaz."
• "Create a method to compare two strings for similarity, with the input parameters 'string1' and 'string2'."
• "Select on 'Products'."
• "Run job 'myDemoJob' on Orchestrator."

Automação de testes manuais

After you link Studio to Test Manager, the Test Explorer in Studio Desktop shows all test cases in your Test Manager project, both automated and manual. Autopilot converts manual test cases into automated ones. To learn how, see Creating a coded test case from a manual test case and Creating a low-code test case from a manual test case.

Observação:

Para automatizar testes manuais, use o Studio Desktop versão 2024.10.1 ou superior.

Prepare a consistent object repository

Autopilot uses UI Automation to convert manual test steps into automated steps. To reference the UI elements under test, Autopilot needs an object repository reference for each of them. Use identical control names in your manual test steps and in your object repository.

For example, for the manual step "Select on 'Submit loan application'", Autopilot generates a Select activity and looks for an object repository element named Submit loan application. Consistent control names make the conversion smoother.

Use common activity names

A typical manual test step, such as "Type '[e-mail]' into the 'Email' field", contains an action ("Type", recognized as the Type Into activity), data ('[e-mail]'), and a target ('Email', recognized as the object repository element).

The following table lists the manual-step verbs to use and the Studio activities and coded automation APIs they map to.

Etapa de teste manual	Atividade da UiPath	UiPath coded automation API
"'Selecionar' em 'Meu botão'"	Selecionar	Selecionar
"'Digite' 'hello world' em 'Meu campo de texto'"	Type Into	Type Into
"'Obter texto' de 'Meu rótulo'"	Get Text	GetText
"'Marcar' 'Minha caixa de seleção'"	Marcar	Marcar
"'Selecionar item' '3 anos' de 'Termo de empréstimo'"	Select Item	Select Item

Automate form filling

For a form that requires data input, use the 'Fill Form' keyword to have Autopilot automate it.

Observação:

Essa funcionalidade está disponível apenas ao gerar um caso de teste codificado.

For example, for the step "Fill the form on the screen named 'myFormScreen' with the following values: Email: '[e-mail]', Loan Amount: '10000', Loan Term: '3'", Autopilot runs the 'Fill Form' command in your coded automation and populates the values across the identified controls in the form.

Gerando dados de teste

Autopilot generates synthetic test data for your test cases. Because test data management can consume up to 50% of testing effort, this can save significant time.

Observação:

Para gerar dados de teste sintéticos, use o Studio Desktop versão 2024.10.1 ou superior.

Create arguments for your data fields

When generating test data, Autopilot considers the existing arguments in your workflow together with the instructions in your prompt, and aims for high coverage within the test case. Use arguments wherever feasible, instead of local variables or fixed string values, so that the generated data aligns with the dynamic requirements of your workflow.

Choose a data combination method

By default, Autopilot generates approximately 10 data records, aiming for high code or activity coverage. If your test strategy requires a specific algorithm, such as pairwise or all-combinations, include it in your prompt, for example: "Generate test data for every argument and combine the data fields pairwise."

Customize your dataset

You can instruct Autopilot to perform other tasks, such as generating a specific number of records or including additional fields. For example:

• "Gerar um mínimo de 25 registros de dados."
• "Incluir o campo 'País' no conjunto de dados, atribuindo um valor distinto de país a cada registro."

Análise de resultados de teste

Autopilot provides insights into failed test cases and recommendations for reducing the failure rate across your test portfolio. The more test results you include when generating the report, especially failed test cases, the more effective the insights. The goal is to understand the main reasons your tests are failing.

Each report section covers one area of your test results:

• Common Errors — Groups similar error messages semantically to highlight the most frequent issues.
• Error Patterns — Categorizes failed test cases into broader themes to reveal recurring and systemic problems.
• Recommendations — Provides actionable recommendations to guide your next steps in stabilizing test execution.