Evaluating your workflows

Practices for evaluations that prove a workflow and its agents do what you expect, before you deploy and after every change.

An AI agent can answer differently each time it runs, even with the same input. Evaluations are how you prove that a workflow still does what it should: you run it against sample inputs with the results you expect, and evaluators score each one. These practices make evaluations worth their time. Evaluations describes how to set them up.

Start from real runs

The quickest way to build a dataset is from runs that already happened. In the Execution trace of a debug run, select Add to evaluations to turn the run's input and output into a data point. When the output was right, keep it as the expected result. When it was wrong, correct it first.

Test the whole workflow first

A dataset tests either the whole workflow or a single agent node:

• The whole workflow checks what a user gets, from the trigger to the end. Start here.
• A single agent node helps when you change that agent's instructions or model and want to see the effect in isolation.

A whole-workflow dataset starts from a trigger, and its entry point is the trigger's node ID, for example manualTrigger1 in Studio Web or start in Studio Desktop. Check the ID on the trigger's Advanced tab.

Pick the evaluator for what you check

What you want to check	Evaluator type
A fixed value, such as an order status or a category	Output validation, such as Exact Match or Contains Text
That the answer means the right thing, even when the wording differs	large language model (LLM) judge, such as LLM Judge Output
How the agent got there, such as which tools it called and in what order	LLM Judge Trajectory, or the Tool call evaluators

Fixed checks are cheaper and always give the same score, so use them where the expected result is exact. Use an LLM judge where several answers can be right.

Cover more than the expected case

• Add edge cases and failures. Include the inputs that should be rejected, the incomplete ones, and the unusual ones. That's where agents go wrong.
• Use real data, not placeholders. A check that passes on placeholder text proves nothing, because the agent never saw a real input.
• Test at the largest size you allow. Include the biggest input the workflow should accept, not only the typical one.

Run them at the right moments

• Before you deploy. Hold the release until the evaluations pass.
• After every change to an agent's instructions, its model, or its tools. A change that improves one case can break another.

Next steps

• Evaluations: datasets, evaluators, and evaluation runs.
• Output mocking: replace a node's output with sample data in a debug run.
• Debugging basics: read a run and find what went wrong.
• Community: ask questions and share workflows with other builders on the UiPath Forum.