# Evaluate agent trajectory > Rerun the existing evaluation set on the tool-using quest classifier, then build a tool-aware set scored by semantic similarity and trajectory evaluators. With the agent calling its tool only when it should, you can evaluate it. ## Step 6 - Rerun your existing evaluation set Your agent already has an evaluation set: the five cases from the low-code lab. Before you build anything new, rerun it to check what the change did to behavior you had already accepted. 1. In the project explorer, open the agent's **Evaluation Sets** node, and find the set from the low-code lab. 2. Select **Run** for the evaluation set. 3. When the run finishes, select **View all results**, then select the most recent run. Expect the returned tiers to hold, and expect the semantic similarity scores to barely move. For example, the rats and dragon cases can drop from 100% to 98%. Their saved outputs come from the agent before it had a tool, so the expected reasoning does not cite a challenge rating and the new reasoning does. The meaning is close enough that semantic similarity hardly registers the change. The exception is `heroic-relic`, which can score lower: its new reasoning cites a guard's challenge rating and drops details the saved answer included, such as possible magical defenses. The agent now reaches its answer a different way: it calls a tool, then sets the tier from the challenge rating it gets back. A semantic similarity score compares answers, so it can't see that change. To check how the agent got to its answer, you need a trajectory evaluator, which the next step adds in a new set that describes the new behavior. :::important Evaluation scores are non-deterministic. The evaluator is itself a language model, so scores vary between runs. A two-point drop like this one is within that normal variation. ::: --- ## Step 7 - Build a tool-aware evaluation set The new set checks two things: what the agent answered, and how it got there. - **Semantic similarity** - compares the agent's output to the expected output by meaning, not exact wording. - **Trajectory** - scores how the agent reached its answer from its tool calls: whether it called the tool, what it searched for, and in what order. It tells you whether the tool description and system prompt are doing their job. ### Capture the remaining runs You already have three runs from Step 5. In the **Debug** panel, run the agent on two more quests: ```text Escort a merchant caravan through the hill country to the coast. ``` ```text Recover a stolen relic from a noble's estate. The guards are many, the owner is powerful, and nobody who has tried has come back. ``` Neither quest names a monster to look up. The relic quest is the harder test: the guards are people the party may have to fight, so one lookup for guards is reasonable, but the relic and its owner aren't threats to search for. An agent that over-uses its tool searches for those too. ### Create a new evaluation set and add the debug runs 1. In **Evaluation Sets**, create a new evaluation set and name it `Quest Classifier with Tools`. 2. Return to the agent's **Definition** to access the agent's trace history. 3. Open the **History** tab in the bottom panel, and select the trace for the run that you want to add. 4. Select **Add to evaluation set** and choose the new set from the drop-down. 5. In the **Add Trace** dialog, review the inputs and outputs and select **Save** to save the trace as an evaluation. 6. Repeat this process for the other four runs from Steps 5 and 7 - adding each of them to the same set. :::important Save only runs you agree with. The evaluation set treats each saved output as the correct answer. Check that every run reached the expected tier and called the tool only when it should have. ::: ![History tab with the relic quest trace selected and the Add to evaluation set menu open, with Quest Classifier with Tools chosen as the target set](https://dev-assets.cms.uipath.com/assets/images/getting-started/agents-tools/agents-tools-step-07a-3ee24274.gif) ### Describe the expected behavior The trajectory evaluator scores each run against a written description of what the agent should do. Give each case a name and an expected behavior from this table: | Name | Expected behavior | | -------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | | `legendary-ancient-dragon` | Calls Monster Query before answering, with a search term naming the ancient red dragon rather than a general term such as "dragon". | | `trivial-rats` | Calls Monster Query before answering, with a search term for rats, such as "rat", "rats", or "swarm of rats". | | `trivial-letter` | The tool-call history should be empty. No creature is named, so the agent should not call Monster Query, and an empty history is the correct result. | | `standard-caravan` | The tool-call history should be empty. No creature is named, so the agent should not call Monster Query, and an empty history is the correct result. | | `no-creature-relic` | Calls Monster Query only to look up the guards, or does not look up anything at all. Does not search for the relic or its owner. | :::note Describe tool use, not the answer. The trajectory evaluator reads only the agent's tool-call history and never sees the final answer, so describe whether the agent calls a tool and what it searches for, and allow reasonable variations in search terms. When the agent shouldn't call a tool, say that an empty history is the correct result, or the evaluator reads it as missing evidence. ::: 1. Open **Evaluation Sets** and select **View details** to open the new evaluation set you created. 2. For each case, select **Edit** to open its settings, and then: 1. Rename the case to its name in the table. 2. Select the **Trajectory** checkbox, if it is not already selected. 3. Under **Agent tools**, leave **Simulate** off for Monster Query, so the tool makes real calls. The trajectory evaluator is less useful when tool calls are mocked. 4. Paste the expected behavior into the **Behavior and output notes** field. 5. Select **Save**. ![Edit evaluation panel for the ancient dragon case, with the Monster Query Simulate toggle off under Agent tools and the Trajectory checkbox selected](https://dev-assets.cms.uipath.com/assets/images/getting-started/agents-tools/agents-tools-step-07b-53677b9f.gif) ### Attach both evaluators and run the set 1. In the new set, select the pencil icon next to **Applied Evaluators**. 2. Check that the list includes both a semantic similarity evaluator and a trajectory evaluator. If the project has no trajectory evaluator yet, create one from the same dialog. 3. Select **Evaluate set** to run all of the evaluations in the list. 4. When the run finishes, select the most recent run. Each case shows a semantic similarity score and a trajectory score, from 0 to 100. 5. Select any case to see the run, its scores, and each evaluator's explanation. ![Results for the Quest Classifier with Tools set, with Default Trajectory Evaluator averaging 99.4% and Default Evaluator 98.6% across the five cases](https://dev-assets.cms.uipath.com/assets/images/getting-started/agents-tools/agents-tools-step-07c-62f94a9c.png) :::important A missing evaluator fails silently. If you forget to attach the trajectory evaluator, the set still runs and shows only semantic similarity scores. If you see one score per case, revisit **Applied Evaluators**. ::: Expect high scores from both evaluators with these descriptions. In the run in the preceding screenshot, semantic similarity averaged 98.6% and trajectory averaged 99.4%. Run the set more than once before you trust a number: the trajectory evaluator is also a language model, and in testing, the same case with the same description scored 45% on one run and 75% on the next. To see how the evaluator scored a case, open it and read its **justification**. :::note You may see a justification mention a redundant first call that returned empty. The evaluator's view of the run lists each tool call twice, even though the agent called the tool once (check the **Execution Trail** to confirm). Until that's fixed, avoid expected behaviors that limit the number of tool calls. ::: A low trajectory score with a high semantic similarity score means the agent reached a reasonable answer, but not the way you asked. Typical causes are a search term that is too general (the `dragon` problem from Step 3), or a tool call on a quest that names no creature. When this situation occurs: 1. Fix the tool description, the system prompt, or the expected behavior. 2. Rerun the set. 3. Compare the results with the previous run. --- ## Outcome Your quest classifier calls Monster Query only when a quest names a creature, cites the challenge rating the tool returns, and has a tool-aware evaluation set that scores both its answers and its tool use. ## What's next - [Building a LangGraph agent](agents-langgraph.md) - build this same quest classifier, with the same System Reference Document (SRD) lookup tool, as a coded agent in Python. - [Agent evaluations](https://docs.uipath.com/agents/automation-cloud/latest/user-guide/agent-evaluations) - reference for evaluation sets, evaluator types, and simulated tool calls. - [Open5e API docs](https://open5e.com/api-docs) - explore the full monster search API and the other fields it can return. - [UiPath Community](https://community.uipath.com) - forums, how-tos, and developer discussion.