# Evaluating model performance

> Assess IXP model performance in the Extract node of UiPath Maestro™ Flow using the evaluation summary and the field and document metrics of each evaluation run.

## Overview

You can assess the performance of the model of a Flow **Extract** node in the following locations:

* **Definition** tab - Shows a summary of the latest evaluation, including the F1 score.
* **Eval runs** tab - Lists all evaluation runs, including the document error rate of each run, and shows the field group, field, and document level performance of each run.

## Node health score

The node health score summarizes how well the current model version of a Flow **Extract** node performs. It appears in the following places:

* The properties pane of the Flow **Extract** node.
* The **Definition** tab. To open it, hover over the Flow **Extract** node, and select **Open full view**.

![The Definition section of the properties pane of the Extract document node, with the health score highlighted.](https://dev-assets.cms.uipath.com/assets/images/ixp/ixp-flow-node-health-score-properties-a591b181.webp)

![The Definition tab of the Extract document node, with the health score highlighted.](https://dev-assets.cms.uipath.com/assets/images/ixp/ixp-flow-node-health-score-definition-52c8bfe3.webp)

The health score shows the following:

* **F1 score** - The average of the F1 scores of all fields in the taxonomy.
* **Rating** - **Good**, **Fair**, or **Poor**.
* **Annotated docs** - The number of annotated documents.
* The model version the score comes from, and the extraction model used by that version.

The F1 score is color-coded as follows:

| **Color** | **F1 score** |
|---|---|
| **Green** | 80% or higher |
| **Yellow** | 40% to 79% |
| **Red** | Below 40% |

The health score is generated from evaluation runs that include all annotated documents in the dataset of the node. Evaluation runs on a subset of documents do not affect the health score.

To view the evaluation runs behind the score, select the model name, for example, Gemini 2.5 Flash. The runs open in the **Eval runs** tab.

:::note
Healthy models have a **Good** score and no field performance warnings.
:::

## Evaluation in the Eval runs tab

Each evaluation run outputs the performance of a model version tested on a selection of the annotated documents in the dataset of the node. A new model version is created when you run an evaluation that includes changes not yet evaluated, such as changes to your taxonomy, including instructions, or to the model configuration. Making these changes does not create a model version by itself. For details, check [Changes not evaluated](running-an-evaluation.md#changes-not-evaluated).

### Viewing evaluation runs

To view the evaluation runs of a node, proceed as follows:

1. Hover over the Flow **Extract** node, and select **Open full view**.
2. Navigate to the **Eval runs** tab.

   ![The Eval runs tab of the Extract document node, with the list of evaluation runs highlighted.](https://dev-assets.cms.uipath.com/assets/images/ixp/ixp-flow-eval-runs-list-a768d14c.webp)

The list shows the following details for each evaluation run:

* **Name** - The name of the evaluation run.
* **Version** - The model version evaluated in the run. Runs that match the current node definition show the **Matches current def** label.
* **Status** - The status of the run: **Completed**, **Cancelled**, **Running**, or **Failed**.
* **F1 score** - The average F1 score across all fields.
* **Doc error rate** - The average of the error rates of the annotated documents.
* **Documents** - The number of documents evaluated.
* **Started** - When the run started.
* **Duration** - How long the run took.

To filter the list by status, select the filter icon, and select one or more of the following options: **Running**, **Completed**, **Failed**, or **Cancelled**.

![The Eval runs tab, with the filter icon and the Filter list of run statuses highlighted.](https://dev-assets.cms.uipath.com/assets/images/ixp/ixp-flow-eval-runs-filter-499cf5ea.webp)

To show or hide columns, select the columns icon, and then select the columns to display.

![The Eval runs tab, with the columns icon and the Columns list highlighted.](https://dev-assets.cms.uipath.com/assets/images/ixp/ixp-flow-eval-runs-columns-2bdcb04a.webp)

To start a new evaluation, select **Run evaluation**. For details, check [Running an evaluation](running-an-evaluation.md).

### Doc error rate

You can view the document error rate of each evaluation run in the **Doc error rate** column of the **Eval runs** tab.

![The Eval runs tab, with the Doc error rate column highlighted.](https://dev-assets.cms.uipath.com/assets/images/ixp/ixp-flow-eval-runs-doc-error-rate-1db3c923.webp)

The document error rate is the ratio of errors the model made on a document to the total annotations on that document, that is, how far the predictions of the model differ from user annotations. Error rates are only available for annotated documents.

The value in the **Doc error rate** column of the **Eval runs** tab is computed as the simple average of the error rate of each fully annotated document. It provides a quick indicator of how error-prone documents are when the evaluated model version processes them, which helps evaluate readiness to deploy.

:::note
The document error rate can exceed 100%, because it is calculated as follows:
* Document error rate = Number of errors ÷ Total annotations

Errors include not only fields that were annotated and extracted incorrectly, but also any extra predictions the model made that have no corresponding annotation. For example, an additional row the model predicted that a reviewer did not label.

Because these extra predictions count as errors but do not add to the annotation total, the number of errors can exceed the number of annotations, resulting in an error rate above 100%.
:::

### Managing evaluation runs

To manage an evaluation run, proceed as follows:

1. In the Flow **Extract** node, navigate to the **Eval runs** tab.
2. Select the ellipsis in the **Actions** column of the run, and select one of the following options:
   * **View details** - Opens the details of the run. Alternatively, select the row of the run.
   * **Re-run** - Runs the evaluation again.
   * **Export as JSON** - Exports the run as a JSON file.
   * **Export as CSV** - Exports the run as a CSV file.
   * **Export Eval Run Results** - Exports the results of the run.
   * **Delete** - Deletes the run.

   ![The Eval runs tab, with the ellipsis of a run and its list of actions highlighted.](https://dev-assets.cms.uipath.com/assets/images/ixp/ixp-flow-eval-run-actions-253df975.webp)

You can export data from evaluation runs for offline analysis, custom filtering, or sharing results with stakeholders.

You can also revert the **Extract** node to a previous model version from the **Eval runs** tab.

### Evaluation run details

The details of an evaluation run help you evaluate how well a model version performs on annotated documents before you deploy it.

To open the details of an evaluation run, proceed as follows:

1. Hover over the Flow **Extract** node, and select **Open full view**.
2. In the **Eval runs** tab, select the row of an evaluation run. The details of the run open, including its **Document performance** table. The page shows the following details of the run:

   * Latency, which is the extraction latency. Hover over the value to view the latency percentiles across documents.
   * The run score, which is the average of the F1 scores of all fields.

   ![The details of an evaluation run, with the By document and By field views and the latency and run score highlighted.](https://dev-assets.cms.uipath.com/assets/images/ixp/ixp-flow-eval-run-details-13c1209c.webp)

3. Select one of the following views:
   * **By document** - Shows the **Document performance** table, with one row per document. By default, the table shows the following columns: **F1 score**, **Doc error rate**, **Total errors**, **Total annotations**, and **Annotated values**.
   * **By field** - Shows the **Field performance** table.

### Field performance table

The field performance table, available in the **Eval runs** tab, in the **By field** view of an evaluation run, is the primary way to analyze model performance. It displays one row per field or field group, along with performance and error metrics calculated across the annotated documents. The table does not take into account unannotated and partially annotated documents when calculating metrics.

The table helps answer questions such as:
* Which fields limit the overall model performance?
* Are errors concentrated in a few fields or spread broadly?
* Did a recent model change improve or degrade specific fields?

The table includes several categories of metrics that help you analyze model performance from different perspectives. Each category answers a specific diagnostic question about how your model behaves across fields and documents.

By default, the table shows the following columns:

* **F1 score**
* **Error rate**
* **Total errors**
* **Total annotations**
* **Annotated values**
* **Docs with errors**
* **Docs annotated**

The F1 score of each field and field group is color-coded using the same ranges as the [node health score](#node-health-score):

* **Green** - The field performs well.
* **Yellow** - The performance of the field is less than satisfactory.
* **Red** - The field is performing poorly.

To show or hide metrics, select **Columns**, and then select the metrics from each category.

![The Field performance table of an evaluation run, with the By field view and the Columns option highlighted.](https://dev-assets.cms.uipath.com/assets/images/ixp/ixp-flow-eval-run-by-field-9f1f5cf8.webp)

To find a particular field group or field, use the **Search fields** box.

### Filtering and sorting

To filter rows in the table, select one or more of the available quick filters in the **Eval runs** tab. Each quick filter shows the number of rows it matches:

* **Annotated values \< 10**
* **Field F1 score \< 50**
* **Field F1 score within 50–70**

You can also sort the table by any metric in the table. When a sort is applied, values are sorted within their respective field group. For example, sorting the table by F1 score sorts the fields within each field group relative to one another.

## Performance metrics

The purpose of the performance metrics is to evaluate the overall quality of extraction for each field or field group.

The performance metrics are described as follows:

* **F1 score** - The harmonic mean of precision and recall: F1 = 2 × (Precision × Recall) / (Precision + Recall). F1 score only remains high when both precision and recall are high. In practice, this makes F1 a strong overall quality indicator for extraction tasks where you care about avoiding incorrect values and avoiding missed values. Therefore, F1 is a useful first metric to review to analyze field performance changes across model versions.
* **Precision** - Measures how often predicted values are correct: Precision = True positives / (True positives + False positives). True positives are predictions that match the annotated value, excluding values annotated as missing.
* **Recall** - Measures how often the model finds a value when it exists: Recall = True positives / (True positives + False negatives). False negatives are annotated values that the model did not predict, excluding values annotated as missing.
* **Error rate** - Total errors / Total annotations. Values marked as missing are included in the count of errors and annotations.
* **Error rate excluding missing** - (Total errors – Extra predictions) / Annotated values. Annotated values marked as missing are excluded.

## Predictions and errors

The purpose of the predictions and errors metrics is to understand the volume and composition of errors that contribute to poor performance.

The metrics are described as follows:

* **Total errors** - Total number of errors for a field across all error classes: Total errors = Incorrect values + Missed predictions + Extra predictions.
* **Total predictions** - Total number of predicted values for a field: Total predictions = Correct values + Correct missing + Incorrect values.
* **Incorrect values** - Number of predictions where the extracted value does not match the annotation. Excludes predictions and annotated values marked as missing.
* **Missed predictions** - Number of annotated values that the model failed to extract.
* **Extra predictions** - Number of predicted values that the model should not have extracted, or did not have a corresponding annotation or annotation marked as missing.
* **Correct values** - Number of predicted values that exactly match the annotation.
* **Correct missing** - Number of instances where the model correctly predicted that a value is missing.

## Annotations

The purpose of the annotations metrics is to provide context for how much labeled data supports each metric and how reliable performance scores are.

The metrics are described as follows:

* **Total annotations** - Total number of annotations, including values marked as missing: Total annotations = Annotated values + Annotated values marked as missing.
* **Annotated values** - Total number of annotated field values, excluding those marked as missing.
* **Annotated as missing** - Total number of times a field was explicitly labeled as missing.

## Document metrics

The purpose of document metrics is to understand how errors are distributed across documents rather than just across predictions.

The metrics are described as follows:

* **Docs with errors** - Total number of documents where the field has at least one error.
* **Docs annotated** - Total number of documents in which the field has at least one annotated field value.
* **% Docs with errors** - Percentage of annotated documents that contain at least one error for the field: Docs with errors / Docs annotated.

## Comparing model versions

When multiple evaluation runs are available, you can compare the performance of a run against a previous run. This way, you can better understand the impact of changes to field instructions, changes in annotations, or model configuration updates.

By default, the table compares the selected run against the previous run. Score differences appear next to each value, with an arrow that indicates whether the value increased or decreased.

To compare against a different run, proceed as follows:

1. In the **By field** view of an evaluation run, select the **Compare with** dropdown.

   ![The By field view of an evaluation run, with the Compare with dropdown and the score differences next to each value highlighted.](https://dev-assets.cms.uipath.com/assets/images/ixp/ixp-flow-eval-run-compare-with-deltas-b499e3ae.webp)

2. Select one of the following options:
   * A previous run, listed by run number, dataset, date, and model version.
   * **No comparison** - Hides score differences.

   ![The Compare with dropdown of an evaluation run, open and showing the No comparison option and the previous runs, highlighted.](https://dev-assets.cms.uipath.com/assets/images/ixp/ixp-flow-eval-run-compare-with-options-e0048658.webp)

## Example scenarios

### Scenario 1: Low F1 + Low Precision, but Recall is moderate or high

#### What you observe

F1 is low, Precision is low, and Recall is moderate or high.

#### What it usually means

* The model is extracting values for a field, but there are more values predicted than you expect to be found.
* Common root causes:
  * Field instruction is too broad or ambiguous. For example, the field instruction is **capture the amount**, but it does not specify which amount.
  * The document has similar values that can be confused for one another, for example, subtotal versus total, ship-to versus bill-to.

#### What to do next

Compare the incorrect values and extra predictions to identify whether the issue is tied to extracting the wrong value (non-zero incorrect values count) or the value should not have been extracted at all (non-zero extra predictions count).

Tighten field instructions with disambiguators, such as labels, keywords, and formatting constraints.

### Scenario 2: High Missed Predictions (Recall is low), Precision is moderate or high

#### What you observe

* Recall is low and Precision is moderate or high (F1 is typically low or moderate).
* Missed predictions is high, often more than incorrect values or extra predictions.

#### What it usually means

* The model is failing to extract values that are present.
* Common root causes:
  * Field instruction is too narrow, which means over-constrained examples or too-specific label requirements.
  * The value appears in multiple formats, such as dates and IDs, and the instruction does not cover variants.

#### What to do next

* Use Missed predictions + Annotated values to confirm this is a recall problem, that is, that the values exist but are not found. Check Annotated values to confirm there is a reasonable number of annotated data points for the field, and Missed predictions to confirm that the model is struggling to find values as opposed to predicting them incorrectly.
* Expand instructions to include acceptable variants: alternative labels or synonyms, multiple formatting patterns, location hints (for example, **near applicant details** or **under the borrower section**).

### Scenario 3: High Error rate but Low Docs with errors (errors concentrated in a few documents)

#### What you observe

* Error rate is high or Total errors is high.
* Docs with errors is low relative to Docs annotated.
* Often one field looks bad but only fails on a small subset of documents.

#### What it usually means

* Errors are driven by outlier documents, not systemic field behavior.
* Common root causes:
  * A specific document or format behaves differently than the rest.
  * OCR or quality issues in a small number of documents, such as blurry scans, skew, and handwritten overlays.
  * The field is present in most documents but formatted unusually in a few, for example, multi-line versus single-line.

#### What to do next

* Compare Docs with errors and Docs annotated, and optionally % Docs with errors, to confirm concentration.
* Switch to the **By document** view of the evaluation run, sort documents by score, and inspect the documents with the lowest score to identify if the field is performing poorly on a specific subset.

### Scenario 4: Large swings in performance between versions with few annotations

#### What you observe

* Large differences in F1 or error rate between model versions (up or down), but Annotated values is low, Docs annotated is low, or both.

#### What it usually means

* The field metrics are not stable yet due to small sample size.
* Common root causes:
  * Too few examples - 1–2 documents can significantly change rates.
  * Field is rarely present, that is, many missing cases and few true values.
  * A handful of difficult documents dominate the metric.

#### What to do next

* Check Annotated values, Docs annotated, and Annotated as missing to validate low coverage.
* Treat the metrics as directional, not definitive, until coverage increases.
* Add more labeled data specifically for that field: prioritize documents where the field is present, and include a diverse set of samples or variants.
* Use version comparisons only after coverage is sufficient to reduce variability-driven noise.