# uip eval eval-set & evaluation

> Syntax and options for managing Orchestrator eval sets and the evaluations (data points) inside them with `uip eval eval-set` and `uip eval evaluation`.

Part of the [`uip eval`](./uip-eval.md) subsystem. This page covers the two resources that define *what gets evaluated*:

- **`eval eval-set`** — a named, process-scoped collection of evaluation data points, linked to one or more [evaluators](./uip-eval-evaluators.md) via `--evaluator-refs`. An eval set also carries batching/timeout settings used when it's run.
- **`eval evaluation`** — one data point inside an eval set: an input payload, an optional expected output/behavior, and optional per-evaluator criteria overrides. [`execute-and-evaluate`](./uip-eval.md#uip-eval-execute-and-evaluate) and [`schedule`](./uip-eval-schedule.md) runs work through an eval set's evaluations, scoring each one with the eval set's linked evaluators.

## Synopsis

```text
uip eval eval-set list --process-key <guid> [--limit <n>] [--offset <n>] [--tenant <tenant>]
uip eval eval-set get <evalSetId> --process-key <guid> [--tenant <tenant>]
uip eval eval-set create --process-key <guid> --workload-id <guid> --folder-key <guid> --name <name> [options...] [--tenant <tenant>]
uip eval eval-set update <evalSetId> --process-key <guid> [options...] [--tenant <tenant>]
uip eval eval-set delete <evalSetId> --process-key <guid> [--tenant <tenant>]

uip eval evaluation list --process-key <guid> --eval-set-id <guid> [--limit <n>] [--offset <n>] [--tenant <tenant>]
uip eval evaluation get <evaluationId> --process-key <guid> --eval-set-id <guid> [--tenant <tenant>]
uip eval evaluation create --process-key <guid> --eval-set-id <guid> --folder-key <guid> --name <name> --inputs <json> [options...] [--tenant <tenant>]
uip eval evaluation update <evaluationId> --process-key <guid> --eval-set-id <guid> [options...] [--tenant <tenant>]
uip eval evaluation delete <evaluationId> --process-key <guid> --eval-set-id <guid> [--tenant <tenant>]
```

## uip eval eval-set list

List eval sets defined for a process.

### Options

| Long | Value | Required | Description |
|---|---|---|---|
| `--process-key <guid>` | GUID | **yes** | Process key. Use [`uip or processes list`](./uip-or-processes.md) to find one. |
| `--limit <n>` | integer | no | Maximum items to return. Default `100`. |
| `--offset <n>` | integer | no | Items to skip. Default `0`. |
| `--tenant <tenant>` | name | no | UiPath tenant name. |

### Example

```bash
uip eval eval-set list --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09
```

### Data shape (--output json)

```json
{
  "Code": "EvalSetList",
  "Data": [
    {
      "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04",
      "Name": "Smoke Tests",
      "Description": "Basic smoke tests",
      "BatchSize": 10,
      "EvaluatorRefs": ["ev-1", "ev-2"],
      "CreatedAt": "2026-08-01T10:00:00Z"
    }
  ],
  "Pagination": { "Returned": 1, "Limit": 100, "Offset": 0 }
}
```

`BatchSize` is `"-"` when unset. `UpdatedAt` is present only once the eval set has been updated at least once.

## uip eval eval-set get

Get one eval set's details.

### Arguments

| Name | Required | Purpose |
|---|---|---|
| `<evalSetId>` | yes | Eval set ID (GUID). |

### Options

| Long | Value | Required | Description |
|---|---|---|---|
| `--process-key <guid>` | GUID | **yes** | Process key. |
| `--tenant <tenant>` | name | no | UiPath tenant name. |

### Example

```bash
uip eval eval-set get f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04 --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09
```

### Data shape (--output json)

```json
{
  "Code": "EvalSetDetails",
  "Data": {
    "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04",
    "Name": "Smoke Tests",
    "Description": "Basic smoke tests",
    "BatchSize": 10,
    "EvaluatorRefs": ["ev-1", "ev-2"],
    "CreatedAt": "2026-08-01T10:00:00Z"
  }
}
```

## uip eval eval-set create

Create an eval set for a process.

### Options

| Long | Value | Required | Description |
|---|---|---|---|
| `--process-key <guid>` | GUID | **yes** | Process key. |
| `--workload-id <guid>` | GUID | **yes** | Workload ID. |
| `--folder-key <guid>` | GUID | **yes** | Folder key. |
| `--name <name>` | string | **yes** | Eval set name. |
| `--description <text>` | string | no | Eval set description. |
| `--batch-size <n>` | integer ≥ 1 | no | Max concurrent evaluations per run. |
| `--timeout-minutes <n>` | integer ≥ 1 | no | Timeout in minutes per evaluation. |
| `--evaluator-refs <refs...>` | space-separated IDs | no | [Evaluator](./uip-eval-evaluators.md) IDs to link to this eval set. Variadic — pass multiple IDs separated by spaces, not commas. |
| `--tenant <tenant>` | name | no | UiPath tenant name. |

An internal `externalId` (a random UUID) is generated automatically on every create — it isn't user-settable.

### Example

```bash
uip eval eval-set create --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 \
  --workload-id a1b2c3d4-0000-0000-0000-000000000001 \
  --folder-key f1f2f3f4-0000-0000-0000-000000000001 \
  --name "Smoke Tests" --evaluator-refs ev-1 ev-2
```

### Data shape (--output json)

```json
{
  "Code": "EvalSetCreated",
  "Data": { "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04", "Name": "Smoke Tests" }
}
```

## uip eval eval-set update

Update an existing eval set. At least one updatable field is required beyond `--process-key`; fields you omit keep their current value (the command reads the existing eval set first, then applies your changes on top).

### Arguments

| Name | Required | Purpose |
|---|---|---|
| `<evalSetId>` | yes | Eval set ID (GUID). |

### Options

| Long | Value | Required | Description |
|---|---|---|---|
| `--process-key <guid>` | GUID | **yes** | Process key. |
| `--name <name>` | string | no* | New eval set name. |
| `--description <text>` | string | no* | New description. |
| `--batch-size <n>` | integer ≥ 1 | no* | New max concurrent evaluations. |
| `--timeout-minutes <n>` | integer ≥ 1 | no* | New timeout in minutes. |
| `--evaluator-refs <refs...>` | space-separated IDs | no* | Replaces the full evaluator-refs list (not additive). |
| `--tenant <tenant>` | name | no | UiPath tenant name. |

\* At least one of `--name`, `--description`, `--batch-size`, `--timeout-minutes`, or `--evaluator-refs` is required — the command fails fast if none are passed.

### Example

```bash
uip eval eval-set update f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04 \
  --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 \
  --name "Updated Name" --evaluator-refs ev-1 ev-2
```

### Data shape (--output json)

```json
{
  "Code": "EvalSetUpdated",
  "Data": {
    "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04",
    "Name": "Updated Name",
    "EvaluatorRefs": ["ev-1", "ev-2"]
  }
}
```

## uip eval eval-set delete

Delete an eval set.

### Arguments

| Name | Required | Purpose |
|---|---|---|
| `<evalSetId>` | yes | Eval set ID (GUID). |

### Options

| Long | Value | Required | Description |
|---|---|---|---|
| `--process-key <guid>` | GUID | **yes** | Process key. |
| `--tenant <tenant>` | name | no | UiPath tenant name. |

### Example

```bash
uip eval eval-set delete f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04 --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09
```

### Data shape (--output json)

```json
{ "Code": "EvalSetDeleted", "Data": { "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04" } }
```

Deleting an eval set does not cascade-delete or otherwise warn about schedules or evaluators that reference it — the CLI issues a plain DELETE with no dependency check.

## uip eval evaluation list

List the evaluations (data points) inside an eval set.

### Options

| Long | Value | Required | Description |
|---|---|---|---|
| `--process-key <guid>` | GUID | **yes** | Process key. |
| `--eval-set-id <guid>` | GUID | **yes** | Eval set ID. |
| `--limit <n>` | integer | no | Maximum items to return. Default `100`. |
| `--offset <n>` | integer | no | Items to skip. Default `0`. |
| `--tenant <tenant>` | name | no | UiPath tenant name. |

### Example

```bash
uip eval evaluation list --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 --eval-set-id f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04
```

### Data shape (--output json)

```json
{
  "Code": "EvaluationList",
  "Data": [
    {
      "EvaluationId": "c1d2e3f4-0000-0000-0000-000000000001",
      "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04",
      "Name": "Greeting test",
      "Inputs": { "input": "hello" },
      "ExpectedOutput": { "content": "Hi there!" },
      "ExpectedBehavior": "",
      "CreatedAt": "2026-08-01T10:00:00Z"
    }
  ],
  "Pagination": { "Returned": 1, "Limit": 100, "Offset": 0 }
}
```

## uip eval evaluation get

Get one evaluation's details.

### Arguments

| Name | Required | Purpose |
|---|---|---|
| `<evaluationId>` | yes | Evaluation ID (GUID). |

### Options

| Long | Value | Required | Description |
|---|---|---|---|
| `--process-key <guid>` | GUID | **yes** | Process key. |
| `--eval-set-id <guid>` | GUID | **yes** | Eval set ID. |
| `--tenant <tenant>` | name | no | UiPath tenant name. |

### Example

```bash
uip eval evaluation get c1d2e3f4-0000-0000-0000-000000000001 \
  --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 --eval-set-id f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04
```

### Data shape (--output json)

Same object shape as one entry in `evaluation list`'s `Data` array.

## uip eval evaluation create

Add an evaluation (data point) to an eval set.

### Options

| Long | Value | Required | Description |
|---|---|---|---|
| `--process-key <guid>` | GUID | **yes** | Process key. |
| `--eval-set-id <guid>` | GUID | **yes** | Eval set ID. |
| `--folder-key <guid>` | GUID | **yes** | Folder key. |
| `--name <name>` | string | **yes** | Evaluation name. |
| `--inputs <json>` | JSON | **yes** | Input values, e.g. `'{"input":"hello"}'`. |
| `--expected-output <json>` | JSON | no | Expected output for comparison-style evaluators. |
| `--expected-behavior <text>` | string | no | Expected agent behavior description, for trajectory evaluators. |
| `--evaluation-criterias <json>` | JSON | no | Per-evaluator criteria overrides. |
| `--tenant <tenant>` | name | no | UiPath tenant name. |

Malformed JSON on `--inputs`, `--expected-output`, or `--evaluation-criterias` fails fast client-side before any network call, with an example of the expected shape in the error for `--inputs`.

### Example

```bash
uip eval evaluation create --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 \
  --eval-set-id f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04 \
  --folder-key f1f2f3f4-0000-0000-0000-000000000001 \
  --name "Greeting test" --inputs '{"input":"hello"}' --expected-output '{"content":"Hi there!"}'
```

### Data shape (--output json)

```json
{
  "Code": "EvaluationCreated",
  "Data": {
    "EvaluationId": "c1d2e3f4-0000-0000-0000-000000000001",
    "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04",
    "Name": "Greeting test"
  }
}
```

## uip eval evaluation update

Update an existing evaluation. At least one updatable field is required beyond `--process-key`/`--eval-set-id`; omitted fields keep their current value.

### Arguments

| Name | Required | Purpose |
|---|---|---|
| `<evaluationId>` | yes | Evaluation ID (GUID). |

### Options

| Long | Value | Required | Description |
|---|---|---|---|
| `--process-key <guid>` | GUID | **yes** | Process key. |
| `--eval-set-id <guid>` | GUID | **yes** | Eval set ID. |
| `--name <name>` | string | no* | New evaluation name. |
| `--inputs <json>` | JSON | no* | New input values. |
| `--expected-output <json>` | JSON | no* | New expected output. |
| `--expected-behavior <text>` | string | no* | New expected behavior description. |
| `--evaluation-criterias <json>` | JSON | no* | New per-evaluator criteria overrides. |
| `--tenant <tenant>` | name | no | UiPath tenant name. |

\* At least one of `--name`, `--inputs`, `--expected-output`, `--expected-behavior`, or `--evaluation-criterias` is required.

### Example

```bash
uip eval evaluation update c1d2e3f4-0000-0000-0000-000000000001 \
  --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 --eval-set-id f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04 \
  --inputs '{"input":"updated"}' --expected-output '{"content":"new response"}'
```

### Data shape (--output json)

```json
{
  "Code": "EvaluationUpdated",
  "Data": {
    "EvaluationId": "c1d2e3f4-0000-0000-0000-000000000001",
    "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04",
    "Name": "Greeting test"
  }
}
```

## uip eval evaluation delete

Remove an evaluation from an eval set.

### Arguments

| Name | Required | Purpose |
|---|---|---|
| `<evaluationId>` | yes | Evaluation ID (GUID). |

### Options

| Long | Value | Required | Description |
|---|---|---|---|
| `--process-key <guid>` | GUID | **yes** | Process key. |
| `--eval-set-id <guid>` | GUID | **yes** | Eval set ID. |
| `--tenant <tenant>` | name | no | UiPath tenant name. |

### Example

```bash
uip eval evaluation delete c1d2e3f4-0000-0000-0000-000000000001 \
  --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 --eval-set-id f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04
```

### Data shape (--output json)

```json
{ "Code": "EvaluationDeleted", "Data": { "EvaluationId": "c1d2e3f4-0000-0000-0000-000000000001" } }
```

## Related

- [uip eval](./uip-eval.md) — overview, `execute-and-evaluate`, and `run`.
- [uip eval schedule](./uip-eval-schedule.md) — recurring evaluation runs against an eval set.
- [uip eval evaluator](./uip-eval-evaluators.md) — the scoring logic an eval set's `--evaluator-refs` point to.
- [`uip or processes`](./uip-or-processes.md) — resolve a `--process-key`.

## See also

- [Orchestrator tool overview](./uip-or.md)
- [Global options](./global-options.md)
- [Exit codes](./exit-codes.md)