# uip eval evaluator

> Syntax and options for `uip eval evaluator`, which manages the scoring evaluators used to grade Orchestrator process runs.

An **evaluator** is a scoring mechanism attached to a process (by `--process-key`) that grades a run's output — for example, an LLM-judge comparison against an expected answer, or an exact-match check. Evaluators are referenced by ID from [`eval-set`](./uip-eval-sets-evaluations.md) entries and from `execute-and-evaluate` calls (see the [eval overview](./uip-eval.md)), which is what actually runs a scored evaluation. This page only covers authoring/managing evaluator definitions themselves.

Every verb requires `--process-key <guid>` — evaluators are scoped to one process. Find process keys with [`uip or processes list`](./uip-or-processes.md).

## Synopsis

```text
uip eval evaluator list --process-key <guid> [--limit <number>] [--offset <number>] [--tenant <tenant>]
uip eval evaluator get <evaluatorId> --process-key <guid> [--tenant <tenant>]
uip eval evaluator create --process-key <guid> --workload-id <guid> --folder-key <guid> --name <name> --description <text> --evaluator-type-id <id> --evaluator-config <json> [--version <version>] [--tenant <tenant>]
uip eval evaluator update <evaluatorId> --process-key <guid> [--name <name>] [--description <text>] [--evaluator-type-id <id>] [--evaluator-config <json>] [--version <version>] [--tenant <tenant>]
uip eval evaluator delete <evaluatorId> --process-key <guid> [--tenant <tenant>]
```

## uip eval evaluator list

List evaluators defined for a process.

### Options

| Long | Value | Required | Description |
|---|---|---|---|
| `--process-key <guid>` | GUID | **yes** | Process key. Use `uip or processes list` to find keys. |
| `--limit <number>` | integer | no | Maximum number of items to return. Default `100`. |
| `--offset <number>` | integer | no | Number of items to skip. Default `0`. |
| `--tenant <tenant>` | name | no | UiPath tenant name. |

### Example

```bash
uip eval evaluator list --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09
```

### Data shape (--output json)

```json
{
  "Code": "EvaluatorList",
  "Data": [
    {
      "EvaluatorId": "a1b2c3d4-0000-0000-0000-000000000001",
      "Name": "Semantic Similarity",
      "Description": "LLM-based output comparison",
      "EvaluatorTypeId": "uipath-llm-judge-output-semantic-similarity",
      "Version": "1.0",
      "CreatedAt": "2026-08-01T10:00:00Z"
    }
  ],
  "Pagination": { "Returned": 1, "Limit": 100, "Offset": 0 }
}
```

`UpdatedAt` is included per item only when the evaluator has been modified since creation.

## uip eval evaluator get

Get one evaluator's full details by ID.

### Arguments

| Name | Required | Purpose |
|---|---|---|
| `<evaluatorId>` | yes | Evaluator ID (GUID). |

### Options

| Long | Value | Required | Description |
|---|---|---|---|
| `--process-key <guid>` | GUID | **yes** | Process key. |
| `--tenant <tenant>` | name | no | UiPath tenant name. |

### Example

```bash
uip eval evaluator get a1b2c3d4-0000-0000-0000-000000000001 --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09
```

### Data shape (--output json)

```json
{
  "Code": "EvaluatorDetails",
  "Data": {
    "EvaluatorId": "a1b2c3d4-0000-0000-0000-000000000001",
    "Name": "Semantic Similarity",
    "Description": "LLM-based output comparison",
    "EvaluatorTypeId": "uipath-llm-judge-output-semantic-similarity",
    "Version": "1.0",
    "CreatedAt": "2026-08-01T10:00:00Z"
  }
}
```

## uip eval evaluator create

Create an evaluator on a process.

### Options

| Long | Value | Required | Description |
|---|---|---|---|
| `--process-key <guid>` | GUID | **yes** | Process key. |
| `--workload-id <guid>` | GUID | **yes** | Workload ID. |
| `--folder-key <guid>` | GUID | **yes** | Folder key. |
| `--name <name>` | string | **yes** | Evaluator name. |
| `--description <text>` | string | **yes** | Evaluator description. |
| `--evaluator-type-id <id>` | string | **yes** | Evaluator type. Source's own examples show `uipath-exact-match`, `uipath-llm-judge-output-semantic-similarity`, `uipath-llm-judge-trajectory-similarity` — this is not a client-validated enum, so other type IDs may exist server-side; the three shown are simply what the CLI's own help text names. |
| `--evaluator-config <json>` | JSON object | **yes** | Type-specific configuration. Shape depends on `--evaluator-type-id` — the semantic-similarity example below shows `name`/`prompt`/`model`/`targetOutputKey`; other types likely take different keys, not enumerated client-side. |
| `--version <version>` | string | no | Evaluator version. Default `1.0`. |
| `--tenant <tenant>` | name | no | UiPath tenant name. |

An `externalId` (UUID) is generated automatically on every create — it isn't a flag, and isn't shown in `list`/`get` output.

### Example

```bash
uip eval evaluator create --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 \
  --workload-id a1b2c3d4-0000-0000-0000-000000000001 \
  --folder-key f1f2f3f4-0000-0000-0000-000000000001 \
  --name "Semantic Similarity" --description "LLM-based output comparison" \
  --evaluator-type-id uipath-llm-judge-output-semantic-similarity \
  --evaluator-config '{"name":"Semantic","prompt":"Score 0-100...","model":"gpt-4.1-2025-04-14","targetOutputKey":"*"}' \
  --version 1.0
```

### Data shape (--output json)

```json
{
  "Code": "EvaluatorCreated",
  "Data": {
    "EvaluatorId": "a1b2c3d4-0000-0000-0000-000000000001",
    "Name": "Semantic Similarity",
    "EvaluatorTypeId": "uipath-llm-judge-output-semantic-similarity"
  }
}
```

## uip eval evaluator update

Update one or more fields on an existing evaluator. Unspecified fields are read from the current evaluator and re-sent unchanged (a full-object PUT under the hood, not a partial patch) — `--evaluator-config` itself is not merged: passing it replaces the entire config object, not just the keys you name.

### Arguments

| Name | Required | Purpose |
|---|---|---|
| `<evaluatorId>` | yes | Evaluator ID (GUID). |

### Options

| Long | Value | Required | Description |
|---|---|---|---|
| `--process-key <guid>` | GUID | **yes** | Process key. |
| `--name <name>` | string | no* | New name. |
| `--description <text>` | string | no* | New description. |
| `--evaluator-type-id <id>` | string | no* | New evaluator type. |
| `--evaluator-config <json>` | JSON object | no* | New config — replaces the whole object. |
| `--version <version>` | string | no* | New version. |
| `--tenant <tenant>` | name | no | UiPath tenant name. |

\* At least one of `--name`/`--description`/`--evaluator-type-id`/`--evaluator-config`/`--version` is required — omitting all five fails with an explicit error before any network call.

### Example

```bash
uip eval evaluator update a1b2c3d4-0000-0000-0000-000000000001 \
  --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 \
  --name "Updated Evaluator" \
  --evaluator-config '{"name":"Updated","prompt":"New prompt"}'
```

### Data shape (--output json)

```json
{
  "Code": "EvaluatorUpdated",
  "Data": {
    "EvaluatorId": "a1b2c3d4-0000-0000-0000-000000000001",
    "Name": "Updated Evaluator",
    "EvaluatorTypeId": "uipath-llm-judge-output-semantic-similarity"
  }
}
```

## uip eval evaluator delete

Delete an evaluator.

:::note
Deleting an evaluator does not check whether it's still referenced by an [`eval-set`](./uip-eval-sets-evaluations.md) entry or a [`schedule`](./uip-eval-schedule.md) — the API performs no reference check client-side. Confirm nothing depends on it first.
:::

### Arguments

| Name | Required | Purpose |
|---|---|---|
| `<evaluatorId>` | yes | Evaluator ID (GUID). |

### Options

| Long | Value | Required | Description |
|---|---|---|---|
| `--process-key <guid>` | GUID | **yes** | Process key. |
| `--tenant <tenant>` | name | no | UiPath tenant name. |

### Example

```bash
uip eval evaluator delete a1b2c3d4-0000-0000-0000-000000000001 --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09
```

### Data shape (--output json)

```json
{
  "Code": "EvaluatorDeleted",
  "Data": { "EvaluatorId": "a1b2c3d4-0000-0000-0000-000000000001" }
}
```

## Related

- [uip eval](./uip-eval.md) — overview, `execute-and-evaluate`, and `run`.
- [uip eval schedule](./uip-eval-schedule.md) — recurring evaluation runs.
- [uip eval-set / evaluation](./uip-eval-sets-evaluations.md) — grouping evaluators into a reusable evaluation configuration.
- [uip or processes](./uip-or-processes.md) — find process keys.

## See also

- [Orchestrator tool overview](./uip-or.md)
- [Global options](./global-options.md)
- [Exit codes](./exit-codes.md)