uip eval eval-set & evaluation

Syntax and options for managing Orchestrator eval sets and the evaluations (data points) inside them with `uip eval eval-set` and `uip eval evaluation`.

Part of the uip eval subsystem. This page covers the two resources that define what gets evaluated:

• eval eval-set — a named, process-scoped collection of evaluation data points, linked to one or more evaluators via --evaluator-refs. An eval set also carries batching/timeout settings used when it's run.
• eval evaluation — one data point inside an eval set: an input payload, an optional expected output/behavior, and optional per-evaluator criteria overrides. execute-and-evaluate and schedule runs work through an eval set's evaluations, scoring each one with the eval set's linked evaluators.

Zusammenfassung

uip eval eval-set list --process-key <guid> [--limit <n>] [--offset <n>] [--tenant <tenant>]
uip eval eval-set get <evalSetId> --process-key <guid> [--tenant <tenant>]
uip eval eval-set create --process-key <guid> --workload-id <guid> --folder-key <guid> --name <name> [options...] [--tenant <tenant>]
uip eval eval-set update <evalSetId> --process-key <guid> [options...] [--tenant <tenant>]
uip eval eval-set delete <evalSetId> --process-key <guid> [--tenant <tenant>]

uip eval evaluation list --process-key <guid> --eval-set-id <guid> [--limit <n>] [--offset <n>] [--tenant <tenant>]
uip eval evaluation get <evaluationId> --process-key <guid> --eval-set-id <guid> [--tenant <tenant>]
uip eval evaluation create --process-key <guid> --eval-set-id <guid> --folder-key <guid> --name <name> --inputs <json> [options...] [--tenant <tenant>]
uip eval evaluation update <evaluationId> --process-key <guid> --eval-set-id <guid> [options...] [--tenant <tenant>]
uip eval evaluation delete <evaluationId> --process-key <guid> --eval-set-id <guid> [--tenant <tenant>]

uip eval eval-set list

List eval sets defined for a process.

Optionen

Long	Wert	Erforderlich	Beschreibung
--process-key <guid>	GUID	ja	Process key. Use uip or processes list to find one.
--limit <n>	Integer	nein	Maximum items to return. Default 100.
--offset <n>	Integer	nein	Items to skip. Default 0.
--tenant <tenant>	name	nein	UiPath tenant name.

Beispiel

uip eval eval-set list --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09

Datenform (--output json)

{
  "Code": "EvalSetList",
  "Data": [
    {
      "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04",
      "Name": "Smoke Tests",
      "Description": "Basic smoke tests",
      "BatchSize": 10,
      "EvaluatorRefs": ["ev-1", "ev-2"],
      "CreatedAt": "2026-08-01T10:00:00Z"
    }
  ],
  "Pagination": { "Returned": 1, "Limit": 100, "Offset": 0 }
}

BatchSize is "-" when unset. UpdatedAt is present only once the eval set has been updated at least once.

uip eval eval-set get

Get one eval set's details.

Argumente

Name	Erforderlich	Zweck
<evalSetId>	ja	Eval set ID (GUID).

Optionen

Long	Wert	Erforderlich	Beschreibung
--process-key <guid>	GUID	ja	Process key.
--tenant <tenant>	name	nein	UiPath tenant name.

Beispiel

uip eval eval-set get f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04 --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09

Datenform (--output json)

{
  "Code": "EvalSetDetails",
  "Data": {
    "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04",
    "Name": "Smoke Tests",
    "Description": "Basic smoke tests",
    "BatchSize": 10,
    "EvaluatorRefs": ["ev-1", "ev-2"],
    "CreatedAt": "2026-08-01T10:00:00Z"
  }
}

uip eval eval-set create

Create an eval set for a process.

Optionen

Long	Wert	Erforderlich	Beschreibung
--process-key <guid>	GUID	ja	Process key.
--workload-id <guid>	GUID	ja	Workload ID.
--folder-key <guid>	GUID	ja	Folder key.
--name <name>	string	ja	Eval set name.
--description <text>	string	nein	Eval set description.
--batch-size <n>	integer ≥ 1	nein	Max concurrent evaluations per run.
--timeout-minutes <n>	integer ≥ 1	nein	Timeout in minutes per evaluation.
--evaluator-refs <refs...>	space-separated IDs	nein	Evaluator IDs to link to this eval set. Variadic — pass multiple IDs separated by spaces, not commas.
--tenant <tenant>	name	nein	UiPath tenant name.

An internal externalId (a random UUID) is generated automatically on every create — it isn't user-settable.

Beispiel

uip eval eval-set create --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 \
  --workload-id a1b2c3d4-0000-0000-0000-000000000001 \
  --folder-key f1f2f3f4-0000-0000-0000-000000000001 \
  --name "Smoke Tests" --evaluator-refs ev-1 ev-2

Datenform (--output json)

{
  "Code": "EvalSetCreated",
  "Data": { "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04", "Name": "Smoke Tests" }
}

uip eval eval-set update

Update an existing eval set. At least one updatable field is required beyond --process-key; fields you omit keep their current value (the command reads the existing eval set first, then applies your changes on top).

Argumente

Name	Erforderlich	Zweck
<evalSetId>	ja	Eval set ID (GUID).

Optionen

Long	Wert	Erforderlich	Beschreibung
--process-key <guid>	GUID	ja	Process key.
--name <name>	string	no*	New eval set name.
--description <text>	string	no*	Neue Beschreibung.
--batch-size <n>	integer ≥ 1	no*	New max concurrent evaluations.
--timeout-minutes <n>	integer ≥ 1	no*	New timeout in minutes.
--evaluator-refs <refs...>	space-separated IDs	no*	Replaces the full evaluator-refs list (not additive).
--tenant <tenant>	name	nein	UiPath tenant name.

* At least one of --name, --description, --batch-size, --timeout-minutes, or --evaluator-refs is required — the command fails fast if none are passed.

Beispiel

uip eval eval-set update f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04 \
  --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 \
  --name "Updated Name" --evaluator-refs ev-1 ev-2

Datenform (--output json)

{
  "Code": "EvalSetUpdated",
  "Data": {
    "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04",
    "Name": "Updated Name",
    "EvaluatorRefs": ["ev-1", "ev-2"]
  }
}

uip eval eval-set delete

Delete an eval set.

Argumente

Name	Erforderlich	Zweck
<evalSetId>	ja	Eval set ID (GUID).

Optionen

Long	Wert	Erforderlich	Beschreibung
--process-key <guid>	GUID	ja	Process key.
--tenant <tenant>	name	nein	UiPath tenant name.

Beispiel

uip eval eval-set delete f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04 --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09

Datenform (--output json)

{ "Code": "EvalSetDeleted", "Data": { "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04" } }

Deleting an eval set does not cascade-delete or otherwise warn about schedules or evaluators that reference it — the CLI issues a plain DELETE with no dependency check.

uip eval evaluation list

List the evaluations (data points) inside an eval set.

Optionen

Long	Wert	Erforderlich	Beschreibung
--process-key <guid>	GUID	ja	Process key.
--eval-set-id <guid>	GUID	ja	Eval set ID.
--limit <n>	Integer	nein	Maximum items to return. Default 100.
--offset <n>	Integer	nein	Items to skip. Default 0.
--tenant <tenant>	name	nein	UiPath tenant name.

Beispiel

uip eval evaluation list --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 --eval-set-id f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04

Datenform (--output json)

{
  "Code": "EvaluationList",
  "Data": [
    {
      "EvaluationId": "c1d2e3f4-0000-0000-0000-000000000001",
      "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04",
      "Name": "Greeting test",
      "Inputs": { "input": "hello" },
      "ExpectedOutput": { "content": "Hi there!" },
      "ExpectedBehavior": "",
      "CreatedAt": "2026-08-01T10:00:00Z"
    }
  ],
  "Pagination": { "Returned": 1, "Limit": 100, "Offset": 0 }
}

uip eval evaluation get

Get one evaluation's details.

Argumente

Name	Erforderlich	Zweck
<evaluationId>	ja	Evaluation ID (GUID).

Optionen

Long	Wert	Erforderlich	Beschreibung
--process-key <guid>	GUID	ja	Process key.
--eval-set-id <guid>	GUID	ja	Eval set ID.
--tenant <tenant>	name	nein	UiPath tenant name.

Beispiel

uip eval evaluation get c1d2e3f4-0000-0000-0000-000000000001 \
  --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 --eval-set-id f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04

Datenform (--output json)

Same object shape as one entry in evaluation list's Data array.

uip eval evaluation create

Add an evaluation (data point) to an eval set.

Optionen

Long	Wert	Erforderlich	Beschreibung
--process-key <guid>	GUID	ja	Process key.
--eval-set-id <guid>	GUID	ja	Eval set ID.
--folder-key <guid>	GUID	ja	Folder key.
--name <name>	string	ja	Evaluation name.
--inputs <json>	JSON	ja	Input values, e.g. '{"input":"hello"}'.
--expected-output <json>	JSON	nein	Expected output for comparison-style evaluators.
--expected-behavior <text>	string	nein	Expected agent behavior description, for trajectory evaluators.
--evaluation-criterias <json>	JSON	nein	Per-evaluator criteria overrides.
--tenant <tenant>	name	nein	UiPath tenant name.

Malformed JSON on --inputs, --expected-output, or --evaluation-criterias fails fast client-side before any network call, with an example of the expected shape in the error for --inputs.

Beispiel

uip eval evaluation create --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 \
  --eval-set-id f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04 \
  --folder-key f1f2f3f4-0000-0000-0000-000000000001 \
  --name "Greeting test" --inputs '{"input":"hello"}' --expected-output '{"content":"Hi there!"}'

Datenform (--output json)

{
  "Code": "EvaluationCreated",
  "Data": {
    "EvaluationId": "c1d2e3f4-0000-0000-0000-000000000001",
    "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04",
    "Name": "Greeting test"
  }
}

uip eval evaluation update

Update an existing evaluation. At least one updatable field is required beyond --process-key/--eval-set-id; omitted fields keep their current value.

Argumente

Name	Erforderlich	Zweck
<evaluationId>	ja	Evaluation ID (GUID).

Optionen

Long	Wert	Erforderlich	Beschreibung
--process-key <guid>	GUID	ja	Process key.
--eval-set-id <guid>	GUID	ja	Eval set ID.
--name <name>	string	no*	New evaluation name.
--inputs <json>	JSON	no*	New input values.
--expected-output <json>	JSON	no*	New expected output.
--expected-behavior <text>	string	no*	New expected behavior description.
--evaluation-criterias <json>	JSON	no*	New per-evaluator criteria overrides.
--tenant <tenant>	name	nein	UiPath tenant name.

* At least one of --name, --inputs, --expected-output, --expected-behavior, or --evaluation-criterias is required.

Beispiel

uip eval evaluation update c1d2e3f4-0000-0000-0000-000000000001 \
  --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 --eval-set-id f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04 \
  --inputs '{"input":"updated"}' --expected-output '{"content":"new response"}'

Datenform (--output json)

{
  "Code": "EvaluationUpdated",
  "Data": {
    "EvaluationId": "c1d2e3f4-0000-0000-0000-000000000001",
    "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04",
    "Name": "Greeting test"
  }
}

uip eval evaluation delete

Remove an evaluation from an eval set.

Argumente

Name	Erforderlich	Zweck
<evaluationId>	ja	Evaluation ID (GUID).

Optionen

Long	Wert	Erforderlich	Beschreibung
--process-key <guid>	GUID	ja	Process key.
--eval-set-id <guid>	GUID	ja	Eval set ID.
--tenant <tenant>	name	nein	UiPath tenant name.

Beispiel

uip eval evaluation delete c1d2e3f4-0000-0000-0000-000000000001 \
  --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 --eval-set-id f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04

Datenform (--output json)

{ "Code": "EvaluationDeleted", "Data": { "EvaluationId": "c1d2e3f4-0000-0000-0000-000000000001" } }

Zugehörig

• uip eval — overview, execute-and-evaluate, and run.
• uip eval schedule — recurring evaluation runs against an eval set.
• uip eval evaluator — the scoring logic an eval set's --evaluator-refs point to.
• uip or processes — resolve a --process-key.

Siehe auch

• Orchestrator tool overview
• Globale Optionen
• Exitcodes