uip eval eval-set & evaluation

Syntax and options for managing Orchestrator eval sets and the evaluations (data points) inside them with `uip eval eval-set` and `uip eval evaluation`.

Part of the uip eval subsystem. This page covers the two resources that define what gets evaluated:

• eval eval-set — a named, process-scoped collection of evaluation data points, linked to one or more evaluators via --evaluator-refs. An eval set also carries batching/timeout settings used when it's run.
• eval evaluation — one data point inside an eval set: an input payload, an optional expected output/behavior, and optional per-evaluator criteria overrides. execute-and-evaluate and schedule runs work through an eval set's evaluations, scoring each one with the eval set's linked evaluators.

Synopsis

uip eval eval-set list --process-key <guid> [--limit <n>] [--offset <n>] [--tenant <tenant>]
uip eval eval-set get <evalSetId> --process-key <guid> [--tenant <tenant>]
uip eval eval-set create --process-key <guid> --workload-id <guid> --folder-key <guid> --name <name> [options...] [--tenant <tenant>]
uip eval eval-set update <evalSetId> --process-key <guid> [options...] [--tenant <tenant>]
uip eval eval-set delete <evalSetId> --process-key <guid> [--tenant <tenant>]

uip eval evaluation list --process-key <guid> --eval-set-id <guid> [--limit <n>] [--offset <n>] [--tenant <tenant>]
uip eval evaluation get <evaluationId> --process-key <guid> --eval-set-id <guid> [--tenant <tenant>]
uip eval evaluation create --process-key <guid> --eval-set-id <guid> --folder-key <guid> --name <name> --inputs <json> [options...] [--tenant <tenant>]
uip eval evaluation update <evaluationId> --process-key <guid> --eval-set-id <guid> [options...] [--tenant <tenant>]
uip eval evaluation delete <evaluationId> --process-key <guid> --eval-set-id <guid> [--tenant <tenant>]

uip eval eval-set list

List eval sets defined for a process.

Options

Long	Value	Required	Description
--process-key <guid>	GUID	yes	Process key. Use uip or processes list to find one.
--limit <n>	integer	no	Maximum items to return. Default 100.
--offset <n>	integer	no	Items to skip. Default 0.
--tenant <tenant>	name	no	UiPath tenant name.

Example

uip eval eval-set list --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09

Data shape (--output json)

{
  "Code": "EvalSetList",
  "Data": [
    {
      "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04",
      "Name": "Smoke Tests",
      "Description": "Basic smoke tests",
      "BatchSize": 10,
      "EvaluatorRefs": ["ev-1", "ev-2"],
      "CreatedAt": "2026-08-01T10:00:00Z"
    }
  ],
  "Pagination": { "Returned": 1, "Limit": 100, "Offset": 0 }
}

BatchSize is "-" when unset. UpdatedAt is present only once the eval set has been updated at least once.

uip eval eval-set get

Get one eval set's details.

Arguments

Name	Required	Purpose
<evalSetId>	yes	Eval set ID (GUID).

Options

Long	Value	Required	Description
--process-key <guid>	GUID	yes	Process key.
--tenant <tenant>	name	no	UiPath tenant name.

Example

uip eval eval-set get f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04 --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09

Data shape (--output json)

{
  "Code": "EvalSetDetails",
  "Data": {
    "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04",
    "Name": "Smoke Tests",
    "Description": "Basic smoke tests",
    "BatchSize": 10,
    "EvaluatorRefs": ["ev-1", "ev-2"],
    "CreatedAt": "2026-08-01T10:00:00Z"
  }
}

uip eval eval-set create

Create an eval set for a process.

Options

Long	Value	Required	Description
--process-key <guid>	GUID	yes	Process key.
--workload-id <guid>	GUID	yes	Workload ID.
--folder-key <guid>	GUID	yes	Folder key.
--name <name>	string	yes	Eval set name.
--description <text>	string	no	Eval set description.
--batch-size <n>	integer ≥ 1	no	Max concurrent evaluations per run.
--timeout-minutes <n>	integer ≥ 1	no	Timeout in minutes per evaluation.
--evaluator-refs <refs...>	space-separated IDs	no	Evaluator IDs to link to this eval set. Variadic — pass multiple IDs separated by spaces, not commas.
--tenant <tenant>	name	no	UiPath tenant name.

An internal externalId (a random UUID) is generated automatically on every create — it isn't user-settable.

Example

uip eval eval-set create --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 \
  --workload-id a1b2c3d4-0000-0000-0000-000000000001 \
  --folder-key f1f2f3f4-0000-0000-0000-000000000001 \
  --name "Smoke Tests" --evaluator-refs ev-1 ev-2

Data shape (--output json)

{
  "Code": "EvalSetCreated",
  "Data": { "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04", "Name": "Smoke Tests" }
}

uip eval eval-set update

Update an existing eval set. At least one updatable field is required beyond --process-key; fields you omit keep their current value (the command reads the existing eval set first, then applies your changes on top).

Arguments

Name	Required	Purpose
<evalSetId>	yes	Eval set ID (GUID).

Options

Long	Value	Required	Description
--process-key <guid>	GUID	yes	Process key.
--name <name>	string	no*	New eval set name.
--description <text>	string	no*	New description.
--batch-size <n>	integer ≥ 1	no*	New max concurrent evaluations.
--timeout-minutes <n>	integer ≥ 1	no*	New timeout in minutes.
--evaluator-refs <refs...>	space-separated IDs	no*	Replaces the full evaluator-refs list (not additive).
--tenant <tenant>	name	no	UiPath tenant name.

* At least one of --name, --description, --batch-size, --timeout-minutes, or --evaluator-refs is required — the command fails fast if none are passed.

Example

uip eval eval-set update f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04 \
  --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 \
  --name "Updated Name" --evaluator-refs ev-1 ev-2

Data shape (--output json)

{
  "Code": "EvalSetUpdated",
  "Data": {
    "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04",
    "Name": "Updated Name",
    "EvaluatorRefs": ["ev-1", "ev-2"]
  }
}

uip eval eval-set delete

Delete an eval set.

Arguments

Name	Required	Purpose
<evalSetId>	yes	Eval set ID (GUID).

Options

Long	Value	Required	Description
--process-key <guid>	GUID	yes	Process key.
--tenant <tenant>	name	no	UiPath tenant name.

Example

uip eval eval-set delete f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04 --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09

Data shape (--output json)

{ "Code": "EvalSetDeleted", "Data": { "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04" } }

Deleting an eval set does not cascade-delete or otherwise warn about schedules or evaluators that reference it — the CLI issues a plain DELETE with no dependency check.

uip eval evaluation list

List the evaluations (data points) inside an eval set.

Options

Long	Value	Required	Description
--process-key <guid>	GUID	yes	Process key.
--eval-set-id <guid>	GUID	yes	Eval set ID.
--limit <n>	integer	no	Maximum items to return. Default 100.
--offset <n>	integer	no	Items to skip. Default 0.
--tenant <tenant>	name	no	UiPath tenant name.

Example

uip eval evaluation list --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 --eval-set-id f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04

Data shape (--output json)

{
  "Code": "EvaluationList",
  "Data": [
    {
      "EvaluationId": "c1d2e3f4-0000-0000-0000-000000000001",
      "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04",
      "Name": "Greeting test",
      "Inputs": { "input": "hello" },
      "ExpectedOutput": { "content": "Hi there!" },
      "ExpectedBehavior": "",
      "CreatedAt": "2026-08-01T10:00:00Z"
    }
  ],
  "Pagination": { "Returned": 1, "Limit": 100, "Offset": 0 }
}

uip eval evaluation get

Get one evaluation's details.

Arguments

Name	Required	Purpose
<evaluationId>	yes	Evaluation ID (GUID).

Options

Long	Value	Required	Description
--process-key <guid>	GUID	yes	Process key.
--eval-set-id <guid>	GUID	yes	Eval set ID.
--tenant <tenant>	name	no	UiPath tenant name.

Example

uip eval evaluation get c1d2e3f4-0000-0000-0000-000000000001 \
  --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 --eval-set-id f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04

Data shape (--output json)

Same object shape as one entry in evaluation list's Data array.

uip eval evaluation create

Add an evaluation (data point) to an eval set.

Options

Long	Value	Required	Description
--process-key <guid>	GUID	yes	Process key.
--eval-set-id <guid>	GUID	yes	Eval set ID.
--folder-key <guid>	GUID	yes	Folder key.
--name <name>	string	yes	Evaluation name.
--inputs <json>	JSON	yes	Input values, e.g. '{"input":"hello"}'.
--expected-output <json>	JSON	no	Expected output for comparison-style evaluators.
--expected-behavior <text>	string	no	Expected agent behavior description, for trajectory evaluators.
--evaluation-criterias <json>	JSON	no	Per-evaluator criteria overrides.
--tenant <tenant>	name	no	UiPath tenant name.

Malformed JSON on --inputs, --expected-output, or --evaluation-criterias fails fast client-side before any network call, with an example of the expected shape in the error for --inputs.

Example

uip eval evaluation create --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 \
  --eval-set-id f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04 \
  --folder-key f1f2f3f4-0000-0000-0000-000000000001 \
  --name "Greeting test" --inputs '{"input":"hello"}' --expected-output '{"content":"Hi there!"}'

Data shape (--output json)

{
  "Code": "EvaluationCreated",
  "Data": {
    "EvaluationId": "c1d2e3f4-0000-0000-0000-000000000001",
    "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04",
    "Name": "Greeting test"
  }
}

uip eval evaluation update

Update an existing evaluation. At least one updatable field is required beyond --process-key/--eval-set-id; omitted fields keep their current value.

Arguments

Name	Required	Purpose
<evaluationId>	yes	Evaluation ID (GUID).

Options

Long	Value	Required	Description
--process-key <guid>	GUID	yes	Process key.
--eval-set-id <guid>	GUID	yes	Eval set ID.
--name <name>	string	no*	New evaluation name.
--inputs <json>	JSON	no*	New input values.
--expected-output <json>	JSON	no*	New expected output.
--expected-behavior <text>	string	no*	New expected behavior description.
--evaluation-criterias <json>	JSON	no*	New per-evaluator criteria overrides.
--tenant <tenant>	name	no	UiPath tenant name.

* At least one of --name, --inputs, --expected-output, --expected-behavior, or --evaluation-criterias is required.

Example

uip eval evaluation update c1d2e3f4-0000-0000-0000-000000000001 \
  --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 --eval-set-id f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04 \
  --inputs '{"input":"updated"}' --expected-output '{"content":"new response"}'

Data shape (--output json)

{
  "Code": "EvaluationUpdated",
  "Data": {
    "EvaluationId": "c1d2e3f4-0000-0000-0000-000000000001",
    "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04",
    "Name": "Greeting test"
  }
}

uip eval evaluation delete

Remove an evaluation from an eval set.

Arguments

Name	Required	Purpose
<evaluationId>	yes	Evaluation ID (GUID).

Options

Long	Value	Required	Description
--process-key <guid>	GUID	yes	Process key.
--eval-set-id <guid>	GUID	yes	Eval set ID.
--tenant <tenant>	name	no	UiPath tenant name.

Example

uip eval evaluation delete c1d2e3f4-0000-0000-0000-000000000001 \
  --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 --eval-set-id f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04

Data shape (--output json)

{ "Code": "EvaluationDeleted", "Data": { "EvaluationId": "c1d2e3f4-0000-0000-0000-000000000001" } }

Related

• uip eval — overview, execute-and-evaluate, and run.
• uip eval schedule — recurring evaluation runs against an eval set.
• uip eval evaluator — the scoring logic an eval set's --evaluator-refs point to.
• uip or processes — resolve a --process-key.

See also

• Orchestrator tool overview
• Global options
• Exit codes