uip eval eval-set & evaluation

Syntax and options for managing Orchestrator eval sets and the evaluations (data points) inside them with `uip eval eval-set` and `uip eval evaluation`.

Part of the uip eval subsystem. This page covers the two resources that define what gets evaluated:

• eval eval-set — a named, process-scoped collection of evaluation data points, linked to one or more evaluators via --evaluator-refs. An eval set also carries batching/timeout settings used when it's run.
• eval evaluation — one data point inside an eval set: an input payload, an optional expected output/behavior, and optional per-evaluator criteria overrides. execute-and-evaluate and schedule runs work through an eval set's evaluations, scoring each one with the eval set's linked evaluators.

概要

uip eval eval-set list --process-key <guid> [--limit <n>] [--offset <n>] [--tenant <tenant>]
uip eval eval-set get <evalSetId> --process-key <guid> [--tenant <tenant>]
uip eval eval-set create --process-key <guid> --workload-id <guid> --folder-key <guid> --name <name> [options...] [--tenant <tenant>]
uip eval eval-set update <evalSetId> --process-key <guid> [options...] [--tenant <tenant>]
uip eval eval-set delete <evalSetId> --process-key <guid> [--tenant <tenant>]

uip eval evaluation list --process-key <guid> --eval-set-id <guid> [--limit <n>] [--offset <n>] [--tenant <tenant>]
uip eval evaluation get <evaluationId> --process-key <guid> --eval-set-id <guid> [--tenant <tenant>]
uip eval evaluation create --process-key <guid> --eval-set-id <guid> --folder-key <guid> --name <name> --inputs <json> [options...] [--tenant <tenant>]
uip eval evaluation update <evaluationId> --process-key <guid> --eval-set-id <guid> [options...] [--tenant <tenant>]
uip eval evaluation delete <evaluationId> --process-key <guid> --eval-set-id <guid> [--tenant <tenant>]

uip eval eval-set list

List eval sets defined for a process.

オプション

長押し	値 (Value)	Required	説明
--process-key <guid>	GUID	○	Process key. Use uip or processes list to find one.
--limit <n>	integer	×	Maximum items to return. Default 100.
--offset <n>	integer	×	Items to skip. Default 0.
--tenant <tenant>	name	×	UiPath tenant name.

例

uip eval eval-set list --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09

データシェイプ(--output json)

{
  "Code": "EvalSetList",
  "Data": [
    {
      "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04",
      "Name": "Smoke Tests",
      "Description": "Basic smoke tests",
      "BatchSize": 10,
      "EvaluatorRefs": ["ev-1", "ev-2"],
      "CreatedAt": "2026-08-01T10:00:00Z"
    }
  ],
  "Pagination": { "Returned": 1, "Limit": 100, "Offset": 0 }
}

BatchSize is "-" when unset. UpdatedAt is present only once the eval set has been updated at least once.

uip eval eval-set get

Get one eval set's details.

引数

名前	Required	目的
<evalSetId>	○	Eval set ID (GUID).

オプション

長押し	値 (Value)	Required	説明
--process-key <guid>	GUID	○	Process key.
--tenant <tenant>	name	×	UiPath tenant name.

例

uip eval eval-set get f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04 --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09

データシェイプ(--output json)

{
  "Code": "EvalSetDetails",
  "Data": {
    "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04",
    "Name": "Smoke Tests",
    "Description": "Basic smoke tests",
    "BatchSize": 10,
    "EvaluatorRefs": ["ev-1", "ev-2"],
    "CreatedAt": "2026-08-01T10:00:00Z"
  }
}

uip eval eval-set create

Create an eval set for a process.

オプション

長押し	値 (Value)	Required	説明
--process-key <guid>	GUID	○	Process key.
--workload-id <guid>	GUID	○	Workload ID.
--folder-key <guid>	GUID	○	Folder key.
--name <name>	string	○	Eval set name.
--description <text>	string	×	Eval set description.
--batch-size <n>	integer ≥ 1	×	Max concurrent evaluations per run.
--timeout-minutes <n>	integer ≥ 1	×	Timeout in minutes per evaluation.
--evaluator-refs <refs...>	space-separated IDs	×	Evaluator IDs to link to this eval set. Variadic — pass multiple IDs separated by spaces, not commas.
--tenant <tenant>	name	×	UiPath tenant name.

An internal externalId (a random UUID) is generated automatically on every create — it isn't user-settable.

例

uip eval eval-set create --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 \
  --workload-id a1b2c3d4-0000-0000-0000-000000000001 \
  --folder-key f1f2f3f4-0000-0000-0000-000000000001 \
  --name "Smoke Tests" --evaluator-refs ev-1 ev-2

データシェイプ(--output json)

{
  "Code": "EvalSetCreated",
  "Data": { "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04", "Name": "Smoke Tests" }
}

uip eval eval-set update

Update an existing eval set. At least one updatable field is required beyond --process-key; fields you omit keep their current value (the command reads the existing eval set first, then applies your changes on top).

引数

名前	Required	目的
<evalSetId>	○	Eval set ID (GUID).

オプション

長押し	値 (Value)	Required	説明
--process-key <guid>	GUID	○	Process key.
--name <name>	string	no*	New eval set name.
--description <text>	string	no*	新しい説明。
--batch-size <n>	integer ≥ 1	no*	New max concurrent evaluations.
--timeout-minutes <n>	integer ≥ 1	no*	New timeout in minutes.
--evaluator-refs <refs...>	space-separated IDs	no*	Replaces the full evaluator-refs list (not additive).
--tenant <tenant>	name	×	UiPath tenant name.

* At least one of --name, --description, --batch-size, --timeout-minutes, or --evaluator-refs is required — the command fails fast if none are passed.

例

uip eval eval-set update f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04 \
  --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 \
  --name "Updated Name" --evaluator-refs ev-1 ev-2

データシェイプ(--output json)

{
  "Code": "EvalSetUpdated",
  "Data": {
    "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04",
    "Name": "Updated Name",
    "EvaluatorRefs": ["ev-1", "ev-2"]
  }
}

uip eval eval-set delete

Delete an eval set.

引数

名前	Required	目的
<evalSetId>	○	Eval set ID (GUID).

オプション

長押し	値 (Value)	Required	説明
--process-key <guid>	GUID	○	Process key.
--tenant <tenant>	name	×	UiPath tenant name.

例

uip eval eval-set delete f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04 --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09

データシェイプ(--output json)

{ "Code": "EvalSetDeleted", "Data": { "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04" } }

Deleting an eval set does not cascade-delete or otherwise warn about schedules or evaluators that reference it — the CLI issues a plain DELETE with no dependency check.

uip eval evaluation list

List the evaluations (data points) inside an eval set.

オプション

長押し	値 (Value)	Required	説明
--process-key <guid>	GUID	○	Process key.
--eval-set-id <guid>	GUID	○	Eval set ID.
--limit <n>	integer	×	Maximum items to return. Default 100.
--offset <n>	integer	×	Items to skip. Default 0.
--tenant <tenant>	name	×	UiPath tenant name.

例

uip eval evaluation list --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 --eval-set-id f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04

データシェイプ(--output json)

{
  "Code": "EvaluationList",
  "Data": [
    {
      "EvaluationId": "c1d2e3f4-0000-0000-0000-000000000001",
      "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04",
      "Name": "Greeting test",
      "Inputs": { "input": "hello" },
      "ExpectedOutput": { "content": "Hi there!" },
      "ExpectedBehavior": "",
      "CreatedAt": "2026-08-01T10:00:00Z"
    }
  ],
  "Pagination": { "Returned": 1, "Limit": 100, "Offset": 0 }
}

uip eval evaluation get

Get one evaluation's details.

引数

名前	Required	目的
<evaluationId>	○	Evaluation ID (GUID).

オプション

長押し	値 (Value)	Required	説明
--process-key <guid>	GUID	○	Process key.
--eval-set-id <guid>	GUID	○	Eval set ID.
--tenant <tenant>	name	×	UiPath tenant name.

例

uip eval evaluation get c1d2e3f4-0000-0000-0000-000000000001 \
  --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 --eval-set-id f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04

データシェイプ(--output json)

Same object shape as one entry in evaluation list's Data array.

uip eval evaluation create

Add an evaluation (data point) to an eval set.

オプション

長押し	値 (Value)	Required	説明
--process-key <guid>	GUID	○	Process key.
--eval-set-id <guid>	GUID	○	Eval set ID.
--folder-key <guid>	GUID	○	Folder key.
--name <name>	string	○	Evaluation name.
--inputs <json>	JSON	○	Input values, e.g. '{"input":"hello"}'.
--expected-output <json>	JSON	×	Expected output for comparison-style evaluators.
--expected-behavior <text>	string	×	Expected agent behavior description, for trajectory evaluators.
--evaluation-criterias <json>	JSON	×	Per-evaluator criteria overrides.
--tenant <tenant>	name	×	UiPath tenant name.

Malformed JSON on --inputs, --expected-output, or --evaluation-criterias fails fast client-side before any network call, with an example of the expected shape in the error for --inputs.

例

uip eval evaluation create --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 \
  --eval-set-id f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04 \
  --folder-key f1f2f3f4-0000-0000-0000-000000000001 \
  --name "Greeting test" --inputs '{"input":"hello"}' --expected-output '{"content":"Hi there!"}'

データシェイプ(--output json)

{
  "Code": "EvaluationCreated",
  "Data": {
    "EvaluationId": "c1d2e3f4-0000-0000-0000-000000000001",
    "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04",
    "Name": "Greeting test"
  }
}

uip eval evaluation update

Update an existing evaluation. At least one updatable field is required beyond --process-key/--eval-set-id; omitted fields keep their current value.

引数

名前	Required	目的
<evaluationId>	○	Evaluation ID (GUID).

オプション

長押し	値 (Value)	Required	説明
--process-key <guid>	GUID	○	Process key.
--eval-set-id <guid>	GUID	○	Eval set ID.
--name <name>	string	no*	New evaluation name.
--inputs <json>	JSON	no*	New input values.
--expected-output <json>	JSON	no*	New expected output.
--expected-behavior <text>	string	no*	New expected behavior description.
--evaluation-criterias <json>	JSON	no*	New per-evaluator criteria overrides.
--tenant <tenant>	name	×	UiPath tenant name.

* At least one of --name, --inputs, --expected-output, --expected-behavior, or --evaluation-criterias is required.

例

uip eval evaluation update c1d2e3f4-0000-0000-0000-000000000001 \
  --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 --eval-set-id f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04 \
  --inputs '{"input":"updated"}' --expected-output '{"content":"new response"}'

データシェイプ(--output json)

{
  "Code": "EvaluationUpdated",
  "Data": {
    "EvaluationId": "c1d2e3f4-0000-0000-0000-000000000001",
    "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04",
    "Name": "Greeting test"
  }
}

uip eval evaluation delete

Remove an evaluation from an eval set.

引数

名前	Required	目的
<evaluationId>	○	Evaluation ID (GUID).

オプション

長押し	値 (Value)	Required	説明
--process-key <guid>	GUID	○	Process key.
--eval-set-id <guid>	GUID	○	Eval set ID.
--tenant <tenant>	name	×	UiPath tenant name.

例

uip eval evaluation delete c1d2e3f4-0000-0000-0000-000000000001 \
  --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 --eval-set-id f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04

データシェイプ(--output json)

{ "Code": "EvaluationDeleted", "Data": { "EvaluationId": "c1d2e3f4-0000-0000-0000-000000000001" } }

関連

• uip eval — overview, execute-and-evaluate, and run.
• uip eval schedule — recurring evaluation runs against an eval set.
• uip eval evaluator — the scoring logic an eval set's --evaluator-refs point to.
• uip or processes — resolve a --process-key.

参照

• Orchestrator tool overview
• グローバル オプション
• 終了コード