uip eval eval-set & evaluation

Syntax and options for managing Orchestrator eval sets and the evaluations (data points) inside them with `uip eval eval-set` and `uip eval evaluation`.

Part of the uip eval subsystem. This page covers the two resources that define what gets evaluated:

• eval eval-set — a named, process-scoped collection of evaluation data points, linked to one or more evaluators via --evaluator-refs. An eval set also carries batching/timeout settings used when it's run.
• eval evaluation — one data point inside an eval set: an input payload, an optional expected output/behavior, and optional per-evaluator criteria overrides. execute-and-evaluate and schedule runs work through an eval set's evaluations, scoring each one with the eval set's linked evaluators.

大纲

uip eval eval-set list --process-key <guid> [--limit <n>] [--offset <n>] [--tenant <tenant>]
uip eval eval-set get <evalSetId> --process-key <guid> [--tenant <tenant>]
uip eval eval-set create --process-key <guid> --workload-id <guid> --folder-key <guid> --name <name> [options...] [--tenant <tenant>]
uip eval eval-set update <evalSetId> --process-key <guid> [options...] [--tenant <tenant>]
uip eval eval-set delete <evalSetId> --process-key <guid> [--tenant <tenant>]

uip eval evaluation list --process-key <guid> --eval-set-id <guid> [--limit <n>] [--offset <n>] [--tenant <tenant>]
uip eval evaluation get <evaluationId> --process-key <guid> --eval-set-id <guid> [--tenant <tenant>]
uip eval evaluation create --process-key <guid> --eval-set-id <guid> --folder-key <guid> --name <name> --inputs <json> [options...] [--tenant <tenant>]
uip eval evaluation update <evaluationId> --process-key <guid> --eval-set-id <guid> [options...] [--tenant <tenant>]
uip eval evaluation delete <evaluationId> --process-key <guid> --eval-set-id <guid> [--tenant <tenant>]

uip eval eval-set list

List eval sets defined for a process.

选项

长	值	必填	描述
--process-key <guid>	GUID	是	Process key. Use uip or processes list to find one.
--limit <n>	整数	否	Maximum items to return. Default 100.
--offset <n>	整数	否	Items to skip. Default 0.
--tenant <tenant>	name	否	UiPath tenant name.

示例

uip eval eval-set list --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09

数据形状（--输出 json）

{
  "Code": "EvalSetList",
  "Data": [
    {
      "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04",
      "Name": "Smoke Tests",
      "Description": "Basic smoke tests",
      "BatchSize": 10,
      "EvaluatorRefs": ["ev-1", "ev-2"],
      "CreatedAt": "2026-08-01T10:00:00Z"
    }
  ],
  "Pagination": { "Returned": 1, "Limit": 100, "Offset": 0 }
}

BatchSize is "-" when unset. UpdatedAt is present only once the eval set has been updated at least once.

uip eval eval-set get

Get one eval set's details.

参数

名称	必填	用途
<evalSetId>	是	Eval set ID (GUID).

选项

长	值	必填	描述
--process-key <guid>	GUID	是	Process key.
--tenant <tenant>	name	否	UiPath tenant name.

示例

uip eval eval-set get f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04 --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09

数据形状（--输出 json）

{
  "Code": "EvalSetDetails",
  "Data": {
    "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04",
    "Name": "Smoke Tests",
    "Description": "Basic smoke tests",
    "BatchSize": 10,
    "EvaluatorRefs": ["ev-1", "ev-2"],
    "CreatedAt": "2026-08-01T10:00:00Z"
  }
}

uip eval eval-set create

Create an eval set for a process.

选项

长	值	必填	描述
--process-key <guid>	GUID	是	Process key.
--workload-id <guid>	GUID	是	Workload ID.
--folder-key <guid>	GUID	是	Folder key.
--name <name>	字符串	是	Eval set name.
--description <text>	字符串	否	Eval set description.
--batch-size <n>	integer ≥ 1	否	Max concurrent evaluations per run.
--timeout-minutes <n>	integer ≥ 1	否	Timeout in minutes per evaluation.
--evaluator-refs <refs...>	space-separated IDs	否	Evaluator IDs to link to this eval set. Variadic — pass multiple IDs separated by spaces, not commas.
--tenant <tenant>	name	否	UiPath tenant name.

An internal externalId (a random UUID) is generated automatically on every create — it isn't user-settable.

示例

uip eval eval-set create --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 \
  --workload-id a1b2c3d4-0000-0000-0000-000000000001 \
  --folder-key f1f2f3f4-0000-0000-0000-000000000001 \
  --name "Smoke Tests" --evaluator-refs ev-1 ev-2

数据形状（--输出 json）

{
  "Code": "EvalSetCreated",
  "Data": { "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04", "Name": "Smoke Tests" }
}

uip eval eval-set update

Update an existing eval set. At least one updatable field is required beyond --process-key; fields you omit keep their current value (the command reads the existing eval set first, then applies your changes on top).

参数

名称	必填	用途
<evalSetId>	是	Eval set ID (GUID).

选项

长	值	必填	描述
--process-key <guid>	GUID	是	Process key.
--name <name>	字符串	no*	New eval set name.
--description <text>	字符串	no*	新说明。
--batch-size <n>	integer ≥ 1	no*	New max concurrent evaluations.
--timeout-minutes <n>	integer ≥ 1	no*	New timeout in minutes.
--evaluator-refs <refs...>	space-separated IDs	no*	Replaces the full evaluator-refs list (not additive).
--tenant <tenant>	name	否	UiPath tenant name.

* At least one of --name, --description, --batch-size, --timeout-minutes, or --evaluator-refs is required — the command fails fast if none are passed.

示例

uip eval eval-set update f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04 \
  --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 \
  --name "Updated Name" --evaluator-refs ev-1 ev-2

数据形状（--输出 json）

{
  "Code": "EvalSetUpdated",
  "Data": {
    "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04",
    "Name": "Updated Name",
    "EvaluatorRefs": ["ev-1", "ev-2"]
  }
}

uip eval eval-set delete

Delete an eval set.

参数

名称	必填	用途
<evalSetId>	是	Eval set ID (GUID).

选项

长	值	必填	描述
--process-key <guid>	GUID	是	Process key.
--tenant <tenant>	name	否	UiPath tenant name.

示例

uip eval eval-set delete f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04 --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09

数据形状（--输出 json）

{ "Code": "EvalSetDeleted", "Data": { "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04" } }

Deleting an eval set does not cascade-delete or otherwise warn about schedules or evaluators that reference it — the CLI issues a plain DELETE with no dependency check.

uip eval evaluation list

List the evaluations (data points) inside an eval set.

选项

长	值	必填	描述
--process-key <guid>	GUID	是	Process key.
--eval-set-id <guid>	GUID	是	Eval set ID.
--limit <n>	整数	否	Maximum items to return. Default 100.
--offset <n>	整数	否	Items to skip. Default 0.
--tenant <tenant>	name	否	UiPath tenant name.

示例

uip eval evaluation list --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 --eval-set-id f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04

数据形状（--输出 json）

{
  "Code": "EvaluationList",
  "Data": [
    {
      "EvaluationId": "c1d2e3f4-0000-0000-0000-000000000001",
      "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04",
      "Name": "Greeting test",
      "Inputs": { "input": "hello" },
      "ExpectedOutput": { "content": "Hi there!" },
      "ExpectedBehavior": "",
      "CreatedAt": "2026-08-01T10:00:00Z"
    }
  ],
  "Pagination": { "Returned": 1, "Limit": 100, "Offset": 0 }
}

uip eval evaluation get

Get one evaluation's details.

参数

名称	必填	用途
<evaluationId>	是	Evaluation ID (GUID).

选项

长	值	必填	描述
--process-key <guid>	GUID	是	Process key.
--eval-set-id <guid>	GUID	是	Eval set ID.
--tenant <tenant>	name	否	UiPath tenant name.

示例

uip eval evaluation get c1d2e3f4-0000-0000-0000-000000000001 \
  --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 --eval-set-id f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04

数据形状（--输出 json）

Same object shape as one entry in evaluation list's Data array.

uip eval evaluation create

Add an evaluation (data point) to an eval set.

选项

长	值	必填	描述
--process-key <guid>	GUID	是	Process key.
--eval-set-id <guid>	GUID	是	Eval set ID.
--folder-key <guid>	GUID	是	Folder key.
--name <name>	字符串	是	Evaluation name.
--inputs <json>	JSON	是	Input values, e.g. '{"input":"hello"}'.
--expected-output <json>	JSON	否	Expected output for comparison-style evaluators.
--expected-behavior <text>	字符串	否	Expected agent behavior description, for trajectory evaluators.
--evaluation-criterias <json>	JSON	否	Per-evaluator criteria overrides.
--tenant <tenant>	name	否	UiPath tenant name.

Malformed JSON on --inputs, --expected-output, or --evaluation-criterias fails fast client-side before any network call, with an example of the expected shape in the error for --inputs.

示例

uip eval evaluation create --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 \
  --eval-set-id f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04 \
  --folder-key f1f2f3f4-0000-0000-0000-000000000001 \
  --name "Greeting test" --inputs '{"input":"hello"}' --expected-output '{"content":"Hi there!"}'

数据形状（--输出 json）

{
  "Code": "EvaluationCreated",
  "Data": {
    "EvaluationId": "c1d2e3f4-0000-0000-0000-000000000001",
    "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04",
    "Name": "Greeting test"
  }
}

uip eval evaluation update

Update an existing evaluation. At least one updatable field is required beyond --process-key/--eval-set-id; omitted fields keep their current value.

参数

名称	必填	用途
<evaluationId>	是	Evaluation ID (GUID).

选项

长	值	必填	描述
--process-key <guid>	GUID	是	Process key.
--eval-set-id <guid>	GUID	是	Eval set ID.
--name <name>	字符串	no*	New evaluation name.
--inputs <json>	JSON	no*	New input values.
--expected-output <json>	JSON	no*	New expected output.
--expected-behavior <text>	字符串	no*	New expected behavior description.
--evaluation-criterias <json>	JSON	no*	New per-evaluator criteria overrides.
--tenant <tenant>	name	否	UiPath tenant name.

* At least one of --name, --inputs, --expected-output, --expected-behavior, or --evaluation-criterias is required.

示例

uip eval evaluation update c1d2e3f4-0000-0000-0000-000000000001 \
  --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 --eval-set-id f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04 \
  --inputs '{"input":"updated"}' --expected-output '{"content":"new response"}'

数据形状（--输出 json）

{
  "Code": "EvaluationUpdated",
  "Data": {
    "EvaluationId": "c1d2e3f4-0000-0000-0000-000000000001",
    "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04",
    "Name": "Greeting test"
  }
}

uip eval evaluation delete

Remove an evaluation from an eval set.

参数

名称	必填	用途
<evaluationId>	是	Evaluation ID (GUID).

选项

长	值	必填	描述
--process-key <guid>	GUID	是	Process key.
--eval-set-id <guid>	GUID	是	Eval set ID.
--tenant <tenant>	name	否	UiPath tenant name.

示例

uip eval evaluation delete c1d2e3f4-0000-0000-0000-000000000001 \
  --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 --eval-set-id f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04

数据形状（--输出 json）

{ "Code": "EvaluationDeleted", "Data": { "EvaluationId": "c1d2e3f4-0000-0000-0000-000000000001" } }

相关内容

• uip eval — overview, execute-and-evaluate, and run.
• uip eval schedule — recurring evaluation runs against an eval set.
• uip eval evaluator — the scoring logic an eval set's --evaluator-refs point to.
• uip or processes — resolve a --process-key.

另请参阅

• Orchestrator tool overview
• 全局选项
• 退出代码