uip eval eval-set & evaluation

Syntax and options for managing Orchestrator eval sets and the evaluations (data points) inside them with `uip eval eval-set` and `uip eval evaluation`.

Part of the uip eval subsystem. This page covers the two resources that define what gets evaluated:

• eval eval-set — a named, process-scoped collection of evaluation data points, linked to one or more evaluators via --evaluator-refs. An eval set also carries batching/timeout settings used when it's run.
• eval evaluation — one data point inside an eval set: an input payload, an optional expected output/behavior, and optional per-evaluator criteria overrides. execute-and-evaluate and schedule runs work through an eval set's evaluations, scoring each one with the eval set's linked evaluators.

Synthèse

uip eval eval-set list --process-key <guid> [--limit <n>] [--offset <n>] [--tenant <tenant>]
uip eval eval-set get <evalSetId> --process-key <guid> [--tenant <tenant>]
uip eval eval-set create --process-key <guid> --workload-id <guid> --folder-key <guid> --name <name> [options...] [--tenant <tenant>]
uip eval eval-set update <evalSetId> --process-key <guid> [options...] [--tenant <tenant>]
uip eval eval-set delete <evalSetId> --process-key <guid> [--tenant <tenant>]

uip eval evaluation list --process-key <guid> --eval-set-id <guid> [--limit <n>] [--offset <n>] [--tenant <tenant>]
uip eval evaluation get <evaluationId> --process-key <guid> --eval-set-id <guid> [--tenant <tenant>]
uip eval evaluation create --process-key <guid> --eval-set-id <guid> --folder-key <guid> --name <name> --inputs <json> [options...] [--tenant <tenant>]
uip eval evaluation update <evaluationId> --process-key <guid> --eval-set-id <guid> [options...] [--tenant <tenant>]
uip eval evaluation delete <evaluationId> --process-key <guid> --eval-set-id <guid> [--tenant <tenant>]

uip eval eval-set list

List eval sets defined for a process.

Options

Long	Valeur (Value)	Requis	Description
--process-key <guid>	GUID	oui	Process key. Use uip or processes list to find one.
--limit <n>	Integer	non	Maximum items to return. Default 100.
--offset <n>	Integer	non	Items to skip. Default 0.
--tenant <tenant>	name	non	UiPath tenant name.

Exemple

uip eval eval-set list --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09

Format des données (--sortie json)

{
  "Code": "EvalSetList",
  "Data": [
    {
      "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04",
      "Name": "Smoke Tests",
      "Description": "Basic smoke tests",
      "BatchSize": 10,
      "EvaluatorRefs": ["ev-1", "ev-2"],
      "CreatedAt": "2026-08-01T10:00:00Z"
    }
  ],
  "Pagination": { "Returned": 1, "Limit": 100, "Offset": 0 }
}

BatchSize is "-" when unset. UpdatedAt is present only once the eval set has been updated at least once.

uip eval eval-set get

Get one eval set's details.

Arguments

Nom	Requis	Objectif
<evalSetId>	oui	Eval set ID (GUID).

Options

Long	Valeur (Value)	Requis	Description
--process-key <guid>	GUID	oui	Process key.
--tenant <tenant>	name	non	UiPath tenant name.

Exemple

uip eval eval-set get f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04 --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09

Format des données (--sortie json)

{
  "Code": "EvalSetDetails",
  "Data": {
    "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04",
    "Name": "Smoke Tests",
    "Description": "Basic smoke tests",
    "BatchSize": 10,
    "EvaluatorRefs": ["ev-1", "ev-2"],
    "CreatedAt": "2026-08-01T10:00:00Z"
  }
}

uip eval eval-set create

Create an eval set for a process.

Options

Long	Valeur (Value)	Requis	Description
--process-key <guid>	GUID	oui	Process key.
--workload-id <guid>	GUID	oui	Workload ID.
--folder-key <guid>	GUID	oui	Folder key.
--name <name>	string	oui	Eval set name.
--description <text>	string	non	Eval set description.
--batch-size <n>	integer ≥ 1	non	Max concurrent evaluations per run.
--timeout-minutes <n>	integer ≥ 1	non	Timeout in minutes per evaluation.
--evaluator-refs <refs...>	space-separated IDs	non	Evaluator IDs to link to this eval set. Variadic — pass multiple IDs separated by spaces, not commas.
--tenant <tenant>	name	non	UiPath tenant name.

An internal externalId (a random UUID) is generated automatically on every create — it isn't user-settable.

Exemple

uip eval eval-set create --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 \
  --workload-id a1b2c3d4-0000-0000-0000-000000000001 \
  --folder-key f1f2f3f4-0000-0000-0000-000000000001 \
  --name "Smoke Tests" --evaluator-refs ev-1 ev-2

Format des données (--sortie json)

{
  "Code": "EvalSetCreated",
  "Data": { "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04", "Name": "Smoke Tests" }
}

uip eval eval-set update

Update an existing eval set. At least one updatable field is required beyond --process-key; fields you omit keep their current value (the command reads the existing eval set first, then applies your changes on top).

Arguments

Nom	Requis	Objectif
<evalSetId>	oui	Eval set ID (GUID).

Options

Long	Valeur (Value)	Requis	Description
--process-key <guid>	GUID	oui	Process key.
--name <name>	string	no*	New eval set name.
--description <text>	string	no*	Nouvelle description.
--batch-size <n>	integer ≥ 1	no*	New max concurrent evaluations.
--timeout-minutes <n>	integer ≥ 1	no*	New timeout in minutes.
--evaluator-refs <refs...>	space-separated IDs	no*	Replaces the full evaluator-refs list (not additive).
--tenant <tenant>	name	non	UiPath tenant name.

* At least one of --name, --description, --batch-size, --timeout-minutes, or --evaluator-refs is required — the command fails fast if none are passed.

Exemple

uip eval eval-set update f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04 \
  --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 \
  --name "Updated Name" --evaluator-refs ev-1 ev-2

Format des données (--sortie json)

{
  "Code": "EvalSetUpdated",
  "Data": {
    "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04",
    "Name": "Updated Name",
    "EvaluatorRefs": ["ev-1", "ev-2"]
  }
}

uip eval eval-set delete

Delete an eval set.

Arguments

Nom	Requis	Objectif
<evalSetId>	oui	Eval set ID (GUID).

Options

Long	Valeur (Value)	Requis	Description
--process-key <guid>	GUID	oui	Process key.
--tenant <tenant>	name	non	UiPath tenant name.

Exemple

uip eval eval-set delete f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04 --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09

Format des données (--sortie json)

{ "Code": "EvalSetDeleted", "Data": { "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04" } }

Deleting an eval set does not cascade-delete or otherwise warn about schedules or evaluators that reference it — the CLI issues a plain DELETE with no dependency check.

uip eval evaluation list

List the evaluations (data points) inside an eval set.

Options

Long	Valeur (Value)	Requis	Description
--process-key <guid>	GUID	oui	Process key.
--eval-set-id <guid>	GUID	oui	Eval set ID.
--limit <n>	Integer	non	Maximum items to return. Default 100.
--offset <n>	Integer	non	Items to skip. Default 0.
--tenant <tenant>	name	non	UiPath tenant name.

Exemple

uip eval evaluation list --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 --eval-set-id f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04

Format des données (--sortie json)

{
  "Code": "EvaluationList",
  "Data": [
    {
      "EvaluationId": "c1d2e3f4-0000-0000-0000-000000000001",
      "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04",
      "Name": "Greeting test",
      "Inputs": { "input": "hello" },
      "ExpectedOutput": { "content": "Hi there!" },
      "ExpectedBehavior": "",
      "CreatedAt": "2026-08-01T10:00:00Z"
    }
  ],
  "Pagination": { "Returned": 1, "Limit": 100, "Offset": 0 }
}

uip eval evaluation get

Get one evaluation's details.

Arguments

Nom	Requis	Objectif
<evaluationId>	oui	Evaluation ID (GUID).

Options

Long	Valeur (Value)	Requis	Description
--process-key <guid>	GUID	oui	Process key.
--eval-set-id <guid>	GUID	oui	Eval set ID.
--tenant <tenant>	name	non	UiPath tenant name.

Exemple

uip eval evaluation get c1d2e3f4-0000-0000-0000-000000000001 \
  --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 --eval-set-id f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04

Format des données (--sortie json)

Same object shape as one entry in evaluation list's Data array.

uip eval evaluation create

Add an evaluation (data point) to an eval set.

Options

Long	Valeur (Value)	Requis	Description
--process-key <guid>	GUID	oui	Process key.
--eval-set-id <guid>	GUID	oui	Eval set ID.
--folder-key <guid>	GUID	oui	Folder key.
--name <name>	string	oui	Evaluation name.
--inputs <json>	JSON	oui	Input values, e.g. '{"input":"hello"}'.
--expected-output <json>	JSON	non	Expected output for comparison-style evaluators.
--expected-behavior <text>	string	non	Expected agent behavior description, for trajectory evaluators.
--evaluation-criterias <json>	JSON	non	Per-evaluator criteria overrides.
--tenant <tenant>	name	non	UiPath tenant name.

Malformed JSON on --inputs, --expected-output, or --evaluation-criterias fails fast client-side before any network call, with an example of the expected shape in the error for --inputs.

Exemple

uip eval evaluation create --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 \
  --eval-set-id f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04 \
  --folder-key f1f2f3f4-0000-0000-0000-000000000001 \
  --name "Greeting test" --inputs '{"input":"hello"}' --expected-output '{"content":"Hi there!"}'

Format des données (--sortie json)

{
  "Code": "EvaluationCreated",
  "Data": {
    "EvaluationId": "c1d2e3f4-0000-0000-0000-000000000001",
    "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04",
    "Name": "Greeting test"
  }
}

uip eval evaluation update

Update an existing evaluation. At least one updatable field is required beyond --process-key/--eval-set-id; omitted fields keep their current value.

Arguments

Nom	Requis	Objectif
<evaluationId>	oui	Evaluation ID (GUID).

Options

Long	Valeur (Value)	Requis	Description
--process-key <guid>	GUID	oui	Process key.
--eval-set-id <guid>	GUID	oui	Eval set ID.
--name <name>	string	no*	New evaluation name.
--inputs <json>	JSON	no*	New input values.
--expected-output <json>	JSON	no*	New expected output.
--expected-behavior <text>	string	no*	New expected behavior description.
--evaluation-criterias <json>	JSON	no*	New per-evaluator criteria overrides.
--tenant <tenant>	name	non	UiPath tenant name.

* At least one of --name, --inputs, --expected-output, --expected-behavior, or --evaluation-criterias is required.

Exemple

uip eval evaluation update c1d2e3f4-0000-0000-0000-000000000001 \
  --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 --eval-set-id f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04 \
  --inputs '{"input":"updated"}' --expected-output '{"content":"new response"}'

Format des données (--sortie json)

{
  "Code": "EvaluationUpdated",
  "Data": {
    "EvaluationId": "c1d2e3f4-0000-0000-0000-000000000001",
    "EvalSetId": "f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04",
    "Name": "Greeting test"
  }
}

uip eval evaluation delete

Remove an evaluation from an eval set.

Arguments

Nom	Requis	Objectif
<evaluationId>	oui	Evaluation ID (GUID).

Options

Long	Valeur (Value)	Requis	Description
--process-key <guid>	GUID	oui	Process key.
--eval-set-id <guid>	GUID	oui	Eval set ID.
--tenant <tenant>	name	non	UiPath tenant name.

Exemple

uip eval evaluation delete c1d2e3f4-0000-0000-0000-000000000001 \
  --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 --eval-set-id f3a7d219-8b4c-4e62-a951-7d3f6e2c8b04

Format des données (--sortie json)

{ "Code": "EvaluationDeleted", "Data": { "EvaluationId": "c1d2e3f4-0000-0000-0000-000000000001" } }

Associé

• uip eval — overview, execute-and-evaluate, and run.
• uip eval schedule — recurring evaluation runs against an eval set.
• uip eval evaluator — the scoring logic an eval set's --evaluator-refs point to.
• uip or processes — resolve a --process-key.

Voir également

• Orchestrator tool overview
• Options globales
• Codes de sortie