uip eval evaluator

Syntax and options for `uip eval evaluator`, which manages the scoring evaluators used to grade Orchestrator process runs.

An evaluator is a scoring mechanism attached to a process (by --process-key) that grades a run's output — for example, an LLM-judge comparison against an expected answer, or an exact-match check. Evaluators are referenced by ID from eval-set entries and from execute-and-evaluate calls (see the eval overview), which is what actually runs a scored evaluation. This page only covers authoring/managing evaluator definitions themselves.

Every verb requires --process-key <guid> — evaluators are scoped to one process. Find process keys with uip or processes list.

Synopsis

uip eval evaluator list --process-key <guid> [--limit <number>] [--offset <number>] [--tenant <tenant>]
uip eval evaluator get <evaluatorId> --process-key <guid> [--tenant <tenant>]
uip eval evaluator create --process-key <guid> --workload-id <guid> --folder-key <guid> --name <name> --description <text> --evaluator-type-id <id> --evaluator-config <json> [--version <version>] [--tenant <tenant>]
uip eval evaluator update <evaluatorId> --process-key <guid> [--name <name>] [--description <text>] [--evaluator-type-id <id>] [--evaluator-config <json>] [--version <version>] [--tenant <tenant>]
uip eval evaluator delete <evaluatorId> --process-key <guid> [--tenant <tenant>]

uip eval evaluator list

List evaluators defined for a process.

Options

Long	Value	Required	Description
--process-key <guid>	GUID	yes	Process key. Use uip or processes list to find keys.
--limit <number>	integer	no	Maximum number of items to return. Default 100.
--offset <number>	integer	no	Number of items to skip. Default 0.
--tenant <tenant>	name	no	UiPath tenant name.

Example

uip eval evaluator list --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09

Data shape (--output json)

{
  "Code": "EvaluatorList",
  "Data": [
    {
      "EvaluatorId": "a1b2c3d4-0000-0000-0000-000000000001",
      "Name": "Semantic Similarity",
      "Description": "LLM-based output comparison",
      "EvaluatorTypeId": "uipath-llm-judge-output-semantic-similarity",
      "Version": "1.0",
      "CreatedAt": "2026-08-01T10:00:00Z"
    }
  ],
  "Pagination": { "Returned": 1, "Limit": 100, "Offset": 0 }
}

UpdatedAt is included per item only when the evaluator has been modified since creation.

uip eval evaluator get

Get one evaluator's full details by ID.

Arguments

Name	Required	Purpose
<evaluatorId>	yes	Evaluator ID (GUID).

Options

Long	Value	Required	Description
--process-key <guid>	GUID	yes	Process key.
--tenant <tenant>	name	no	UiPath tenant name.

Example

uip eval evaluator get a1b2c3d4-0000-0000-0000-000000000001 --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09

Data shape (--output json)

{
  "Code": "EvaluatorDetails",
  "Data": {
    "EvaluatorId": "a1b2c3d4-0000-0000-0000-000000000001",
    "Name": "Semantic Similarity",
    "Description": "LLM-based output comparison",
    "EvaluatorTypeId": "uipath-llm-judge-output-semantic-similarity",
    "Version": "1.0",
    "CreatedAt": "2026-08-01T10:00:00Z"
  }
}

uip eval evaluator create

Create an evaluator on a process.

Options

Long	Value	Required	Description
--process-key <guid>	GUID	yes	Process key.
--workload-id <guid>	GUID	yes	Workload ID.
--folder-key <guid>	GUID	yes	Folder key.
--name <name>	string	yes	Evaluator name.
--description <text>	string	yes	Evaluator description.
--evaluator-type-id <id>	string	yes	Evaluator type. Source's own examples show uipath-exact-match, uipath-llm-judge-output-semantic-similarity, uipath-llm-judge-trajectory-similarity — this is not a client-validated enum, so other type IDs may exist server-side; the three shown are simply what the CLI's own help text names.
--evaluator-config <json>	JSON object	yes	Type-specific configuration. Shape depends on --evaluator-type-id — the semantic-similarity example below shows name/prompt/model/targetOutputKey; other types likely take different keys, not enumerated client-side.
--version <version>	string	no	Evaluator version. Default 1.0.
--tenant <tenant>	name	no	UiPath tenant name.

An externalId (UUID) is generated automatically on every create — it isn't a flag, and isn't shown in list/get output.

Example

uip eval evaluator create --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 \
  --workload-id a1b2c3d4-0000-0000-0000-000000000001 \
  --folder-key f1f2f3f4-0000-0000-0000-000000000001 \
  --name "Semantic Similarity" --description "LLM-based output comparison" \
  --evaluator-type-id uipath-llm-judge-output-semantic-similarity \
  --evaluator-config '{"name":"Semantic","prompt":"Score 0-100...","model":"gpt-4.1-2025-04-14","targetOutputKey":"*"}' \
  --version 1.0

Data shape (--output json)

{
  "Code": "EvaluatorCreated",
  "Data": {
    "EvaluatorId": "a1b2c3d4-0000-0000-0000-000000000001",
    "Name": "Semantic Similarity",
    "EvaluatorTypeId": "uipath-llm-judge-output-semantic-similarity"
  }
}

uip eval evaluator update

Update one or more fields on an existing evaluator. Unspecified fields are read from the current evaluator and re-sent unchanged (a full-object PUT under the hood, not a partial patch) — --evaluator-config itself is not merged: passing it replaces the entire config object, not just the keys you name.

Arguments

Name	Required	Purpose
<evaluatorId>	yes	Evaluator ID (GUID).

Options

Long	Value	Required	Description
--process-key <guid>	GUID	yes	Process key.
--name <name>	string	no*	New name.
--description <text>	string	no*	New description.
--evaluator-type-id <id>	string	no*	New evaluator type.
--evaluator-config <json>	JSON object	no*	New config — replaces the whole object.
--version <version>	string	no*	New version.
--tenant <tenant>	name	no	UiPath tenant name.

* At least one of --name/--description/--evaluator-type-id/--evaluator-config/--version is required — omitting all five fails with an explicit error before any network call.

Example

uip eval evaluator update a1b2c3d4-0000-0000-0000-000000000001 \
  --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09 \
  --name "Updated Evaluator" \
  --evaluator-config '{"name":"Updated","prompt":"New prompt"}'

Data shape (--output json)

{
  "Code": "EvaluatorUpdated",
  "Data": {
    "EvaluatorId": "a1b2c3d4-0000-0000-0000-000000000001",
    "Name": "Updated Evaluator",
    "EvaluatorTypeId": "uipath-llm-judge-output-semantic-similarity"
  }
}

uip eval evaluator delete

Delete an evaluator.

Note:

Deleting an evaluator does not check whether it's still referenced by an eval-set entry or a schedule — the API performs no reference check client-side. Confirm nothing depends on it first.

Arguments

Name	Required	Purpose
<evaluatorId>	yes	Evaluator ID (GUID).

Options

Long	Value	Required	Description
--process-key <guid>	GUID	yes	Process key.
--tenant <tenant>	name	no	UiPath tenant name.

Example

uip eval evaluator delete a1b2c3d4-0000-0000-0000-000000000001 --process-key 9e4b2f17-7c3a-4d81-b592-3f6e8a1d5c09

Data shape (--output json)

{
  "Code": "EvaluatorDeleted",
  "Data": { "EvaluatorId": "a1b2c3d4-0000-0000-0000-000000000001" }
}

Related

• uip eval — overview, execute-and-evaluate, and run.
• uip eval schedule — recurring evaluation runs.
• uip eval-set / evaluation — grouping evaluators into a reusable evaluation configuration.
• uip or processes — find process keys.

See also

• Orchestrator tool overview
• Global options
• Exit codes