Subscribe

UiPath Insights

The UiPath Insights Guide

Performance and scalability

Overview


This topic covers performance and scalability metrics for both Insights Standalone and the Automation Suite deployment models, intended for administrators. Performance examines ways in which data is ingested and processed in the background, alongside dashboard loading times and hardware requirements.

The largest scale at which Insights can operate


The assumption of the total events that are supported over a two year period, with the current hardware requirements is the following:

  • Up to 100 million jobs;
  • Up to 1 billion job events;
  • Up to 100 million queue items;
  • Up to 500 million queue item events;
  • Up to 1 billion robot logs.

Performance test data set


A performance test has been conducted with the following configuration:

  • 100 million Jobs, 1 billion job events, 1 billion robot logs, 100 million queue items, 500 million queue item events. For each job, there are 10 related job events, 10 robot logs, 1 queue item, and 5 queue item events.
  • On average, the raw robot logs size is 4KB each, while the queue item (including custom data via Analytics, Output, and Specific Data) is 9KB.
  • There are 400 processes, of which a single process generates 20% of the data.
  • The total disk usage in Azure SQL is about 5.7TB.

Initial size of data

#Database Table NameSchema NameNumber of RowsUsed Disk Space (GB)
1RobotLogsdbo10000044164095.45
2QueueItemsdbo106332315916.56
3JobEventsdbo1022790964499.09
4QueueItemEventsdbo520882100200.08
5Jobsdbo10569787828.33

Size of data after backfilling (moving data from the raw store in dbo to columnstore read)

🚧

Important

For optimal performance, tables in the read schema use a columnstore index, meaning that the data is compressed so the size consumption is decreased compared to the dbo schema.

#Database Table NameSchema NameNumber of RowsUsed Disk Space (GB)
1RobotLogsread100000441045.71
2QueueItemsread106328486.5
3JobEventsread102227909529.62
4QueueItemEventsread5208690004.86
5Jobsread1056974428.09

Hardware requirements

An azure SQL configuration (40cores, 200GB memory, 7TB disk vCore purchasing model - Azure SQL Database) was used to run the performance tests as follows:

  • Insights DB, Azure SQL Hyperscale, 40cores and 0 replicas;
  • Azure SQL Hyperscale was used as Insights database, supporting more than 5TB of disk;
  • The MAX DOP was set to 16 by
    ALTER DATABASE SCOPED CONFIGURATION SET MAXDOP = 16;
  • Insights VM, Insights 22.10, Standard D8s v4 (8 vcpus, 32 GiB memory)
  • Looker VM, Looker 22.10, Standard D4s v3 (4 vcpus, 16 GiB memory)
  • Orchestrator VM, 22.10, Standard D4s v3 (4 vcpus, 16 GiB memory)
  • Orchestrator DB VM, Standard D8s v3 (8 vcpus, 32 GiB memory)

Performance test

The following performance characteristics have been evaluated:

  • “Read path” performance : how fast the dashboards are rendering information.
  • “Write path” performance : how fast the data can be ingested into the system, and how fast the data can be processed before becoming available for consumption. This happens in the background and executes periodically in batches. When this background processing is running it consumes server resources leaving less resources for the “read path”. This is the reason dashboards can load slower when ingestion or data processing is running. This test also included configuring custom variables from robot logs and queues for use in dashboards.

Insights background processes

Insights runs the following background processes on the SQL server:

  • By default, ingestion from the Orchestrator database to the Insights database occurs every 15 minutes.
  • The transformation pipeline moves data to the read schema to prepare jobs, queue ite, and robot logs to be consumed by a dashboard every 10 minutes.
  • The transformation pipeline extracts custom variables from robot logs and queues to be consumed by a dashboard when the custom fields configuration is changed. In the performance tests, the change occurs every 6 hours to simulate the worst case scenario. Under normal product usage, the custom fields configuration is not changed often.
    On average, job events, queue item events, and robot logs generate from 1-10 new items every second (e.g., 5 new queue items every second).

📘

Note

Background jobs running may impact the loading time of a dashboard.

Template loading time

The tests measured loading time of the pre-built templates. When measuring dashboard loading time, two metrics have been recorded:

  • p90 (ninetieth percentile) load time which accounts for worst case when a user reconfigures custom fields that start an expensive background job that consumes server resources.
  • Average load time.

Each dashboard has multiple widgets. In some of the dashboards, most widgets load significantly faster than outlier widgets. This is the case for Business ROI and Business ROI Queues dashboards.

All dashboards are loaded with the default time provided in the filter.

DashboardNumber of queried rows*p90 (seconds)Average time (seconds)
Business ROI (This Quarter)**28 million1917
Business ROI Queues (This Quarter)**28 million1210
Processes (last 30 days)20 million2520
Queues (last 30 days)10 million98
Robots (last 30 days)5 million1312

*By default, each dashboard only processes a part of the whole data set (e.g., last 30 days out of the data spread across 2 years). We estimate the number of rows each dashboard processes with default filter selection.

**Business ROI and Business ROI Queues have one slow widget (Saved Monthly widget). With the slow widget, the Business ROI loading time is 399 seconds at p90 and 324 seconds on average. Business ROI Queues is 193 seconds at p90 and 145 seconds on average.

Updated about a month ago


Performance and scalability


Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.