Deployment in the Cloud

There are multiple enterprise cloud deployment options available to host your Orchestrator, such as Amazon Web Services (AWS), Microsoft Azure, or Google Cloud Platform (GCP). Here we detail a large, scalable deployment using the Azure Infrastructure as a Service (IaaS) offerings. The following services are required:

• VM Availability Set for Orchestrator
• VM Availability Set for Elasticsearch
• Azure SQL Server
• Azure Load Balancer
• Azure Redis Cache for multi-node deployments
• Distributed DNS Service (such as Cloudflare)

Architecture

Note:

The architecture examples below contain optional and/or differing components (e.g. CyberArk, UiPath High Availability Add-on).

The Jumpbox depicted is not required but is a recommended best practice for your production environments, providing isolation and security.

Single-Node Architecture

Multi-node Architecture

Hardware Requirements

This section describes the hardware configurations used for the performance testing listed in Scaling Your Deployment, below.

Orchestrator Nodes

Each Orchestrator node must be configured as follows:

VCPUs	RAM (GB)	SSD (GB)
16	32	128

SQL Server

The SQL Server virtual machine specifications must scale in line with the number of Orchestrator nodes:

Orchestrator Nodes	VCPUs	RAM (GB)
1 - 2	8	16
5	16	32
10	16	64

Elasticsearch Availability Set

The Elasticsearch availability set is comprised of 3 master nodes and 6 data nodes, for a total of 9 nodes, each with the following specifications:

VCPUs	RAM (GB)	OS SSD (GB)	Data SSD (TB)
8	16	128 (with 5000 IOPS and 100 MB/s Throughput)	1 (with 5000 IOPS and 200 MB/s Throughput)

Software Requirements

Software	Version
Operating System	Windows Server 2016
Databases	SQL Server 2017
Logging	Elasticsearch 6.4.0

The versions listed above are those used for the deployments and performance tested loads described. For all versions compatible with Orchestrator, see here.

Load Balancing

For multi-node deployments, it is recommended to use two Azure Standard load balancers:

• One for the Orchestrator servers;
• One for the Elasticsearch servers.

Azure Redis Cache

Important:

Multi-node Orchestrator deployments use the RESP (REdis Serialization Protocol) for communication, and thus can be configured using any solution implementing this protocol, such as Azure Redis Cache in this example.

High availability deployments of Orchestrator are supported by UiPath only if the UiPath High Availability Add-on is used.

For multi-node deployments, it is recommended to use two separate Redis instances:

• Azure Redis Cache Premium with a 6GB cache - the primary node used for session state and user-entity associations;
• Azure Redis Cache Basic - used to scale the SignalR service.

Configuration

UiPath.Orchestrator.dll.config Settings

1. Set all Orchestrator nodes to report to Elasticsearch instead of their respective Windows Event Log by setting the writeTo parameter in the serverdiagnostics index as follows:

   <logger name="*" minlevel="Warn" writeTo="serverElasticBuffer"/>

2. Enable Quartz job clustering as follows:

   <add key="quartz.jobStore.clustered" value="true" />

3. In the Deployment section, configure the NuGet repository as follows:

   <add key="NuGet.Repository.Type" value="Composite" />
   <add key="Deployment.Libraries.AllowTenantPublish" value="true" />
   <add key="Processes.AllowUpdateWithRunningJobs" value="true" />

4. In the Load Balancer section, configure the Redis instances as follows, being sure to replace the connection string and password with your respective values:

   <add key="LoadBalancer.UseSqlServer" value="false"/>
   <add key="LoadBalancer.UseRedis" value="true"/>
   <add key="LoadBalancer.Redis.ConnectionString" value="<your.redis.cache.windows.net>:6379,password=****"/>
   <add key="LoadBalancer.SignalR.UseRedisScaleout" value="true"/>
   <add key="LoadBalancer.SignalR.RedisScaleout.ConnectionString" value="<your.second.Redis.Server>:6379,password=********"/>

5. Set the Max Pool Size in the database connection string as follows:

   <add connectionString="Data Source=172.16.100.12;Initial Catalog=UiPath;User ID=admin;Password=*******;Max Pool Size=700" name="Default" providerName="System.Data.SqlClient" />

   Note: Be sure to replace the Data Source, Initial Catalog, User ID, and Password parameters with the respective values of your Orchestrator Azure service. See here for more details.
6. Set the Elasticsearch buffer size as follows:

   <target name="robotElasticBuffer" xsi:type="BufferingWrapper" bufferSize="1000" flushTimeout="5000">

Scaling Your Deployment

The number of nodes needed in your Orchestrator scale set depends on the number of Robots being deployed:

Orchestrator Scale Set Nodes	No. of Robots
1	up to 4,000
2	up to 8,000
5	up to 24,000
10	up to 48,000

These deployments were tested using the hardware and software configurations above to exhibit no performance loss under the specified load below.

Performance Testing

Static Data

Static Data refers to the initial Orchestrator load existing.

Entity	One Node	Two Nodes	Five Nodes	Ten Nodes
Tenants	40	80	240	480
Environments	2,000	4,000	12,000	24,000
Robots Attended Floating Unattended	4,000 1,600 1,600 800	8,000 3,200 3,200 1,600	24,000 9,600 9,600 4,800	48,000 19,200 19,200 9,600
Packages	8,000	16,000	48,000	96,000
Processes	4,000	8,000	24,000	48,000
Queues	200	420	1,200	2,400
Queue Items	1,120,000	1,500,000	3,000,000	5,000,000
Assets	40,000	80,000	240,000	480,000
Schedules	400	800	2,400	4,800

Dynamic Data

Dynamic data refers to the data added to or changed in Orchestrator as processes are executed.

Entity	One Node	Two Nodes	Five Nodes	Ten Nodes
Queue Items (per day)	112,000	175,000	672,000	1,250,000
Jobs (per minute)	2,000	4,000	12,000	24,000
Logs (per minute)	20,000	20,000	20,000	25,000
Nuget Downloads (Maximum per minute)	2,000	4,000	12,000	24,000
Robots Connected (Maximum)	4,000	8,000	24,000	48,000
Heartbeat (per minute)	10,000	20,000	60,000	120,000
SignalR Notifications (per minute)	2,000	4,000	12,000	24,000
Busy Robots	2,000	4,000	12,000	24,000
Available Robots	2,000	4,000	12,000	24,000