Live streaming and remote control

While a job is in progress, you can access the live stream of the robot running the automation. If troubleshooting or debugging is needed, you can take control of the execution using your own mouse and keyboard, without pausing the automation.

The live streaming and remote control features are available for Automation Cloud Robots - Serverless and Windows Robots version 2023.4+. Due to technical limitations, we can not offer full support for Linux robots.

Note:

• On machines with High-density Robots, you can start a live stream on one robot only.

• This functionality cannot be used with Windows background jobs.

Prerequisites for Windows Robots 2023.10 and older

1. Enable SignalR. Details...
2. Install a VNC server by following these steps:
   1. Download TightVNC. The version we currently support is 2.8.75.
   2. Run the installer, and, once you get to the Choose Setup Type screen, select Custom.
   3. Click Next.
   4. Make sure that the Register TightVNC Server as a system service option (under TightVNC Service configuration) is not selected.
   5. If you are using a physical machine, a monitor must be connected to it.

Network requirements

The minimum network requirements for a live streaming and remote control session to work smoothly can vary significantly, due to a number of factors:

• The robot's screen resolution

• The changes driven by your automation - the more dynamic the changes in images and text, the higher the volume of traffic

• The latency between your machine and the Automation Cloud region

• The latency between your robot and the Automation Cloud region

Considering the above, we recommend a 3MB/s network bandwidth for a 1920x1080 screen resolution.

Enabling the features

Live streaming and remote control actions are controlled by the following options displayed in the Additional Settings tab of the process creation or editing window:

• The Allow live streaming toggle

  This is disabled by default.

• When Allow live streaming is enabled, the Allow remote control when live streaming option is also displayed, with two radio buttons:
  • Disabled (this is the default selection)
  • Enabled

General permissions

These features are controlled by the Live streaming & Remote control permission, with the following options:

• View - allows you to view live streams of running jobs.
• Edit - allows you to gain remote control over running jobs.
• Create - allows you to enable the features at the process level.

Live streaming

The live stream can be accessed from the contextual menu of the desired job while it is running, and it opens in a new browser tab.

Auditing

Live streaming activity is included in the audit, allowing you to see:

• if a live stream was accessed.
• the time that the live stream was accessed.
• the user that accessed it (along with the permission that the user had at access time).

Permissions

To access an available live stream, you need to have the folder-level Live Streaming & Remote Control permission with the View option enabled.

Remote control

This option allows you to take control of a running automation through its live stream, so as to troubleshoot any issues, such as long-running workflows and broken or blocked processes. It is only available if live streaming is also enabled.Taking remote control does not pause the job execution, so you can interfere with the automation as necessary.

Permissions

To take control over a running automation, you need to have the folder-level Live Streaming & Remote Control permission with the Edit option enabled.

Security considerations

The live streaming connection between the client and the target machine is tunneled through TLS via the VNC proxy, meaning that the server is secured. Additionally, every time a live streaming session is started, a unique JWT Token is generated and used to authenticate the request. The only instance of unsecured traffic is on the robot machine for the localhost connection.

On the robot side, a unique password is generated for each job, and set for the VNC server. The server only runs during the live streaming session. When the user disconnects from the session, the VNC server closes as well.

Network flow

1. When you make a request from the Orchestrator interface to start live streaming, Orchestrator sends a command to the robot to initiate a new remote control session.
2. The robot starts a VNC server that listens on a dynamic port. The VNC server is started as loopback-only, which disables remote connections. The Orchestrator command also includes a password for VNC authentication, that is shared between the VNC server and the browser client.
3. The robot then starts the following:
   1. another program that connects to the local VNC server on a dynamic port
   2. a UiPath live streaming proxy
   • The program forwards all TCP traffic to the live streaming proxy by converting it into the WebSocket Secure protocol (wss).
4. The Orchestrator VNC client connects to the same live streaming proxy whose job is to pair the two websockets: one from the VNC client in the browser, and the other from the robot. It then creates a tunnel between the two websockets and forwards any traffic from one websocket to the other.
   

Error scenarios

These are some issues that you might occasionally experience when you live stream and/or take remote control of a running execution, along with our proposed solutions.