Live streaming and remote control via TightVNC

Important:

This applies to Robot versions 2023.4 and 2023.10.

The live streaming and remote control features are available for:

• Windows Robots versions 2023.4 and 2023.10
• Automation Cloud Robots - VM

Due to technical limitations, we can not offer full support for Linux robots, but the main use cases are available in this iteration.

Note: This functionality cannot be used with Windows background jobs.

Prerequisites for Windows Robots 2023.10 and older

1. Enable SignalR. Details...
2. Install a VNC server by following these steps:
   1. Download TightVNC. The version we currently support is 2.8.75.
   2. Run the installer, and, once you get to the Choose Setup Type screen, select Custom.
   3. Click Next.
   4. Make sure that the Register TightVNC Server as a system service option (under TightVNC Service configuration) is not selected.
   5. If you are using a physical machine, a monitor must be connected to it.

Network requirements

The minimum network requirements for a live streaming and remote control session to work smoothly can vary significantly, due to a number of factors:

• The robot's screen resolution

• The changes driven by your automation - the more dynamic the changes in images and text, the higher the volume of traffic

• The latency between your machine and the Automation Cloud region

• The latency between your robot and the Automation Cloud region

Considering the above, we recommend a 3MB/s network bandwidth for a 1920x1080 screen resolution.

Security considerations

The live streaming connection between the client and the target machine is tunneled through TLS via the VNC proxy, meaning that the server is secured. Additionally, every time a live streaming session is started, a unique JWT Token is generated and used to authenticate the request. The only instance of unsecured traffic is on the robot machine for the localhost connection.

On the robot side, a unique password is generated for each job, and set for the VNC server. The server only runs during the live streaming session. When the user disconnects from the session, the VNC server closes as well.

Network flow

1. When you make a request from the Orchestrator interface to start live streaming, Orchestrator sends a command to the robot to initiate a new remote control session.
2. The robot starts a VNC server that listens on a dynamic port. The VNC server is started as loopback-only, which disables remote connections. The Orchestrator command also includes a password for VNC authentication, that is shared between the VNC server and the browser client.
3. The robot then starts the following:
   1. Another program that connects to the local VNC server on a dynamic port
   2. a UiPath live streaming proxy

   The program forwards all TCP traffic to the live streaming proxy by converting it into the WebSocket Secure protocol (wss).

4. The Orchestrator VNC client connects to the same live streaming proxy whose job is to pair the two websockets: one from the VNC client in the browser, and the other from the robot. It then creates a tunnel between the two websockets and forwards any traffic from one websocket to the other.

   The VNC client uses two ports: 443 and 5900.

   Port 443 must be open to connect to an auxiliary online service.

   Port 5900 operates locally on the same machine to tunnel traffic between two applications.

   For the communication to work, these ports need to be opened bidirectionally to allow both incoming and outgoing traffic.

   

TightVNC configuration

Before the session starts on the local machine, HTTP connections are disabled, and the VNC server is configured to not accept connections outside the local machine.