- Overview
- Requirements
- Installation
- Post-installation
- Cluster administration
- Managing products
- Managing the cluster in ArgoCD
- Setting up the external NFS server
- Automated: Enabling the Backup on the Cluster
- Automated: Disabling the Backup on the Cluster
- Automated, Online: Restoring the Cluster
- Automated, Offline: Restoring the Cluster
- Manual: Enabling the Backup on the Cluster
- Manual: Disabling the Backup on the Cluster
- Manual, Online: Restoring the Cluster
- Manual, Offline: Restoring the Cluster
- Additional configuration
- Migrating objectstore from persistent volume to raw disks
- Monitoring and alerting
- Migration and upgrade
- Migration options
- Step 1: Moving the Identity organization data from standalone to Automation Suite
- Step 2: Restoring the standalone product database
- Step 3: Backing up the platform database in Automation Suite
- Step 4: Merging organizations in Automation Suite
- Step 5: Updating the migrated product connection strings
- Step 6: Migrating standalone Insights
- Step 7: Deleting the default tenant
- B) Single tenant migration
- Product-specific configuration
- Best practices and maintenance
- Troubleshooting
- How to Troubleshoot Services During Installation
- How to Uninstall the Cluster
- How to clean up offline artifacts to improve disk space
- How to clear Redis data
- How to enable Istio logging
- How to manually clean up logs
- How to clean up old logs stored in the sf-logs bundle
- How to disable streaming logs for AI Center
- How to debug failed Automation Suite installations
- How to delete images from the old installer after upgrade
- How to automatically clean up Longhorn snapshots
- How to disable TX checksum offloading
- How to address weak ciphers in TLS 1.2
- Unable to run an offline installation on RHEL 8.4 OS
- Error in Downloading the Bundle
- Offline installation fails because of missing binary
- Certificate issue in offline installation
- First installation fails during Longhorn setup
- SQL connection string validation error
- Prerequisite check for selinux iscsid module fails
- Azure disk not marked as SSD
- Failure After Certificate Update
- Automation Suite not working after OS upgrade
- Automation Suite Requires Backlog_wait_time to Be Set 1
- Volume unable to mount due to not being ready for workloads
- RKE2 fails during installation and upgrade
- Failure to upload or download data in objectstore
- PVC resize does not heal Ceph
- Failure to Resize Objectstore PVC
- Rook Ceph or Looker pod stuck in Init state
- StatefulSet volume attachment error
- Failure to create persistent volumes
- Storage reclamation patch
- Backup failed due to TooManySnapshots error
- All Longhorn replicas are faulted
- Setting a timeout interval for the management portals
- Update the underlying directory connections
- Cannot Log in After Migration
- Kinit: Cannot Find KDC for Realm <AD Domain> While Getting Initial Credentials
- Kinit: Keytab Contains No Suitable Keys for *** While Getting Initial Credentials
- GSSAPI Operation Failed With Error: An Invalid Status Code Was Supplied (Client's Credentials Have Been Revoked).
- Alarm Received for Failed Kerberos-tgt-update Job
- SSPI Provider: Server Not Found in Kerberos Database
- Login Failed for User <ADDOMAIN><aduser>. Reason: The Account Is Disabled.
- ArgoCD login failed
- Failure to get the sandbox image
- Pods not showing in ArgoCD UI
- Redis Probe Failure
- RKE2 Server Fails to Start
- Secret Not Found in UiPath Namespace
- After the Initial Install, ArgoCD App Went Into Progressing State
- MongoDB pods in CrashLoopBackOff or pending PVC provisioning after deletion
- Unexpected Inconsistency; Run Fsck Manually
- Degraded MongoDB or Business Applications After Cluster Restore
- Missing Self-heal-operator and Sf-k8-utils Repo
- Unhealthy Services After Cluster Restore or Rollback
- RabbitMQ pod stuck in CrashLoopBackOff
- Prometheus in CrashloopBackoff state with out-of-memory (OOM) error
- Missing Ceph-rook metrics from monitoring dashboards
- Using the Automation Suite Diagnostics Tool
- Using the Automation Suite Support Bundle Tool
- Exploring Logs
Certificates overview
This page describes all the certificates required by an Automation Suite installation as well as the principle of the certificate rotation process.
https://automationsuite.mycompany.com/identity
.
While two different Automation Suite products must use the FQDN of the cluster, they can also contain multiple microservices. These microservices can use internal URLs to communicate with each other.
The following diagram and flow explain how the client connects to a service and how the authentication is done via the Identity Service.
- The client makes a connection with the service using URL, i.e., Orchestrator, Apps, Insights, etc. using the following URL:
https://automationsuite.mycompany.com/myorg/mytenant/service_
. - Istio intercepts the call, and based on the path of the
service_
, forwards the call to the specific service. - The service calls Identity Service to authenticate the incoming request from the robot via
https://automationsuite.mycompany.com/myorg/mytenant/identity_
. - Istio intercepts the call, and based on the path
identity_
, forwards the request to Identity Service. - Identity Service returns the response with the result to Istio.
- Istio returns the response to the service. Since the call is made using the HTTPS protocol, Istio returns the response with the TLS certificate so that the connection is secure. If the service trusts the server certificate returned by Istio, it approves the response. Otherwise, the service rejects the response.
- The service prepares the response and sends it back to Istio.
-
Istio forwards the request back to the client. If the client machine trusts the certificate, then the entire request is successful. Otherwise the request fails.
This section describes a scenario where a robot tries to connect to Orchestrator in Automation Suite. The following diagram and flow explain how the robot connects to Orchestrator, and how authentication is done via Identity Server.
- Robot makes a connection with Orchestrator using the following URL:
https://automationsuite.mycompany.com/myorg/mytenant/orchestrator_
- Istio intercepts the call, and based on the
orchestrator_
path, it forwards it to the Orchestrator service. - The Orchestrator service calls Identity Server to authenticate the incoming request from the robot via
https://automationsuite.mycompany.com/myorg/mytenant/identity_
. - Istio intercepts the call, and based on the
identity_
path, it forwards the request to Identity Server. - Identity Server returns the response with the results to Istio.
- Istio returns the response to Orchestrator. Since the call is made using the HTTPS protocol, Istio returns the response with the TLS certificate, so that connection is secure. If Orchestrator trusts the server certificate returned by Istio, it also approves the response. Otherwise, Orchestrator rejects the response.
- Orchestrator prepares the response and sends it back to Istio.
-
Istio forwards the request back to robot. If the robot machine trusts the certificate, then the entire request is successful. Otherwise, the request fails.
In this example, the container has its own operating system (RHEL OS), and Service can represent an Orchestrator running on top of RHEL OS.
/etc/pki/ca-trust/ca/
.
This path is where RHEL OS stores all certificates. Every container will have its own Certificate Trust Store. As part of the Automation Suite configuration, we inject the entire chain certificate that contains the root certificate, all the intermediate certificates, as well as the leaf certificate, and we store them in this path. Since services trust the root and intermediate certificates, they automatically trust any other certificates created by the root and intermediate certificates.
There are hundreds of containers running within Automation Suite. Manually adding certificates for each of these containers for all the services would be a demanding task. However, Automation Suite includes a shared volume and an Init container cert-trustor to help with this task. Init is a specialized container that runs before app containers in a Pod, and its lifecycle ends as soon as it completes its job.
In the following example, the Orchestrator service is running in one pod. As a reminder, a pod can contain more than one container. In this pod, we inject one more Init container called Cert-trustor. This container will contain the root certificate, the intermediate certificates, and the leaf certificate.
/etc/pki/ca-trust/ca/source/anchors
.
/etc/pki/ca-trust/ca/source/anchors
location and terminates.
Certificates will be available for Orchestrator service through the shared volume.
As part of the Automation Suite installation, the following certificates are generated:
- Self-signed certificate generated at the time of installation, which is valid for 3 months. You must replace the self-signed certificate with a domain certificate post-installation. See Managing certificates.
- Identity Server certificate for signing JWT tokens used in authentication. If the certificate for signing the JWT token is not provided, Automation Suite uses the currently configured TLS certificate (can be the self-signed one or customer provided one), which expired in 90 days. If you want to have your own certificate for signing identity token, you can configure using the instructions in Managing certificates.
- Internal certificate for MongoDB generated via the cert manager. Automation Suite provides an internal certificate for MongoDB that is valid for 3 years. The certificate will rotate automatically, and no manual intervention is required. For more details, see MongoDB certificate renewal.
- RKE2 certificates are generated, and by default expire in 12 months. If the certificates are already expired or they expire in less than 90 days, they are rotated when RKE2 is restarted.
- If enabled, SAML2 Authentication protocol can use a service certificate.
- If you configure Active Directory using a username and password, LDAPS (LDAP Over SSL) is optional. If you opt for LDAPS, you must provide a certificate. This certificate will be added into Automation Suite’s Trusted Root Certification Authorities. For details, see Microsoft documentation.
This certificate will be added into Automation Suite’s Trusted Root Certification Authorities.
The certificates are stored in two places:
istio-ingressgateway-certs
inistio-system
uipath
namespace
istio-system
and uipath
namespaces, you must run the sudo ./configureUiPathAS.sh tls-cert update
command.
uipath
namespace cannot access the secrets stored inistio-system
namespace. Hence, certificates are copied in both namespaces.
uipath
namespace, we mount the certificates to the pods that need certificates and restart the pods so they can use the new certificates.
For single-node evaluation installations, the update will scale down the pods. All the pods will be shut down and restarted. This operation will cause downtime.
For multi-node HA-ready production installations, the update happens using the rolling deployment method. If microservices have two pod for high availability purpose, the update will delete one of the pod, and a new version of the pod will come up. Once the new one is started successfully, the old one will removed. There will be a brief downtime period while the old pod is not yet terminated.
rootCA.crt
and tls.crt
are used. Certificates are used in ArgoCD and Docker Registry, and they are then stored in both Docker and ArgoCD namespaces.
You can verify the secrets using the following command :
# For docker registry
kubectl -n docker-registry get secrets docker-registry-tls -o yaml
# For Argocd
argocd cert list --cert-type https
# For docker registry
kubectl -n docker-registry get secrets docker-registry-tls -o yaml
# For Argocd
argocd cert list --cert-type https
- Understanding how trust certificates work
- Understanding how communication works
- Understanding how robots and Orchestrator communicate
- Understanding the container architecture related to certificates
- Container level
- Pod level
- Inventory of all certificates in Automation Suite
- Certificates Generated During Installation
- Additional certificates
- Understanding how the certificate update/rotation works
- Online installation
- Offline installation