- Overview
- Requirements
- Installation
- Post-installation
- Cluster administration
- Managing products
- Managing the cluster in ArgoCD
- Setting up the external NFS server
- Automated: Enabling the Backup on the Cluster
- Automated: Disabling the Backup on the Cluster
- Automated, Online: Restoring the Cluster
- Automated, Offline: Restoring the Cluster
- Manual: Enabling the Backup on the Cluster
- Manual: Disabling the Backup on the Cluster
- Manual, Online: Restoring the Cluster
- Manual, Offline: Restoring the Cluster
- Additional configuration
- Migrating objectstore from persistent volume to raw disks
- Monitoring and alerting
- Migration and upgrade
- Migration options
- Step 1: Moving the Identity organization data from standalone to Automation Suite
- Step 2: Restoring the standalone product database
- Step 3: Backing up the platform database in Automation Suite
- Step 4: Merging organizations in Automation Suite
- Step 5: Updating the migrated product connection strings
- Step 6: Migrating standalone Insights
- Step 7: Deleting the default tenant
- Performing a single tenant migration
- Product-specific configuration
- Best practices and maintenance
- Troubleshooting
- How to Troubleshoot Services During Installation
- How to Uninstall the Cluster
- How to clean up offline artifacts to improve disk space
- How to clear Redis data
- How to enable Istio logging
- How to manually clean up logs
- How to clean up old logs stored in the sf-logs bucket
- How to disable streaming logs for AI Center
- How to debug failed Automation Suite installations
- How to delete images from the old installer after upgrade
- How to automatically clean up Longhorn snapshots
- How to disable TX checksum offloading
- How to address weak ciphers in TLS 1.2
- Unable to run an offline installation on RHEL 8.4 OS
- Error in Downloading the Bundle
- Offline installation fails because of missing binary
- Certificate issue in offline installation
- First installation fails during Longhorn setup
- SQL connection string validation error
- Prerequisite check for selinux iscsid module fails
- Azure disk not marked as SSD
- Failure After Certificate Update
- Automation Suite not working after OS upgrade
- Automation Suite Requires Backlog_wait_time to Be Set 1
- Volume unable to mount due to not being ready for workloads
- RKE2 fails during installation and upgrade
- Failure to upload or download data in objectstore
- PVC resize does not heal Ceph
- Failure to Resize Objectstore PVC
- Rook Ceph or Looker pod stuck in Init state
- StatefulSet volume attachment error
- Failure to create persistent volumes
- Storage reclamation patch
- Backup failed due to TooManySnapshots error
- All Longhorn replicas are faulted
- Setting a timeout interval for the management portals
- Update the underlying directory connections
- Cannot Log in After Migration
- Kinit: Cannot Find KDC for Realm <AD Domain> While Getting Initial Credentials
- Kinit: Keytab Contains No Suitable Keys for *** While Getting Initial Credentials
- GSSAPI Operation Failed With Error: An Invalid Status Code Was Supplied (Client's Credentials Have Been Revoked).
- Alarm Received for Failed Kerberos-tgt-update Job
- SSPI Provider: Server Not Found in Kerberos Database
- Login Failed for User <ADDOMAIN><aduser>. Reason: The Account Is Disabled.
- ArgoCD login failed
- Failure to get the sandbox image
- Pods not showing in ArgoCD UI
- Redis Probe Failure
- RKE2 Server Fails to Start
- Secret Not Found in UiPath Namespace
- After the Initial Install, ArgoCD App Went Into Progressing State
- MongoDB pods in CrashLoopBackOff or pending PVC provisioning after deletion
- Unexpected Inconsistency; Run Fsck Manually
- Degraded MongoDB or Business Applications After Cluster Restore
- Missing Self-heal-operator and Sf-k8-utils Repo
- Unhealthy Services After Cluster Restore or Rollback
- RabbitMQ pod stuck in CrashLoopBackOff
- Prometheus in CrashloopBackoff state with out-of-memory (OOM) error
- Missing Ceph-rook metrics from monitoring dashboards
- Pods cannot communicate with FQDN in a proxy environment
- Using the Automation Suite Diagnostics Tool
- Using the Automation Suite support bundle
- Exploring Logs
In a multi-node HA-ready production setup, High Availability (HA) is enabled by default. However, the Redis-based in-memory cache used by cluster services runs on a single node and represents a single point of failure. Therefore, if you have not purchased a High Availability Add-on (HAA) license, a cache node failure or restart will result in downtime for the entire cluster. To prevent such an incident, you can purchase HAA, which enables redundant, multi-node HA-ready production deployment of the cache.
All installations include the HAA software with a single-node license. This license is free of cost, no purchase required.
If you wish to enable HAA across multiple nodes, then purchasing an HAA license is required. This will implement full high availability for the cluster in a multi-node HA-ready production setup.
HAA is based on Redis technology.
To do that, take the following steps:
- Purchase an HAA license. Contact UiPath for details.
- Update the following fields in the
cluster.config.jsonfile:-
fabric.redis.license- enter the HAA license converted to a single base64 string. In bash you can do that usingecho 'license_text_here' | base64 -w0. -
fabric.redis.ha- usetrueto enable HAA and make sure to also configure thefabric.redis.licenseparameter. This enables HAA database replication and increases the number of HAA pods to 3. By default,fabric.redis.hais set tofalse.Note:If
redis.hais enabled,redis.licenseneeds to be set to a license that supports more than two shards."fabric": { "redis": { "ha": "true", "license": Base64String } }"fabric": { "redis": { "ha": "true", "license": Base64String } }
-
- Rerun the fabric installer:
-
online installation:
./install-uipath.sh -i cluster_config.json -f -o output.json --accept-license-agreement./install-uipath.sh -i cluster_config.json -f -o output.json --accept-license-agreement -
offline installation:
./install-uipath.sh -i cluster_config.json -f --install-type offline -o output.json --accept-license-agreement./install-uipath.sh -i cluster_config.json -f --install-type offline -o output.json --accept-license-agreement
Updating the Redis License
To update the Redis license, take the following steps:
-
Setting up kubectl and ArgoCD access
- Enable kubectl access on the primary node. See Enabling kubectl for instructions.
- Enable access to ArgoCD. See Accessing ArgoCD for instructions.
-
Checking the current license status
To check the status of the current license, run the following Shell command:
kubectl get rec -n redis-system redis-cluster -o jsonpath='{.status.licenseStatus}' | jqkubectl get rec -n redis-system redis-cluster -o jsonpath='{.status.licenseStatus}' | jq-
Clusters deployed after the current license included in the installer expired will show the trial 4-shard license that expires in 30 days, as in the following example:
-
Clusters that were already running when the license expired will show the following status:
-
-
Updating the existing license
-
To update the existing license, run the following Shell command:
kubectl patch application fabric-installer -n argocd \ --type=json -p '[{"op":"add","path":"/spec/source/helm/parameters/-","value":{"name": "global.redis.license", "value": "<LICENSE_KEY_IN_BASE64>"}}]'kubectl patch application fabric-installer -n argocd \ --type=json -p '[{"op":"add","path":"/spec/source/helm/parameters/-","value":{"name": "global.redis.license", "value": "<LICENSE_KEY_IN_BASE64>"}}]' -
To see if the change was applied, access ArgoCD. See Accessing ArgoCD for instructions.
-
If the fabric-installer application appears out of sync, and the sync process was not triggered automatically, click the Sync button yourself. This may be happen if you are using an older Automation Suite version.
Note:There is a small delay between the moment the ArgoCD UI shows the app synced and when the Redis operator successfully applies the new license.
-
To see logs from the Redis operator when it tries to apply the license, run the following command:
kubectl logs -n redis-system --since=300s -l name=redis-enterprise-operator -c redis-enterprise-operator --tail=-1 | grep licensekubectl logs -n redis-system --since=300s -l name=redis-enterprise-operator -c redis-enterprise-operator --tail=-1 | grep license -
If you try to apply an expired license, or you run the installer that ships with an expired license, you will get the following output:
-
To update the Redis license used by older installers before running them, update the
fabric.redis.licensekey in the<installer_folder>/defaults.json:
-
-
Checking that the new license is applied
To check if the new license is applied, run the following Shell command:
kubectl get rec -n redis-system redis-cluster -o jsonpath='{.status.licenseStatus}' | jqkubectl get rec -n redis-system redis-cluster -o jsonpath='{.status.licenseStatus}' | jqIn the following image you can see that the Redis cluster switched from the trial 30-day license to a single-shard 10-year license.