Using the monitoring stack

The monitoring stack for Automation Suite clusters includes Prometheus, Grafana, and Alertmanager, which are automatically installed unless you choose to bring your own monitoring stack.

Note:

Node failures might lead to a Kubernetes shutdown, which would disrupt Prometheus alerts. To prevent this, we recommend setting up a separate alert on the RKE2 server.

This page describes a series of monitoring scenarios optimized to work with the monitoring tool, which is bundled with the Automation Suite cluster. If you choose to bring your own monitoring stack, make sure to follow the official documentation of your tools to monitor the health of the cluster.

Important:

When using collectors to export metrics to third-party tools, enabling application monitoring may disrupt the functionality of Automation Suite.

Accessing the monitoring tools

Overview

The monitoring stack for Automation Suite clusters includes Prometheus, Grafana, and Alert Manager.

You can access the Automation Suite monitoring tools individually using the following URLs:

Application	Tool	URL	Example
Metrics	Prometheus	`https://monitoring.fqdn/metrics`	`https://monitoring.automationsuite.mycompany.com/metrics`
Dashboard	Grafana	`https://monitoring.fqdn/grafana`	`https://monitoring.automationsuite.mycompany.com/grafana`
Alert Management	Alert Manager	`https://monitoring.fqdn/alertmanager`	`https://monitoring.automationsuite.mycompany.com/alertmanager`

Authentication

To access the monitoring tools for the first time, follow the instructions in Accessing Automation Suite.

Checking currently firing alerts

To view the alerts, navigate to Prometheus using https://monitoring.fqdn/metrics and click the Alerts tab. Here you can see all the alerts configured in Automation Suite.

To view the active alerts, filter the alert status by clicking the Firing checkbox and the Show annotations checkbox at the top. Here you can see all the alerts that are firing currently and their corresponding messages.

Silencing alerts

If alerts are too noisy, you can silence them. To do that, take the following steps:

Click the Alertmanager tile in the upper left corner of the Monitoring dashboard. The following screen is displayed.
Find the alert in question, and select Silence.
Fill in the Creator and Comment details, and click Create. The alert should no longer show on the Monitoring Dashboard or be reported to any of the configured receivers.

Configuring the alerts

Adding a new email configuration

To add a new email configuration after an installation, run the following command:

./uipathctl config alerts add-email \
  --name test \
  --to "admin@example.com" \
  --from "admin@example.com" \
  --smtp server.mycompany.com \
  --username admin \
  --password somesecret \
  --require-tls \
  --ca-file <path_to_ca_file> \
  --cert-file <path_to_cert_file> \
  --key-file <path_to_key_file> \
  --send-resolved
  --ASEA./uipathctl config alerts add-email \
  --name test \
  --to "admin@example.com" \
  --from "admin@example.com" \
  --smtp server.mycompany.com \
  --username admin \
  --password somesecret \
  --require-tls \
  --ca-file <path_to_ca_file> \
  --cert-file <path_to_cert_file> \
  --key-file <path_to_key_file> \
  --send-resolved
  --ASEA

Flag	Description	Example
`name`	The name of the email configuration	`testconfig`
`to`	The email address of the receiver	`admin@example.com`
`from`	The email address of the sender	`admin@example.com`
`SMTP`	SMTP server URL or IP address and port number	`server.mycompany.com:567`
`username`	Authentication username	`admin`
`password`	Authentication password	`securepassword`
`require-tls`	Boolean flag to denote that TLS is enabled at the SMTP server.	N/A
`ca-file`	File path containing the CA Certificate of the SMTP server. This is optional if the CA is private.	`./ca-file.crt`
`cert-file`	File path containing the certificate of the SMTP server. This is optional if the certificate is private.	`./cert-file.crt`
`key-file`	File path containing the private key of the certificate of the SMTP server. This is required if the certificate is private.	`./key-file.crt`
`send-resolved`	Boolean flag to send an email once the alert is resolved.	N/A
`ASEA`	Boolean flag to indicate that you installed Automation Suite on EKS or AKS.	N/A

Removing an email configuration

To remove an email configuration, you must run the following command. Make sure to pass the name of the email configuration you want to remove.

./uipathctl config alerts remove-email --name test --ASEA./uipathctl config alerts remove-email --name test --ASEA

Updating an email configuration

To update an email configuration, you must run the following command. Make sure to pass the name of the email configuration you want to update and the additional optional parameters you want to edit. These parameters are the same as the ones for adding a new email configuration. You can pass one or more flags at the same time.

./uipathctl config alerts update-email --name test --ASEA [additional_flags]./uipathctl config alerts update-email --name test --ASEA [additional_flags]

Accessing Grafana dashboard

To access Grafana dashboards, you must retrieve your credentials and use them to log in:

Username:

kubectl -n cattle-monitoring-system get secrets/rancher-monitoring-grafana -o "jsonpath={.data.admin-user}" | base64 -d; echokubectl -n cattle-monitoring-system get secrets/rancher-monitoring-grafana -o "jsonpath={.data.admin-user}" | base64 -d; echo

Password:

kubectl -n cattle-monitoring-system get secrets/rancher-monitoring-grafana -o "jsonpath={.data.admin-password}" | base64 -d; echokubectl -n cattle-monitoring-system get secrets/rancher-monitoring-grafana -o "jsonpath={.data.admin-password}" | base64 -d; echo

Monitoring Persistent Volumes

You can monitor persistent volumes via the Kubernetes / Persistent Volumes dashboard. You can keep track of the free and used space for each volume.

You can also check the status of each volume by clicking the PersistentVolumes item within the Storage menu of the Cluster Explorer.

Monitoring hardware utilization

To check the hardware utilization per node, you can use the Nodes dashboard. Data on the CPU, Memory, Disk, and Network is available.

You can monitor the hardware utilization for specific workloads using the Kubernetes / Compute Resources / Namespace (Workloads) dashboard. Select the uipath namespace to get the needed data.

Creating shareable visual snapshot of a Grafana chart

Click the downwards pointing arrow next to the chart title, and then select Share.
Click the Snapshot tab, and set the Snapshot name,Expire, and Timeout.
Click Publish to snapshot.raintank.io.