# Alerts

> Configure alert receivers in Automation Suite to prevent downtime using uipathctl.

You must configure alert receivers to prevent issues that may lead to downtime.

## Configuring alerts

:::note
You can find `uipathctl` in the Automation Suite installation folder: `.../UiPathAutomationSuite/UiPath_Installer/bin`. 

Before starting configuring alerts, make sure to [enable kubectl](https://docs.uipath.com/automation-suite/automation-suite/2.2510/installation-guide/accessing-automation-suite#enabling-kubectl).
:::

### Adding an email receiver

To add a new email configuration after an installation, run the following command:

```
./uipathctl config alerts add-email \
  --name test \
  --to "admin@example.com" \
  --from "admin@example.com" \
  --smtp server.mycompany.com \
  --username admin \
  --password somesecret \
  --require-tls \
  --ca-file <path_to_ca_file> \
  --cert-file <path_to_cert_file> \
  --key-file <path_to_key_file> \
  --send-resolved
```

To send email notifications to multiple recipients, you can create multiple email receivers. The following table describes the flags, their expected values, and examples for configuring an email receiver.

| Flag | Value | Example |
| --- | --- | --- |
| `name` | The name of the email configuration | `testconfig` |
| `to` | The email address of the receiver | `admin@example.com` |
| `from` | The email address of the sender | `admin@example.com` |
| `SMTP` | SMTP server URL or IP address and port number | `server.mycompany.com:567` |
| `username` | Authentication username | `admin` |
| `password` | Authentication password | `securepassword` |
| `require-tls` | Boolean flag to denote that TLS is enabled at the SMTP server. | N/A |
| `ca-file` | File path containing the CA Certificate of the SMTP server. This is optional if the CA is private. | `./ca-file.crt` |
| `cert-file` | File path containing the certificate of the SMTP server. This is optional if the certificate is private. | `./cert-file.crt` |
| `key-file` | File path containing the private key of the certificate of the SMTP server. This is required if the certificate is private. | `./key-file.crt` |
| `send-resolved` | Boolean flag to send an email once the alert is resolved. | N/A |

### Removing an email receiver

To remove an email configuration, you must run the following command. Make sure to pass the name of the email configuration you want to remove.

```
./uipathctl config alerts remove-email --name test
```

### Updating an email receiver

To update an email configuration, you must run the following command. Make sure to pass the name of the email configuration you want to update and the additional optional parameters you want to edit. These parameters are the same as the ones for adding a new email configuration. You can pass one or more flags at the same time.

```
./uipathctl config alerts update-email --name test [additional_flags]
```

### Adding webhooks to send alerts to external tools

Alert Manager configurations are stored in a secret in Base64-encoded format. To update the secret, take the following steps:

1. Get the existing Alert Manager configuration in a file called `am_config.yaml`.
   
   ```
   kubectl get secret alertmanager-config -n monitoring -o jsonpath="{.data.alertmanager\.yaml}" | base64 --decode > am_config.yaml
   ```
2. Configure webhooks. For details, refer to [Alert Manager webhook configuration](https://prometheus.io/docs/alerting/latest/configuration/#webhook_config).
3. Encode the configuration to Base64 format.
   
   ```
   openssl base64 -in am_config.yaml -out am_config_base64.yaml
   ```
4. Update the `alertmanager.yaml` attribute in the secret. Replace the existing Base64-encoded string with the one in `am_config_base64.yaml`.
5. Check that the configurations are updated in the status section in Alert Manager.

For troubleshooting, refer to the Alert Manager and Prometheus pod logs.

## Testing alerts

To test the alerts, take the following steps:

1. Log in to Alert Manager by accessing `https://monitoring.fqdn/alertmanager`.
2. Copy the `_oauth2_proxy=<value>` cookie.
3. Create a test alert via Postman, as shown in the following example:
   
   ```
   Method: Post
   URL: https://monitoring.fqdn/alertmanager/api/v2/alerts
   Headers: 
   - Content-Type: application/json
   - Cookie: _oauth2_proxy=<value>
   Body: [{"labels":{"alertname":"Acme Alert", "severity":"warning"}}]
   ```
4. You should now see the alert in Alert Manager.

## Checking currently firing alerts

To view the alerts, navigate to Prometheus using `https://monitoring.fqdn/metrics` and select the **Alerts** tab. Here you can see all the alerts configured in Automation Suite.

* To view the active alerts, filter the alert status by using the **Firing** checkbox.
* For more details about certain alerts, expand the alerts you want to view.

  ![Prometheus Alerts tab showing currently firing alerts](https://dev-assets.cms.uipath.com/assets/images/automation-suite/automation-suite-docs-image-556773-1135b33a.webp)

## Silencing alerts

If alerts are too noisy, you can silence them. To do that, take the following steps:

1. Access Alert Manager using `https://monitoring.fqdn/alertmanager`.
2. Find the alert in question, and select **Silence**.
3. Fill in the **Creator** and **Comment** details, and select **Create**. The alert should no longer show on the **Monitoring Dashboard** or be reported to any of the configured receivers.

## Troubleshooting alerts

If you have issues with the alert receiver configuration, check the logs by running the following command:

```
kubectl logs -n monitoring -l app.kubernetes.io/name=kube-prometheus-stack-prometheus-operator --tail=-1
```