Storage alerts

kubernetes-system

KubernetesDiskPressure

This alert indicates that disk usage is very high on the Kubernetes node.

If this alert fires, try to see which pod is consuming more disk.

KubernetesMemoryPressure

This alert indicates that memory usage is very high on the Kubernetes node.

If this alert fires, try to see which pod is consuming more memory.

KubePersistentVolumeFillingUp

When Warning: The available space is less than 30% and is likely to fill up within four days.

When Critical: The available space is less than 10%.

For any services that run out of space, data may be difficult to recover, so volumes should be resized before hitting 0% available space.

For instructions, see Configuring the cluster.

For Prometheus-specific alerts, see PrometheusStorageUsage for more details and instructions.

KubePersistentVolumeErrors

PersistentVolume is not able to be provisioned. This means any service requiring the volume would not start. Check for other errors with Longhorn and/or Ceph storage and contact UiPath® Support.

node-exporter

NodeFilesystemSpaceFillingUp

The filesystem on a particular node is filling up. Provision more space by adding a disk or mounting unused disks.

NodeFilesystemAlmostOutOfSpace

The filesystem on a particular node is filling up. Provision more space by adding a disk or mounting unused disks.

NodeFilesystemFilesFillingUp

The filesystem on a particular node is filling up. Provision more space by adding a disk or mounting unused disks.

NodeFilesystemAlmostOutOfFiles

The filesystem on a particular node is filling up. Provision more space by adding a disk or mounting unused disks.

NodeNetworkReceiveErrs

There is a problem with the physical network interface on the node. If the issues persist, it may need to be replaced.

NodeNetworkTransmitErrs

There is a problem with the physical network interface on the node. If the issues persist, it may need to be replaced.

ceph.rules, cluster-state-alert.rules

CephClusterErrorState

This alert indicates that the Ceph storage cluster has been in error state for more than 10m.

This alert reflects that the rook-ceph-mgr job has been in error state for an unacceptable amount of time. Check for other alerts that might have triggered prior to this one and troubleshoot those first.

CephMonQuorumAtRisk

This alert indicates that storage cluster quorum is low.

Multiple mons work together to provide redundancy; this is possible because each keeps a copy of the metadata. The cluster is deployed with 3 mons, and requires 2 or more mons to be up and running for quorum and for the storage operations to run. If quorum is lost, access to data is at risk.

If this alert fires, check if any OSDs are in terminating state, if there are any, force delete those pods, and wait for some time for the operator to reconcile. If the issue persists, contact UiPath® support.

cluster-utilization-alert.rules

CephClusterNearFull

This alert indicates that the Ceph storage cluster utilization has crossed 75% and will become read-only at 85%.

If this alert fires, free up some space in Ceph by deleting some unused datasets in AI Center or Task Mining or expand the storage available for Ceph PVC by following the instructions in .

Before resizing PVC, make sure you meet the storage requirements. For details, see Evaluating your storage needs.

CephClusterCriticallyFull

This alert indicates that Ceph storage cluster utilization has crossed 80% and will become read-only at 85%.

If this alert fires, free up some space in Ceph by deleting some unused datasets in AI Center or Task Mining or expand the storage available for Ceph PVC by following the instructions in .

Before resizing PVC, make sure you meet the storage requirements. For details, see Evaluating your storage needs.

CephClusterReadOnly

This alert indicates that Ceph storage cluster utilization has crossed 85% and will become read-only now. Free up some space or expand the storage cluster immediately.

If this alert fires, free up some space in Ceph by deleting some unused datasets in AI Center or Task Mining or expand the storage available for Ceph PVC by following the instructions in .

Before resizing PVC, make sure you meet the storage requirements. For details, see Evaluating your storage needs.

osd-alert.rules

CephOSDCriticallyFull

When the alert severity is Critical, the available space is less than 20%.

For any services that run out of space, data may be difficult to recover, so you should resize volumes before hitting 10% available space. See the following instructions: Configuring the cluster.

CephOSDNearFull

This alert indicates that the Ceph storage cluster utilization has crossed 75% and will become read-only at 85%.

If this alert fires, free up some space in Ceph by deleting some unused datasets in AI Center or Task Mining or expand the storage available for Ceph PVC by following the instructions in .

Before resizing PVC, make sure you meet the storage requirements. For details, see Evaluating your storage needs.

PersistentVolumeUsageNearFull

This alert indicates that the Ceph storage cluster utilization has crossed 75% and will become read-only at 85%.

If this alert fires, free up some space in Ceph by deleting some unused datasets in AI Center or Task Mining or expand the storage available for Ceph PVC by following the instructions in .

Before resizing PVC, make sure you meet the storage requirements. For details, see Evaluating your storage needs.

persistent-volume-alert.rules

PersistentVolumeUsageCritical

If this alert fires, free up some space in Ceph by deleting some unused datasets in AI Center or Task Mining or expand the storage available for Ceph PVC by following the instructions in .

Before resizing PVC, make sure you meet the storage requirements. For details, see Evaluating your storage needs.

pool-quota.rules

CephPoolQuotaBytesCriticallyExhausted

This alert indicates that Ceph storage pool usage has crossed 90%.

If this alert fires, free up some space in CEPH by deleting some unused datasets in AI Center or Task Mining or expand the storage available for Ceph PVC by following the instructions in .

Before resizing PVC, make sure you meet the storage requirements. For details, see Evaluating your storage needs.

host-disk

LowDiskForRancherPartition

This alert indicates that the free space for the /var/lib/rancher partition is less than:

• 35% – the severity of the alert is warning
• 25% – the severity of the alert is critical

If this alert fires, increase the size of the disk.

LowDiskForKubeletPartition

This alert indicates that the free space for the /var/lib/kubelet partition is less than:

• 35% – the severity of the alert is warning

• 25% – the severity of the alert is critical

  If this alert fires, increase the size of the disk.

LowDiskForVarPartition

This alert indicates that the free space for the /var partition is less than:

• 35% – the severity of the alert is warning
• 25% – the severity of the alert is critical

If this alert fires, increase the size of the disk.