MongoDB Certificate Renewal for Apps Users

Overview

MongoDB uses two types of certificates for a secure connection within the Automation Suite cluster:

• CA root certificate
• TLS certificate

The MongoDB CA certificate for Automation Suite versions prior to 2021.10.3 is only valid for 60 days. Since there is no auto-renewal process in place, a manual procedure is needed to update the certificate. Follow the steps from the Manual certificate update section to renew the certificates.

Note: The manually updated certificates are only valid for another 90 days. After this, the certificates have to be manually updated again.

For Automation Suite versions 2021.10.4 and later, the expiry is updated to three years. For fresh installs of 2021.10.4 and later, the certificate renewal is automatic.

For environments upgraded from version 2021.10.3 and earlier, some manual steps are needed. Follow the steps from the Certificate rotation section to update the certificates.

Manual Certificate Update

Note: Follow this procedure to update MongoDB certificates manually only if you are using Automation Suite up to and including 2021.10.3.

Online Environment

This procedure is applicable for online environments.

Prerequisites

Before starting the procedure, create a script file named mongo-cert-rotation-script.sh containing the information below.

mongo-cert-rotation-script.sh

export KUBECONFIG=/etc/rancher/rke2/rke2.yaml PATH=$PATH:/var/lib/rancher/rke2/bin:/usr/local/bin
curl -sSL -o kubectl-cert-manager.tar.gz https://github.com/cert-manager/cert-manager/releases/download/v1.6.0/kubectl-cert_manager-linux-amd64.tar.gz
tar xzf kubectl-cert-manager.tar.gz
sudo mv kubectl-cert_manager /usr/local/bin
function update_additional_secrets() {
  #extract updated pem file name
  newPemFileName=$(kubectl -n mongodb get secret mongodb-replica-set-server-certificate-key -o json | jq -r '.data'| jq -r keys[0])
  echo "New pem file name ${newPemFileName}"
  #extract stale pem file name
  oldPemFile=$(kubectl -n mongodb get secret mongodb-replica-set-config -o json | jq -r '.data."cluster-config.json"' | base64 -d | jq -r '.processes[0].args2_6.net.tls.certificateKeyFile')
  oldPemFileName=$(basename "$oldPemFile")
  echo "Stale pem file name ${oldPemFileName}"
  if [[ "$oldPemFileName" != "$newPemFileName" ]]; then
    echo "Pem file entries do not match. replacing"
    #extract replica set secret  cluser config json to file
    kubectl -n mongodb get secret mongodb-replica-set-config -o json | jq -r '.data."cluster-config.json"' | base64 -d > /tmp/clusterConfig.json
    #replace old pem file name with new in the json file
    sed -i -e "s@$oldPemFileName@$newPemFileName@g" /tmp/clusterConfig.json
    #encode the json
    encodedUpdatedClusterConfig=$(jq -r '. | @base64 | "\)\)(.)"' /tmp/clusterConfig.json)
    #patch replica set secret with updated cluster config
    kubectl -n mongodb patch secret mongodb-replica-set-config --type='json' -p='[{"op" : "replace" ,"path" : "/data/cluster-config.json" ,"value" : "'"$encodedUpdatedClusterConfig"'"}]'
  else
    echo "Pem file entries match; not updating"
  fi
}
function rotate_secrets(){
    NAMESPACE="mongodb"
    
    #shellcheck disable=SC2154
    HOME_DIR=$(eval echo "~$whoami")
    echo "extracting certs and secrets from relevant files"
    #cleanup if the dir already exists
    rm -rf "$HOME_DIR"/tmp/.certs || true
    mkdir -p "$HOME_DIR"/tmp/.certs/
    kubectl -n "$NAMESPACE" get secret/tls-ca-key-pair -o jsonpath='{.data.ca\)\).crt}' | base64 -d > "$HOME_DIR"/tmp/.certs/ca.crt
    kubectl -n "$NAMESPACE" create configmap mongo-ca --from-file="$HOME_DIR/tmp/.certs/ca.crt" --dry-run=client -o yaml | kubectl apply -f -
    
    kubectl -n mongodb label  configmap mongo-ca config-discovery=yes 2>/dev/null || true
    
}
PREVIOUS_REVISION=$(kubectl -n mongodb get cert cert-manager-tls-certificate -o json | jq -r '.status.revision')
echo "Previous Version"
echo $PREVIOUS_REVISION
kubectl cert-manager renew --namespace=mongodb --all
sleep 60
CURRENT_REVISION=$(kubectl -n mongodb get cert cert-manager-tls-certificate -o json | jq -r '.status.revision')
echo "Current Version"
echo $CURRENT_REVISION
#Validate if Cert gets renewed
if [[ "${PREVIOUS_REVISION}" != "${CURRENT_REVISION}" ]]; then
  echo "Cert Renewal Successful. Previous Revision: $PREVIOUS_REVISION Current Revision: $CURRENT_REVISION"
  echo "Rotating secrets"
  rotate_secrets
  echo "Secrets rotated"
  echo "Updating additional secrets"
  update_additional_secrets
  echo "Additional secrets updated"
  echo "Rolling restart mongodb replica set"
  kubectl rollout restart sts mongodb-replica-set -n mongodb
  echo "Mongodb replica successfully Restarted"
  echo "Rolling restart apps server"
  kubectl rollout restart  -n uipath deployment apps-server
  echo "Apps server successfully restarted"
  echo "Rolling restart apps-wsserver"
  kubectl rollout restart  -n uipath deployment apps-wsserver
  echo "Apps wsserver successfully restarted"
else
  echo "Cert Renewal UnSuccessful. Previous Revision: $PREVIOUS_REVISION Current Revision: $CURRENT_REVISION"
fi
rm -rf /usr/local/bin/kubectl-cert_manager

Procedure

1. Copy the script from Prerequisites (mongo-cert-rotation-script.sh) to the virtual machine (VM) node by running the following commands:

   scp <path-to-cert-script> <username>@<node-fqdn>:~

2. Connect to the machine using SSH.

   ssh <username>@<node-fqdn>

3. Check the current expiration and renewal dates by running the command below. Look for the notBefore and notAfter fields.

   export KUBECONFIG=/etc/rancher/rke2/rke2.yaml PATH=$PATH:/var/lib/rancher/rke2/bin:/usr/local/bin
   kubectl -n mongodb describe certs

4. Copy the script from the home directory to the installer directory by running the following commands:

   mv /home/<username>/mongo-cert-rotation-script.sh <installer-dir>
   cd <installer-dir>

5. Run the script using the following commands:

   chmod u+x mongo-cert-rotation-script.sh
   ./mongo-cert-rotation-script.sh

6. Check the new expiration and renewal dates by running the command below. Look for the notBefore and notAfter fields.

   kubectl -n mongodb describe certs

Offline Environment

This procedure is applicable for offline (airpgapped) environments.

Prerequisites

Before starting the procedure, create a script file named mongo-cert-rotation-script.sh containing the information below.

mongo-airgap-cert-rotation-script.sh

export KUBECONFIG=/etc/rancher/rke2/rke2.yaml PATH=$PATH:/var/lib/rancher/rke2/bin:/usr/local/bin
tar xzf kubectl-cert-manager.tar.gz
sudo mv kubectl-cert_manager /usr/local/bin
function update_additional_secrets() {
  #extract updated pem file name
  newPemFileName=$(kubectl -n mongodb get secret mongodb-replica-set-server-certificate-key -o json | jq -r '.data'| jq -r keys[0])
  echo "New pem file name ${newPemFileName}"
  #extract stale pem file name
  oldPemFile=$(kubectl -n mongodb get secret mongodb-replica-set-config -o json | jq -r '.data."cluster-config.json"' | base64 -d | jq -r '.processes[0].args2_6.net.tls.certificateKeyFile')
  oldPemFileName=$(basename "$oldPemFile")
  echo "Stale pem file name ${oldPemFileName}"
  if [[ "$oldPemFileName" != "$newPemFileName" ]]; then
    echo "Pem file entries do not match. replacing"
    #extract replica set secret  cluser config json to file
    kubectl -n mongodb get secret mongodb-replica-set-config -o json | jq -r '.data."cluster-config.json"' | base64 -d > /tmp/clusterConfig.json
    #replace old pem file name with new in the json file
    sed -i -e "s@$oldPemFileName@$newPemFileName@g" /tmp/clusterConfig.json
    #encode the json
    encodedUpdatedClusterConfig=$(jq -r '. | @base64 | "\)\)(.)"' /tmp/clusterConfig.json)
    #patch replica set secret with updated cluster config
    kubectl -n mongodb patch secret mongodb-replica-set-config --type='json' -p='[{"op" : "replace" ,"path" : "/data/cluster-config.json" ,"value" : "'"$encodedUpdatedClusterConfig"'"}]'
  else
    echo "Pem file entries match; not updating"
  fi
}
function rotate_secrets(){
    NAMESPACE="mongodb"
    
    #shellcheck disable=SC2154
    HOME_DIR=$(eval echo "~$whoami")
    echo "extracting certs and secrets from relevant files"
    #cleanup if the dir already exists
    rm -rf "$HOME_DIR"/tmp/.certs || true
    mkdir -p "$HOME_DIR"/tmp/.certs/
    kubectl -n "$NAMESPACE" get secret/tls-ca-key-pair -o jsonpath='{.data.ca\)\).crt}' | base64 -d > "$HOME_DIR"/tmp/.certs/ca.crt
    kubectl -n "$NAMESPACE" create configmap mongo-ca --from-file="$HOME_DIR/tmp/.certs/ca.crt" --dry-run=client -o yaml | kubectl apply -f -
    
    kubectl -n mongodb label  configmap mongo-ca config-discovery=yes 2>/dev/null || true
    
}
PREVIOUS_REVISION=$(kubectl -n mongodb get cert cert-manager-tls-certificate -o json | jq -r '.status.revision')
echo "Previous Version"
echo $PREVIOUS_REVISION
kubectl cert-manager renew --namespace=mongodb --all
sleep 60
CURRENT_REVISION=$(kubectl -n mongodb get cert cert-manager-tls-certificate -o json | jq -r '.status.revision')
echo "Current Version"
echo $CURRENT_REVISION
#Validate if Cert gets renewed
if [[ "${PREVIOUS_REVISION}" != "${CURRENT_REVISION}" ]]; then
  echo "Cert Renewal Successful. Previous Revision: $PREVIOUS_REVISION Current Revision: $CURRENT_REVISION"
  echo "Rotating secrets"
  rotate_secrets
  echo "Secrets rotated"
  echo "Updating additional secrets"
  update_additional_secrets
  echo "Additional secrets updated"
  echo "Rolling restart mongodb replica set"
  kubectl rollout restart sts mongodb-replica-set -n mongodb
  echo "Mongodb replica successfully Restarted"
  echo "Rolling restart apps server"
  kubectl rollout restart  -n uipath deployment apps-server
  echo "Apps server successfully restarted"
  echo "Rolling restart apps-wsserver"
  kubectl rollout restart  -n uipath deployment apps-wsserver
  echo "Apps wsserver successfully restarted"
else
  echo "Cert Renewal UnSuccessful. Previous Revision: $PREVIOUS_REVISION Current Revision: $CURRENT_REVISION"
fi
rm -rf /usr/local/bin/kubectl-cert_manager

Procedure

1. Download the kubectl-cert-manager.tar file on your system by running the following command:

   curl -sSL -o kubectl-cert-manager.tar.gz https://github.com/cert-manager/cert-manager/releases/download/v1.6.0/kubectl-cert_manager-linux-amd64.tar.gz

2. Copy the certificate manager binary to the VM node by running the following command:

   scp <path-to-kubectl-cert-manager> <username>@<node-fqdn>:~

3. Copy the script from Prerequisites (mongo-airgap-cert-rotation-script.sh) to the VM node by running the following commands:

   scp <path-to-mongo-airgap-cert-rotation> <username>@<node-fqdn>:~

4. Connect to the machine using SSH.

   ssh <username>@<node-fqdn>

5. Copy the kubectl-cert-manager.tar file to the installer directory by running the following command:

   mv /home/<username>/kubectl-cert_manager.tar.gz <installer-dir>

6. Copy the mongo-airgap-cert-rotation-script.sh script to the installer directory by running the following command:

Note: Make sure that both mongo-airgap-cert-rotation-script.sh and kubectl-cert-manager.tar files are at the same path.

mv /home/<username>/mongo-airgap-cert-rotation-script.sh <installer-dir>
cd <installer-dir>

7. Check the current expiration and renewal dates by running the command below. Look for the notBefore and notAfter fields.

export KUBECONFIG=/etc/rancher/rke2/rke2.yaml PATH=$PATH:/var/lib/rancher/rke2/bin:/usr/local/bin
kubectl -n mongodb describe certs

8. Run the script using the following commands:

chmod u+x mongo-airgap-cert-rotation-script.sh
./mongo-airgap-cert-rotation-script.sh

9. Check the new expiration and renewal dates by running the command below. Look for the notBefore and notAfter fields.

kubectl -n mongodb describe certs

Certificate validity dates

When checking for certificate validity, the following dates are displayed:

Not After - this is the expiry date and time of your certificate validiy period.

Not Before - this is the starting date and time of your certificate validity period.

Renewal Time - this is the recommended date and time for manually renewing your certificate.

For example:

Certificate 1 in the file

Name:        cert-manager-tls-certificate

Namespace:   mongodb

Not After:              2023-03-08T23:19:49Z

Not Before:             2022-03-08T23:19:49Z

Renewal Time:           2023-02-06T23:19:49Z

Certificate Rotation

Note: Follow this procedure to update MongoDB certificates manually only if you upgraded from Automation Suite up to and including 2021.10.3 to a later version (2021.10.4 at least).

1. SSH to the VM node.

   ssh <username>@<node-fqdn>

2. Assume the super user role by running the command below.

   sudo su

3. Go to the installation (for fresh installations) or upgrade (for upgraded environments) directory by running the command below.

   cd <Installation/Upgrade Directory>

4. Copy the rotate-cert.sh script below to the installation directory.

   ./configureUiPathAS.sh mongodb rotate-certificate
   kubectl -n mongodb get secret/tls-ca-key-pair -o jsonpath='{.data.ca\)\).crt}' | base64 -d > /tmp/.certs/ca.crt
   kubectl -n mongodb create configmap mongo-ca --from-file="/tmp/.certs/ca.crt" --dry-run=client -o yaml | kubectl replace -f -
   kubectl rollout restart sts mongodb-replica-set -n mongodb

5. Run the certificate rotation script by using the command below.

   chmod u+x rotate-cert.sh
   ./rotate-cert.sh

The certificate rotation script takes around 5-10 minutes to complete. The new certificates generated by the script are valid for three years from the time of creation and are automatically renewed as per the timeline mentioned above.