Saving robot logs to Elasticsearch

Orchestrator is set up by default to store robot logs in the database. However, if you want to switch this storage to Elasticsearch, a basic configuration process is available: simply provide the required URL, username, and password. This setup should suffice for most use cases. For further customization, you may leverage the advanced configuration mechanism on top of the basic setup.

Basic configuration

If you opt for this configuration, you allow robot logs to be written to the Elasticsearch target instead of the database, and set up Orchestrator to retrieve the logs from Elasticsearch.

You can configure robot logs to be saved to an Elasticsearch server by editing the input.json file.

The basic configuration supports Elasticsearch version 7.x. For Elasticsearch 8.x, you need to use the advanced configuration.

The following table lists out the orchestrator.orchestrator_robot_logs_elastic parameters:

Parameter	Description
orchestrator_robot_logs_elastic	Elasticsearch section configuration. This contains the following properties: uri, username, password.
elastic_uri	The address of the Elasticsearch instance that should be used. It should be provided in the form of a URI. If provided, then username and password are also required.
elastic_auth_username	The Elasticsearch username, used for authentication.
elastic_auth_password	The Elasticsearch password, used for authentication.

Applying the configuration via the uipathctl installer

Update your input.json to add the orchestrator_robot_logs_elastic section under orchestrator.

{
    ...
    "orchestrator": {
        ...
        "orchestrator_robot_logs_elastic": {
            "elastic_uri": "uri",
            "elastic_auth_username": "user",
            "elastic_auth_password": "pass"
        }
        ...
    }
    ...
}

To run the uipathctl installer to apply the configuration, use the following command:

uipathctl manifest apply input.json --only orchestrator --versions versions.json

Advanced configuration

To access the advanced configuration settings, you must first complete the basic configuration. This process allows you to customize properties of the NLog target that communicates with Elasticsearch. However, redirecting robot logs requires additional settings. These necessary adjustments are performed through the basic configuration.

Important:

Any changes you make per the following steps bcan negatively affect the functionality and stability of the entire system. It is advisable to only make changes if you understand their consequences.

The advanced configuration allows you to fully customize your NLog.config target:

1. Follow the basic configuration steps.

2. Edit the nlog.json sample parameters to update the robotElasticBuffer target with the desired configuration. See the following sections for samples of nlog.json.
3. Update the nlog.json parameters:

   uipathctl config orchestrator update-config --nlog-config nlog.json

nlog.json

Orchestrator includes a pre-set NLog target for robot logs, called robotElasticBuffer. You can modify any properties of this target using the custom nlog.json file. It is not necessary to provide the entire target configuration; instead, add or update only the desired values.

Elasticsearch 7.x configuration sample

You can skip the configuration of uri, requireAuth, username, and password fields, since they are already set in the basic configuration step.

{
  "Nlog": {
    "targets": {
      "robotElasticBuffer": {
        "flushTimeout": 1000,
        "bufferSize": 1000,
        "slidingTimeout": false,
        "target": {
          "uri": "https://elastic.example.com:9200",
          "requireAuth": true,
          "username": "elastic-user",
          "password": "elastic-password",
          "index": "${event-properties:item=indexName}-${date:format=yyyy.MM}",
          "documentType": "logEvent",
          "includeAllProperties": true,
          "layout": "${message}",
          "excludedProperties": "agentSessionId,tenantId,indexName"
        }
      }
    }
  }
}

Elasticsearch 8.x configuration sample

You can skip the configuration of uri, requireAuth, username, and password fields, since they are already set in the basic configuration step.

{
  "Nlog": {
    "targets": {
      "robotElasticBuffer": {
        "flushTimeout": 1000,
        "bufferSize": 1000,
        "slidingTimeout": false,
        "target": {
          "uri": "https://elastic.example.com:9200",
          "requireAuth": true,
          "username": "elastic-user",
          "password": "elastic-password",
          "index": "${event-properties:item=indexName}-${date:format=yyyy.MM}",
          "documentType": "",
          "includeAllProperties": true,
          "layout": "${message}",
          "excludedProperties": "agentSessionId,tenantId,indexName"
        }
      }
    }
  }
}

Managing Orchestrator custom configuration

This section provides additional information on how to provide the custom configuration to Orchestrator.

When the existing configuration scenarios are not enough for the required use case, there is another way to change theOrchestrator configuration: uipathctl config orchestrator.

Orchestrator configuration

The Orchestrator configuration for appsettings or for NLog is loaded from multiple configuration files that are loaded one after the other. Each new loaded file overwrites the keys from the previous ones. The order in which the files are loaded is:

• built-in configuration file from the Orchestrator image

• orchestrator configmap
• orchestrator secret
• orchestrator-customconfig configmap

The built-in configuration file for NLog contains a target for writing the robot logs to the database, and configures the NLog robot logs rule to use this target. The built-in appsettings file configures Orchestrator to read the RobotLogs from this target when they are needed for display. The NLog built-in configuration file also contains also robotElasticBuffer target, partially configured and not used.

The orchestrator configmap and secret are managed by ArgoCD via the helm chart. Each time ArgoCD synchronizes the application, these configs are regenerated using the current parameters and values for the orchestrator ArgoCD application.

When the basic configuration enables Elasticsearch, the argo parameters are changed, and the orchestrator configmap and secret are updated to provide the final configuration required to enable the robotElasticBuffer.

The advanced configuration means adding overrides in the orchestrator-customconfig configmap. Every new or modified value is merged in the existing configuration, but values cannot be removed (they can be set empty but not completely removed).

NLog default configuration

The default configuration provided by Orchestrator for NLog includes the pre-configured robotElasticBuffer target.

{
  "NLog": {
    "targets": {
      "robotElasticBuffer": {
        "type": "BufferingWrapper",
        "flushTimeout": 5000,
        "target": {
          "type": "ElasticSearch",
          "name": "robotElastic",
          "requireAuth": false,
          "uri": "",
          "username": "",
          "password": "",
          "index": "${event-properties:item=indexName}-${date:format=yyyy.MM}",
          "documentType": "logEvent",
          "includeAllProperties": true,
          "layout": "${message}",
          "excludedProperties": "agentSessionId,tenantId,indexName"
        }
      },
    }
  }
}

Orchestrator custom configuration via uipathctl

To view more information about updating Orchestrator custom config, run the following command:

uipathctl config orchestrator --help

Output:

Configure orchestrator custom config

Usage:
  uipathctl config orchestrator [command]

Available Commands:
  get-config    Get the current orchestrator custom config
  update-config Update orchestrator config

Flags:
  -h, --help   help for orchestrator

Global Flags:
      --context string      name of the kubeconfig context to use
      --kubeconfig string   kubectl configuration file (default: ~/.kube/config)
      --log-format string   log format. one of [text,json] (default "text")
      --log-level string    set log level. one of [trace,debug,info,error] (default "error")
  -q, --quiet               suppress all output except for errors and warnings
      --timeout duration    timeout of the command (default 1h0m0s)

Note: The advanced configuration also supports Elasticsearch 8.x.