Subscribe

UiPath Process Mining

The UiPath Process Mining Guide

Loading data using DataBridgeAgent

Introduction

Similar to extracting data and preparing a dataset for use in AppOne or a Discovery Accelerator, you must extract data from the source system and create a dataset for use in Process Mining (Cloud). This page describes how you can extract data from your source system for use in Process Mining (Cloud) using DataBridgeAgent.

Downloading the DataBridgeAgent

Use the following link to download the DataBridgeAgent https://download.uipath.com/ProcessMining/versions/21.10/DataBridgeAgent/DataBridgeAgent-21.10.0.zip.

DataBridgeAgent

DataBridgeAgent is a stand-alone package that can be used to extract data from your source system to SnowFlake for use in Process Mining (Cloud). It contains the UiPath Process Mining build and the on-premises connector used to prepare the dataset. Currently, the following on-premises connectors are included in DataBridgeAgent:

Difference from an on-premise installation

DataBridgeAgent is different from an on-premise installation since it:

  • is a command-line tool, and therefore does not have a user interface;
  • does not require a separate license;
  • does not require IIS.

📘

Note

The DataBridgeAgent already includes the connectors. You do not need to download and deploy the connector itself.

Prerequisites

See also System requirements.

Shared Access Signature (SAS) uri

To be able to upload the data to Automation CloudTM you need to have the SAS uri of the Azure blob storage to which the extracted data needs to be uploaded.

The SAS is a url that includes all the information in its query parameters needed to authenticate access to the Azure blob storage location See the illustration below for an example.

Obtaining the SAS uri for a new app

Follow these steps to obtain the SAS uri for a new app.

Step

Action

1

Open the Process Mining Portal. See Process Mining Portal

2

Create a new process app.
Note: Select the applicable Discovery Accelerator in the Process App type step.

3

In the Upload data using extractor step, click on imageimage to copy the upload uri to the clipboard.

4

Finish the steps to create the app.

Obtaining the SAS url for an existing app.

Follow these steps to obtain the SAS url for an existing app.

Step

Action

1

On the Process Apps tab, locate the app for which you want to obtain the SAS uri in the list of process apps.

2

Click on imageimage and select Upload data from the context menu.

3

Enable the Upload data using extractor option in the Data source step and click on Ingest data and publish app button.

4

In the Upload data using extractor step, click on imageimage to copy the upload uri to the clipboard.

5

Click on Finish.

Extracting data from an SAP source system

If you want to use data directly from your SAP system in Automation Cloud you must set up the SAP extraction before you can use DataBridgeAgent. See Set up Native SAP Extraction for a detailed description.

Extracting data via loading .csv files

If you want to use data loaded from .csv files always make sure that:

  • a separate .csv file is available for each table.
  • the file names of the .csv files are the same as the names of the input tables of the connector.
  • all the fields used in Process Mining are present in the .csvfile.
  • the fields in the .csv files have the same names as the field names used in Process Mining column.

The CSV settings can be defined in the CSV parameters of the DataBridgeAgent.

Extracting data via an ODBC connection

If you want to use data from an external data source via an ODBC connection, make sure that the required ODBC driver is installed on the server.

🚧

Important

All databases that have a 64-bit version can be accessed using an ODBC connection. The required 64-bit ODBC driver must be installed on the UiPath Process Mining server, and the server must be able to access that specific system.

Using a credential store

A Credential store allows you to use passwords without showing the actual password. See Set up a Credential Store for a detailed description.

By default, an encrypted credential store is used. See Setting up an encrypted file-based credential store for more information.

📘

Note

To change the Server Settings you edit the <EXTRACTORDIR>\databases\ServerSettings.json file.

Configuring the DataBridgeAgent

Follow these steps to configure DataBridgeAgent.

Step

Action

1

Download DataBridgeAgent.

2

On the server, create a folder for the DataBridgeAgent. For instance, D:\processmining\P2P_data\.

Note: In the remainder of this guide, we will refer to this directory as <EXTRACTORDIR>.

3

Place the installation package in the <EXTRACTORDIR> folder.
• Right-click on the installation package.
• Select Extract All….

4

Right-click on the file <EXTRACTORDIR>\datarun.json and select Open.

5

Enter a value for the following settings:
azureURL
Connector
Input type
Use credential store

Generic parameters

Below is an overview of the generic parameters for DataBridgeAgent.

Parameter

Description

azureURL

The SAS URL of the Azure blob storage to which the extracted data needs to be uploaded. See Loading data with Databridge agent

Connector

The location of the connector used to load the data and to create the dataset.

Input type

Can be either:
SAP see SAP parameters
CSV see CSV parameters
ODBC see ODBC parameters
Note: depending on the preferred input type, you must enter the settings in the corresponding section.

Use credential store

Indicate whether or not a credential store is used for password storage.
Note: if set to true you specify the password identifier in de SAP Password or ODBC Password field.

Reporting currency

The currency in which price-related values are displayed.

Exchange rate type

The exchange rate type that is used for currency conversion.

Language

The language in which data is extracted from the source system.

Extraction start date

The start date of the extraction period of the data.
Note: In cases where only a subset of the data is needed, it is recommended to limit the amount of data loaded, while this may improve the loading times.

Extraction end date

The end date of the extraction period of the data.
Note: In cases where only a subset of the data is needed, it is recommended to limit the amount of data loaded, while this may improve the loading times.

SAP parameters

Below is an overview of the parameters that can be used for SAP datasources.

Parameter

Description

SAP Host

The hostname or IP address of the SAP application server.

SAP SystemNumber

The two-digit number between 00 and 99 that identifies the designated instance.

SAP Username

The username of the account that is being used to log in to the SAP instance.

SAP Password

The password of the user above.
Note: If you use a credential store, you must enter the password identifier from the credential store, instead of the password.

SAP Client

The client that is being used.

CSV parameters

Below is an overview of the parameters that can be used for CSV datasources.

SAP Setting

Description

CSV Data path

Data path in the Server Data that points to the place where the .csv files are stored. For example P2P/ if all files can be found in the folder named P2P.

CSV Suffix

A regular expression containing the files extension of the file to read in. May contain a suffix up to 2 digits that are added to the name of the table.

CSV Delimeter

The delimiter character that is used to separate the fields.

CSV Quotation character

The quote character that is used to identify fields that are wrapped in quotes.

CSV Has header

Indicate whether the first line of the .CSV file is a header line.

ODBC parameters

Below is an overview of the parameters that can be used for ODBC datasources.

Parameter

Description

ODBC Driver

The name of the ODBC driver to use for this connection.

ODBC Username

Username to be used to connect to the external datasource.

ODBC Password

Password to be used to connect to the external datasource.
Note: If you use a credential store, you must enter the password identifier from the credential store, instead of the password.

ODBC Connection parameters

Any other parameters are passed as specified to the ODBC driver. Use the format param1=value1 (;param2=value2)

Creating the dataset and upload it for use in Automation Cloud

To load the data and upload it to Azure for use in Automation Cloud you must run the EXTRACTORDIR\datarun.bat file.

The time taken for this task will depend highly on the data volumes loaded.
The output is uploaded to the blob storage which triggers processing in Snowflake.

Starting the file manually

Follow this step to start the data run.

Step

Action

1

Double-click on the EXTRACTORDIR\datarun.bat file to start the data run.

Schedule a task for the data run

Instead of running the file manually, you can use Windows Task Scheduler to schedule a task that runs the batch script for automated data refreshes.

📘

Note

It is recommended to select the local SYSTEM user account for running the task and to select the Run with highest priority option in the Create Task window when creating the task in Windows Scheduler.

The output is uploaded to the blob storage which triggers processing in Snowflake.

🚧

Important

If you add an interval for which there is no data available, the dashboards will show an error message.

Troubleshooting

The data run also creates a log file that can help in case of unexpected results or if the data upload with DataBridgeAgent fails. The file <EXTRACTORDIR>\datarun.txt contains the logs of the last data run.

Updated 8 days ago

Loading data using DataBridgeAgent


Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.