Loading Data Using DataBridgeAgent
Introduction
Similar to extracting data and preparing a dataset for use in a Process Mining on-premises (standalone) Discovery Accelerator, you must extract data from the source system and create a dataset for use in Process Mining on-premises (Automation Suite). This page describes how you can extract data from your source system for use in Process Mining (Automation Suite) using DataBridgeAgent.DataBridgeAgent is used to extract data from the source system, and exports .tsv
files. These .tsv
files are loaded into the Microsoft SQL Server database using CData Sync. See the illustration below for an overview of the
data flow.
Only use DataBridgeAgent if you have a custom .mvp
connector. Otherwise, use CData Sync, Theobald Xtract Universal for SAP, or files to upload data.
Using .mvp Connectors
In Process Mining (Automation Suite), an existing Process Mining on-premises (standalone) connector (.mvp
file) can be loaded using DataBridgeAgent. This enables you to reuse the existing Data transformations, and load the output data into a process app in Process Mining Automation Suite.
See Adding a Custom Connector to DataBridgeAgent for a detailed description on how to add a .mvp
connector file created in Process Mining on-premises (standalone) to DataBridgeAgent.
Downloading DataBridgeAgent
Use the following link to download DataBridgeAgent: https://download.uipath.com/ProcessMining/versions/21.10.3/DataBridgeAgent/DataBridgeAgent-21.10.3.zip.
You can only use .mvp
connectors of a Process Mining on-premises standalone version that is the same version as the DataBridgeAgent. (v2021.10.3).
DataBridgeAgent is a standalone tool. It is recommended to create a separate folder on the server to use DataBridgeAgent.
Make sure to install DataBridgeAgent in a folder on the C:\
drive. Since DataBridgeAgent contains connectors that have rather long names, file paths can become very large, which may cause issues when running the
datarun.bat
file to create the dataset.
Do not install DataBridgeAgent in the in the Process Mining installation folder (<PLATFORMDIR>
). Also, do not install DataBridgeAgent in the Downloads folder.
DataBridgeAgent
DataBridgeAgent is a standalone package that can be used to extract data from your source system to Microsoft SQL Server for use in Process Mining. It contains the UiPath Process Mining on-premises (standalone) build and the on-premises (standalone) .mvp
connector used to prepare the dataset. Currently, the following on-premises (standalone) connectors are included in DataBridgeAgent:
- SAP Connector for Purchase-to-Pay Discovery Accelerator; See Using DataBridgeAgent With SAP Connector for Purchase-to-Pay Discovery Accelerator.
- SAP Connector for Order-to-Cash Discovery Accelerator; See Using DataBridgeAgent With SAP Connector for Order-to-Cash Discovery Accelerator.
If you want to use your own custom .mvp
connector, you can add it to DataBridgeAgent. In this case, you need to configure DataBrigdeAgent for use with your connector. See Adding a Custom Connector to DataBridgeAgent.
DataBridgeAgent automatically converts input field names to lowercase and special characters to underscores. If you use DataBridgeAgent to load data, make sure to edit the transformations of your process app and set all input fields to lowercase. For example:
{{ pm_utils.to_varchar('Cases_base_raw."Case_ID"') }} as "Case_ID", must be changed to
{{ pm_utils.to_varchar('Cases_base_raw."case_id"') }} as "Case_ID",.
Difference from an on-premises (standalone) installation
DataBridgeAgent is different from an on-premise (standalone) installation since it:
- is a command-line tool, and therefore does not have a user interface;
- does not require a separate license;
-
does not require IIS.
Note: The DataBridgeAgent already includes the connectors. You do not need to download and deploy the connector itself.
Prerequisites
See also System Requirements.
Setup Credentials for the SQL Server Database
Uploading data to Process Mining (Automation Suite) requires access to the SQL Server database location to which the extracted data needs to be uploaded. To set up a SQL Server destination connection you need the following setup parameters to authenticate access to the SQL Server database location.
Server
Database
Schema
Role
Extracting Data From an SAP Source System
If you want to use data directly from your SAP system in Automation Suite you must set up the SAP extraction before you can use DataBridgeAgent. See Set up Native SAP Extraction for a detailed description.
Extracting Data Via Loading .csv Files
If you want to use data loaded from .csv
files always make sure that:
- a separate
.csv
file is available for each table. - the file names of the
.csv
files are the same as the names of the input tables of the connector. - all the fields used in Process Mining are present in the
.csv
file. - the fields in the
.csv
files have the same names as the field names used in Process Mining column.
The CSV settings can be defined in the CSV parameters of DataBridgeAgent.
Extracting Data Via an ODBC Connection
If you want to use data from an external data source via an ODBC connection, make sure that the required ODBC driver is installed on the server.
Using a Credential Store
A Credential store allows you to use passwords without showing the actual password. See Set up a Credential Store for a detailed description.
By default, an encrypted credential store is used. See Setting up an encrypted file-based credential store for more information.
<EXTRACTORDIR>\databases\ServerSettings.json
file.
Next Steps
- Introduction
- Using .mvp Connectors
- Downloading DataBridgeAgent
- DataBridgeAgent
- Difference from an on-premises (standalone) installation
- Prerequisites
- Setup Credentials for the SQL Server Database
- Extracting Data From an SAP Source System
- Extracting Data Via Loading .csv Files
- Extracting Data Via an ODBC Connection
- Using a Credential Store
- Next Steps