Loading data using DataBridgeAgent

Attention:

DataBridgeAgent is only needed if you are migrating from an on-premises stand-alone Process Mining version.

Only use DataBridgeAgent if you have a custom .mvp connector. Otherwise, use CData Sync, Theobald Xtract Universal for SAP, or files to upload data.

Important:

Using a proxy server is not supported for loading data using DataBridgeAgent.

Important:

Always make sure that the data is in the required input format for the app template. See App Templates. This can be done either in the Data transformations, or by renaming the output fields of the on-premises stand-alone connector (.mvp).

Introduction

Similar to extracting data and preparing a dataset for use in a on-premises (stand-alone) Discovery Accelerator, you must extract data from the source system and create a dataset for use in Process Mining. This page describes how you can extract data from your source system for use in Process Mining using DataBridgeAgent.

DataBridgeAgent is used to extract data from the source system, and exports .tsv files. These .tsv files are loaded into the Microsoft SQL Server database using CData Sync. See the illustration below for an overview of the data flow.

Important: Only use DataBridgeAgent if you have a custom .mvp connector. Otherwise, use CData Sync, Theobald Xtract Universal for SAP, or file to upload data.

Using .mvp Connectors

In Process Mining (Automation Suite), an existing Process Mining on-premises (standalone) connector (.mvp file) can be loaded using DataBridgeAgent. This enables you to reuse the existing Data transformations, and load the output data into a process app in Process Mining Automation Suite.

See Adding a custom connector to DataBridgeAgent for a detailed description on how to add a .mvp connector file created in Process Mining on-premises (standalone) to DataBridgeAgent.

Downloading DataBridgeAgent

Use the following link to download DataBridgeAgent: https://download.uipath.com/ProcessMining/versions/21.10.3/DataBridgeAgent/DataBridgeAgent-21.10.3.zip

Note:

You can only use .mvp connectors of a Process Mining on-premises standalone version that is the same version as the DataBridgeAgent. (v2021.10.3).

DataBridgeAgent is a standalone tool. It is recommended to create a separate folder on the server to use DataBridgeAgent.

Note:

Make sure to install DataBridgeAgent in a folder on the C:\ drive. Since DataBridgeAgent contains connectors that have rather long names, file paths can become very large, which may cause issues when running the datarun.bat file to create the dataset.

Do not install DataBridgeAgent in the in the Process Mining installation folder (PLATFORMDIR). Also, do not install DataBridgeAgent in the Downloads folder.

DataBridgeAgent

DataBridgeAgent is a standalone package that can be used to extract data from your source system to the blob storage for use in Process Mining Automation Suite. It contains the Process Mining on-premises (stand-alone) build and the on-premises .mvp connector used to prepare the dataset. Currently, the following on-premises .mvp connectors are included in DataBridgeAgent:

SAP Connector for Purchase-to-Pay Discovery Accelerator; See Using DataBridgeAgent with SAP Connector for Purchase-to-Pay Discovery Accelerator.
SAP Connector for Order-to-Cash Discovery Accelerator; See Using DataBridgeAgent with SAP Connector for Order-to-Cash Discovery Accelerator.

If you want to use your own custom .mvp connector, you can add it to DataBridgeAgent. In this case, you need to configure DataBrigdeAgent for use with your connector. See Adding a custom connector to DataBridgeAgent.

Important:

DataBridgeAgent automatically converts input field names to lowercase and special characters to underscores. If you use DataBridgeAgent to load data, make sure to edit the transformations of your process app and set all input fields to lowercase. For example:

{{ pm_utils.to_varchar('Cases_base_raw."Case_ID"') }} as "Case_ID", must be changed to

{{ pm_utils.to_varchar('Cases_base_raw."case_id"') }} as "Case_ID",.

Difference from an on-premise installation

DataBridgeAgent is different from an on-premise (stand-alone) installation since it:

is a command-line tool, and therefore does not have a user interface;
does not require a separate license;
does not require IIS.

Note: The DataBridgeAgent already includes the connectors. You do not need to download and deploy the connector itself.

Prerequisites

See System requirements.

Setup credentials for the SQL Server database

Uploading data to Process Mining (Automation Suite) requires access to the SQL Server database location to which the extracted data needs to be uploaded. To set up a SQL Server destination connection you need the following setup parameters to authenticate access to the SQL Server database location.

Server
Database
Schema
Role

See Retrieving the SQL Server database parameters

Extracting data from an SAP source system

If you want to use data directly from your SAP system in Process Mining Automation Suite you must set up the SAP extraction before you can use DataBridgeAgent. See Set up Native SAP Extraction for a detailed description.

Extracting data via loading .csv files

If you want to use data loaded from .csv files always make sure that:

a separate .csv file is available for each table.
the file names of the .csv files are the same as the names of the input tables of the connector.
all the fields used in Process Mining are present in the .csvfile.
the fields in the .csv files have the same names as the field names used in Process Mining column.

The CSV settings can be defined in the CSV parameters of the DataBridgeAgent.

Extracting data via an ODBC connection

If you want to use data from an external data source via an ODBC connection, make sure that the required ODBC driver is installed on the server.

Note: All databases that have a 64-bit version can be accessed using an ODBC connection. The required 64-bit ODBC driver must be installed on the Process Mining server, and the server must be able to access that specific system.