Subscribe

UiPath Process Mining

The UiPath Process Mining Guide

Loading data using Theobald Xtract Universal

Introduction

This page describes how to use Theobald Xtract Universal to load data from your SAP source system into a process app in Process Mining (Cloud). Refer to the official Theobald Software website for more information on Theobald Xtract Universal.

Theobald Xtract Universal provides fast data access and extractions to SAP ECC and SAP HANA data sources. For general instructions on how to use Theobald Xtract Universal, see the official Theobald Software documentation Getting Started with Theobald Xtract Universal.

📘

Note:

Loading data using Theobald Xtract Universal is currently only available for the SAP Purchase-to-Pay app template.

Prerequisites

It is assumed that you have:

📘

Note

Loading data using Theobald Xtract Universal requires that the command line tool (xu.exe) can be executed without additional authentication.

Shared Access Signature (SAS) uri

To be able to upload the data to Automation CloudTM you need to have the Shared Access Signature (SAS) uri of the Azure blob storage to which the extracted data needs to be uploaded. See Obtaining the SAS uri of a process app.

Loading data using Theobald Xtract Universal

Setting up data load using Theobald Xtract Universal requires several steps to be performed.

  1. Import the template extractions
  2. Configure the source
  3. Configure the extraction script
  4. Set up the data uploader
  5. Run the extraction script

The steps are described in detail below.

1: Importing the template extractions

Follow these steps to import the template extractions in Theobald Xtract Universal.

StepAction
1Download the XU_template_extractions.zip file. This XU_template_extractions.zip file contains the folders destinations, extractions, and sources.
2Copy the folders from the downloaded XU_template_extractions.zip file to the config folder of Xtract Universal. The location of this folder is [XU_installation_path]/config.
3Open Xtract Universal Designer and click on the refresh button in to load all extractions.

A list of all extractions that each extract one table from the SAP system is displayed. The template source is named SAP and the template destination is named CSV. See the illustration below.

10611061

2: Configuring the source

Follow these steps to configure the SAP source in Theobald Xtract Universal.

StepAction
1Go to Server -> Manage sources and edit the SAP source.
2Fill in the settings to connect to your SAP system.

3: Configuring the extraction script

Follow these steps to configure the extraction script.

StepAction
1Download the extract_theobald.zip file. This extract_theobald.zip file contains the extract_theobald.ps1 script and the config.json file.
2Unzip the content of the extract_theobald.zip file on the server where Theobald Xtract Universal is installed.
3Configure the variables for the script in the config.json as listed below.
VariableDescription
xu_locationThe Xtract Universal installation folder. This is the folder where you can find the xu.exe.
xu_serverAddress of the server where Xtract Universal is installed. The default is localhost.
xu_portPort of the server where Xtract Universal is installed. The default port of an installation is 8065.
azure_urlThe SAS URL of the Azure blob storage to which the extracted data needs to be uploaded.
extraction_start_dateStart date to be used to limit extractions. Only tables where we a date filter is applied are filtered on this date. Format is YYYYMMDD. The default is no limitation indicated by the start date 19700101.
Note: extraction_start_date must have a value.
extraction_end_dateEnd date to be used to limit extractions. Only tables where we a date filter is applied are filtered on this date. Format is YYYYMMDD. The default is no limitation indicated by the end date 20991231.
Note: extraction_end_date must have a value.
languageThe language in which data is extracted from the source system. The default is E.
exchange_rate_typeThe exchange rate type that is used for currency conversion. The default is M.

4: Setting up the data uploader

Follow these steps to set up the data uploader.

StepAction
1The Theobald Xtract Universal extraction extracts data from the SAP system and puts the data in .csv files in an output folder of the Theobald Xtract Universal installation folder. The data uploader will upload the files to the azure blob storage.
2Download the DataUploader. The DataUploader.zip file contains a node.exe and datarun.js.
3Save the node.exe and datarun.js files in the same folder as the extract_theobald.ps1 script.

5: Running the extraction script

To load the data and upload it to Azure for use in Process Mining (Cloud) you must run the extract_theobald.ps1 file.
The time taken for this task will depend highly on the data volumes loaded. The output is uploaded to the blob storage which triggers processing in Snowflake.

Starting the file manually

Follow this step to start the data run.

StepAction
1Double-click on the extract_theobald.ps1 file to start the data run.

Schedule a task for the data run

Instead of running the file manually, you can use Windows Task Scheduler to schedule a task that runs the batch script for automated data refreshes.

📘

Note

It is recommended to select the local SYSTEM user account for running the task and to select the Run with highest priority option in the Create Task window when creating the task in Windows Scheduler.

The output is uploaded to the blob storage which triggers processing in Snowflake.

🚧

Important

If you add an interval for which there is no data available, the dashboards will show an error message.

Troubleshooting

The data run also creates a log file that can help in case of unexpected results or if the data upload with the data uploader fails. The file extraction.log contains the logs of the last data run. For each extraction that failed, a return code is displayed. See the official Theobald Software documentation Call via Commandline for more information. When something with the setup is incorrect, this is mentioned in the error message.

Extraction issues with CDPOS for non-HANA systems

In some older, non-HANA systems, CDPOS is a cluster table and may result in a “join with pool table” error such as below.

940940

The extraction of the CDPOS table includes by default a join with the CDHDR table to limit the number of records that need to be extracted. Follow these steps to overcome this error.

  1. Edit the CDPOS extraction in Xtract Universal Designer to remove the CDDHR from the tables.
  2. Replace the CDHDR~UDATE between @extraction_start_date and @extraction_end_date with:

CHANGENR between @CHANGENR_min and @CHANGENR_max

  1. In the extract_theobald.ps1, replace &"$xu_location\xu.exe" -s "$xu_server" -p "$xu_port" -n "CDPOS_raw" -o "extraction_start_date=$extraction_start_date" -o "extraction_end_date=$extraction_end_date" with the code displayed below.
$CDHDR = Import-Csv -Path "$xu_location\output\CDHDR_raw.csv" -Delimiter "`t"
$CDHDR_sorted = $CDHDR | Sort-Object -Property CHANGENR 
$CHANGENR_min=$CDHDR_sorted.GetValue(0).CHANGENR
$CHANGENR_max=$CDHDR_sorted.GetValue($CDHDR_sorted.Count-1).CHANGENR
&"$xu_location\xu.exe" -s "$xu_server" -p "$xu_port" -n "CDPOS_raw" -o "CHANGENR_min=$CHANGENR_min" -o "CHANGENR_max=$CHANGENR_max"

Updated 11 days ago

Loading data using Theobald Xtract Universal


Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.