Loading Data Using Theobald Xtract Universal

Introduction

This page describes how to use Theobald Xtract Universal (XU) to load data from your SAP source system into a process app. Refer to the official Theobald Software website for more information on Theobald Xtract Universal (XU).

Theobald Xtract Universal provides fast data access and extractions to SAP ECC and SAP HANA data sources. For general instructions on how to use Theobald Xtract Universal, see the official Theobald Software documentation Getting Started with Theobald Xtract Universal.

Prerequisites

It is assumed that you have:

• a valid Theobald Xtract Universal license.
  Note:

  The license is provided via UiPath for the purpose of Process Mining.

• installed Theobald Xtract Universal. See the official Theobald Software documentation for more information on installing an updating Theobald Xtract Universal and customizing the function module for table extraction in SAP.

Note: Loading data using Theobald Xtract Universal requires that the command line tool (xu.exe) can be executed without additional authentication.

Setting up Theobald Xtract Universal

Theobald Xtract Universal is the SAP certified extraction tool used for data extraction from SAP source systems, integrating with SAP through Remote Function Calls (RFC). Theobald Xtract Universal contains the read-only custom function module Z_THEO_READ_TABLE-transport1 that is to be installed in SAP.

You can use the following link to download the custom module: Z_THEO_READ_TABLE-transport1.zip.

Attention:

The latest version of the Z_THEO_READ_TABLE-transport1.zip file will always be included when you download Xtract Universal.

Follow these steps to set up Theobald Xtract Universal.

1. Download the latest trial version of Theobald Xtract Universal from our website, see Download Trial Version. Install Theobald Xtract Universal on a Windows Server / Virtual Machine.

2. Check if the network access to your SAP system is open, see SAP TCP/IP Ports.

3. Grant SAP user rights to access to the SAP instance, see Authority Objects - SAP User Rights for Table.

4. The following components in Theobald Xtract Universal require customization in SAP: Z_THEO_READ_TABLE.

5. Configure Theobald Xtract Universal for the required app template(s). See App Templates.

Note:

Steps 2-4 will require validation from your SAP team.

Shared Access Signature (SAS) Uri

To be able to upload the data to Process Mining Cloud you need to have the Shared Access Signature (SAS) uri of the Azure blob storage to which the extracted data needs to be uploaded. See Obtaining the Authentication Credentials for an App.

Loading Data Using Theobald Xtract Universal

Setting up data load using Theobald Xtract Universal requires several steps to be performed.

1. Import the template extractions;
2. Configure the source;
3. Configure the extraction script;
4. Set up the data uploader;
5. Run the extraction script.

The steps are described in detail below.

Importing the Template Extractions

Follow these steps to import the template extractions in Theobald Xtract Universal.

1. Go to the App Templates page and open the documentation page for your app template.

   Note:

   You can log in using the default user.

   

   

2. Locate the section Loading data using Theobald Xtract Universal and download the XU_template_extractions.zip file for you app template. This XU_template_extractions.zip file contains the folders destinations, extractions, and sources.

3. Copy the folders from the downloaded XU_template_extractions.zip file to the config folder of Xtract Universal. The location of this folder is [XU_installation_path]/config.

4. Open Theobald Xtract Universal Designer and click on the refresh button to load all extractions.

A list of all extractions that each extract one table from the SAP system is displayed. The template source is named SAP and the template destination is named CSV.

Configuring the Source

Follow these steps to configure the SAP source in Theobald Xtract Universal.

1. Go to Server - Manage sources and edit the SAP source.
2. Fill in the settings on the General and Authentication tabs to connect to your SAP system.
3. Test the connection.

Configuring the Extraction Script

Follow these steps to configure the extraction script.

1. Go to the App Templates page and open the documentation page for your app template.

2. Locate the section Loading data using Theobald Xtract Universal and download the extract_theobald.zip file for your app template. This extract_theobald.zip file contains the extract_theobald.ps1 script and the config.json file.

3. Unzip the content of the extract_theobald.zip file on the server where Theobald Xtract Universal is installed.

4. Configure the variables for the script in the config.json as listed below.

Variable	Description
xu_location	The Xtract Universal installation folder. This is the folder where you can find the xu.exe. For example: C:\\MyFolder.
xu_server	Address of the server where Xtract Universal is installed. The default is localhost.
xu_port	Port of the server where Xtract Universal is installed. The default port of an installation is 8065.
sas_uri	The SAS uri of the Azure blob storage to which the extracted data needs to be uploaded. See Retrieving the Credentials for the Azure Blob Storage.
end_of_upload_api	The API that is called to start data processing in Process Mining, once all data has been uploaded. See Retrieving the Credentials for the Azure Blob Storage.
extraction_start_date	Start date to be used to limit extractions. Only tables where a date filter is applied are filtered on this date. Format is YYYYMMDD. The default is no limitation indicated by the start date 19700101. Note: extraction_start_date must have a value.
extraction_end_date	End date to be used to limit extractions. Only tables where a date filter is applied are filtered on this date. Format is YYYYMMDD. The default is no limitation indicated by the end date 20991231. Note: extraction_end_date must have a value.
language	The language in which data is extracted from the source system. The default is E.
exchange_rate_type	The exchange rate type that is used for currency conversion. The default is M.

Setting up the Data Uploader

The Theobald Xtract Universal extraction extracts data from the SAP system and puts the data in .csv files in an output folder of the Theobald Xtract Universal installation folder. The data uploader will upload the files to the azure blob storage.

Follow these steps to set up the data uploader.

1. Download the DataUploader.
2. Save the file in the same folder as the extract_theobald.ps1 script.

Running the Extraction Script

To load the data and upload it to for use in Process Mining you must run the extract_theobald.ps1 file.

The time taken for this task will depend highly on the data volumes loaded.

The output is uploaded to the blob storage which triggers processing in Snowflake.

Starting the file manually

Follow this step to start the data run.

1. Run the extract_theobald.ps1 file to start the data run.

The data run also creates a log file that can help in case of unexpected results or if the data upload with the data uploader fails. The file extraction.log contains the logs of the last data run. For each extraction that failed, a return code is displayed. See the official Theobald Software documentation Call via Commandline for more information. When something with the setup is incorrect, this is mentioned in the error message.

Schedule a task for the data run

Instead of running the file manually, you can use Windows Task Scheduler to schedule a task that runs the batch script for automated data refreshes.

Note: It is recommended to select the local SYSTEM user account for running the task and to select the Run with highest priority option in the Create Task window when creating the task in Windows Scheduler.

The output is uploaded to the blob storage which triggers processing in Snowflake.

Important: If you add an interval for which there is no data available, the dashboards will show an error message.

When the extraction is successful, data ingestion may fails because of some faulty data being extracted. You can filter out this data should using Theobald Xtract Universal using the WHERE clause. Refer to the official Theobald documentation for more information on the WHERE Clause.

Note:

If you have a WHERE clause that uses variables (for example the extraction date) you cannot filter via the UI.