Data transformations

Editing transformations locally

When you create a process app from an app template, you want to make sure the data used in the process app reflects your business process in the correct way. With Data transformations you can customize the data used for Process Mining.

You can either customize the transformations from within Process Mining using Data transformations, or you can edit them locally on your desktop, see Editing data transformations in a local environment

This page describes how to customize data transformations from within Process Mining. Data transformations are integrated in the dashboard editor. See Working with the dashboard editor.

Data transformations enable you to:

• add new fields for group by and filters. For example, Quality check type.

• add events. For example, Invoice quality check done.

• add tags. For example, Failed to meet SLA, Four eyes violation.

• add business logic for calculating KPIs. For example, Case_has_maverick_buying.

Prerequisites

When starting editing data transformations it is strongly recommended that you:

• have in-depth knowledge of SQL;

• are familiar with the source system that is used for data extraction.

Opening Data transformations

1. Select the Data transformations button in the upper right corner of the the dashboard editor to open the Data transformations.

Note:

If you open the in-line editor for the first time, it will take some time to load the editor.

The in-line data transformations editor is displayed.

Input data

The input data panel shows the input tables that have been loaded into the process app as raw data. Select a table to see the fields and the data contents in the data preview. The data preview enables you to check if the input data looks as expected.

Note:

The preview shows 1000 records of data. If you want to filter for specific data, create a temporary debug query, see Data transformations.

Note:

The Preview panel shows the data of the last data run in which this table was recalculated. If you have made recent changes, then start a new data run to view its results, see Editing and testing data transformations.

Transformations

1. Select Transformations to view the structure of the transformations and to display the .sql files.

See Structure of transformations for more information on the structure of the transformations.

Each .sql file defines a new table in the data model. If you select the a .sql file in the Transformations panel, the SQL query is displayed in the Transformations and a preview of the data file you are editing is displayed in the Preview panel. See the illustration below for an example.

When editing a query, you can see a preview of the data of the last time the query was run in the Preview panel.

Important:

If there are any unsaved changes in the transformations, the Save button is enabled. Select Save to save the changes. The status of the transofrmation will be set to Up to date.

Selecting visible columns

You can select the columns you want to display in the Preview panel.

Follow these steps to change the visible columns.

Step	Action
1	Select Columns to display the list of columns.
2	Select the columns you want to display in the Preview panel from the list.

Viewing and editing the data model

Important:

Always make sure your data model adheres the requirements. See Data model requirements.

1. Select Data model to view the data model of your process app.

See Editing and testing data transformations for more information on how to change the data model.

Validation checking

A notification is displayed if your data model does not meet the requirements.

See Data model requirements.

Adding a table

1. Select + Add table. The Add table dialog is displayed.

2. Select the table that defines the new output table.

3. Select the Primary key for the new table, and select Done.

See Adding tables for more information on how to add a table in the transformations.

Adding relations

1. Select the table that you want to relate to another table.

   The Edit table panel is displayed.

2. In the Edit table panel, select + Add new to create a new relation.

3. Select the field that you want to use in this table from the Key list.

4. Select the Table you want to connect to and select the field that use to connect from the Key list.

5. Select Apply.

Changing the key for a table

1. Select the table for which you want to change the key.

   The Edit table panel is displayed.

2. In the Edit table panel, locate the relation for which you want to change the key.

3. Select the new field that is to be used as the Key to join the tables.

4. Select Apply.

Note:

If you change the primary key, the field that you select as the new primary key is duplicated in the table. You can delete the original field in Data Manager.

Deleting a relation

1. Select the table for which you want to delete an outgoing relation.

   The Edit table panel is displayed.

2. In the Edit table panel, locate the relation you want to delete and select Delete relation.

3. Select Apply.

Deleting a table

1. Select the table that you want to delete in the data model editor.

   The Edit table panel is displayed.

2. Select Delete table.

   A confirmation message is displayed.

3. Select Delete to continue.

The table and the relations are deleted from the data model.

Making the new data model available for use in dashboards

1. Select Save to save the data model.

2. Select Apply to dashboards to run the transformations and make new the table available for use in dashboards.

   Note:

   This may take several minutes. If the run finishes successfully, the changes to the data model will show up in the Data Manager.

Note:

The Save option is only enabled, after you made any changes to the data model.

Exporting and importing transformations

Exporting transformations

Follow these steps to export the transformations from the process app.

Steps	Action
1	Go to the Transformations panel and select the Import or export transformations icon to open the menu.
2	Select the Export transformations option.

The transformations are that are used for the last full data run are exported and downloaded to your default download folder as a .zip file.

Importing transformations

Follow these steps to import the customized transformations in the process app.

Step	Action
1	Go to the Transformations panel and select the Import or export transformations icon to open the menu.
2	Note: If you have added new input tables or new input fields for existing tables to the data transformations, you must upload a new dataset before importing the data transformations. Select the Import transformations option.

The transformations are imported and and run, and the new data is displayed in the process app.

Important:

When the transformations are imported, the new transformations are automatically run. This will immediately affect the data displayed in the published process app. It is strongly recommended to test the new transformations in a separate test process app to prevent any errors.

Running an erroneous transformation will result in the published app not being visible for end users.

Note:

If you select Cancel in the Import successful message dialog, the imported transformations will be available in the Data transformations editor, but the transformations are not run.

Viewing the transformations Log

The Log panel shows what happens in the background. The Log panel is refreshed automatically every couple of seconds. See the illustration below for an example.

Log levels

The log contains different log levels to indicate the severity of the message:

Log level	Description	Data run status
Information	An Information message contains helpful information on the progress of the datarun.	Success
Warning	A Warning refers to a potential problem in your data that might affect what will be displayed on a chart in the published process app. It is advised to resolve any warnings to prevent any potential future problems.	Success
Error	An Error refers to a mistake in your data that prevents the process app from loading the data or running the transformations. You must resolve all errors to enable a successful data run.	Failed

Filtering logs

You can use the Filter menu to change the log level. See the illustration below for an example.

Saving the log file

You can select the icon to copy the contents of the log file and paste in, for example, a Notepad file that you can save on your computer. This enables you to view the messages when working on solving the issues causing errors or warnings.

Note:

This also enables you to share the contents of the log file, for example if you need support.