Setting up a local test environment

Introduction

To edit data transformations, a local test setup is recommended for an improved edit and test experience for data engineers.

A local test environment includes the following set of tools.

Tool	Usage
dbt	to execute the data transformations on a local database.
Visual Studio Code	to edit the SQL code of the data transformations.
SQL Server (Express edition)	for testing the data transformations.
SQL Server Management Studio	for reviewing the result of the data transformations in the local database.

Prerequisites

To install and run a dbt project, you need Python 3.9. You can download Python 3.9 from the official Python website.

Create a Python virtual environment

It is recommended to create a Python virtual environment in which you will install dbt. It is good practice to create a dedicated folder for your transformations. You can also use this folder to create the Python virtual environment.

Step	Action
1	Open Windows Explorer and create a folder where your virtual environment will be located. For example, C:\My_transformations.
2	Open a Windows Command Prompt.

Run the commands described in the steps below to create a Python Virtual environment.

Step	Action	Command
1	Install the Python package virtualenv.	py -m pip install virtualenv
2	Go to the folder where you want to create the environment.	cd [path_to_your_folder]. For example, cd C:\My_transformations
3	Create a virtual environment (named venv).	py -m virtualenv venv

Install Dbt

Make sure the virtual environment is still activated. Follow these steps to install dbt to run transformations.

Important:

Running scripts must be enabled on your system. Follow these steps:

1. Open PowerShell in Administrator mode.

2. Enter the command Set-ExecutionPolicy RemoteSigned or Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser.

3. Enter Y when prompted.

Step	Action	Command
1	Activate the virtual environment.	venv\Scripts\activate
2	Install the dbt package	pip install dbt-sqlserver==1.1.0
3	Check whether the installation is successful	dbt --version

Visual Studio Code

Visual Studio Code is the recommended code editor for editing data transformations.

Install Visual Studio Code

You can download Visual Studio Code from Download Visual Studio Code webpage.

Extensions

After you have installed Visual Studio Code, install the following extensions to make it easier to work with dbt:

• Dbt Power User

Follow these steps to install an extension in Visual Studio Code.

Step	Action
1	Start Visual Studio Code
2	Go to the Extensions panel (CTRL+SHIFT+X). See the illustration below.
3	Search for the dbt Power User extension by start typing the name in the Search Extensions in Marketplace text box.
4	Click on Install.

See the illustration below.

Set Python interpreter path

To run Python from the virtual environment that was created, the path needs to be set in Visual Studio Code.

Follow these steps.

Step	Action
1	Go to File > Preference > Settings.
2	Search for Python.
3	In the Default Interpreter Path, make sure to check the path to the python.exe. It should be located in the [path_to_your_folder]\venv\Scripts folder. See the illustration below for an example.

Microsoft SQL Server

Microsoft SQL Server is the required database to test the transformations. This database server is not provided as part of the Process Mining product. For editing and testing the transformation you can also use SQL Server Express. You can download Microsoft SQL Server Express from the official Microsoft SQL Server downloads webpage.

Note: It is recommended to test the transformations using small development datasets. This makes it possible to test these using a SQL Server with minimal requirements. If you do not have a Microsoft SQL Server available, or if you want to test the transformations on a local desktop machine, it is recommended to use Microsoft SQL Server Express.

Note:

By default, SQL Server is case insensitive whereas Process Mining is case sensitive. You are advised to change the behavior of your local SQL Server database to match this behavior, to prevent any problems. This can be accomplished by setting the right collation during installation. The default value for the collation is dependent on your locale.

To change the collation to become case sensitive, replace the CI part with CS, e.g. Latin1_General_CI_AS should be changed to Latin1_General_CS_AS. If you already installed SQL Server, follow instructions describe in Set or change the server collation to update the collation.

Minimum requirements

Below is an overview of the minimum requirements for the SQL Server for a local development environment. If you want to use more than 10M records in your development environment, please use the Capacity calculator to calculate the corresponding system requirements for the SQL Server.

Type	Minimum requirements (10M events)
Version	SQL Server 2019 or above
CPU	8 vCPU/cores (16 recommended)
Memory	32 GiB RAM
Disks	Low latency & dedicated disks are recommended
Data disks	256 GiB
Temp DB	128 GiB

Capacity calculator

You can use the Capacity calculator to determine the hardware requirements for setting up a dedicated Microsoft SQL Server machine for Process Mining. See Capacity calculator.

SQL Server Management Studio (SSMS)

To view the database tables and to manage the SQL Server infrastructure, you can download SQL Server Management Studio (SSMS), which can be installed on any computer.