Use generic script datasources

Introduction

UiPath Process Mining contains functionality that supports processing data with external tools, such as R script, Python, JavaScript, Ruby, Swift, etc. that can be used as a datasource in your application.

As an application developer you can define which attributes need to be exported to the external process.

Note: Only attributes which are explicitly made available for external processing are available for use in the external process.

Processing data with external tools

Overview

When the datasource is reloaded, the UiPath Process Mining platform will output the defined attributes into a file. The platform then executes the external process with the arguments supplied by the developer.

The external process should do its processing by reading the data from the file and writing the result to standard-output channel. The external process can show messages such as progress indications, warnings, and/or errors by writing to its standard-error channel.

The data on the standard-output channel of the external process is interpreted by the UiPath Process Mining platform as a semi-colon separated and double-quote-quoted data.

See illustration below for an overview of the data flow from the UiPath Process Mining platform to the external process, and back again.

Generic script datasources

The datasource uses themvscript driver, which means it is a script datasource. The script parameter used by the mvscript driver refers to the generic script. For example, the connection string:

‘driver={mvscript}; script=Rscript’

refers to a Rscript datasource.

 

In this guide, we refer to these datasources as a generic script datasource. In principle, the ‘generic’ script does not only support scripts, but any executable that, given the right combination of arguments:

• can read data from a file, and
• output its result as a semi-colon separated and double-quote-quoted data.

However, the main target for the generic script datasource is running custom scripts written in R, Python, etc.

Server Settings

Before you can use the generic script datasource, you must configure the available handlers in the Server Settings. To do this, add a key GenericScriptHandlers to the Server Settings. The value of this key is an object, where:

• each key denotes an identifier for the script type,
• the value for each key denotes the path to the script’s interpreter executable.

The identifier can be any string. However, it is recommended to use the extension of your script files, because the UiPath Process Mining platform will try to auto-detect the correct handler from the script file’s extension. Below is an example of the GenericScriptHandlers Server Setting.

"GenericScriptHandlers": {
    "r": "D:/R/R.exe",
    "py": "P:/Python/bin/python.exe",
    "perl": "C:/Strawberry/perl/bin/perl.exe"
}

The keys with the extensions ‘.r’ and ‘.py’ will automatically select the R and Python executables.

Note: For Perl scripts, which typically have the extension ‘.pl’, the autodetection will not select the “perl” handler. In this case the advanced query parameter scriptType can be specified to identify the desired handler.

Permissions

The Windows user that runs Microsoft (i.e. IIS_IUSRS) must have the correct permission to execute the executables referred from the script handlers. Otherwise, the executable cannot be found.

Data sources

To configure a generic script datasource, you must create a connection string table which uses the mvscript driver with the generic script. The generic script has several parameters, of which typically only the scriptFile and inputData parameters need to be specified, as the rest can be inferred by the UiPath Process Mining platform.

 

Parameter overview

The following table gives an overview of the generic script parameters.

Parameter	Description	Mandatory Y/N
scriptFile	The path to your script. For example, resources/my_script.r. Note: instead of scriptFile you can also use the scriptText argument.	N
scriptType	Which handler to use for the scriptFile. Should be one of the handlers in the GenericScriptHandlers.	N
inputData	The data that should be passed to the script for processing.	Y
debugPath	An absolute path to a folder. If this parameter is specified, UiPath Process Mining will output: • the input file (as given to the external process), and • the external process’s output (as read from the external process’ standard output) in this folder for debugging purposes.	N
args	An array of additional arguments to pass to the script’s handler, e.g. when using Python, this can be used to pass the -I flag to enable isolated mode. • At least one of the array elements should be the text <SCRIPT> to indicate where on the command line the path to the script file should be inserted. • At least one of the array elements should be the text <INPUT> to indicate where on the command line the path to the input file should be inserted.	N

Table Scope

The location of the script is interpreted as relative to the table scope. Therefor the table scope must be set to either Server or Workspace. When it is set to None, the path to the script file does not exist and data loading will fail. You can set the table scope in the Advanced table options.

Data

The data can be exported in any way that is most convenient to the receiving external process.

Follow these steps to export the data that you want to process in an external script.

Step	Action
1	Determine the fields that need to be available in the script.
2	Create a new global table in your application.
3	Create a look-up expression attribute in the global table for each field of Step 1.

Note: This table can then be exported so that the script can read it in. The csvtable() function can be used to convert the data into a CSV-like format. Since this function expects a list of (text) records the result type of the look-up expression must be a list of text, i.e. one record with all data encoded in a list.

API Interface

The interface between UiPath Process Mining and any external process accessed through the generic script datasource is defined as described below.

Step	Description
1	UiPath Process Mining creates a file containing the data specified by the inputData parameter.
2	The external process: • reads the data from the file; • performs its processing; • writes the result to its standard-output channel in a semi-colon separated and double-quote-quoted format. The first line of output should be a header specifying column names. For example: "Case_ID"; "Amount" "110553"; "73650" "110565"; "14420" "110689"; "262370" "110754"; "3500" "110939"; "50270" "111155"; "2400" "111251"; "27530" "111251"; "27530" "111397"; "123450" "111418"; "100"
3	The external process writes any status messages to its standard-error channel.
4	The external process exits with either a status code of zero, indicating success, or a non-zero status code, indicating failure.
5	UiPath Process Mining evaluates the status code returned by the external process If the status code is: • zero, the platform will parse the external process’s standard-output channel as semi-colon separated and double-quote-quoted data. • non-zero, the platform will display an error message, including any messages written to the standard-error channel by the external process. The table load is aborted.