Extension for Java

Supported versions

The Java Extension is compatible with any version of JRE's, from Java 3 to Java 17.

When trying to indicate or validate selectors in a Java application, the Robot checks if the extension files are loaded in the target application process.

The files may exist there through various mechanisms:

• by having the extension installed in the JRE (for Java 3 to 8) through the extension mechanism.
• by loading them from a custom folder (for Java 3 to 17, with different steps to follow for Java 9+).

If the extension files are not there, an injection mechanism is used to try to start our bridge.

Note:

The Java Extension installer embedded in Studio (and also the ScreenScrapeJavaSupport tool) relies on the extension mechanism to have the extension loaded by the JRE, so it can be used for automating Java-based applications running with Java 3 to 8.

Starting with Java 9, the extension mechanism has been deprecated and the installer is not able to add the necessary files to the JRE. These files must be loaded from a custom folder (more info here).

Starting with UIAutomation package version 22.2.0-preview, with the support of the default injection mechanism, it is no longer mandatory to install the Extension for Java for automating Java-based applications.

You should install the Java extension from Studio or the command line only when the injection mechanism does not work, and when Java applications are using Java 3 to 8. For Java applications using Java 9 or greater, follow the steps described in Installing the Extension in a custom folder.

Injection mechanism

Until UiPath.UIAutomation.Activities version 22.2.0-preview, the injection mechanism didn't work in all cases. Loading the extension from a custom folder was the main method used for applications launched with Java 9+.

Starting with UiPath.UIAutomation.Activities version 22.2.0-preview, the injection mechanism has become more reliable and is able to automate applications that couldn't be automated before, like .jnlp applications launched with third-party applications, such as OWS or IcedTea Web.

The injection mechanism is used by default and the Java extension installation should only be used as a backup method.

Installing the Extension in the JRE from Studio or the command line

For Java 8 or lower

The UiPath Extension for Java is installed as follows:

From Studio

1. Close any running Java applications.
2. Open Studio.
3. Navigate to Tools > UiPath Extensions.
4. Select Java > Install. A confirmation message appears, informing you that the extension has been installed.

From the command line

Run this command C:\Program Files\UiPath\Studio\UiPath\SetupExtensions.exe /Java in a command prompt window with administrator rights.

Studio and UI Explorer also ask to install the UiPath Extension for Java when you try to select a control inside a Java app and when the bridge is not detected. If you choose to install the UiPath Extension for Java when prompted by UI Explorer, the Java Bridge files get deployed only in the JRE directory which launched the target Java application instead of the JDK directory.

Note: To install this extension, you must have administrator rights.

To check if the extension is not working properly, open UIExplorer and try to select controls inside a Java app. If the entire window is selected, it means that the extension is not working properly.

You can tell whether or not the extension is properly installed by the generated selectors for Java controls:

• Valid Selector for Java Controls: <wnd app=”...” /><java role=”...” />
• Invalid Selector for Java Controls: <wnd app=”...” /><ctrl role=”client” />

Note: The UiPath Extension for Java is compatible only with AWT apps (e.g. Java Swing, Oracle Forms, Fujitsu JBK), while SWT apps are automated with Active Accessibility. To detect the type of your Java application, check the cls attribute (window class name) in UI Explorer. The class name should be similar to SunAwtFrame or javax.swing.

Processes can interact with Java applications regardless of their DPI scaling.

The UiPath Extension for Java can also be installed using the ScreenScrapeJavaSupport tool.

For Java 9 or greater

Use the installation in a custom folder, as described in the next section.

Installing the Extension in a custom folder

For Java 8 or lower

To have the UiPath Extension for Java installed in a custom location, perform the following steps:

1. Create a new folder on your machine in which you wish to install the UiPath Extension for Java (such as C:\UiPathJavaExt) and copy the following files to it:
   1. UiPathBridge.jar - located in the %UserProfile%\.nuget\packages\uipath\20.xx.xxxx\build\UiPath\JavaSupport directory.
   2. UiPathJavaBridgeV8.dll and UiPathJavaBridgeV8_x64.dll located in the %UserProfile%\.nuget\packages\uipath\20.xx.xxxx\build\UiPath directory. It is not necessary to copy both files, but it can prevent issues which can occur due to version mismatch between the JRE and the UiPathJavaBridge.
2. The Java Extension Setup mechanism used by UiPath to install the extension deploys the UiPathBridge.jar and accessibility.properties files in the Java installation directory (usually C:\Program Files (x86)\Java\jre1.8.0_xx). To use the UiPath Extension for Java from a custom folder, the dependency for these files must be loosened, as follows:
   1. For the UiPathBridge.jar file, you need to specify the folder in which you copied the UiPathBridge.jar and UiPathJavaBridgev8.dll (such as C:\UiPathJavaExt) when you start the Java application which uses the extension. This is done through the -Djava.ext.dirs = C:\UiPathJavaExt argument.
   2. For the accessibility.properties file, there are two methods:
      • Add the assistive_technologies=UiPath.UiPathBridge parameter to the accessibility.properties global configuration file located in the %userprofile% directory. Please note that this file is different from the one deployed by the UiPath Extension for Java, and its settings have precedence over the ones set by the Java installation.
      • Use the -Djavax.accessibility.assistive_technologies=UiPath.UiPathBridge argument when you run the target Java application.

System properties need to be properly passed to the target Java application. This can be done through several methods:

• Include the -Djava.ext.dirs=C:\UiPathJavaExt -Djavax.accessibility.assistive_technologies=UiPath.UiPathBridge system property in the _JAVA_OPTIONS environment variable. The JRE loads the properties from the environment variable and the UiPath Extension for Java loaded in the target application.
• Use the C:\Program Files (x86)\Java\jre1.8.0_xx\bin\java.exe -Djavax.ext.dirs=C:\UiPathJavaExt -Djavax.accessibility.assistive_technologies=UiPath.UiPathBridge -jar SwingSet2.jar command inside command prompt. A confirmation message appears that the UiPath Extension for Java is successfully loaded.
• Include the -Djava.ext.dirs=C:\UiPathJavaExt -Djavax.accessibility.assistive_technologies=UiPath.UiPathBridge system property in the JAVA_TOOL_OPTIONS environment variable. Please note that this method only works with Java 5 or greater.

Passing the properties to a Java applet is done with one of the following methods:

1. Using the appletviewer.exe application.
   This method requires you to first create the .java.policy file in the %userprofile% folder, and should contain the following:

   grant { 
       permission java.net.URLPermission "url_of_the_Java_applet", "<em>:</em>"; 
   };

   Once this is done, the target applet needs to be started via appletviewer.exe with this command:

   C:\Program Files (x86)\Java\jre1.8.0_xx\bin\appletviewer.exe -J-Djava.ext.dirs=C:\UiPathJavaExt -J-Djavax.accessibility.assistive_technologies=UiPath.UiPathBridge url_of_the_java_applet

2. Using the Java Control Panel which is delivered with the JRE.

   The Java Control Panel allows you to specify the runtime parameters for each registered JRE. As such, you need to specify the

   -Djava.ext.dirs=C:\UiPathJavaExt -Djavax.accessibility.assistive_technologies=UiPath.UiPathBridge

   parameter for the target applet. This makes the UiPath Extension for Java load every time the Java applet starts.

The table below present the compatibility between the methods used above, Java versions, and different types of Java applications:

Application Type	_JAVA_OPTIONS	JAVA_TOOL_OPTIONS	Command Line Parameters	Java Control Panel - Runtime Parameters
Java Desktop Application	Java 3 to Java 8	Java 5 to Java 8	Java 3 to Java 8	Not Supported
Java Contained Application (.exe)	Java 3 to Java 8	Java 5 to Java 8	Java 3 to Java 8 (1)	Not Supported
Java Applet	Java 3 to Java 8	Java 5 to Java 8	Not Supported	Java 3 to Java 8
Java Oracle Forms	Not Supported	Not Supported	Not Supported	Not Supported

(1) - Depending on the command line tool you use, additional parameters might need to be passed through the environment variables.

For Java 9 or greater

Starting with Java 9, the Java Extension Mechanism is removed, which means that the -Djava.ext.dirs argument can not be used on Java 9 and greater applications.

The main idea behind loading the extension can be split into 3 requirements:

1. Add the UiPathBridge.jar and UiPathServiceProvider.jar to the classpath of the application.
2. Make UiPathJavaBridgeV8.dll and UiPathJavaBridgeV8_x64 visible to UiPathBrdige.jar. This can be done in various ways:
   • place the files in the same folder with UiPathBridge.jar;
   • place the files in the folder above UiPathBridge.jar;
   • add the two files to a folder and reference its path via an environment variable called UIPATH_JB_PATH;

     It is not necessary to reference both files, but it can prevent issues that can occur due to version mismatch between the JRE and the UiPath Java Bridge.

3. Configure the application to load assistive technology. This is done via the accessibility.properties file which is placed inside the JREs.
   Note: The UiPathJavaServiceProvider.jar file is available in the latest UiPath.UIAutomation.Activities packages (newer than 21.2-preview). If you are using a lower version of UIAutomation, the file can be obtained by temporarily upgrading an empty workflow file to 21.2-preview. This causes unpacking a newer version of the UiPath.nuget package to the user machine (%UserProfile%\.nuget\packages\uipath). After getting the file, the package can be downgraded to the version used before the upgrade.

Fulfilling the requirements above depends directly on the way the Java application is started. There are 3 common ways of starting the application and loading the extension.

Applications started from .jar, .bat, or .ps1 files

Let's consider the simple case when the app is started directly from simple .jar files: java -jar SwingSet2.jar

The following steps need to be performed:

1. Create a new folder on your machine in which you wish to install the UiPath Extension for Java (such as C:\UiPathJavaExt) and copy the following files to it:
   • UiPathBridge.jar and UiPathJavaServiceProvider.jar - located in the %UserProfile%\.nuget\packages\uipath\20.xx.xxxx\build\UiPath\JavaSupport directory.
   • UiPathJavaBridgeV8.dll and UiPathJavaBridgeV8_x64.dll - located in the %UserProfile%\.nuget\packages\uipath\20.xx.xxxx\build\UiPath directory.
2. Add the previously created folder to the classpath of the target Java application. This is done with the -cp, -classpath command. For this example, we consider SwingSet2 the target Java application:

   C:\Program Files (x86)\Java\jre1.9.0_xx \bin\java.exe -cp SwingSet2.jar;C:\UiPathJavaExt\* -Djavax.accessibility.assistive_technologies=UiPathJavaServiceProvider SwingSet2

If the application is started via a .bat or a .ps1 file, then the file must be analyzed with a text editor. The file that starts the application contains a line similar to the command java -jar SwingSet2.jar, but also includes many other parameters. The command can be changed to include the files in the classpath and to specify the assistive technology parameter. Changes inside those files should be done with caution. In some cases, the classpath and the additional parameters are picked from environment variables, which is a less invasive method.

Applications started from .exe files

When the app is started via an .exe file, you must find an alternative way of loading the files to the classpath and specifying the assistive technology parameter.

1. To load the files from the classpath, you can search for environment variables that the executable reads or for configuration files that specify where the dependencies are loaded. There is nothing standard here, and it depends a lot on the level of customization of the executable.
2. The assistive technology can be specified in this case in a manner similar to the one that the extension installer does. The standard accessibility.properties file, can be moved to the %UserProfile% folder and renamed to .accessbility.properties.

Applications started from .jnlp files

Starting with UiAutomation v22.2, interacting with .jnlp type applications is done by default due to an upgrade of the existing injection mechanism.

Application scaling support

Scaling is fully supported for Java 9+ applications via parameters. Consider adding the following parameters to the Java application you want to automate to enable scaling support:

• sun.java2d.uiScale.enabled - When unused or set to the exact string value of "true", it is considered true and the following scaling configuration is taken into account. When set to any other value, but "true", (such as True, false, False, abc) then it is considered false and no additional scaling adjustments are made.
• sun.java2d.uiScale - Values can be passed in three forms:
  • As ratio (0.5, 2.3, 1.5)
  • As percentages (100%, 150%)
  • As dpi values (96dpi, 120dpi). It is considered if scaling is enabled. It modifies scaling on both axis.
• sun.java2d.win.uiScaleX, sun.java2d.win.uiScaleY Values can be passed as for the sun.java2d.uiScale parameter. Both of them are necessary to be considered by the Java application. They are more specific than the general configuration of scaling As such, if all 3 parameters are set, only these are taken into account.