- Overview
- UI Automation
- Applications and technologies automated with UI Automation
- Project compatibility
- UI-ANA-016 - Pull Open Browser URL
- UI-ANA-017 - ContinueOnError True
- UI-ANA-018 - List OCR/Image Activities
- UI-DBP-006 - Container Usage
- UI-DBP-013 - Excel Automation Misuse
- UI-DBP-030 - Forbidden Variables Usage In Selectors
- UI-PRR-001 - Simulate Click
- UI-PRR-002 - Simulate Type
- UI-PRR-003 - Open Application Misuse
- UI-PRR-004 - Hardcoded Delays
- UI-REL-001 - Large Idx in Selectors
- UI-SEC-004 - Selector Email Data
- UI-SEC-010 - App/Url Restrictions
- UI-USG-011 - Non Allowed Attributes
- UX-SEC-010 - App/Url Restrictions
- UX-DBP-029 - Insecure Password Use
- UI-PST-001 - Audit Log Level in Project Settings
- UiPath Browser Migration Tool
- Clipping region
- Computer Vision Recorder
- Activate
- Anchor Base
- Attach Browser
- Attach Window
- Block User Input
- Callout
- Check
- Click
- Click Image
- Click Image Trigger
- Click OCR Text
- Click Text
- Click Trigger
- Close Application
- Close Tab
- Close Window
- Context Aware Anchor
- Copy Selected Text
- Element Attribute Change Trigger
- Element Exists
- Element Scope
- Element State Change Trigger
- Export UI Tree
- Extract Structured Data
- Find Children
- Find Element
- Find Image
- Find Image Matches
- Find OCR Text Position
- Find Relative Element
- Find Text Position
- Get Active Window
- Get Ancestor
- Get Attribute
- Get Event Info
- Get From Clipboard
- Get Full Text
- Get OCR Text
- Get Password
- Get Position
- Get Source Element
- Get Text
- Get Visible Text
- Go Back
- Go Forward
- Go Home
- Google Cloud Vision OCR
- Hide Window
- Highlight
- Hotkey Trigger
- Hover
- Hover Image
- Hover OCR Text
- Hover Text
- Image Exists
- Indicate On Screen
- Inject .NET Code
- Inject Js Script
- Invoke ActiveX Method
- Key Press Trigger
- Load Image
- Maximize Window
- Microsoft Azure Computer Vision OCR
- Microsoft OCR
- Microsoft Project Oxford Online OCR
- Minimize Window
- Monitor Events
- Mouse Trigger
- Move Window
- Navigate To
- OCR Text Exists
- On Element Appear
- On Element Vanish
- On Image Appear
- On Image Vanish
- Open Application
- Open Browser
- Refresh Browser
- Replay User Event
- Restore Window
- Save Image
- Select Item
- Select Multiple Items
- Send Hotkey
- Set Clipping Region
- Set Focus
- Set Text
- Set To Clipboard
- Set Web Attribute
- Show Window
- Start Process
- System Trigger
- Take Screenshot
- Tesseract OCR
- Text Exists
- Tooltip
- Type Into
- Type Secure Text
- Use Foreground
- Wait Attribute
- Wait Element Vanish
- Wait Image Vanish
- Application Event Trigger
- Check/Uncheck
- Check App State
- Check Element
- Click
- Click Event Trigger
- Drag and Drop
- Extract Table Data
- For Each UI Element
- Get Attribute
- Get Browser Data
- Get Text
- Get URL
- Go to URL
- Highlight
- Hover
- Inject Js Script
- Keyboard Shortcuts
- Keypress Event Trigger
- Mouse Scroll
- Navigate Browser
- Select Item
- Set Browser Data
- Set Runtime Browser
- Set Text
- Take Screenshot
- Type Into
- Use Application/Browser
- Perform browser search and retrieve results using UI Automation APIs
- Web Browsing
- Find Images
- Click Images
- Trigger and Monitor Events
- Create and Override Files
- HTML Pages: Extract and Manipulate Information
- Window Manipulation
- Automated List Selection
- Find and Manipulate Window Elements
- Manage Text Automation
- Load and Process Images
- Manage Mouse Activated Actions
- Automate Application Runtime
- Automated Run of a Local Application
- Browser Navigation
- Web Automation
- Trigger Scope Example
- Enable UI Automation support in DevExpress
- Computer Vision Local Server
- Mobile Automation
- Release notes
- Project compatibility
- Get Log Types
- Get Logs
- Get Page Source
- Get Device Orientation
- Get Session Identifier
- Install App
- Manage Current App
- Manage Other App
- Open DeepLink
- Open URL
- Mobile Device Connection
- Directional Swipe
- Draw Pattern
- Positional Swipe
- Press Hardware Button
- Set Device Orientation
- Take Screenshot
- Take Screenshot Part
- Element Exists
- Execute Command
- Get Attribute
- Get Selected Item
- Get Text
- Set Selected Item
- Set Text
- Swipe
- Tap
- Type Text
- Terminal
UI Automation Activities
v23.10
Release date: August 7, 2024
Release date: June 7, 2024
Release date: April 3, 2024
BasicTreeTable
and JxTreeTable
Java controls.
DesignTime ScreenId
is now sent in the header of the HTTP
request.
- The OCR for Chinese, Japanese, and Korean was not working as a custom OCR for Computer Vision in Unified Target when used with a Computer Vision API key.
- The OCR for Chinese, Japanese, and Korean was not working as a custom OCR in the Use Application/Browser activity when the project name contained Japanese characters.
- The Appeared event type from
the Application Event Trigger activity could not be monitored inside
iframe
HTML elements. - When the
enabled
attribute was selected, the Get Attribute activity returned a string output instead of boolean. Backwards compatibility is ensured.
Release date: February 26, 2024
- A
Value cannot be null
exception was thrown when removing the OCR engine activity from the CV Screen Scope after having indicated a target. - A
There is no row at position 0
exception was thrown when removing the OCR engine activity from the CV Screen Scope and the action of hovering over table cells persisted.
Release date: December 21, 2023
Added support for UiPath Autopilot™ for Developers, currently available in Preview exclusively in Studio Web. Instead of building a new automation from scratch, you can now simply describe the task you want to automate using natural language and watch as Autopilot's generative AI skillset converts your description into a fully automated workflow.
Additionaly, this update contains improvements for coded automations support in Studio.
Release date: December 11, 2023
Release date: December 6, 2023
- The Java Extension couldn't be reinstalled after upgrading Studio to v2023.4 if an error occurred when trying to uninstall it. With the new changes, the Uninstall button on the Studio > Tools > UiPath Extensions page changes to Install, despite any uninstall errors. Running the install again upgrades the Java extension properly even if any errors occur after uninstalling.
-
The workaround implemented for the UiPath MV3 browser extension initialization failure in Private mode is no longer necessary starting with Edge v119, as Microsoft confirmed that the original bug has been fixed in this version. The workaround now only applies for Edge versions between 115 and 118.
Release date: October 23, 2023
When the browser closes unexpectedly or when it's closed during Windows shut down, the Restore pages pop-up window is displayed. This behavior can break the automation if no specific logic is implemented on browser opening (for example, checking if the pop-up window appears and closing it with a Click activity). We implemented a change that prevents the Restore pages pop-up window from appearing when opening the browser.
Automating multiple browser instances running with multiple user profiles at the same time is now supported. This feature allows users to design automations with multiple browser instances opened under different profiles for Chrome, Edge, and Firefox.
UiPathRemoteRuntime.msi
.
The following activities have been hidden from the Modern experience:
- OCR Text Exists
- Click OCR Text
- Double Click OCR Text
- Hover OCR Text
- Find OCR Text Position
To see these activities in the Activities panel, select the Classic filter.
For situations in which you need to identify the type of native events trigged when you interact with certain UI elements in an application, we introduced a new tool integrated with UI Explorer, namely Event Inspection Tool, which can be used for configuring the Application Event Trigger activity. The tool captures and monitors all the selected event types, enabling you to analyze the actions performed and, if applicable, diagnose issues.
- HTML events:
- Download changed
- Tab activated
- Tab created
- Tab removed
- Tab updated
- WEBCTRL events:
- Selection change
- UI-based events:
- Appeared
- Disappeared
- Text changed
- Focus gained
- Focus lost
- Invoked
- Selected
- Deselected
- Toggled
We also added support for WEBCTRL based native events in native embedded browsers (such as Slack, Discord, or Electron test app).
The Application Event Trigger activity can now be used with coded workflows. As a result, we also introduced a way to monitor multiple target elements with a single Application Event Trigger and allow the number of monitored targets to be dynamic (the monitored targets may change after design time). Prior to this, a triggered workflow with an Application Event Trigger only monitored a single target element, requiring you to create a triggered workflow for each target element you wanted your automation to monitor.
We added support for the following SAP WebGUI elements: SAP Menu items (using the Select Item activity), SAP Toolbar buttons (using the Click Toolbar Button activity), and SAP Trees (including stable identification of the Trees and reliable automation across all browsers and all SAP FIORI versions). Note than you can use the existing capabilities to automate various other SAP WebGUI elements, such as ABAP lists, Context menus, Pictures, Textedit controls, Generic Object Services, Dialog boxes, and Table controls.
ColumnName
. Additionally, you can
filter on columns and rows to use any other property to identify elements. Two
property names were renamed to support this improvement: Column Name became
Column Name / Filter, and Row Number became Row Number /
Filter.
We added support for automating legacy versions of Oracle Siebel CRM Applications (e.g. 7 or 8).
We added support for the CTable component of the Clarity Java application.
Added the Copy to Clipboard option for all UI Automation dialog messages.
We improved the error messages displayed for nested remote sessions.
In this version of the activity package, we optimized core issues by accounting for each targeting method's relevancy in terms of targeting power and resilience to UI changes and we addressed the main limitations in the 23.4.x. As a result, UI Automation Table Extraction is now Computer Vision boosted when indicating full tables and table extraction is also enabled for pure Computer Vision scenarios; you can change the OCR engine as you can now select a custom OCR engine besides the embedded UiPath Screen OCR; Computer Vision is enabled by default in scopeless activities further boosting fully loaded targets (having both selectors and Computer Vision descriptors); and we optimized the experience of indicating UI elements by updating the reconciling logic between Driver and Computer Vision regarding bounding boxes. We have thus continued to pursue our vision of a unified UI Automation user experience where both pure Computer Vision and combined Computer Vision plus Driver powered automations can be created without the need for the user to be aware of the intricacies of each targeting method. More specific details can be found below.
A new hierarchical concept is now applied to Unified Target, taking into account each targeting method and its relevancy in terms of targeting power and resilience to UI changes. Hence, selectors are now considered the primary targeting method, Computer Vision is a secondary targeting method, and Image is a tertiary targeting method (and is now disabled by default). Before this, all targeting methods were trying to identify the target element simultaneously and whichever targeting method was able to find it first was used. Also, a new setting, Wait for primary targeting method until timeout, has been added in Project Settings > UI Automation Modern. The setting is set to True by default and allows the primary targeting method to consume its full timeout until any fallback targeting method can be used.
Computer Vision is now integrated in the Table Extraction wizard. You can now simply indicate full tables without having to indicate a relevant table cell first. Since Computer Vision is able to see entire tables, this functionality is now also used at design-time to correct faulty selector-based extraction when indicating a full table. At runtime, however, we don't have a Computer Vision fallback mechanism in place just yet when selector-based extraction has issues extracting full tables. You can extract pure Computer Vision tables as well without having to switch to a CV Screen Scope activity or a CV Extract Table activity for scenarios where selectors are not available at all, such as remote desktops without the remote run-time installed. On top of that, you can now also extract tables spanning across multiple pages in pure Computer Vision scenarios by indicating the Next button in the wizard.
Using the same logic as in the CV Extract Table activity, extracting scrollable tables in pure Computer Vision scenarios is now also possible in the Table Extraction wizard, as well as extracting scrollable text using CV Get Text. The Log Table Scroll Data project setting (Project Settings > Computer Vision > Robot Logging) has been renamed to Log Scrollable Content Extraction Data and it now covers Computer Vision Table Extraction, Table Extraction for scrollable tables in pure Computer Vision scenarios, and CV Get Text with scrollable text content.
In the Use Application/Browser activity, in addition to the Computer Vision embedded OCR engine, namely UiPath Screen OCR, you can now use a custom OCR engine to enhance the text extraction in Computer Vision when used in Unified Target. The OCR engine can be changed in each Use Application/Browser activity by going to the activity card > choosing the hamburger menu that appears next to the informative screenshot after indicating an UI element > selecting OCR Engine > Custom OCR Engine. When this option is selected, a panel is added in the activity card where you can drop a different OCR engine activity than the embedded one. This only affects the Computer Vision targets, as the rest of the OCR activities still use the default OCR set in Project Settings > OCR section.
As the happy path is having fully loaded targets, Computer Vision is now auto-enabled for indicating in scopeless activities (that can be used outside an Use Application/Browser scope) when it is also enabled in Project Settings.
When Computer Vision is enabled as a targeting method in Project Settings, the App/Web Recorder now captures Computer Vision descriptors as well.
We adapted our reconciling logic between Driver and Computer Vision regarding bounding boxes by matching the two targets to avoid indicating a Computer Vision-only target and to favor fully loaded targets. Selecting Computer Vision-only targets should now be reserved for scenarios like remote desktop with no access to selectors.
Strict selector failures are now displayed in the logs as warnings alongside ten suggestions of the closest selector matches that can be used to replace the faulty selectors. This feature is set to True by default in Project Settings > UI Automation Modern > Show closest matches in the 'Element not found' error message.
The Computer Vision screenshot no longer needs an initial delay, so now the start of an indicate session is faster.
If you prefer to code when building automations, you can now achieve this using coded automations. Coded automations support UiPath services, such as UI Automation activities, and allow you to create hybrid automations, by integrating them with low-code automations.
Visit the UI Automation APIs section to browse through the available APIs and Coded Automations Introduction to learn how to create coded automations.
-
Fixed the broken Active Accessibility support for web pages containing IFrame elements or PDFs in Chrome and Edge version 117, which caused
<ctrl />
selectors to fail.Also added the option to pass extra command-line arguments to the browser opened using the Open Browser / Use Browser activities. This can be done by setting the appropriate environment variable using the Set Environment Variable activity:UIPATH_EXTRA_CMD_ARGS_CHROME
UIPATH_EXTRA_CMD_ARGS_EDGE
UIPATH_EXTRA_CMD_ARGS_FIREFOX
These variables can be used to force-enable the Active Accessibility support in Chrome and Edge to avoid issues like the broken Chrome/Edge Active Accessibility support from versions 114 and 117.
For Chrome, set the environment variableUIPATH_EXTRA_CMD_ARGS_CHROME
with the value--force-renderer-accessibility=complete
.For Edge, set the environment variableUIPATH_EXTRA_CMD_ARGS_EDGE
with the value--force-renderer-accessibility=complete
. - MV3 browser extensions failed to initialize when using Microsoft Edge (versions 115 and newer) in Private mode. We have implemented a workaround for this issue that affected all MV3 extensions, not just the UiPath Extension for Microsoft Edge.
- We addressed some extension initialization issues affecting the Open Browser and Use Browser activities by adding six UI Automation Classic – Browser project settings: Extra arguments for OpenBrowser Chrome/Edge, Custom executable path for OpenBrowser Chrome/Edge, and Enable retry for OpenBrowser Chrome/Edge.
- The Table Cell Scope activity didn't recognize column names when SAP WinGUI was set in Japanese.
- Data Scraping returned empty cells for Salesforce reports.
- Support for Chrome/Edge running in App Mode is now available in the self-hosted MV2 browser extension installed via Group Policy Offline.
-
When installing extensions with Admin rights,
UiPath.Common
is now automatically deployed to%ProgramFiles(x86)%\UiPath\UiPath.Common
. When installing extensions without Admin rights,UiPath.Common
is now automatically deployed to%LocalAppData%\Programs\UiPath\UiPath.Common
. -
We added support for web pages using the ServiceNow Framework.
- The
tableCol
tag in the selector prevented some table elements from being identifiable. - You can now automate applications running under a different Windows User (other than the one running UiPath Robot/Studio), without having to run Robot/Studio as administrator.
- In a cross-platform Use Browser activity, the browser connection failed or became unresponsive when more than one tab was opened at design time on the serverless machine where the connection got confirmed, the user was logged out of Studio, the user deleted her/his personal workspace in Orchestrator, or when encountering a communication error with Orchestrator.
- Adding a selector from an activity to Object Repository failed with the "The operation was canceled" error message when the reuse search took more than three seconds.
- The Use Application/Browser activity threw an exception at design time in StudioX due to the asterisk indicating a property field is required.
- The Send Hotkey activity caused the .xaml file to be shown as unsaved.
- Image screenshot was still captured even when Image was disabled as a targeting method in Project Settings.
-
The
You Are Calling The Computer Vision Server Too Often [429]
error message displayed when reaching the Computer Vision rate limit has been changed toError: Rate limit exceeded. If you're on a free tier, consider upgrading to paid license. If already on a paid license, please contact UiPath for further upgrade options
. - The modern Get Text activity did not extract text in pure Computer Vision scenarios.