- Overview
- UI Automation
- Applications and technologies automated with UI Automation
- Project compatibility
- UI-ANA-016 - Pull Open Browser URL
- UI-ANA-017 - ContinueOnError True
- UI-ANA-018 - List OCR/Image Activities
- UI-DBP-006 - Container Usage
- UI-DBP-013 - Excel Automation Misuse
- UI-DBP-030 - Forbidden Variables Usage In Selectors
- UI-PRR-001 - Simulate Click
- UI-PRR-002 - Simulate Type
- UI-PRR-003 - Open Application Misuse
- UI-PRR-004 - Hardcoded Delays
- UI-REL-001 - Large Idx in Selectors
- UI-SEC-004 - Selector Email Data
- UI-SEC-010 - App/Url Restrictions
- UI-USG-011 - Non Allowed Attributes
- UX-SEC-010 - App/Url Restrictions
- UX-DBP-029 - Insecure Password Use
- UI-PST-001 - Audit Log Level in Project Settings
- UiPath Browser Migration Tool
- Clipping region
- Computer Vision Recorder
- Activities index
- Activate
- Anchor Base
- Attach Browser
- Attach Window
- Block User Input
- Callout
- Check
- Click
- Click Image
- Click Image Trigger
- Click OCR Text
- Click Text
- Click Trigger
- Close Application
- Close Tab
- Close Window
- Context Aware Anchor
- Copy Selected Text
- Element Attribute Change Trigger
- Element Exists
- Element Scope
- Element State Change Trigger
- Export UI Tree
- Extract Structured Data
- Find Children
- Find Element
- Find Image
- Find Image Matches
- Find OCR Text Position
- Find Relative Element
- Find Text Position
- Get Active Window
- Get Ancestor
- Get Attribute
- Get Event Info
- Get From Clipboard
- Get Full Text
- Get OCR Text
- Get Password
- Get Position
- Get Source Element
- Get Text
- Get Visible Text
- Go Back
- Go Forward
- Go Home
- Google Cloud Vision OCR
- Hide Window
- Highlight
- Hotkey Trigger
- Hover
- Hover Image
- Hover OCR Text
- Hover Text
- Image Exists
- Indicate On Screen
- Inject .NET Code
- Inject Js Script
- Invoke ActiveX Method
- Key Press Trigger
- Load Image
- Maximize Window
- Microsoft Azure Computer Vision OCR
- Microsoft OCR
- Microsoft Project Oxford Online OCR
- Minimize Window
- Monitor Events
- Mouse Trigger
- Move Window
- Navigate To
- OCR Text Exists
- On Element Appear
- On Element Vanish
- On Image Appear
- On Image Vanish
- Open Application
- Open Browser
- Refresh Browser
- Replay User Event
- Restore Window
- Save Image
- Select Item
- Select Multiple Items
- Send Hotkey
- Set Clipping Region
- Set Focus
- Set Text
- Set To Clipboard
- Set Web Attribute
- Show Window
- Start Process
- System Trigger
- Take Screenshot
- Tesseract OCR
- Text Exists
- Tooltip
- Type Into
- Type Secure Text
- Use Foreground
- Wait Attribute
- Wait Element Vanish
- Wait Image Vanish
- Application Event Trigger
- Check/Uncheck
- Check App State
- Check Element
- Click
- Click Event Trigger
- Drag and Drop
- Extract Table Data
- For Each UI Element
- Get Attribute
- Get Browser Data
- Get Text
- Get URL
- Go to URL
- Highlight
- Hover
- Inject Js Script
- Keyboard Shortcuts
- Keypress Event Trigger
- Mouse Scroll
- Navigate Browser
- Select Item
- Set Browser Data
- Set Runtime Browser
- Set Text
- Take Screenshot
- Type Into
- Use Application/Browser
- Perform browser search and retrieve results using UI Automation APIs
- Web Browsing
- Find Images
- Click Images
- Trigger and Monitor Events
- Create and Override Files
- HTML Pages: Extract and Manipulate Information
- Window Manipulation
- Automated List Selection
- Find and Manipulate Window Elements
- Manage Text Automation
- Load and Process Images
- Manage Mouse Activated Actions
- Automate Application Runtime
- Automated Run of a Local Application
- Browser Navigation
- Web Automation
- Trigger Scope Example
- Enable UI Automation support in DevExpress
- Computer Vision Local Server
- Mobile Automation
- Release notes
- Project compatibility
- Get Log Types
- Get Logs
- Get Page Source
- Get Device Orientation
- Get Session Identifier
- Install App
- Manage Current App
- Manage Other App
- Open DeepLink
- Open URL
- Mobile Device Connection
- Directional Swipe
- Draw Pattern
- Positional Swipe
- Press Hardware Button
- Set Device Orientation
- Take Screenshot
- Take Screenshot Part
- Element Exists
- Execute Command
- Get Attribute
- Get Selected Item
- Get Text
- Set Selected Item
- Set Text
- Swipe
- Tap
- Type Text
- Terminal
UI Automation Activities
OUT OF SUPPORTv21.10
Release date: October 5, 2023
- We added support for the CTable component of the Clarity Java application.
-
Fixed the broken Active Accessibility support for web pages containing IFrame elements or PDFs in Chrome and Edge version 117, which caused
<ctrl />
selectors to fail.Also added the option to pass extra command-line arguments to the browser opened using the Open Browser / Use Browser activities. This can be done by setting the appropriate environment variable using the Set Environment Variable activity:UIPATH_EXTRA_CMD_ARGS_CHROME
UIPATH_EXTRA_CMD_ARGS_EDGE
UIPATH_EXTRA_CMD_ARGS_FIREFOX
Release date: June 8, 2023
- The latest Chrome and Edge updates to version 114 introduced a breaking change to the way support for Active Accessibility is enabled. This caused automation processes to fail when trying to interact with browser screens such as the Print pop-up or PDF viewer.
- A "Value does not fall within the expected range" error occurred when using the Data Scraping wizard (Extract Structured Data activity) in Microsoft Edge in IE Mode.
- An error occurred when using the Data Scraping wizard (Extract Structured Data activity) on a table with an empty column header.
Release date: December 7, 2022
visibleInnerText
attribute is now available in the Selector
Attributes panel from UI Explorer.
Release date: August 3, 2022
- In certain Java environments, the
visibility
attribute was not correctly populated for AWT dialogs, causing the application to behave incorrectly. - When using the Type Into
activity with the Simulate Type input method, the number
8
could not be typed in certain applications. - Automating WinForms CefSharp applications performed very slowly.
- Certain system exceptions caused too many log entries to be recorded, making debugging cumbersome. To fix this, the log level has been changed from Error to Information.
Release date: May 23, 2022
- Multiple accessibility and localization issues have been fixed.
- When automating Microsoft Edge in IE Mode, selector validation failed.
- CV Extract Table was unable to refresh the output DataTable when used in a loop.
- Certain Java Konsis apps could not be targeted properly.
- When attempting to deserialize elements from a remote connection, the UI Automation activities sometimes crashed.
- Trying to paste certain types of strings from UI Explorer into Excel cells did not work.
Release date: January 12, 2022
The UiPath Extension for Remote Desktop Protocol can now also be used with Azure Virtual Desktop client.
- The Suppress Audit Logs Project Setting has been renamed to Log audit information and is set by default to True. Also, all logs generated by this setting are now at Verbose level.
- Fixed a bug for Oracle Forms that caused, in particular cases, the validation of selectors to fail.
- Fixed a bug that prevented you from reading the correct state of a checkbox inside a table in Java applications.
Release date: November 26, 2021
- When Background Mode was used with an Use Application/Browser activity that had the Resize window property set to Minimize, applications were not correctly minimized.
- To clear the text from web pages elements, you can use again empty strings with the Set Text activity, just like in UI Automation v18.4.
Release date: October 25, 2021
The Get Attribute activity has been reworked and now comes as part of the Modern design experience. Changes have also been made to improve the usability of the activity, making it offer a better experience overall.
The Get Text activity now features a wizard that enables you to preview the results of the text extraction, as well as configure the extraction method.
Two new activities, Get Browser Data and Set Browser Data, have been created to make it easier to migrate browser data from a session to another.
Adding or editing variables is now easier than ever. With the enhanced Selection Screen variable support this release brings, you can easily edit your selectors by using variables and arguments, and then choose to propagate the change throughout all instances of the said variable from that descriptor.
You can now refresh the Image selection when indicating in an application, by using the new button added in the Selection Screen.
All of the variable features added to the Selection Screen are now also available when editing selectors in the Selector Editor.
The Resize window property in the Use Application/Browser activity now has the option to minimize the window.
Support for Responsive websites has been added with this update. The Selection Screen now features a check box that enables working with elements that reflow when the browser window is resized.
The Open Application/Browser activity can now open applications in remote environments, making automation with the Modern experience in such environments much easier.
The Modern UI from the Selection Screen is now part of the Classic Experience as well.
Starting with this update, every time you start the Recorder, it starts recording automatically, and you no longer have to indicate the window first, thus making your automation life easier.
Applications provisioned through Microsoft RemoteApp can now be automated with the UI Automation activities.
As Internet Explorer 11 is approaching its upcoming end-of-life, we have developed a migration tool that enables you to convert all of the selectors and properties for your workflows seamlessly into ones fit for modern browsers, such as Edge, Chrome, or Firefox, ensuring you do not lose all of your automations based on this soon-to-be legacy browser.
The Type Into activity can now automatically detect the correct Input Mode based on what results each input mode returns for the application you are automating.
The Verify Execution feature in the Type Into, Click, and Hover activities can now check not only if the target element appeared or disappeared, but also if its text has been modified or if its visual appearance has been modified (status icons changing, etc.), via two new options in the Verify Element dropdown.
A new option has been implemented for the Input Mode property, called Chromium API, which is tailored for use with all Chromium-based browsers, being faster and more reliable than the others.
The App/Web Recorder now supports using the Object Repository when recording, enabling you to use already existing objects or create new ones on the fly.
The new Highlight feature in Object Repository enables you to highlight any object you have captured in the screen where you are capturing, making it easier to identify which element you are working with.
label-for
attribute, regardless of the position of their label.
When validating results in the Selection Screen, multiple selectors are now searched for simultaneously.
The Extract Table Data activity now extracts all attributes when used on standard table extraction. Also, when automatically extracting URLs, the full URLs are now retrieved.
Adding another human ability to our robot workforce, we have developed a new activity, Mouse Scroll, which enables you to either scroll through pages for a fixed number of lines, or to a specified UI element, in applications you are automating.
The Edge Extension now supports Java Applets and ActiveX controls in IE Mode.
The Display Name property of the Use Application/Browser activity now automatically changes according to the indicated target, so you can easily spot which card points to which application at design time. In the situation when you are using nested application cards, you can now choose from a drop-down in the body of every activity the application card you want to reference when indicating on screen.
When recording Type Into with the Modern Experience, a drop-down for Special Keys is now displayed when entering the text you want to type.
When scraping tables with the Extract Table Data activity, the Extract Table Data Wizard now retains previous changes when reopened.
The Attach Browser and Attach Window activities now enable you to set both the Selector and Browser or Window properties at design time without throwing an error. At design time, the Selector property is used, while, at runtime, the Browser or Window properties are used.
*
and ?
expressions for defining patterns for URLs
and application file names to allow or block.
Salesforce Lightning, with specific Salesforce AURA components path and attributes, and reliable selectors, is now supported in Chrome and Edge.
This activity package now supports .NET 5 Windows and Cross-platform projects. For more information, see About Automation Projects. Also, the following OCR engines now support .NET5: Google Cloud Vision OCR, Microsoft Azure Computer Vision OCR, Tesseract OCR.
Microsoft OCR, however, does not support .NET5; when using the UiPath.UIAutomation.Activities package in a .NET5 project, Microsoft OCR is not displayed.
The Mouse Scroll and Keyboard Shortcuts activities have been visually updated, sharing the same UI as the other Modern activities.
The Object Repository now uses Computer Vision technology for the Capture all elements feature, which can extract all the elements of a specified application and add them to your Object Repository. This can greatly reduce the time needed to build the Object Repository libraries, as you no longer have to manually add each element one by one. You can also open any element in the Capture Elements wizard directly in UI Explorer via a dedicated button.
The Use Application/Browser activity now has a new Input Mode, called Background, which enables you to use all its children activities in the background whenever possible.
You can now open the application or browser targeted by the Use Application/Browser activity by using the new button in the body of the activity.
The UI-SEC-010 and UX-SEC-010 Workflow Analyzer rules now also apply to remote target applications. Also, both rules now allow using wildcards in creating rules.
The App/Web Recorder now has an input method called Auto, which automatically selects the best input method available, making it easier to automate different kinds of apps.
UI elements from CefSharp embedded widgets (e.g. SharpBrowser, SAP NWBC with Fiori Launchpad) are now recognized as belonging to the host application instance.
%localappdata%\UiPath\WebdriverExe
folder.
Starting with UiPath.Excel.Activities version 2.11.3 and UiPath.Presentations.Activities version 1.2.3, you can add UI Automation activities inside a Use Excel File or Use PowerPoint File activity to quickly automate the interface of Excel and PowerPoint.
The Check App State activity has been taken up a notch, and can now monitor the entire application screen for changes, not just a single UI element.
The RDP Extension can now be installed per system, removing the previous limitation, where it required per-user installation.
The new Text Changed option in the Verify element property drop-down menu enables you to verify any changed text in the targets of activities that support this, such as Click, Hover, and Type Into.
The Verify Execution property has been added to the Keyboard Shortcuts activity, expanding this feature to the scenario where you want to verify the way keyboard shortcuts are sent by your automation process.
The CV Activities now support table cell identification, enabling you to automate table cells a lot easier, based on either row and column content or indexes.
The new Runtime Browser project setting enables you to use a different browser at runtime than the one used at design time. This is done automatically, as all related properties and selectors are seamlessly adapted to the browser of your choice.
The Desktop Project Settings Targeting Methods (AA, UIA, Win32) have been merged into a single setting, called Targeting Methods – Desktop.
Multiple new project settings or the UI Automation package have been added or updated, to fit your automation needs, as follows:
A new Capture Screenshots setting that specifies whether activities save screenshots of their targets at design time. This setting is present in both Modern and Classic experiences.
A new Robot Logging section, with the Run value and Debug value settings that specify whether the Robot should log extra values.
The Targeting Methods Project Settings have now been split according to the type of the target application. Now you can configure separate settings for Web, Java, SAP, and Desktop applications.
The Verify Execution project setting has a new option which enables you to always perform verifications for Click, Hover, and Type Into activities in a project.
Also, the Input Mode project setting has been split into two separate project settings, Input mode for Edge browser and Input mode for Chrome, Firefox, IE, Edge in IE mode, to make sure the correct input method is selected when using Edge Chromium.
Element Not
Found
error message enables you to log either a detailed or a trimmed down
version of the error message that is displayed when an UI element is not found.
The new SAP Session Attributes Change Trigger activity enables you to create triggers by monitoring changes in SAP session attributes.
Support has been added for Chromium for SAP Business Client. This allows users to use the same features and SAP Fiori capabilities available as when automating with UiPath Studio and Google Chrome.
Support has been extended for SAP ALV Tables with Multiheader. You can now extract data from tables which have their headers split into two or more lines.
The new Expand ALV Tree activity enables you to expand the parent ALV tree to the selected node and use any activities with the resulting elements, making SAP ALV Tree lists a lot easier to automate.
SAP Fiori UI5 attributes are now fully visible and usable with the Modern UI Automation experience.
ColorIndex
, ColorIntensified
, and
ColorInverse
have been introduced for SAP ALV Tables, enabling you
to identify the background color of elements, giving you extra methods of identifying
the elements you want to work with.
Full text scraping is now supported in SAP ALV Tables, which means that you can now also scrape the text that is not visible.
When using the Extract Table Data activity to extract data from SAP ALV Tables, you can now distinguish between the column display name and the column tooltip, to make it easier to correctly identify your target.
tooltip
and text
attributes.
- The
aastate
attributes of<uia/>
selectors were incorrectly retrieving theaastate
value of the ancestor. The attribute’s value is now correctly retrieved, which may cause breaking changes in older workflows. This does not impact the Universal Windows Platform applications selectors which were built correctly. - The Key Press Trigger activity was erroneously detecting some events when the IncludeChildren property is set to False. This may cause breaking changes in older workflows.
- In certain conditions, such as using a maximized Chrome window, the Click activity with the Click offset Anchoring Point property set to BottomRight triggered an extra scroll event. This behaviour is now fixed, and may cause breaking changes in older workflows.
- Performance issues were present when working with certain Java applications.
- Fixed an issue which caused an error to be thrown when extracting data from certain types of webpages (ShadowRoot).
- Fixed an issue which caused UI Explorer to throw an error when using AA for targeting elements in Firefox. When using a new or empty Firefox tab with UI Framework set to Default does not allow selection of individual UI elements. Also, any Firefox tab permits selection of the entire window only when using the AA Framework.
- When Object Repository was enforced in a project, it could still be disabled from the Recorder.
- Fixed a bug that occurred when OCR was run on different OS region formats. Now, the OCR runs as expected and all results are generated correctly, no matter what OS region format is used.
- When using certain versions of dependencies, screenshots of Excel could not be taken.
- The Input Mode property of the Extract Data Table activity was not affected by any changes in the Targeting Method Project Setting.
- Trying to install Java Bridge from UI Explorer when using UI Automation v21.4.x failed.
- Fixed an issue which caused the Wait for page load property to not work properly on browser tabs that were loading.
- Trying to click a certain cell containing a date in a table by using Computer Vision activities failed in the case of certain tables.
- In certain cases, strings were merged between separate cells when extracting values from a table using Computer Vision activities.
- Automating PDFs opened in Firefox failed to extract individual UI elements.
- Fixed an issue that caused
title
HTML attributes to be mistakenly identified in certain types of tables. Added a new attribute,uipath-html-title
, which has the value of thetitle
attribute of the DOM node. - Fixed an issue which caused Type Into to behave differently when the design-time keyboard layout was different than the one used at runtime.
- The
Enter
keystroke was not performed correctly in some activities when running as a XenApp in an App-V container. - Fixed an issue which caused certain activities to perform very slowly when retrieving values when used with newer versions of UI Automation.
- Default selectors in Edge pop-up windows were generated incorrectly.
- Sending the
Enter
keystroke by using SendWindowMessages to buttons failed. - UiPathRemote.exe sometimes crashed in Citrix environments.
- In certain situations, the Click Text activity caused applications to crash.
- Fixed an issue which caused the Recorder to throw an internal error when deleting a keyboard shortcut from the recorded steps.
- Fixed an issue which caused elements inside an iFrame to not be detected in Edge and Chrome.
- Fixed an issue which caused interactive selection with Smart-Sizing and using different resolution scaling to not work properly when automating in RDP.
- Fixed an issue which caused the Attach Browser activity to throw an error when it was configured to use both the Browser and Selector properties.
- Using the Type Into activity with the SimulateType or SendWindowMessages properties when automating a Command Prompt window failed to type the correct text.
- The Silverlight extension might not work on Windows 10 x64 systems where the Windows update 10.0.19043 was applied.
- If you want to use any OCR Engine in Studio v19.10 with UI Automation activities, please install the UiPath.CoreIPC package, version 2.0.1 or higher.
- In a C# process, editing a descriptor does not display changes at design time when using the CV activities.
- When using UI Automation v21.10 with Studio v2019.10, the UX-SEC-010 and UX-DBP-029 Workflow Analyzer rules are not available.
- When using Firefox,
Salesforce-specific
sfl
attributes are not displayed in selectors and selector nodes. - On certain applications that use Windows Forms, when using only the Fuzzy selector method for targeting elements in a minimized app, the target cannot be found.
- Using Background input mode with apps that use Universal Windows Platform causes the apps to be brought to the foreground.
- When using the Modern experience
with SharpBrowser, selectors are not correctly generated, lacking
webctrl
tags. - Indicating an application requires the use of a mouse, and cannot be executed by using the keyboard.
- Variable right click functionality requires the use of a mouse and cannot be performed only by using the keyboard.