Enabling Computer Vision for mobile automation

Computer Vision enables you to use UiPath's neural network in order to identify UI elements and interact with them via Robots, simulating human interaction. In the context of Mobile Automation, the Computer Vision feature enhances the number of target elements that you can identify, allowing you to automate UI elements that are otherwise harder to identify and automate.

See the activities that can use Computer Vision in the table below:

Activity	Description
Get Text	Extracts the text from a specified UI element.
Set Text	Types the specified text into a target UI element.
Tap	Returns a boolean variable that states whether a specified UI element exists.
Swipe	Swipes the screen in a specified direction and for a specified amount of times.
Element Exists	Returns a boolean variable that states whether a specified UI element exists.

To start discovering mobile UI elements using Computer Vision, you need to set up your Computer Vision server, for either cloud or on-premises servers. You can set up the Computer Vision server in the Project Settings menu of the project that you are working in.

The Computer Vision configuration section is split into three other sub-sections:

• Server - the URL for the type of Computer Vision server that you want to connect to: cloud or on-premises. The default value for the Run value and Debug value server fields is the cloud instance of Computer Vision: https://cv.uipath.com.
• Api Key - the Computer Vision's API key, specific for your organization.
• Use Local Server - this configuration is not yet available.

Known limitation

Computer Vision for mobile automation doesn't support CJK (Chinese, Japanese, Korean) languages.