Subscribe

UiPath Test Suite

UiPath Test Suite

Device Interaction

Device Interaction Conditions


When an action is executed, device interaction is locked until its completion.

Starting an Application


To interact with your device you must first start an application. You have three options to choose from in MDM, as listed in the following table.

MethodProcedure
Start an application from a device1. Go to the Devices tab and select a device.
2. Click Start an application on this device.
3. Choose an application from the dropdown and click Connect.
Start an application from a configured app1. Go to the Applications tab and select an application.
2. Click Start application.
3. Choose a device from the dropdown and click Connect.
Start an application using Quick Actions within the Welcome tab1. Go to the Welcome tab and select Start an application.
2. Choose a device and an application from the dropdown.
3. Click Connect.
For more information about Quick Actions, refer to Quick Actions.

Interacting With The Device


You can interact with your device in MDM as you would with an actual physical device, using your mouse or the action bar. The screen and the actions are locked when you perform an action.

504

On-Screen Interaction


The following table lists the on-screen actions that you can take to interact with your device.

ActionDescriptionProcedure
Tap Access an on-screen element.Click an element on the device screen.
Double Tap Perform double tap action on-screen.Double-click an element on the device screen.
Long TapLong press.Click and hold on an element.
Pinch (Zoom In/Out)Zoom in and out. This action is not recorded.Right-click and hold, and then drag to the bottom of the screen to zoom in, or towards the upper part to zoom out.
Swipe Swipe on-screen.Click and drag towards any part of the screen.
Keyboard InputType in your text and send it to the mobile. For example, you can use the search bar to quickly access a website.1. Double-click on a text area to open the input window.
2. Type in your text.
3. Select Press Enter key after sending text to type in and enter your input directly.
4. Click Send Text.

Action Bar


The following table lists the actions that you can take to interact with your device. The action bar is right next to the device.

ActionDescriptionProcedure
iOS HomeReturn to the home screen.Click the iOS Home button.
Android BackGet back to the previous screen/page.Click the Android Back button.
Android HomeReturn to the home screen.Click the Android Home button.
Android Switch AppsSwitch between apps.Click the Android Switch Apps button, and choose your next action.
Change orientation to landscape/portraitToggle between portrait (vertical) and landscape (horizontal) mode for your device. Screen orientation is not supported by all applications.Click the Change Orientation toggle to switch between modes.
RefreshRefresh the device connection.Click the Refresh button.
Alternatively, click the Reconnect button when the connection has been lost.
Save ScreenshotSave the current screen.Click Save Screenshot and choose a location to save the file.
Send TextType in your text and send it to the mobile. For example, you can use the search bar to quickly access a website.1. Select a focusable element and then click Send Text.
2. Type in your text.
3. Select Press Enter key after sending to type in and enter your input directly.
4. Click Send Text to confirm.
Draw Pattern (preview)Draws whatever pattern on the device screen. Several points from the finger path are recorded and displayed on a X-Y axis.1. Select Draw Pattern and draw whatever pattern you want inside a UI element.
2. To import the action in Studio, click Import recorded actions. The pattern that you drew in MDM shows up as a Draw pattern activity, that prints out the points recorded from the finger path.
Selection by identified elementsSelects UI elements that are identified by MDM.Click the Selection by identified elements to choose to select UI elements that are identified by MDM.
Selection by absolute positionUses fixed element coordinates for selecting a UI element. The coordinates are recorded as a Positional Swipe activity.Click Selection by absolute position and choose the default coordinates for selecting a UI element.
Record element actionRecords a manually configured action completed on certain UI elements. It is recommended for recording more granular actions, such as adding an offset to an on-screen tap.1. Click Record element action .
2. Select a UI element on the device’s screen.
3. Click Single tap , and choose a type of on-screen interaction from the drop-down list. For example, choose Single tap.
4. Click
Configure Action to further configure the action, for more granularity.
5. Click
Execute Action to run the action that you’ve just recorded, in the Recorded Actions** panel.
Toggle WebView selectors as native selectorsTries to convert future selectors of type WebView, to selectors of type native, in accordance with the operating system used. This feature makes automating web apps easier.1. Click Toggle WebView selectors as native selectors .
2. Start recording your actions.
Install AppInstall an application on your device.1. Click the horizontal ellipsis to open Advanced Settings.
2. Click the Install App button and enter your app full location.
3. Click Install to confirm.
Manage AppManage the current application.1. Click the horizontal ellipsis to open Advanced Settings.
2. Click the Manage App and select one of the following actions from the dropdown:
Launch: Launch the default app page.
Reset: Restart the app.
3. Click Execute to confirm.
Manage Other AppManage other applications on your device. You can use this when you test integrations between apps.1. Click the horizontal ellipsis to open Advanced Settings.
2. Click the Manage Other Apps and enter your app's name.
3. Choose on of the following commands from the dropdown:
Remove: Uninstall the app.
Activate: Open the app.
Terminate: Close the app.
4. Click Execute to confirm.
Open URLOpen URL in browser.1. Open a browser on your device.
2. Click the horizontal ellipsis to open Advanced Settings.
2. Click the Open URL and enter a full URL. For example: https://www.google.com/.
3. Click Open to confirm.
Open Deep LinkLaunch an app that opens a specific page.1. Click the horizontal ellipsis to open Advanced Settings.
2. Click the Open Deep Link and configure the following:
App: App path
Deep Link: Specify the page that you want to open.
3. Click Open to confirm.
Save SourceSave the XML source file of the device.1. Click the horizontal ellipsis to open Advanced Settings.
2. Click the Save Source and select a location to save the file.
Set Geo LocationSet geographic coordinates.1. Click the horizontal ellipsis to open Advanced Settings.
2. Click the Set Geo Location and use the Latitude, Longitude, Altitude finder on map to get coordinates. Alternatively, you can search for your location.
3. Click Set to confirm.

Record element action

The Record element action feature can be found inside the Action bar. This allows you to record highly granular actions, that can't be captured through a regular on-screen interaction. The Record element action feature works by recording an action that is completed on one, or more UI elements, and configuring it. The configuration of the action depends on the type of on-screen interaction that you choose. For example, if you choose Single Tap as your on-screen interaction, you can configurate into Double Tap or Long Tap.

622

The Record element action feature is recommended for recording more granular actions, such as adding an offset to an on-screen tap.

Using Record element action

To use the Record element action feature:

  1. Click Record element action .
  2. Select a UI element on the device’s screen.
  3. Click Single tap , and choose a type of on-screen interaction from the drop-down list. For example, choose Single tap**.
  4. Click Configure Action to further configure the action, for more granularity.
  5. Click Execute Action to run the action that you’ve just recorded, in the Recorded Actions panel.

WebView selectors as Native selectors

The WebView selectors as native selectors feature can be found inside the Action bar. When toggled, it tries to convert all future selectors from WebView to Native. This allows a better, faster, and more familiar interaction with UI elements inside web views.

Note: Keep in mind that there are scenarios where this selection strategy may not work as expected, especially for complex web pages or web pages with elements that update frequently.

Using WebView selectors as Native selectors

To use WebView selectors as Native selectors:

  1. Click Toggle WebView selectors as native selectors .
  2. Start recording your actions.

Recorded Actions


You can interact with your device through the action bar or by indicating your actions on-screen to record the steps. For example, you can record and manage your actions to prepare mobile automation workflows in MDM. For more information, see Create Workflow Using Recorded Actions.

852

The following table lists the actions that you can take using Recorded Actions.

ActionDescriptionProcedure
ExecuteExecute the action.Click the Play button on an action to execute it.
CopyCopy the action to your clipboard. You can use the Import Recorded Actions in Studio to paste it to your workflow.Click the Copy button to copy the action to your clipboard.
DeleteDelete the action.Click Delete to remove the action from the list. Alternatively, you can click Clear All to remove all actions.
ReorderReorder the list of actions in alternate order.Drag and drop an action to different position within the list.
Wait for page updateWait for the page source to retrieve a screenshot of each action.Toggle the Wait for new screenshot after each action.

Updated about a month ago


Device Interaction


Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.