Manage Mouse Activated Actions

The example below explains how to automate the use of the mouse on a website by using activities like OCR Text Exists, Get OCR Text, or Hover OCR Text. You can find these activities in the UiPath.UIAutomation.Activities package.

This is how the automation process can be built:

1. Open Studio and create a new Process.

2. Drag a Sequence container in the Workflow Designer.

   • Create the following variable:

     Variable Name	Variable Type	Default Value
     ocrTextFound	Boolean

3. Drag an Open Browser activity inside the Sequence container.

   • In the Properties panel, select the IE option from the BrowerType drop-down list.
   • Add the website path in the Url field, in this case, "https://google.com".
   • Select the check box for the NewSession option. This starts a new session of the selected browser.

4. Place a Type Into activity below the Open Browser activity.

   • Inside the activity, click the Indicate on screen option. The GIF below shows all the steps you need to follow:
     
     

     

     
     
   • In the Properties panel, add the value "News" in the Text field.
   • Select the check box for the Simulate Type option. This simulates, in the background, the type by using the technology of the target application.

5. Drag an OCR Text Exists activity below the Type Into activity.

   • Inside the activity, click the Indicate on screen option. The GIF below shows all the steps you need to follow:
     
     

     

     
     
   • In the Properties panel, add the value "Lucky" in the Text field.
   • Add the variable ocrTextFound in the Exists field.
6. Place a Tesseract OCR inside the OCR Text Exists activity.

7. Drag an If activity below the OCR Text Exists activity.

   • Add the variable ocrTextFound in the Condition field.

8. Place a Hover OCR Text inside the Then container of the If activity.

   • Inside the activity, click the Indicate on screen option. The GIF below shows all the steps you need to follow:
     
     

     

     
     
   • Add the value "Lucky" in the Text field.
9. Place a Tesseract OCR inside the Hover OCR Text activity.

10. Drag a Click activity below the Hover OCR Text activity.

    • Inside the activity, click the Indicate on screen option. The GIF below shows all the steps you need to follow:
      
      

      

      
      
    • In the Properties panel, select the check box for the SendWindowMessages option for executing the click action by sending a specific message to the target application.

11. Drag a Click OCR text activity inside the Else field of the If activity.

    • Inside the activity, click the Indicate on screen option. The GIF below shows all the steps you need to follow:
      
      

      

      
      
    • In the Properties panel, add the value "Search" in the Text field.
    • Select the check box for the SendWindowMessages option for executing the click ocr text action by sending a specific message to the target application.
12. Place a Tesseract OCR inside the Hover OCR Text activity.
13. Run the process. A new web browser instance opens and initiates a search after the specified word.

Download example