Object Detection
OS Packages > Image Analysis > ObjectDetection
This is a generic, retrainable deep learning model to perform Object Detection. This ML Package is pretrained on COCO Dataset so you can directly create an ML Skill which can be used for identifying 80 classes of COCO Dataset.
Well, you can also train it on your own data and create an ML Skill and use for performing object detection where it will now work on your data.
This deep learning model uses You only look once (YOLO) which a state-of-the-art and one of the most effective object detection algorithms that also encompasses many of the most innovative ideas evolving from the field of computer vision.
Model Details
Input Type
FILE
Input Description
Full Path of the image file on which you want to detect the objects.
Output Description
JSON with identified object’s class byte array representation (allows you to see box around objects), identified object’s class - name, score (between 0-1)
Example:
{
"Predicted ByteArray":
"/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAIBAQEBAQIBAQECAgICAgQDAgI…TD",
"Predicted Class":
"[{'class': 'book', 'score': ' 0.31'}, {'class': 'dog', 'score': ' 0.53'}, {'class': 'chair', 'score': ' 0.79'}]"
}
{
"Predicted ByteArray":
"/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAIBAQEBAQIBAQECAgICAgQDAgI…TD",
"Predicted Class":
"[{'class': 'book', 'score': ' 0.31'}, {'class': 'dog', 'score': ' 0.53'}, {'class': 'chair', 'score': ' 0.79'}]"
}
Pipelines
This package only supports full pipeline runs.
Dataset Format
Image File
By default, this model will read images of format .jpg, .jpeg. Below are some points of considerations for the input images:
- Having all images of same format
- Having all images of same size, preferred 800*600
- Having at least 100+ count of image for each category of object
XML Annotation File
For each uploaded image there should be a corresponding annotation .XML file which contains the bounding box details of the image. The required file format for the .XML file is Pascal VOC.
For annotating the images, you can use an opensource annotation tool like LabelImg or any other tool of your preference.
Below are some points to be considered while creating the .xml files:
- Its preferred to have single class in the .xml file
- Giving meaningful name to the class (as above)
-
Avoiding any alterations in .xml file
So, this is how your dataset folder will look:
In the above image we can see there are 5 classes – cat, dog, giraffe, horse, zebra and they have corresponding images and xml’s in the dataset folder. Of course, your dataset folder will have more images and xml’s this is just an example to understand the folder structure.
Environment Variables
- learning_rate: change this value to adjust the learning rate, default learning rate is 0.0001
Artifacts
Evaluate function produces one artifact: Here the performance of model is evaluated on map value
- result.txt – A report containing summary information of how the model performed by sharing map (mean average precision) value of each class and total map value
Sample Workflow
You can use this sample Workflow to try this model. Make sure you first deploy the model on your own tenant and then use this workflow with any of your image to send the image to the workflow and automatically identify objects in that image.
Dependencies
- UiPath.MLServices.Activities v1.1.3
- UiPath.Web.Activities v1.4.5
Paper
YOLOv3: An Incremental Improvement by Joseph Redmon, Ali Farhadi