Checkboxes and signatures

Checkboxes and signatures are two elements that play crucial roles in various types of documents, ranging from contractual agreements to registration forms. Understanding how to correctly annotate checkboxes and signatures is important in making the most out of your model.

Checkboxes

When annotating your documents, you can encounter several types of multiple choice fields that use checkboxes:

• Mutually exclusive checkboxes.
• Non-mutually exclusive checkboxes, where you can select more than one option.

An important aspect to consider is the number of choices offered within a given multiple-choice field. In some cases there could be a single option, where the checkbox is either checked or not. However, in many instances, there may be 10, 20, or even more options, often organized into a grid or table format, which is common for health forms.

In terms of annotating these diverse multiple-choice fields, there are four primary methods you can use.

Annotate the option

Let's use an example to understand how you can annotate the options.

Note: Before you begin annotating options, ensure that you've created a regular field for each relevant option.

Consider situations where a document includes year options. In these scenarios, you only have one field, and you only annotate the word that has been selected. For instance, annotate 2018 if the checkbox next to it is checked, or 2019 if that option is chosen. If neither option is selected, then neither should be annotated. A scenario where both options are checked is not possible; documents with such instances should be omitted from the set.

This approach has the advantage that you have a single field, which requires less data. It also doesn't depend upon the successful detection of checkboxes. For example, if a checkbox is mistakenly detected as the letter X, the model can still learn to recognize that it signifies the selection of the option next to it.

However, a potential disadvantage is the necessity to ensure that both options are roughly equally represented, which might not always be the case. For instance, if 90% of the documents in your dataset have 2018 checked, the model's performance could be affected, leading to the failure of this approach. The problem gets worse when you have more options because some of them are almost always rare. In these cases you may need to create fake documents with the rare options checked to balance things out.

Annotate the checkboxes, with separate field for each checkbox

Note: Before you begin annotating options, ensure that you've created a regular field for each relevant option.

In the previous example, you might have created two distinct fields: one labelled 2018 where you consistently annotate the checkbox for that year, and another one labelled 2019 where you continuously annotate the checkbox for 2019, whether it's checked or not. This method's positive aspect is that balance becomes less critical; even if one choice is selected 90% of the time, the model can still learn to identify them because the checkboxes hold fixed positions.

The downside is that you have two fields instead of one. While this may not pose a considerable issue when dealing with two options, handling 10-20 options and consequently creating 10-20 fields rather than a single one can significantly complicate the annotation process. Additionally, this also leads to a more challenging model training process, requiring more training data.

Another drawback is the occasional incorrect detection of the checkbox, which can leads to the need of more complex logic in the workflow to manage all the returned X, V, or K characters. In some cases, the OCR might even merge the checkbox with the word next to it, like X2018, requiring an even more complex RPA logic to handle this situation.

Annotate only the checkboxes with a single multi-value field

Note: To use this approach, create a Multi-value field which you will use to annotate the checkboxes.

Multi-value fields make it easier to annotate, and they are not affected by imbalances in checked options or by a wide variety of selections. However, these fields are still subject to the accuracy of checkbox detection and the potential risk of checkboxes being merged with adjoining options. OCR errors are very hard to defend against.

Annotate the options with a single multi-value field

Note: To use this approach, create a Multi-value field which you will use to annotate the checkboxes.

This approach also simplifies the annotation process and is less sensitive to checkbox detection errors. However, it may be more sensitive to unbalanced options.

All of these options may be appropriate in some situations. Initially, the first option is preferred. As the accuracy of the checkbox detection in UiPath® Document OCR has improved, options two and three are preferred.

Signature detection

Signatures can be identified using UiPath Document OCR, allowing ML models to detect them directly.

You can annotate a signature like any other field in your document. Once the signature is identified by UiPath Document OCR, the ML model learns to recognize the field as a signature.

At inference time, the signature will be retrieved as displayed in the documents. You then have to convert this into a boolean field (Yes/No) using RPA logic.