Evaluate project success

You can track the metrics described in this page to establish the success of your projects.

Main metric

The most relevant metric to follow is Estimated time saved, expressed in number of hours. This metric can be directly tied to ROI, tracks the time saved having a Document Understanding process in place, considering a human processes one document page in the configured time. The metric uses the following formula:

{Estimated time saved} = {number of document processed} * {x} (minutes) - {validation time}

• Number of documents processed: the total number of documents processed using the automation.
• x: the time a user would need to process one document without automation.
• validation time: the time the user spent in Classification or Validation Station, validating all processed documents. This is considered 0 for documents which do not require validation.

Example:

10 minutes = 2 documents * 5 minutes (manual processing) - 0 minutes validation

Secondary metrics

Besides Estimated time saved, there are other important metrics that are worth tracking to evaluate the performance and success of a project.

Table 1. Secondary metrics

Metric	Description	When to track
Number of documents processed	The total number of processed documents.	Use this metric to: Track how many documents you processed via automation. Track how many documents you processed via automation compared with manual processed.
Validation time	The total time spent validating the classification and extraction results.	Use this metric to: Track how much time is still required by a human in the loop. Check if validation is improving with automation and model updates.
Average handling time	The average time required to process a document, including time users spend with validation, considering 0 for the documents which do not require validation. The following formula is used: {Average handling time} = sum of({time spent processing documents} / {number of documents processed}) Example: If 10 documents are processed and it takes 2 minutes per document, 5 documents validated by a human in the loop add 2 extra minutes: (2*5 + 4*5 = 3 minutes Average handling time	Use this metric to track how much time users need to process a document now that you have the automation in place.
Field corrections trend	The number of corrected fields, by field modification (edited value, edited box, marked as missing, etc.), per month. In addition to corrections, the metric also provides a baseline of the total number of fields so you can establish how many of the total fields were corrected.	Use this metric to: Check if the number of modified fields increases over time. Check if newer model versions require more human input or less. Check why is validation taking too long.

Improve project metrics

Check the following tips and tricks on how you can improve your project success metrics:

• To improve Estimated time saved:
  • Increase the volume of processed documents.
  • Improve Validation time and straight-through-processing by adding business rules in place.
• To improve Validation time and Average handling time:
  • Define the logic around which to decide whether or not to send documents for validation to a human in the loop.
  • Fine-tune your model and add training data for fields that are not extracted accurately. Act based on the field corrections trend to figure out which fields require the most attention and focus on improving the extraction there.
  • Use the Generative Validation feature.