Using Entities in your Application

Communications Mining provides multiple ways of fetching predictions, including predicted entities. Please consult the data download overview to understand which method will work best for your use case.

Whichever method you choose, you need to be aware of the following edge-cases, and handle them in your application:

• Not all expected entities are present in the response
• The response contains multiple matches for one or more entities
• Not all entities present in the response are correct

In this section we will go through each one of these edge-cases in more detail.

Not all entities are present in the response

You should expect to handle cases where not all expected entities are present. In the example below, the email has the policy number, but doesn't have the insured organization name. Your application should be able to handle such partial information.

Figure 1. Missing Insured Organization

The response contains multiple matches for one or more entities

You should also expect to handle the opposite of the previous case, namely cases where a comment has more entities than expected. In the example below, even though we expect one policy number and insured organization name per email, the email has multiple policy numbers.

Figure 2. Multiple matches for the same entity

Note that you can use the metadata in the response when handling such cases. For example, we can choose to preferentially pick policy numbers that appear in the email subject over those that appear in the email body. The example below shows the response that the API will return for our example email.

{
  "predictions": [
    {
      "uid": "aa05ba2250de48e3.7588b85f68f81c3b",
      "labels": [...],
      "entities": [
        {
          "id": "6a1d11118b60868e",
          "name": "policy-number",
          "span": {
            "content_part": "body",
            "message_index": 0,
            "utf16_byte_start": 200,
            "utf16_byte_end": 222,
            "char_start": 100,
            "char_end": 111
          },
          "kind": "policy-number",
          "formatted_value": "GHI-0204963"
        },
        {
          "id": "6a1d11118b60868e",
          "name": "policy-number",
          "span": {
            "content_part": "subject",
            "message_index": 0,
            "utf16_byte_start": 0,
            "utf16_byte_end": 22,
            "char_start": 0,
            "char_end": 11
          },
          "kind": "policy-number",
          "formatted_value": "GHI-0068448"
        },
        {...},
        {...},
        {...}
      ]
    }
  ],
  "model": {
    "version": 31,
    "time": "2021-07-14T15:00:57.608000Z"
  },
  "status": "ok"
}

Not all entities present in the response are correct

Finally, since entities are extracted using machine learning, you should expect to receive wrong matches. The number of wrong matches will depend on the entity you are using. The Validation page of your dataset provides validation statistics to understand how an entity will perform.

Figure 3. Entity Validation