- API docs
- CLI
- Integration guides
- Blog
- How machines learn to understand words: a guide to embeddings in NLP
- Prompt-based learning with Transformers
- Efficient Transformers II: knowledge distillation & fine-tuning
- Efficient Transformers I: attention mechanisms
- Deep hierarchical unsupervised intent modelling: getting value without training data
- Fixing annotating bias with Communications Mining
- Active learning: better ML models in less time
- It's all in the numbers - assessing model performance with metrics
- Why model validation is important
- Comparing Communications Mining and Google AutoML for conversational data intelligence
Using general fields in your application
Communications Mining provides multiple ways of fetching predictions, including predicted general fields. Please consult the data download overview to understand which method will work best for your use case.
Whichever method you choose, you need to be aware of the following edge-cases, and handle them in your application:
- Not all expected general fields are present in the response
- The response contains multiple matches for one or more general fields
- Not all general fields present in the response are correct
In this section we will go through each one of these edge-cases in more detail.
Note that you can use the metadata in the response when handling such cases. For example, we can choose to preferentially pick policy numbers that appear in the email subject over those that appear in the email body. The example below shows the response that the API will return for our example email.
{
"predictions": [
{
"uid": "aa05ba2250de48e3.7588b85f68f81c3b",
"labels": [...],
"entities": [
{
"id": "6a1d11118b60868e",
"name": "policy-number",
"span": {
"content_part": "body",
"message_index": 0,
"utf16_byte_start": 200,
"utf16_byte_end": 222,
"char_start": 100,
"char_end": 111
},
"kind": "policy-number",
"formatted_value": "GHI-0204963"
},
{
"id": "6a1d11118b60868e",
"name": "policy-number",
"span": {
"content_part": "subject",
"message_index": 0,
"utf16_byte_start": 0,
"utf16_byte_end": 22,
"char_start": 0,
"char_end": 11
},
"kind": "policy-number",
"formatted_value": "GHI-0068448"
},
{...},
{...},
{...}
]
}
],
"model": {
"version": 31,
"time": "2021-07-14T15:00:57.608000Z"
},
"status": "ok"
}
{
"predictions": [
{
"uid": "aa05ba2250de48e3.7588b85f68f81c3b",
"labels": [...],
"entities": [
{
"id": "6a1d11118b60868e",
"name": "policy-number",
"span": {
"content_part": "body",
"message_index": 0,
"utf16_byte_start": 200,
"utf16_byte_end": 222,
"char_start": 100,
"char_end": 111
},
"kind": "policy-number",
"formatted_value": "GHI-0204963"
},
{
"id": "6a1d11118b60868e",
"name": "policy-number",
"span": {
"content_part": "subject",
"message_index": 0,
"utf16_byte_start": 0,
"utf16_byte_end": 22,
"char_start": 0,
"char_end": 11
},
"kind": "policy-number",
"formatted_value": "GHI-0068448"
},
{...},
{...},
{...}
]
}
],
"model": {
"version": 31,
"time": "2021-07-14T15:00:57.608000Z"
},
"status": "ok"
}