- Getting started
- Balance
- Clusters
- Concept drift
- Coverage
- Datasets
- General fields (previously entities)
- Labels (predictions, confidence levels, hierarchy, etc.)
- Models
- Streams
- Model Rating
- Projects
- Precision
- Recall
- Reviewed and unreviewed messages
- Sources
- Taxonomies
- Training
- True and false positive and negative predictions
- Validation
- Messages
- Administration
- Manage sources and datasets
- Understanding the data structure and permissions
- Create a data source in the GUI
- Uploading a CSV file into a source
- Create a new dataset
- Multilingual sources and datasets
- Enabling sentiment on a dataset
- Amend a dataset's settings
- Delete messages via the UI
- Delete a dataset
- Export a dataset
- Using Exchange Integrations
- Preparing data for .CSV upload
- Model training and maintenance
- Understanding labels, general fields and metadata
- Label hierarchy and best practice
- Defining your taxonomy objectives
- Analytics vs. automation use cases
- Turning your objectives into labels
- Building your taxonomy structure
- Taxonomy design best practice
- Importing your taxonomy
- Overview of the model training process
- Generative Annotation (NEW)
- Dastaset status
- Model training and annotating best practice
- Training with label sentiment analysis enabled
- Train
- Introduction to Refine
- Precision and recall explained
- Precision and recall
- How does Validation work?
- Understanding and improving model performance
- Why might a label have low average precision?
- Training using Check label and Missed label
- Training using Teach label (Refine)
- Training using Search (Refine)
- Understanding and increasing coverage
- Improving Balance and using Rebalance
- When to stop training your model
- Using general fields
- Generative extraction
- Using analytics and monitoring
- Automations and Communications Mining
- Licensing information
- FAQs and more
Communications Mining User Guide
Monitoring
User permissions required: ‘Modify datasets’ to enable Tone analysis.
Tone analysis is the process of understanding and identifying the emotional tone or attitude expressed in a message.
The primary purpose of tone analysis is to gain insights into the emotional state of the writer to understand how they're feeling about a specific subject or topic.
Leveraging the platform's tone analysis capabilities allows users to be proactive in customer experience management, and can help users anticipate and address customer needs based on the emotions expressed within their communications.
Label sentiment is typically only appropriate for customer feedback related datasets. This is because they contain many more identifiable expressions of sentiment than other datasets, which tend to be much more neutral by nature.
For all other use cases (e.g. email inbox analysis and automation), tone should be used because the emotional tone expressed in a message may not simply be a positive vs. a negative sentiment. It could be a complex spectrum of emotions, better represented by a scoring system. The majority of communications in these datasets are also neutral in tone, which is not captured by label sentiment analysis.
Communications Mining's tone model is trained to look for specific expressions of sentiment, both positive and negative, and aggregates them up into an overall 'tone score' between -10 and 10.
This score then becomes a filterable / queryable property for each message, and can be aggregated up at different levels in Reports.
Additionally, the tone score is used as a contributing factor in generating Quality of Service (QoS) scores.
Step 1: Ensure that you have the right permissions to enable QoS ('Quality of Service')
Step 2: Turn tone in the dataset. This can be enabled in the dataset's settings page at any time.
User permissions required: ‘Quality of Service’ AND ‘Pre-trained labels’.
The platform combines the overall sentiment, i.e. tone, and impact of intents in every message to automatically compute a configurable Quality of Service (QoS) score for each message. It also enables real-time QoS dashboards and alerting based on predefined thresholds.
This functionality has many useful applications, including (but not limited to):
- Quality assurance across customer facing teams.
- Identification of prevalent customer issues & recovery opportunities.
- Performance monitoring & proactive customer interventions.
- Prioritisation of agents’ work & time management.
The QoS feature helps to ensure that customers are provided with the highest quality of service, whilst identifying priority areas for improvement. It allows managers to measure agents’ performance, prioritise their work and drive timely interventions.
The platform automatically calculates a Quality of Service (QoS) score between -10 and 10 for every message. It does this by combining the predicted tone score (also -10 to 10) with the combined impact scores across label predictions (weighted by prediction confidence).
Label impacts make up the majority of the QoS score, with tone representing ~10% of the score.
Step 1: Ensure that you have the right permissions to enable QoS ('Quality of Service' AND 'Pre-trained labels' where applicable)
Step 2: Turn on pre-trained labels, on key QoS intents that are applicable to your use case. These can be made trainable or non-trainable.
Step 3: Turn on tone analysis (tone is incorporated into the QoS score). This can be enabled in a dataset’s settings page (accessible via the top navigation bar), on the 'General' tab.
Step 4: Change the label impacts. Every label can be assigned an impact score from -10 to +10 (many are typically left at 0).
You can change the QoS label impact by clicking into a label's settings (via the Explore tab, on the taxonomy bar on the left-hand side), and adjusting the QoS slider on a label.
We can use QoS within the following Communications Mining product features:
- Charts: Charts displaying QoS are available in the Label Summary, Trends, and Segments tabs within Reports, and they can all be added to dashboards for monitoring.
- Filters: QoS and tone scores become a property of each message, meaning they can be filtered on in Reports and Explore.
- Explore: There are sort orders in Explore for QoS and Tone, allowing users to explore qualitative examples of high and low scoring messages.
- Alerts: QoS and tone can be filter inputs to alerts, and alert types are available for tracking changes in QoS scores.
The platform provides real-time alerting capabilities, allowing users to define issues that impact their clients, processes and service quality, and be alerted each time they occur ( e.g. risk events, client complaints, long-running issues, breaks, etc.). These notifications are currently available on the platform only (API integration will be available soon).
This functionality has many useful applications, including (but not limited to):
- Communications volume & Quality of Service monitoring.
- Prioritisation & resolution of issues as soon as they occur.
- Triggering downstream automation.
- Keeping an audit trail of resolved issues & tracking of unresolved issues.
The Alerts feature gives users the visibility into the recurring or high-risk issues within their communication channels. It also enables them to monitor unresolved issues and once the issue is fixed, mark them as resolved.
Alert Centre (within the admin console) houses both the Issues and Alerts pages. Once set up, alerts trigger issues when their criteria are met, and these are tracked in the Issues page.
Active issues will track how many times an alert was triggered whilst the issue remained unresolved. When an issue has been investigated and addressed, it can be marked as resolved. If the alert is triggered again in future, a new issue is created.
Step 1: Permissions: Users need Alerts Admin to create, modify and delete alerts and View Alerts to view alerts and issues raised by them. Please ensure that the appropriate permissions have been assigned.
Step 2: Go to the Alert Center (accessible via the Admin console). Alerts are created and updated in the Alerts page within Alert Centre
Step 3: Link your alert to dataset(s). Alerts are linked to a primary dataset (to determine label filters available) and then any number of additional datasets that you have access to within the tenant
Step 4: Select your alert type. Current alert types are for volume changes, or changes to average quality of service scores
Step 5: Select your filters. All the usual filters can be applied to determine which messages are in consideration for each alert
Alert previews: When setting up alerts, users can preview how many times the alert would have been triggered in a past time period, e.g. 6 months
There are 2 types of alerts available: alerts linked to email volumes and alerts linked to average quality of service scores.
- Volume based alerts focus on communications volume monitoring. For example, number of exceptions or errors, or any instance of a specific high-risk event
- QoS based alerts focus on service quality monitoring. For example, changes in the QoS score for our high-risk or high-value customers
To create an alert, click on the New alert button in the Alerts page.
For each alert, you need to specify:
- Relevant project
- Relevant dataset
- Alert name
- Applicable filters: these are the same filters available in Explore or Reports (e.g. label filters or user property filters). If no filters are applied, the alert will be triggered on the entire dataset
Now you can define the alert's conditions.
Our Alerts functionality enables you to customise individual alert's conditions based on your business needs and objectives.
Firstly, you can choose between the volume based or QoS based alerts:
Secondly, you can indicate your desired level of increase or decrease; and specify the exact value / percentage:
Thirdly, you can select your preferred time period:
Finally, you can define your ideal comparison benchmark:
Our objective is to create an alert that gets triggered each time the average QoS score drops by 0.3 within a single day, compared to the long running average.
- Select QoS alert in the dropdown list.
- Select ‘has a decrease of more than’ in the dropdown list.
- Type in ‘0.3’ in the free text field.
- Select ‘1 day’ in the dropdown list.
- Select ‘long running average’ in the dropdown list.
- Click on 'Preview alert' button.
- Click on 'Create alerts' button.
This particular alert would have triggered one issue in this period; and the issue is still ongoing.
All existing alerts can be edited or deleted, by clicking on the pencil or bin icon next to its name.
We can use QoS within the following Communications Mining product features:
- Issues page: This page tracks all active unresolved issues, and past resolved issues (providing an audit log of past issues).
- Resolving issues: Active issues will count the number of times the issue has been triggered so far, and once addressed can be marked as resolved by an alerts admin.
- Explore examples: When an alert triggers an issue, the issue card allows user to click into examples of messages causing the alert to trigger the issue, enabling investigation and resolution.
- In-platform: Currently issues are only tracked in-platform, but will be available via API soon.
The platform will show you all identified issues, triggered by alerts, in the Issues page by default. However, you can filter on the Active or Resolved issues. Active issues is often used to track all unresolved issues, whereas Resolved issues enables you to keep an audit trail of all past issues.
Active issues can be marked 'resolved' once investigated and fixed. Resolved issues can be re-opened if needed.
Additionally, each issue contains relevant alert details, as well as all messages that match the alert. These can be accessed by clicking on the respective arrow icons.
- Tone analysis
- What is tone?
- When should we use tone vs. label sentiment?
- How does it work?
- How to set this up
- What does it look like?
- Quality of Service
- What is Quality of Service (QoS)?
- How does it work?
- How to set this up
- Tracking and monitoring
- What does it look like?
- Alert Center
- What are alerts?
- What is Alert Centre?
- Quick Guide: How to set up an alert
- Detailed Walkthrough: How to set up alerts
- Available conditions
- Example: Quality of Service (QoS) alert
- How to edit or delete alerts
- Tracking and monitoring
- How to track and resolve issues triggered by Alerts