Monitoring

Tone analysis

User permissions required: ‘Modify datasets’ to enable Tone analysis.

What is tone?

Tone analysis is the process of understanding and identifying the emotional tone or attitude expressed in a message.

The primary purpose of tone analysis is to gain insights into the emotional state of the writer to understand how they're feeling about a specific subject or topic.

Leveraging the platform's tone analysis capabilities allows users to be proactive in customer experience management, and can help users anticipate and address customer needs based on the emotions expressed within their communications.

When should we use tone vs. label sentiment?

Label sentiment is typically only appropriate for customer feedback related datasets. This is because they contain many more identifiable expressions of sentiment than other datasets, which tend to be much more neutral by nature.

For all other use cases (e.g. email inbox analysis and automation), tone should be used because the emotional tone expressed in a message may not simply be a positive vs. a negative sentiment. It could be a complex spectrum of emotions, better represented by a scoring system. The majority of communications in these datasets are also neutral in tone, which is not captured by label sentiment analysis.

How does it work?

Communications Mining's tone model is trained to look for specific expressions of sentiment, both positive and negative, and aggregates them up into an overall 'tone score' between -10 and 10.

This score then becomes a filterable / queryable property for each message, and can be aggregated up at different levels in Reports.

Additionally, the tone score is used as a contributing factor in generating Quality of Service (QoS) scores.

How to set this up

Step 1: Ensure that you have the right permissions to enable QoS ('Quality of Service')

Step 2: Turn tone in the dataset. This can be enabled in the dataset's settings page at any time.

Tone Analysis Toggle

What does it look like?

Example message with Tone enabled

Quality of Service

User permissions required: ‘Quality of Service’ AND ‘Pre-trained labels’.

What is Quality of Service (QoS)?

The platform combines the overall sentiment, i.e. tone, and impact of intents in every message to automatically compute a configurable Quality of Service (QoS) score for each message. It also enables real-time QoS dashboards and alerting based on predefined thresholds.

This functionality has many useful applications, including (but not limited to):

• Quality assurance across customer facing teams.
• Identification of prevalent customer issues & recovery opportunities.
• Performance monitoring & proactive customer interventions.
• Prioritisation of agents’ work & time management.

The QoS feature helps to ensure that customers are provided with the highest quality of service, whilst identifying priority areas for improvement. It allows managers to measure agents’ performance, prioritise their work and drive timely interventions.

How does it work?

The platform automatically calculates a Quality of Service (QoS) score between -10 and 10 for every message. It does this by combining the predicted tone score (also -10 to 10) with the combined impact scores across label predictions (weighted by prediction confidence).

Label impacts make up the majority of the QoS score, with tone representing ~10% of the score.

How to set this up

Step 1: Ensure that you have the right permissions to enable QoS ('Quality of Service' AND 'Pre-trained labels' where applicable)

Step 2: Turn on pre-trained labels, on key QoS intents that are applicable to your use case. These can be made trainable or non-trainable.

QoS Labels

Step 3: Turn on tone analysis (tone is incorporated into the QoS score). This can be enabled in a dataset’s settings page (accessible via the top navigation bar), on the 'General' tab.

'Tone analysis' toggle on the dataset settings page

Step 4: Change the label impacts. Every label can be assigned an impact score from -10 to +10 (many are typically left at 0).

You can change the QoS label impact by clicking into a label's settings (via the Explore tab, on the taxonomy bar on the left-hand side), and adjusting the QoS slider on a label.

Slider to adjust QoS impact

Tracking and monitoring

Example QoS features associated with tracking and monitoring

We can use QoS within the following Communications Mining product features:

• Charts: Charts displaying QoS are available in the Label Summary, Trends, and Segments tabs within Reports, and they can all be added to dashboards for monitoring.
• Filters: QoS and tone scores become a property of each message, meaning they can be filtered on in Reports and Explore.
• Explore: There are sort orders in Explore for QoS and Tone, allowing users to explore qualitative examples of high and low scoring messages.
• Alerts: QoS and tone can be filter inputs to alerts, and alert types are available for tracking changes in QoS scores.

What does it look like?

Example message with QoS enabled

Alert Center

User permissions required: ‘Alerts Admin’ AND ‘View Alerts’.

Note: All users have 'View Alerts' permission by default and are able to view all alerts and issues. Each project's founding user has 'Alerts Admin' permission by default.

What are alerts?

The platform provides real-time alerting capabilities, allowing users to define issues that impact their clients, processes and service quality, and be alerted each time they occur ( e.g. risk events, client complaints, long-running issues, breaks, etc.). These notifications are currently available on the platform only (API integration will be available soon).

This functionality has many useful applications, including (but not limited to):

• Communications volume & Quality of Service monitoring.
• Prioritisation & resolution of issues as soon as they occur.
• Triggering downstream automation.
• Keeping an audit trail of resolved issues & tracking of unresolved issues.

The Alerts feature gives users the visibility into the recurring or high-risk issues within their communication channels. It also enables them to monitor unresolved issues and once the issue is fixed, mark them as resolved.

What is Alert Centre?

Alert Centre (within the admin console) houses both the Issues and Alerts pages. Once set up, alerts trigger issues when their criteria are met, and these are tracked in the Issues page.

Active issues will track how many times an alert was triggered whilst the issue remained unresolved. When an issue has been investigated and addressed, it can be marked as resolved. If the alert is triggered again in future, a new issue is created.

Example Issue triggered by an Alert

Quick Guide: How to set up an alert

Step 1: Permissions: Users need Alerts Admin to create, modify and delete alerts and View Alerts to view alerts and issues raised by them. Please ensure that the appropriate permissions have been assigned.

Step 2: Go to the Alert Center (accessible via the Admin console). Alerts are created and updated in the Alerts page within Alert Centre

Step 3: Link your alert to dataset(s). Alerts are linked to a primary dataset (to determine label filters available) and then any number of additional datasets that you have access to within the tenant

Step 4: Select your alert type. Current alert types are for volume changes, or changes to average quality of service scores

Step 5: Select your filters. All the usual filters can be applied to determine which messages are in consideration for each alert

Example of an Alert being set up

Alert previews: When setting up alerts, users can preview how many times the alert would have been triggered in a past time period, e.g. 6 months

Detailed Walkthrough: How to set up alerts

There are 2 types of alerts available: alerts linked to email volumes and alerts linked to average quality of service scores.

• Volume based alerts focus on communications volume monitoring. For example, number of exceptions or errors, or any instance of a specific high-risk event
• QoS based alerts focus on service quality monitoring. For example, changes in the QoS score for our high-risk or high-value customers

To create an alert, click on the New alert button in the Alerts page.

Alert details + matched messages example

For each alert, you need to specify:

• Relevant project
• Relevant dataset
• Alert name
• Applicable filters: these are the same filters available in Explore or Reports (e.g. label filters or user property filters). If no filters are applied, the alert will be triggered on the entire dataset

Alert configuration window

Now you can define the alert's conditions.

Available conditions

Our Alerts functionality enables you to customise individual alert's conditions based on your business needs and objectives.

Firstly, you can choose between the volume based or QoS based alerts:

Secondly, you can indicate your desired level of increase or decrease; and specify the exact value / percentage:

Thirdly, you can select your preferred time period:

Finally, you can define your ideal comparison benchmark:

Note: Each alert allows you to preview the issues it would trigger. This is particularly helpful when defining the specific conditions of the alert to ensure we are not creating an overly sensitive alert, or one that will never be triggered.

Example: Quality of Service (QoS) alert

Our objective is to create an alert that gets triggered each time the average QoS score drops by 0.3 within a single day, compared to the long running average.

1. Select QoS alert in the dropdown list.
2. Select ‘has a decrease of more than’ in the dropdown list.
3. Type in ‘0.3’ in the free text field.
4. Select ‘1 day’ in the dropdown list.
5. Select ‘long running average’ in the dropdown list.
6. Click on 'Preview alert' button.
7. Click on 'Create alerts' button.

Alert's conditions example

This particular alert would have triggered one issue in this period; and the issue is still ongoing.

Preview Issues window

How to edit or delete alerts

All existing alerts can be edited or deleted, by clicking on the pencil or bin icon next to its name.

Edit Alert window

Edit Alert window

Tracking and monitoring

We can use QoS within the following Communications Mining product features:

• Issues page: This page tracks all active unresolved issues, and past resolved issues (providing an audit log of past issues).
• Resolving issues: Active issues will count the number of times the issue has been triggered so far, and once addressed can be marked as resolved by an alerts admin.
• Explore examples: When an alert triggers an issue, the issue card allows user to click into examples of messages causing the alert to trigger the issue, enabling investigation and resolution.
• In-platform: Currently issues are only tracked in-platform, but will be available via API soon.

How to track and resolve issues triggered by Alerts

The platform will show you all identified issues, triggered by alerts, in the Issues page by default. However, you can filter on the Active or Resolved issues. Active issues is often used to track all unresolved issues, whereas Resolved issues enables you to keep an audit trail of all past issues.

Issues filter

Active issues can be marked 'resolved' once investigated and fixed. Resolved issues can be re-opened if needed.

Additionally, each issue contains relevant alert details, as well as all messages that match the alert. These can be accessed by clicking on the respective arrow icons.

Issue example

Alert details + matched messages example