- Getting Started
- Administration
- Manage Sources and Datasets
- Understanding the data structure and permissions
- Create a data source in the GUI
- Uploading a CSV file into a source
- Create a new dataset
- Multilingual sources and datasets
- Enabling sentiment on a dataset
- Amend a dataset's settings
- Delete messages via the UI
- Delete a dataset
- Export a dataset
- Using Exchange Integrations
- Preparing Data for .CSV Upload
- Model Training and Maintenance
- Understanding labels, entities and metadata
- Label hierarchy and best practice
- Defining your taxonomy objectives
- Analytics vs. automation use cases
- Turning your objectives into labels
- Building your taxonomy structure
- Taxonomy design best practice
- Importing your taxonomy
- Overview of the model training process
- Generative Annotation (NEW)
- Understanding the status of your dataset
- Model training and labelling best practice
- Training with label sentiment analysis enabled
- Train
- Introduction to 'Refine'
- Precision and recall explained
- Precision and recall
- How does Validation work?
- Understanding and improving model performance
- Why might a label have low average precision?
- Training using 'Check label' and 'Missed label'
- Training using Teach label (Refine)
- Training using Search (Refine)
- Understanding and increasing coverage
- Improving Balance and using 'Rebalance'
- When to stop training your model
- Using Analytics & Monitoring
- Automations and Communications Mining
- FAQs and More
Multilingual sources and datasets
Communications Mining now supports multilingual sources and datasets. This means that the models can understand sources that contain multiple different supported languages, without actually having to translate them.
The languages that are currently 'Generally Availability' within multilingual sources and datasets are: English, French, German, Spanish, Italian, Portuguese and Dutch (we'll be expanding this list over time!).
What this means in practice is that if users work and do business in several languages that are supported by the platform, they can train on messages in those languages, rather than translating everything into a single language.
A large list of additional languages are supported In Preview (included at the bottom of this page), meaning that we will be working to fine-tune them over time as our customers and partners begin to use them. A large proportion of these languages will perform very strongly, and will require little to no fine-tuning by our teams to achieve high performance.
Important considerations when looking to use multilingual sources and datasets:
- If a dataset is multilingual, users will not be able to see translations of any messages (as provided for translated datasets), so they will need to be able to understand all of the languages in the dataset to effectively train their model
- Understanding multiple languages is a more complex machine learning problem than understanding a single language, so these datasets may potentially experience a slight drop in performance compared to datasets in a single language
- The platform will only be able to understand language from one of the supported languages listed above. If there are other languages present in the dataset, tagging these messages with labels used on messages in supported languages will be confusing for the platform. It is better to label these as their own specific labels that capture the language as a label, but the platform will not be able to interpret the specifics of the unsupported language
How do you create multilingual sources and datasets?
For both data source and datasets, the language family is selected when they are created, and cannot be changed once they are.
Simply select 'multilingual' from the language family dropdown on the create source or create dataset modal (it's typically the last setting to select).
For more detail on creating a source in the UI, check the Create a data source in the GUI page.
For more detail on creating a dataset, check the Create a new dataset page.
General Availability Languages
- English
- Dutch
- French
- German
- Italian
- Portuguese
- Spanish
- Afrikaans
- Albanian
- Amharic
- Arabic
- Armenian
- Assamese
- Azerbaijani
- Basque
- Belarusian
- Bengali
- Bengali (Romanized)
- Bosnian
- Breton
- Bulgarian
- Burmese
- Burmese
- Catalan
- Chinese (Simplified)
- Chinese (Traditional)
- Croatian
- Czech
- Danish
- Esperanto
- Estonian
- Filipino
- Finnish
- Galician
- Georgian
- Greek
- Gujarati
- Hausa
- Hebrew
- Hindi
- Hindi (Romanized)
- Hungarian
- Icelandic
- Indonesian
- Irish
- Japanese
- Javanese
- Kannada
- Kazakh
- Khmer
- Korean
- Kurdish (Kurmanji)
- Kyrgyz
- Lao
- Latin
- Latvian
- Lithuanian
- Macedonian
- Malagasy
- Malay
- Malayalam
- Marathi
- Mongolian
- Nepali
- Norwegian
- Oriya
- Oromo
- Pashto
- Persian
- Polish
- Punjabi
- Romanian
- Russian
- Sanskrit
- Scottish Gaelic
- Serbian
- Sindhi
- Sinhala
- Slovak
- Slovenian
- Somali
- Sundanese
- Swahili
- Swedish
- Swiss German
- Tamil
- Tamil (Romanized)
- Telugu
- Telugu (Romanized)
- Thai
- Turkish
- Ukrainian
- Urdu
- Urdu (Romanized)
- Uyghur
- Uzbek
- Vietnamese
- Welsh
- Western Frisian
- Xhosa
- Yiddish