Using an automation to upload data

When it comes to uploading communications for automation, there are two key concepts to grasp:

• What data do you need to upload?
• How do you upload this data?

What data do you need to upload?

When designing an automation to upload data to Communications Mining, the first step is to determine what data you truly need to upload. Typically, this data will fall into two main categories: the content of the communication itself and the user properties that go along with it.

Note: You can think of user properties as being metadata.

Communications Mining requires that a communication has at a minimum a body, a timestamp, and an ID. You can see a complete list of the fields that are supported in the Communications Mining Developer Guide, but there are a few key aspects to mention about the content of the communications:

• The value that you’re using for the ID should not be randomly generated, this should ideally come from whatever system that you’re sourcing the data from, for example you might use the Internet Message ID header when syncing from Gmail or the id assigned to a review if uploading reviews from a forum. This will make re-processing the data much easier if you need to do this later
• Timestamps should be provided in UTC in order to ensure consistency across your dataset
• If the subject / body of the email needs to be cleaned up (e.g. due to bad formatting in the source system) this needs to happen before the data is uploaded into the platform.

User properties are helpful when uploading data alongside the communication itself.

When deciding what data to include as user properties, there are two things to consider:

1. What data do I need to do my downstream automation?

   First, consider what data your downstream automation will require. For instance, when syncing emails from Gmail, you'll likely need the message ID, the mailbox the email came from, and perhaps the folder it was stored in. Add all this data as user properties, so your automation can use it later to perform actions on the communication.

2. What data will be useful for Analytics?

   Next, think about what information will be valuable for analytics within the Communications Mining platform itself. Usually, you'll want to chat with your business users about this during the solution design phase. For example, when uploading communications from a CRM tool, including the customer's tier (like gold, silver, bronze) or their geographic location can be really useful. This data allows the business to ask insightful questions, such as 'What do Gold tier customers complain about that bronze tier customers do not?'.