Uploading a PST file

Upload historical emails into Communications Mining from a PST file exported from Exchange, with the export requirements, known limitations, and resolutions for upload errors.

Wichtig:

A PST file is a one-off snapshot, not a live feed. Use a PST upload to unblock model training or to backfill historical emails while a permanent integration is being set up. For ongoing email ingestion, use the Exchange integration instead.

You upload a PST file with the Communications Mining™ CLI, which parses the emails in the file and writes them to a bucket. You then create a source from that bucket and add the source to a dataset.

When to use a PST file

Use a PST file when	Do not use a PST file when
You need to unblock model training before a live integration is available.	You are moving to production. Set up the Exchange integration instead.
You need a historical backfill of emails that the live integration cannot reach, for example because the mailbox uses an in-place archive.	You need real-time or ongoing ingestion. A PST file captures the mailbox at a single point in time.
Your environment blocks the Exchange integration and you need data in the platform quickly.	You need complete conversation threads. A PST export can omit messages from a conversation.

Tipp:

For a historical backfill, keep the Exchange integration enabled for new emails and use the PST file only for the older data. Keeping the sync running during training gives active learning more data to work with.

Bekannte Einschränkungen

A PST upload is not a reliable way to get data into the platform, and it is not a substitute for a live integration. Every PST upload loses some data, so do not expect every email in the file to reach the platform. Where your environment allows it, use the Exchange integration instead.

The following limitations apply:

• PST files exported from Outlook are not supported. An Outlook client export drops the transport headers that Communications Mining requires and omits the Sent Items folder. The CLI can fail to parse such a file entirely. This is the most common cause of large-scale data loss.
• Emails whose only body is RTF are not uploaded. The CLI uses the HTML body when one is present, and falls back to the plain-text body. An email is dropped only when both are missing. Most RTF-composed Outlook mail also carries a plain-text alternative and uploads without a problem. There is no way to detect RTF-only emails in a PST file before you run the upload.
• Attachment content is not uploaded. Only the attachment metadata is uploaded: name, size, and content type. The content type is inferred from the file extension rather than read from the PST file. Attachments that are themselves emails are skipped, so an email whose only attachment is a forwarded message reports no attachments.
• Some data loss is expected. As a rule of thumb, budget for approximately 1% of emails failing. Actual results vary in both directions, so run the upload with --dry-run first and check the emails failed to parse and failed to upload counters that the CLI prints at the end of every run.
• Items that are not emails are counted as failures. The CLI walks every folder in the PST file and requires each item to have both a Message-ID and a Date header. Drafts, calendar items, contacts, and tasks have neither, so they are reported as parse failures. A non-zero failure count is expected.
• Threads can be incomplete. A PST file does not necessarily contain every message in a conversation.
• Character sets other than UTF-8 can be garbled. Japanese (ISO-2022-JP) and Chinese emails are the most affected. The behavior depends on the body type:
  • Plain-text bodies are converted using the codepage of the message. If the conversion fails, the email is dropped and counted as a parse failure, which --dry-run reports.
  • HTML bodies are coerced to UTF-8, so bytes that cannot be decoded become replacement characters. No error is raised, no counter is incremented, and --dry-run does not report it. Check a sample of HTML emails in the platform after the upload.

Wichtig:

To populate a mailbox before you export it, move, copy, or redirect the emails, for example by dragging them between mailboxes in Outlook. Do not forward them. A forwarded message arrives with the previous conversation stripped from the body, which leaves only headers and addresses and is not useful for training.

Exporting the PST file

Export the PST file from Exchange, not from Outlook. A PST file exported from the Outlook client, through File > Open & Export > Import/Export, omits the Sent Items folder and other folders, and can produce a file that the CLI cannot parse.

For an overview of the supported export routes, check Mailbox export procedures.

Microsoft 365 and Exchange Online

Use Microsoft Purview eDiscovery for cloud mailboxes. For details, check Export search results in eDiscovery.

Voraussetzungen:

• The administrator running the export has the eDiscovery Manager role in the Microsoft Purview portal. This role is not assigned by default.
• The organization has an E3 or E5 license.

Steps:

1. In the Microsoft Purview portal, create or open an eDiscovery case.
2. Run a search scoped to the target mailboxes. Leave the keywords empty to return everything in the date range.
3. Select Export on the search.
4. Under Export format, choose Create PSTs for messages.
5. Include the archive mailbox and Recoverable Items to export the full history. The primary and archive mailboxes are merged into a single PST file.
6. Set the maximum PST package size to 1, 2, 5, or 10 GB. Smaller packages are easier to transfer.
7. Download the package from Process manager.

Wichtig:

Export packages expire after 14 days. Extract the package with a third-party archiving tool rather than the built-in Windows extraction, and check the file sizes after the download completes.

On-premises Exchange Server

This procedure applies to Exchange Server 2016, 2019, and Subscription Edition. For details, check Export eDiscovery search results to a PST file.

Voraussetzungen:

• The export machine has .NET Framework 4.7 and the ClickOnce add-in installed in Chrome or Firefox.
• The account running the export does not use multi-factor authentication (MFA). The eDiscovery PST Export Tool does not work with MFA accounts, so create an app password for the account first.

Steps:

1. In the Exchange admin center, go to Compliance management > In-Place eDiscovery & Hold.
2. Select the search, and then select Export to a PST file.
3. In the eDiscovery PST Export Tool, choose a download location.
4. Optionally, select Enable deduplication to write all results to a single PST file, and Include unsearchable items.
5. Wählen Sie Start aus.

Hinweis:

Corporate proxies and firewalls can block or throttle the export download. The Microsoft endpoints involved might need to be added to your network allowlist.

Uploading the PST file

Voraussetzungen

• The Communications Mining CLI is installed. For details, check CLI.
• A context is configured for your Communications Mining endpoint. For details, check Configuring the CLI.
• The PST file is exported from Exchange, as described in the preceding section.
• You have permissions to create buckets and sources in the target project.

Wichtig:

Uploading emails consumes AI Units or Platform Units, so the CLI prompts you to consent to the charge before the upload starts. A dry run consumes nothing but still displays the prompt. Add --yes to suppress the prompt in an unattended or scripted run.

Schritte

1. Create a bucket for the emails. Project and bucket names accept only letters, digits, hyphens, and underscores.

   re -c <context> create bucket <ProjectName>/<BucketName>
   

2. Check that the PST file parses, before you consume any units.

   re -c <context> parse pst --bucket <ProjectName>/<BucketName> --file <PathToPst> --dry-run
   

   The command reads the whole file and reports parse errors without uploading anything. It prints either No parse errors found or a breakdown of each distinct error and the number of emails affected.

3. Upload the PST file into the bucket.

   re -c <context> parse pst --bucket <ProjectName>/<BucketName> --file <PathToPst> --resume-on-error
   

   The --resume-on-error flag skips and counts emails that cannot be processed. Without it, the first email that fails aborts the whole run.

The emails are now in the bucket. To make them available for training, create a source from the bucket and add that source to a dataset.

Upload options and behavior

• One PST file per command. The --file flag takes a single path. Loop in your shell to upload multiple PST files.
• The mailbox name in the platform is the file name of the PST file, including the extension. A file named export (1).pst produces a mailbox named export (1).pst, which is visible to your users. Rename the file to something meaningful before you upload it.
• A large PST file starts slowly. The CLI counts every item in the file before it begins uploading, so the file is read twice and the progress bar takes time to appear. Do not stop the run.
• --num-threads has no effect. The re parse pst command is single-threaded by design.
• Re-running the same PST file into the same bucket is idempotent. Each email is keyed on its Message-ID header and the upload is an upsert, so it is safe to re-run after a failed or interrupted upload.
• Drop --resume-on-error when you diagnose failures. The CLI then stops at the first failure and reports the reason.

Buckets, sources, and datasets

Understanding how emails move through the platform prevents the most common cause of duplicated comments:

• A bucket stores raw emails. This is where PST and Exchange emails land.
• A source is created from a bucket. The emails sync from the bucket into the source, where they are converted into comments.
• A dataset is what you train on. It contains one or more sources.

As a best practice, attach one bucket to one source, and populate each source through one method only, either a bucket or the API. If you attach the same bucket to four sources, every email is duplicated across all four.

For details on creating the source, check Creating or deleting a data source in the GUI.

Duplicate emails when you also use the Exchange integration

Communications Mining de-duplicates emails on the Message-ID header, and the upload is an upsert. Backfilling with a PST file and then enabling the Exchange integration therefore does not produce duplicate comments. Two PST files that share emails do not double up either, provided both carry the same Message-ID. If you do see duplicates, check how many sources the bucket is attached to.

Fehlersuche und ‑behebung

Symptome	Cause and resolution
Every email fails to parse, for example Total emails: 304 / failed to parse: 304 / uploaded: 0.	The PST file was exported from Outlook, which is not supported. Re-export the file from Exchange.
Some emails fail and the reason is not clear.	Re-run the upload without --resume-on-error so that the CLI stops at the first failure and prints the reason. A --dry-run reports the same errors across the whole file without uploading.
You need to know whether a PST file is corrupt.	If the CLI can open the file and iterate its items, look for a parse-level cause instead. If the file cannot be read at all, download it again and check that the local file size matches the source. A partial download resembles a corrupt file.
The CLI reports more emails than the dataset contains.	The PST files contain duplicate Message-ID values, which the platform de-duplicates on upload. No report of what was de-duplicated is available. To confirm, count the unique Message-ID values in the file.
You need to find the RTF-only emails before you run the upload.	This is not detectable up front. Run the upload with --dry-run and read the reported errors.
The upload appears to have stalled.	On a large PST file the CLI counts all items before it uploads, so there is a long quiet phase first. If the run is genuinely stuck, check that the machine is not resource-constrained, and copy the PST file to a local path outside any synced cloud storage folder.
Emails display garbled text.	Known limitation for character sets other than UTF-8, such as Japanese ISO-2022-JP and Chinese. HTML bodies are corrupted silently, so --dry-run does not report them. No workaround is available.
re create bucket or re parse pst returns Unknown context, or targets the wrong tenant.	Every command needs a context: re -c <context>. The -c flag takes the name of a context you already saved, not an organization and tenant string. A context created with re config parse-from-url is named <organization>/<tenant>, and a context created with re config add --name production is named production. List the saved contexts with re config ls.
The upload prompts for confirmation and blocks a scripted run.	The prompt is the consent to the AI Unit or Platform Unit charge. Add --yes.
The CLI cannot reach the platform.	Your network is filtering egress. Add the platform and CLI endpoints to your allowlist, or run the CLI from a machine that has access.
Not all emails were uploaded.	Expected to a degree. Confirm that the export came from Exchange, that --resume-on-error was used, and check the failure counters. Drafts, calendar items, and contacts are counted as failures.

If your issue is not listed, submit a UiPath® Support ticket.