Email

The Email provider in Data Sync connects to an Microsoft Exchange mailbox and returns metadata from a mailbox and optionally the email message in MSG or EML formats.

The following Data Sync providers can be connected to the Email provider to download Email Messages.

  • FileSystem Provider
  • SharePoint Document Library
  • Amazon S3
  • Azure Blob Storage
  • SQL Server Table with Blob Column configuration.

Connecting with Basic Authentication

If your exchange server allows for it you can connect using your username and password by clicking onto the ellipsis in the credentials box.

Connect to Email

Enter your username and password to the fields shown in the window and click OK.

If your account is using MFA you will need to use an app password to connect. To get an app password you can go to https://account.activedirectory.windowsazure.com/AppPasswords.aspx or ask your administrator to get one for you.

Then complete the remaining fields as required. You can leave OAuth credentials blank as these are only required when connecting via OAuth.

Connecting with OAuth

Create Azure App

To begin you need to register an app in your Azure AD portal to grant access to Exchange.

To do this open your Azure AD Portal and go to Manage > App Registrations > New Registration. Now enter in a name for your application and choose the supported account types, for this we only need access to those within our organization so we have selected Accounts in this organizational directory only.

Once complete click on Register to create the app.

New App Registration

You will now be given an overview of your app credentials (Client ID and Tenant ID), but you need to configure the API permissions before you can use the credentials in Data Sync.

Configure Permissions

Go to API Permissions and click onto Add Permission.

Add Permission

Locate Exchange in the list, choose Application permissions and check Calendars.Read.All and full_access_as_app, click onto Add Permissions to add them.

Exchange Permission

Exchange Permissions

Then grant admin consent for the two permissions by clicking onto the Grant admin consent tab and following the instructions on screen.

Create Client Secret

We now need to create the Client Secret that Data Sync will use. To do this go to Certificates & Secrets and under Client secrets click New client secret.

Enter in a name/description for the secret, choose the expiry timescale and then click Add.

Generate Client Secret

Once it has been added your client secret will be shown on screen, copy this and make a note of it as you will not be able to view it again.

Copy Client Secret

These details can now be added to Data Sync to connect to Exchange.

Credentials

To connect via OAuth2 click on the ellipsis(...) in the OAuth box and leave the credentials field blank.

Connect to Email

This will open a new window where you need to enter in the following details:

OAuth2 Credentials

Tenant

Enter in your tenant domain for example simego.com.

Client Id

Enter in your Client ID from the App you created earlier to connect to Exchange. This can be found on the App overview page in Azure AD App Registrations.

Client ID

Client Secret

Enter in your Client Secret that you saved from the App you created earlier.

Once your details have been entered click OK to go back and complete the remaining fields as required.

ExchangeServerUrl

The URL to the Exchange WebServices endpoint. For Office 365 this is https://outlook.office365.com/ews/Exchange.asmx

Mailbox

The Mailbox to Open or empty for the default mailbox.

Folder

The Folder in the Mailbox to read from Inbox or SentItems

DownloadFullMessage

Specifies that the entire message is downloaded before the metadata is extracted. This can make a difference to the Body and BodyHtml values that are returned.

ExportFormat

The data format to export to the target when using File/Blob synchronisation.

Format Description
Eml The message is downloaded in EML format.
Msg The message is downloaded in MSG format.
AttachmentPdf The message is searched for an Attachment with a filename *.pdf and the first one found is returned.
AttachmentDocument The message is searched for an Attachment with a filename *.xls,*.xlsx,*.docx,*.doc,*.xml,*.csv,*.pdf and the first one found is returned.

QueryDays

The number of Day's worth or messages to return in each request (min 1) to the Exchange Server (default 5). The smaller the value to more round-trips to the server. The bigger the number the more likely you are to receive an query too complex or server busy error.

FilterByFrom

A Server Side Exchange Filter to only return messages that were sent by this value.

Example only return messages sent from Amazon aws-receivables-support@email.amazon.com

FilterByTo

A Client Side filter to only include messages in the result that were sent to this recipient.

FilterBySubject

A Client Side RegularExpression filter to only include messages in the result that match this subject.

Example only return Amazon Invoices Amazon Web Services Invoice Available*

FilterByCategory

A Client Side filter to only include messages in the result that contain this Category.

FilterByReceivedDateTime

A Server Side Exchange Filter to only return messages that were received since this time.

Set at Runtime

You can set the FilterByReceivedDateTime property at runtime in Project Automation in the Start() method.

For example setting FilterByReceivedDateTime to the previous 5 days.

public override void Start()
{
	DataSourceA.FilterByReceivedDateTime = DateTime.Today.AddDays(-5).ToString("yyyy-MM-dd HH:mm:ss");
}

Columns

The Columns collection allow you to extract values from the message body PlainText result and create new schema columns for these values.

Note: HTML format email messages do not always have a Plain Text message and therefore this cannot extract from the HTML message.

Property Example Description
Data Type System.String The Schema Column Data Type.
Name InvoiceNo The Schema Column Name.
Expression \d{4} The Expression used to extract the value.
ExtractType Regex, ReadLine, ReadLineAndTrim How to extract the value either via RegularExpression, ReadLine or ReadLineAndTrim
Find order reference Search RegularExpression to set the position of the extract.
Position StartOfDocument Where to start the search from

Example

In this example we extract the Invoice No (9051) from the body of the message into a new column InvoiceNo. We can see the Invoice No comes after order reference and is a string of 4 numbers which can be extracted with the RegularExpression \d{4}.

Connect to Email

We can use the following configuration to extract this from the Body.

Connect to Email