Introduction

Email Collection Module Overview

Purpose

This module enables the automatic collection of emails from a designated mailbox using the IMAP protocol. It facilitates the extraction of relevant information from emails to record events or data directly into the SolutionsInc system.

Key Features

• IMAP Protocol: Utilizes the Internet Message Access Protocol (IMAP) to connect and retrieve emails from your specified mailbox.
• Data Extraction: Automatically extracts relevant information from incoming emails, which can then be used to create records or events within SolutionsInc.
• Event Recording: Supports the automatic logging of events or other data based on the content or metadata of the collected emails.
• Integration: Seamlessly integrates with existing workflows in SolutionsInc, enhancing the efficiency of data entry and event tracking.

---

Benefits

Improved Communication Tracking: Keeps a consistent and organized log of communications and actions stemming from email interactions.

Increased Efficiency: Reduces manual data entry by automating the process of capturing email content, streamlining operations.

Real-Time Data Entry: Ensures that data derived from emails is recorded quickly and accurately in the system.

Setup

Email Collection Module Setup Guide

Step 1: Activation

To start using the Email Collection Module, you must first activate it with an administrator account.

1. Navigate to: Home -> Setup -> Modules.
2. Find the tab where the Email Collection Module is listed.
3. Click on “Activate”.

Note: The module is now activated. If you see a cog icon on the module thumbnail or at the end of the module’s line, click on it to access the specific setup page for the module.

Step 2: Configure IMAP Settings

It is recommended to enable the option to use the PHP-IMAP library for better functionality.

• Enable: “Use the PHP-IMAP library for IMAP instead of native PHP IMAP.”
  • This also allows the use of an OAuth2 connection for IMAP (ensure the OAuth module is also activated).

Step 3: Create an Email Collector

You can create multiple email collectors depending on your requirements.

1. Host of Email IMAP Server:
   • Enter the full name of the IMAP server, e.g., imap.gmail.com.
2. Port of Email IMAP Server / IMAP Encryption:
   • Set to 993 with SSL.
3. Host Charset:
   • Choose UTF-8 (commonly works well). If you’re using some versions of Office365, you may also consider US-ASCII.
4. Authentication Mode:
   • Password: Set an IMAP login and password.
   • OAuth2: Set the IMAP login and specify the OAuth2 entry in the field labelled OAuth Service. Refer to the Module OAuth documentation for creating your OAuth2 entry and ensuring the correct permissions.
5. Source Directory:
   • For collecting received emails, enter “INBOX”.
   • For collecting sent emails, enter “Sent”.
6. Target Directory:
   • It is a best practice to move collected emails into a separate folder to prevent duplication. Create this output target IMAP directory manually before running your email collector.
   • For example, to create a new IMAP directory in Gmail:
     • Sign in to your Gmail account.
     • Click on Settings (the gear icon) -> See all settings.
     • Navigate to the Labels tab.
     • Create a new label and check the box: “Show in IMAP”.

Step 4: Filters

• Choose criteria to define which emails qualify for collection based on specified attributes (e.g., sender, subject, keywords).

Step 5: Operations to Perform

• Select the desired actions to take on collected emails, such as moving, marking as read, or deleting them after being processed.

---

This guide provides step-by-step instructions for activating the Email Collection Module, configuring IMAP settings, and creating an email collector in SolutionsInc. By following these steps, you can efficiently automate the collection of emails and integrate them into your operational workflow.

Usage

Let the job scheduler (Module Scheduled jobs) being executed automatically to run the Email collector and see your data being recorded into SolutionsInc.

To avoid waiting the batch, you can force to execute the email collector manually from Manual Execution and Test Mode

Executing the Email Collector

To avoid waiting for the batch execution:

• Run Now: Navigate to the email collector record and click the “Run now” button to execute it manually.

Test Mode

A Test Mode is also available:

• Activate Test Mode: Click the “Test” button before running your collector. This mode will not modify your mailbox or make any changes in SolutionsInc. Instead, it reads emails and processes them as if it were live, allowing you to:
  • View qualified email selection stages.
  • Access filters applied.
  • See the actions that would be taken without making actual changes.

Best Practice: Set the maximum number of emails to collect to a low value during testing to avoid processing large volumes of data.

Troubleshooting Common Issues

• Connection Setup Error: If you encounter the error “connection setup failed – authenticate” while using Google OAuth and your OAuth token and refresh token appear to work, double-check the following:
  • Verify the Login (email) in your Email Collector setup corresponds exactly to your OAuth credentials.

Example Use Case: Email Collector for a Support Center

In this scenario, you have a support team that shares a common email address:

Email Address: support@mycompany.com

Step 1: Configure Employee Email Accounts

Each employee corresponding to the support centre needs to configure their personal email accounts:

1. Go to Gmail or your email provider’s settings.
2. Navigate to the “Accounts and Import” section.
3. Under “Send mail as,” click “Add another email address.”
4. Enter support@mycompany.com and uncheck “Treat as an alias.”
5. Validate the sender profile via the email sent to support@mycompany.com and enable “Reply from the same address the message was sent to.”

Step 2: Set Up Additional Email for Monitoring

To ensure a direct record of incoming and outgoing correspondence:

1. Create an additional email account such as followsupport@mycompany.com.
2. Add it to the mailing list of support@mycompany.com.
   • Therefore, any email sent to the support centre will be copied to followsupport@mycompany.com, allowing the email collector to monitor these emails.

Step 3: Enable Required Modules

1. Activate the Email Collector Module.
2. Activate the Ticket Module in SolutionsInc.

Step 4: Create an Email Collector Entry

1. Create an Email Collector entry with the credentials for followsupport@mycompany.com.
   • Use Inbox as the input directory.
   • Set the output directory to “Processed emails” (create this folder/label in your email account prior to the initial run).

Step 5: Configure Email Extraction Actions

Content Structure: Imagine the received emails are formatted with:

Organization: A customer or name
Email: acustomeremail@example.com

Set Actions:

Load Third-Party Information:

Action: “Load the third party by searching name/alias/email/id (load only)”

Parameters:

name=EXTRACT:BODY:Organization:\s*([\w\-\s]+)$
email=EXTRACT:BODY:Email:\s*([\w\.\-]+@[a-zA-Z\.\-]+)

Create or Complete a Ticket:

Action: “Create or complete a ticket” (no parameters).

Final Setup

Each time an email is received at support@mycompany.com, or a response is sent by any employee:

• The followsupport@mycompany.com address receives the email, and the Email Collector processes it based on your configuration.
• It will determine whether to create a new help desk ticket or complete an existing ticket with the parsed information.

Example of use case: Create an opportunity for a team of sales

Email Collector Setup for Sales Team

Overview

The objective is to use an email collector to create or update opportunities in your project management system whenever emails are sent to the sales team’s shared email address sales@mycompany.com. This way, all correspondence can be processed to manage customer opportunities efficiently.

Step-by-Step Implementation

Step 1: Configure Employee Email Accounts

1. Sender Email Declaration:
   • Each employee should declare sales@mycompany.com as a sender email in their personal email settings.
   • When setting this up in Gmail:
     • Go to Settings > Accounts and Import > Send mail as > Add another email address.
     • Enter sales@mycompany.com but uncheck the box labelled “Treat as an alias.”
   • This allows employees to send and respond from sales@mycompany.com. Any customer replies will land back in the shared inbox.
2. Setting Up Mailing List:
   • If sales@mycompany.com is guided through a mailing list, create an additional email account, such as followsales@mycompany.com.
   • Include followsales@mycompany.com in the mailing list of sales@mycompany.com. Thus, followsales@mycompany.com will receive emails and replies directed to the sales inbox.

Step 2: Enable Necessary Modules

1. Enable Email Collector Module.
2. Enable Projects Module:
   • Set the option “Allow opportunity management” to On.

Step 3: Create the Email Collector Entry

1. Set Up Email Collector:
   • Create an Email Collector entry using the credentials (either IMAP login or OAuth) of the sales mailbox.
   • Specify Inbox as the input directory.
   • Set an output directory as “Processed emails” (ensure this directory or label is created before the first run).

Step 4: Email Format and Action Configuration

Expect incoming emails to be formatted similar to:

Email Structure:

Organization: A customer or name
Email: acustomeremail@example.com

Set Actions:

• First Action – Load Third Party:
  • Action: “Load the third party by searching name/alias/email/id (create if not found)”
  • Parameters:

name=EXTRACT:BODY:Organization:\s*([\w\-\s]+)$
email=EXTRACT:BODY:Email:\s*([\w\.\-]+@[a-zA-Z\.\-]+)

This action attempts to find a contact in the system’s database (third party) using organization name and email. If no corresponding contact is found, a new record will be created.

Second Action – Create Opportunity:

Action: “Create an opportunity” (no additional parameters required).

This creates a new opportunity in the project management system based on the third party identified or created in the previous step.

Step 5: Processing Workflow

With the above setup:

• Each time an email arrives at sales@mycompany.com, it is also sent to followsales@mycompany.com.
• The Email Collector processes these emails:
  • It extracts the organization name and email.
  • Attempts to find or create a third party based on the extracted data.
  • Creates an opportunity associated with that third party.

Summary

This setup allows the sales team to effectively manage leads and customer relations directly from their shared inbox. Each incoming email is automatically processed to either create or update opportunities in your project management system, ensuring no potential lead is overlooked.

Boundary Value Analysis

Use Cases

The following use cases were tested for some version of SolutionsInc and mail providers:

Use Case	Gmail + SolutionsInc	Errors / Problems
Collect received mails to create a ticket (created from scratch by someone)	OK
Answer the ticket email received by email, this answer appears into the “Email sent”, the collector collect the email sent and link it to the ticket.	OK
Send email from a SolutionsInc ticket. The email must appear into “Email Sent” of the mailer, but the email collector must not collect it from the “Email sent” because it will be a double: When the email is generated and sent from SolutionsInc (so saved into SolutionsInc), and the SMTP platform also save the email into the “Emails sent”, the collector we must not re-import it.	???
Collect and link email to the ticket for email received from the customer answer of our answer (sent from ticket interface).	OK
Collect and link email to the ticket for email received from the customer answer of our answer (sent from the emailer)	OK
Collect and link email to the ticket for email sent from the emailer on a mail that was an answer of a ticket.	OK

Filters

This is a summary of tests conducted with various email services. Depending on your mailbox provider, certain filters may or may not function correctly. If your mail host does not support a specific filter, SolutionsInc may utilize PHP to enable it to work despite this limitation. However, this solution is not always implemented.

Filters / Technology used to filter	Office365	Gmail	SolutionsInc + PHP	Errors / Problems
From		OK	NA
To		OK	NA
Cc		OK	?	Search string not understood – Unknown search criterion: CC
Bcc		OK	?	Search string not understood – Unknown search criterion: BCC
ReplyTo		NA	Filter support with SolutionsInc
Subject		OK	NA
Body		OK	NA
Already read		OK	NA
Unread		OK	NA
Unanswered		OK	NA
Answered		OK	NA
Size (Smaller than)		OK	NA	Search string not understood – Unknown search criterion: SMALLER
Size (Larger than)		OK	NA	Search string not understood – Unknown search criterion: LARGER
Message from a conversation initiated by a first email sent from SolutionsInc		NA	OK
Message from a conversation initiated by a first email NOT sent from SolutionsInc		NA	OK
Message sent from SolutionsInc		NA	OK
Message NOT sent from SolutionsInc		NA	OK	Message not processed but not an error
Is not answer (initial email)		NA	OK
Is an answer of an initial email		NA	OK