Stop Forwarding and Start POSTing

Feb. 25, 2022

Background

Most of our data contribution and parsing clients send emails to a dropbox (e.g bwic@empirasign.com) using a rule-based email forwarding tool. But there is a better way.

Stop Forwarding and Start POSTing

Below are links to example scripts that utilize a new API endpoint /submit-email/ to programmatically send emails to us instead of forwarding to a role-based email address. The benefits of this methodology are manifold. For example, forwarding an email incurs a doubling of storage and bandwidth costs. By default, each forwarded email is saved in your Sent folder (i.e. stored on disk). Email providers have limits on the number of emails sent and received and on overall bandwidth use. Published limits for a few of them are linked below. By transmitting via a script you deploy, you'll have greater precision and control over which items to transmit and how to log such transmissions. Our API endpoint also logs the receipt of such transmissions for fail-safe record keeping.

Example Scripts

Data Contribution via IMAP
gist.github.com/empirasign/6f3feab4730c94be747723f0b56f6502

Data Contribution via Outlook
gist.github.com/empirasign/5aeb506978b8689f053ed21e4417c30d

Save the Fibers

The /submit-email/ endpoint now accepts gzip-encoded payloads. If using the requests library, it's a small change (see below) with potentially large bandwidth savings. In our experience, gzipping email reduces payload size by about 50%.

Code Snippet

# uncompressed API call
resp = requests.post(URL, json=post_data, proxies=_proxies_dict(PROXY_SERVER))
# https://gist.github.com/empirasign/6f3feab4730c94be747723f0b56f6502#file-imap_daq-py-L185

# compressed API call
resp = requests.post(URL,
                     data=zlib.compress(json.dumps(post_data).encode("utf-8"))
                     proxies=_proxies_dict(PROXY_SERVER)
                     headers={"Content-Encoding": "gzip"})

Link to full docs: www.empirasign.com/api-docs/

Coming Soon

The above Outlook script interacts directly with an Outlook client. We'll soon be releasing example scripts that interact directly with Exchange Server. Support for Exchange Web Services and the new Microsoft Graph API will be provided. Interacting directly with Exchange Server allows for easier handling of multiple contributor inboxes as well as code portability. Your data contribution scripts can run on any box and OS you chose. They don't need to run on the client machine or the Exchange Server machine. This is already the case if your back-end email services are IMAP-based.

Developer Resources

We have posted a Developer Resources page. This page is the best place to start when looking for help on anything technical or API-related.
www.empirasign.com/developer-resources/

Email Service Provider Bandwidth Limits

Office 365 Exchange Bandwidth Limits
docs.microsoft.com/en-us/office365/servicedescriptions/exchange-online-service-description/exchange-online-limits

Google Workspace Limits
2.5GB per day for IMAP. That's around 100K messages per day for corps (maybe 25K for Structured Products)
support.google.com/a/answer/1071518?hl=en

Fastmail Professional Limits
80K messages per day
www.fastmail.help/hc/en-us/articles/1500000277382-Account-limits#sending/receiving


1