If you’ve followed my blog then you know I’m a fan of automation and the programming model Azure Functions enable.
Some of my personal automation is built around emails, and SendGrid’s Inbound Parse feature is a real enabler when it comes to automating emails.
The fact that I can host the automation for pennies a month using a SaaS service that I don’t need to update/patch is just the cherry on top.
So far, I have used Inbound Parse to:
- Send myself bug reports via email and have them created as tickets in Azure DevOps (using email-bug-tracker)
- Send and receive emails from my custom domain without having to pay for a mail package (using SendGrid and email-relay)
- Post messages to chat systems when certain (newsletter) emails are received (partially open sourced here, to be fully open sourced in the future)
Overall, I have been really happy with all of these systems, except for the fact that I have to set each up on a custom (sub)domain.
The reason for that is that SendGrid only allows one webhook per domain so I ended up having to host each webhooks on a separate domain (
This makes sense from their perspective because they do retries for up to 72 hours in case of failures and with multiple webhooks guaranteeing exactly once delivery is impossible to achieve.
I recently added two more services that deal with emails and instead of having to setup more subdomains and SendGrid records I wanted all of my functions to run on a single domain.
Because each system really only needs access to 4-5 parts of the emails (from, to, subject, body and maybe attachments) I decided to build an abstraction that:
- allows me to run all my webhooks through a single domain
- simplifies the format each webhook must process
- simplifies sending email using the
Reply-Tofeature (which I wasn’t aware of previously)
Conceptually this system needed to process the incoming emails, determine which webhooks to call (filtering) and invoke the webhooks (forwarding) while also taking care of retries.
Of course I built it as another Azure function that I open sourced on Github as well.
Using a rule engine (much like a regular inbox) you can decide which emails should be processed by which action.
Example action processing of the fanout system
In the example above the fanout system first determines which targets to call (based on the rules & filters) and then executes each action.
The Archive action writes emails into a storage account (as sort of a poor mans email inbox backup).
The second action forwards newsletters to another webhook which then posts them to a matrix room.
And the third action forwards emails to a private inbox.
Of course, these are just examples. I implemented various actions that can be configured & customized.
Rules, filters & actions
In the system a rule is defined as zero or more filters and one or more actions.
With no filter, each email is simply sent to the respective actions whereas using filters (similar to mailbox filters) allows forwarding certain emails to specific actions. Read more on filters on Github.
Actions then allow to perform specific tasks.
So far, I have built 4 actions:
- Archive - store emails in blob storage (as a backup)
- Forward - forward the received request body as is (to allow integration with my various other systems that rely on Inbound Parse)
- Webhook - provides a simplified format of the email (json with from, to, subject, body & attachments properties)
- Email - Allow receiving emails from the domain in a private inbox
The Webhook action is especially helpful because targets now no longer need to parse the (complicated) Inbound Parse format but can instead rely on a simplified model that provides them with all the essential information of the email. The Webhook action even allows minor modifications (e.g. drop attachments & body to only forward the subject/sender to the webhook).
The actions are documented on github as well.
Retries & delivery guarantees
Because each target may fail individually, I needed to add support for retries and keep track of the success status.
As mentioned earlier exactly once delivery is not really possible in a distributed system so I opted for at least once delivery (accepting that webhooks might be triggered multiple times).
I also decided to go the simplest route possible for retries: reusing the retry mechanism that SendGrid already offers.
For each received email I evaluate all the configured rules and determine which targets should receive a message based on their respective filters.
Each target is then called in parallel and marked as successful/failed in a status table. If > 0 actions failed I respond with
(400) Bad Request to SendGrid which triggers their retry mechanism.
On the next SendGrid retry the fanout system only re-executes the actions that weren’t successful before (based on the status table). This behaviour continues until either all actions succeed (in which case
(200) OK is sent to SendGrid and they stop retrying) or SendGrid stops retrying after 72 hours at which point the email is simply not delivered to all targets.
In theory each target should receive the notification exactly once as I mark each action as successful as soon as it completes. Only once all actions are processed will SendGrid receive the overall success/failure notification. Future retries should thus have the previous success/failure state available in the status table and react accordingly.
In practice I have received notifications exactly once when all targets succeeded on first try and when one or more targets failed, I have received notifications mostly once and rarely twice (so far I had two duplicate notifications across ~100 notifications).
Personally, I am okay with rare duplicate messages as I’d rather have duplicate than lost messages.
From vs. Reply-To
Interesting sidenote: I didn’t know about this feature of the email spec until I accidently hit reply on a newsletter recently. To my surprise the
To field was filled with a different email than the newsletter was sent from (firstname.lastname@example.org vs. email@example.com).
Turns out an email sender can set the
Reply-To header to another address and when a recipient hits reply the alternate address is used as the target. Neat!
Based on this new knowledge I decided to add the
With my email-relay it is possible to receive emails from a custom domain in a private inbox (Gmail, Outlook, ..) and to directly reply to them from the inbox but have the email be sent from the domain (without needing to purchase a mail package).
The reply feature was a bit clunky to use because you basically responded to your own domain (with the original sender address in the subject), the email-relay would then use Inbound parse to detect that
A) It’s you (the owner of the domain) sending from your registered private email B) It is a reply to an existing email C) the address the email was sent to should be used as the sender D) The actual recipient is encoded in the subject
and would then parse the recipient address from the subject and send a new email (from the domain, to the recipient, stripping out your private email address).
Since I barely used the feature (and if I did use it it just felt strange to have the actual target email in the subject) and armed with my new knowledge about the
Reply-To feature I decided to build a different type of action directly into the fanout system that is much simpler:
When someone sends me an email to my domain his email is still repackaged and sent from my domain to my private email but now with the original sender in the
When I now hit reply from my private inbox the original sender is injected in the to field, allowing me to reply directly to the sender. More details here.
This feels much more natural (and less complicated).
Just beware that the email is now actually sent from the private inbox instead of the domain - with the email relay your private email was never exposed as the actual email was always sent from the domain (using SendGrid).
However, my personal email is all over my git commits anyway so there really isn’t any necessity to hide it when someone sends me an email to the domain.
Still, some people prefer sending emails from a custom domain - in which case it is still possible to hook-up the email relay behind the email fanout (using Forward action) - or purchase a proper mail package at some domain host. ;)
For me personally I now have a solution that
- handles all my webhooks relying on emails via a single domain (no more need for multiple subdomains)
- has a rule engine to easily filter emails to various webhooks (see my example on Github)
- Supports a simpler Webhook format (so other webhooks don’t need to parse the SendGrid Inbound Parse format)
- Simplifies my email domain handling
You can find the source on Github.