I keep on hearing that hosting your own server is a pain. Email costs were always an issue for me. I manage a few e-commerce website. Initially when they were small I used MailChimp. Then moved to AWS SES. Now I moved to my own self hosted postalserver. My mailing lists are not huge ( 100k recipients) and I send daily and I don't have much of a deliverability issue. Depending on the audience my open rates are between 35 and 55%.
What I learned is:
- Make sure your configuration is perfect. From reverse dns to DKIM. Everything should be aligned.
- Have a clean email list. Validate emails ( MX, etc ), double optin, remove hard bounces, respect unsubscribes to the t ( people don't always unsubscribe using the link, but often just send an email to customer service, provide the option fornthe customer service team to unsubscribe people from marketing).
- Use a email warming service like mailreach. It seems weird but it makes a world of difference. It really works.
I spend less than $50/month all in all with a beefy instance on Hetzner Cloud. For the volume I send it would cost 6x this on SES.
What you're describing is exactly what most people would call "a pain". It is very doable if it is your job, but it's definitely much much more work than having a Gmail or Proton Mail account or whatever other provider if it's about your own personal email.
There are many who would like to not be beholden to anyone else for their own email, but when they try it, they quickly find it's far too much work to actually ensure that when they send an email to, say, apply for a job, it will certainly arrive.
> There are many who would like to not be beholden to anyone else for their own email, but when they try it, they quickly find it's far too much work to actually ensure that when they send an email to, say, apply for a job, it will certainly arrive.
I think not knowing is a big problem. For example, I just use docker-mailserver on a VPS with decent IP reputation and SPF set up (but not DKIM/DMARC) and haven't really had any issues with the big providers.
However, if there were issues, I wouldn't know about those myself, unless I had some code that sends emails to those providers and then checks whether they've been received periodically.
On one hand, it would be nice to have confirmation for when a mail has been handled successfully (delivered and sitting in their inbox folder, not tossed aside and not filtered as spam, regardless of whether they'll actually read it), but then again some software will also block remote content (like tracking pixels) over justified privacy concerns and bad actors would also benefit from said functionality.
If you want cheap dedicated servers, you can get even lower prices (though the value may not be as good): SoYouStart (OVH but in North America) has $30 dedicated servers[0], OVH in France has dedicated servers (Intel Atom tier, but still your own whole computer) for $11 a month[1].
I'm amazed at how email server software is innovating and evolving like never before. We have new implementations of SMTP, IMAP, etc. We even have new protocols like JMAP thanks to the Fastmail guys.
But it also feels like fewer and fewer people are running their own mail servers. Everyone seems to just outsource it to Google or Microsoft. I'm also surprised at how many engineers barely understand how email works. (IE: I have to show them what email headers are and the useful info that can ne found there.)
That's because email delivery is such a weird dance. It's getting harder every year to have your mail delivered without being part of those few services since the protocol is abused so much.
To be fair, you can relay through those services to work around the delivery problem. It's still better than nothing since they now only see your outbound mail, but inbound mail processing & storage is still local & self-hosted.
How does that work exactly? Do you set up your server as higher priority MX and the external provider as backup and then configure your server to relay outbound mail through that other service? And all your server has to have is a valid certificate while DKIM/RDNS/SPF get handled by the external service?
Is there anything to watch out for (e.g. w.r.t. bounce mail) in this setup?
Edit: Maybe you don't even have to set up an MX record for the external provider (unless you want to use it as a fallback for incoming mail).
That's exactly how it works, although you don't need an external backup modern email servers will retry for many days before returning a undeliverable notification if a valid MX record exists. With email receiving and delivering email are two separate tasks that don't need to be done by the same service.
For example I use Amazons SES. They have SMTP credentials you plug into your server and you add the DKIM and SPF txt entries to your DNS. The only thing that points to your own server is the MX record. Gmail will re-write emails to be FROM gmail while others may allow you to do similar to SES.
You don't actually need a valid cert or IPv6 to receive email, everyone only cares about the sending side of things.
Amazon will want to know what automated systems you have to deal with bounces and complaints to protect their service but that's about it
Yup bounce emails are routed just like any other email so they go through the relay. This is actually one of the gotcha's when using fetchmail, you can accidentally send out a lot of unintentional bounce emails from an invalid server if you are not careful.
It's just like doing bulk mail through a third party, except now you're sending your regular mail through them too.
MX records are used to figure out where to deliver mail so you would leave those pointed at your own mail server, no need to set up any MX record pointed to the 3rd party.
You would need DKIM/SPF, etc for sender verification pointed at the 3rd party.
I love to throw out a library and implement something myself so I gain a better understanding of what’s going on and maybe even improve on what was there before. But I can’t do that with email. Other servers are going to accept or reject my mail solely based on what my IP is, irrespective of how good my implementation is or is not. I’ve just never found it worth the time to dig into.
I'm a huge fan of Chasquid. The HAProxy protocol support makes it easy to integrate in Kubernetes.
I uses to use this solution until a few months ago when I decided that self hosting my mail server was too much work. To be fair, it's not Chasquid's fault as operations there are basically 0, but more of a spam / email reputation thing.
Still running my own mail server, for myself and friends. No problems with deliverability thus far (including to Google) with SPF and DKIM set correctly.
The killer problem is often source address, and source address range, reputation. Mail sent from a normal residential ISP will have trouble. Addresses allocated by many VPS providers (and some dedicated server providers) will have spotty reputation. Mail from cheap VPS providers simply won't get through (many block SMTP outgoing and state so on sign-up because they don't want the support burden of people complaining about this). Sending mail from cloud providers can be an issue too (I've had mixed results with VMs in Azure sending out alerts).
If you are stuck with such a source reputation issue then you can use a service like MXRoute (several others are available, this one I've not used but it seems popular on a couple of hosting forums I frequent) as a relay. Obviously that is not free, but it is cheaper than moving to a much more expensive VPS provider or changing your home ISP.
Another reputation issue many have when setting up a new project is that many mail servers explicitly distrust new domains. In this case SPF, DKIM, and a good reputation source address won't make much difference. The definition of “new” varies from place to place but where it is an issue you need a domain registered at least some months ago. This can be mitigated somewhat by users explicitly marking messages from new domains as not spam, but that is not something you can rely upon as said users need to see the message to be able to mark it that way.
can't confirm. I'm hosting a online registration software for parent-teacher conferences and hosting all my mailservers on my business internet line.
If you have DKIM,DMARC,SPF and the reverse DNS of the mailserver is set to the correct domain you have no problem. Well.. except for GMX and yahoo which frequently block me for a few hours when multiple parents register at the same time
The security solutions on the mail system make a line that you need to cross, but it doesn’t guarantee a success.
It’s a hard way to guess how good is the implementation on the receiver, like GMX, Libero or other big country providers with “historical” service that don’t care about them too much.
I agree. Self hosted my email for over a decade and as long as you do the recommended SPF, DKIM and DMARC you have basically no problems.
Occasionally I turn up in spam when I email someone I haven't before, but that's usually due to the .ro in my domain and the forced text mode rather than html.
I've found problems only start occuring when you send transactional email. User signups, notifications, etc. Anything really automated.
I think people like to joke on self hosting email because there are a lot of moving parts and it was hard to diagnose why something went wrong. Until the last decade and early 2010s there were no all in one self hosted solutions that made it easy & available for most to do it.
Google puts random things in Spam, including ones marked 'not spam' or from otherwise reputable senders (like Github, despite receiving their emails for a decade). My monthly credit card bills occasionally are dumped in spam.
Its not really proof of much unless one is constantly getting junked
Back when I used GMail, Google would sometimes mark correspondence e-mails from Google staff as spam. Heck, I even had instances of responses to e-mails I had sent to Google staff marked as spam. Spam filtering is complex and clearly Google has (and continue) to tread a very fine line here.
Last week I saw the same behavior sending email from my work's Google Workspace account to my personal Gmail account. The email stays within Google, I'm logged into both accounts on the same computer, does Google think I am I spamming myself...?
It needs a bunch of milters to have basic functionality, such as DKIM, SPF and DMARC. OpenDKIM and OpenDMARC are not well-maintained, reliable or nice to use.
Postfix was designed years ago, but that doesn't do it bad, it is a beast, it can handle 5k mails per second (DKIM signed) with an small tunning in a modern hardware without problem.
Postfix doesn't have DKIM support though. You'd have to use unmaintained OpenDKIM, no Ed25519, bunch of known bugs. It's not good. 5k/s is also really mediocre on modern hardware.
Apparently there's no support for smarthost / relayhost (or I can't find any traces of it in the documentation). Running your own low traffic mail exchanger without it is often not a pleasant experience.
Does anyone know of an SMTP server which is easily usable with Caddy? I find tinkering with certificates quite bothersome, and I won't give up Caddy as I already host a bunch of apps with it.
The hook functionality[1] looks easy to extend. I was looking for something that could trigger workflows for inbound email, anyone have (alternative) tool recommendations or experience?
