Key takeaways
- Mail lands in the inbox when DNS, authentication and reputation all line up, so the records matter as much as the software.
- Publish MX, SPF, DKIM and DMARC for every domain, and add MTA-STS and TLS reporting for encrypted delivery.
- Never run an open relay: require login on submission ports and tie the sender to the login.
- A panel can publish the records and test every configuration before it goes live, which removes the commonest mistakes.
Most trouble with a self hosted mail server is not the software. Postfix and Dovecot are well known. The trouble is the set of small things around them that receiving servers judge you on. This post lists them in the order you should do them.
Start with the DNS records
Every mail domain needs a set of records, and a missing one is the commonest reason for spam folder delivery:
- MX says where mail for the domain goes.
- SPF lists the servers allowed to send for the domain.
- DKIM signs each message, and the receiver checks the signature against a key in DNS.
- DMARC tells receivers what to do when SPF and DKIM fail, and where to send reports.
- MTA-STS and TLS-RPT ask other servers to use encrypted delivery and report failures.
- Autoconfig and autodiscover let mail apps set themselves up.
Doing this by hand for every domain is where errors creep in. RespireCloud publishes all of them into the domain's zone for you. See mail and webmail and DNS.
Authenticate the senders
Open relay is the fastest way to get listed. Submission on ports 587 and 465 should require a login, and the login should match the sender address, so one account cannot send as another. You also want to be able to hold or block a sender that starts sending in bulk, since a taken over mailbox damages the reputation of the whole server address.
Encrypt and keep keys safe
Use TLS for every connection you can. Keep DKIM private keys on the mail server and out of databases and logs, and rotate them in two steps so mail in flight keeps verifying. Certificates should renew without you, which is covered in our post on automatic wildcard SSL.
Check every change before it goes live
A bad mail configuration stops mail until someone notices. Test each change with the tools the software ships (for Postfix, Dovecot and Rspamd these are postfix check, doveconf and rspamadm configtest) and keep the previous files so you can go back. RespireCloud does this on every change and restores the old files on failure.
Watch the queue and the reputation
Learn the queue controls: list, hold, release, retry and delete. Watch for bounces. Check the server address on public blocklists, and give it a reverse DNS name that matches its mail host name. A new address starts with no reputation, so send slowly at first.
Where a panel helps
A panel will not give a new address a good reputation, and nothing can promise inbox delivery. What it removes is the manual work and the typos. If you move mail from another panel, the migrations feature and the move from cPanel guide cover mailboxes. Mail in RespireCloud is built and verified in our test lab but not yet released, so check the feature page for what is there today.
Questions
Why does my self hosted mail go to spam?
The usual causes are missing or wrong SPF, DKIM or DMARC records, no reverse DNS on the server address, a new or listed IP address, or a server that sends spam because an account was taken over. Fix the records first, then check the address reputation.
Is it worth running my own mail server?
It gives you control and keeps mail on servers you own, but it takes care. If you host customers' mail anyway, a panel that automates the setup makes it practical. If you send large marketing volumes, a sending service is usually a better fit.
What ports does a mail server use?
Commonly port 25 between servers, 587 and 465 for users to send, and 993 for IMAP over TLS. Submission should always require a login.