How Email Works

Email is fifty years old and it shows. It was designed for a network of a few hundred machines operated by people who trusted each other. It now carries billions of messages a day across a global network full of spammers, phishers, and state-sponsored attackers. And yet, despite its age, its flaws, and the dozens of products that have tried to replace it, email remains the backbone of internet communication. You can’t create an account on most services without one. You can’t do business without one. It’s the one protocol that connects everyone.

A brief history of electronic mail

Email predates the internet.

The first electronic messages were sent between users on the same machine in the early 1960s. MIT’s Compatible Time-Sharing System (CTSS), operational from 1961, had a MAIL command that let users leave messages for each other in a shared file. It was essentially a digital noticeboard.

The jump to network email came in 1971, when Ray Tomlinson – a programmer at Bolt, Beranek and Newman (BBN), the company building the ARPANET – wrote a program that could send a message from one machine to another over the network. He chose the @ symbol to separate the user name from the host name, giving us the user@host convention that has survived for over fifty years. The first network email, by Tomlinson’s own account, was something like “QWERTYUIOP” – a test message sent to himself between two PDP-10 machines sitting next to each other.

Through the 1970s, email on ARPANET was ad hoc. Different machines used different formats. There was no standard for message structure, addressing, or routing. RFC 733 (1977) made the first attempt at standardising the message format, but the protocol for actually transferring messages between machines remained informal.

That changed in 1982 with SMTP.

SMTP: the protocol that delivers email

The Simple Mail Transfer Protocol (SMTP), defined in RFC 821 by Jon Postel in August 1982 (and later updated by RFC 5321 in 2008), is the protocol that moves email from one server to another. It’s still the protocol used today. When you send an email, your mail client talks SMTP to your mail server, and your mail server talks SMTP to the recipient’s mail server.

SMTP is a text-based protocol. You can literally have an SMTP conversation by typing commands into a terminal. Let’s walk through what happens when Craig at craig@example.com sends an email to Priya at priya@greenbox.com.au.

First, Craig’s mail server needs to find the mail server responsible for greenbox.com.au. It does this by performing a DNS lookup for the MX records (Mail eXchange records) of greenbox.com.au. An MX record specifies the hostname and priority of a domain’s mail server:

greenbox.com.au.  IN  MX  10 mail.greenbox.com.au.
greenbox.com.au.  IN  MX  20 backup-mail.greenbox.com.au.

The lower number means higher priority. Craig’s server will try mail.greenbox.com.au first. If it’s unreachable, it’ll try backup-mail.greenbox.com.au. This is how email achieves a basic form of redundancy – if the primary mail server is down, a backup can accept the message and hold it for later delivery.

Craig’s server resolves mail.greenbox.com.au to an IP address (another DNS lookup, this time for an A or AAAA record), opens a TCP connection to port 25, and the SMTP conversation begins:

S: 220 mail.greenbox.com.au ESMTP ready
C: EHLO mail.example.com
S: 250-mail.greenbox.com.au Hello mail.example.com
S: 250-SIZE 52428800
S: 250-STARTTLS
S: 250 OK
C: STARTTLS
S: 220 Ready to start TLS
   [TLS handshake occurs]
C: EHLO mail.example.com
S: 250-mail.greenbox.com.au Hello mail.example.com
S: 250 OK
C: MAIL FROM:<craig@example.com>
S: 250 OK
C: RCPT TO:<priya@greenbox.com.au>
S: 250 OK
C: DATA
S: 354 Start mail input; end with <CRLF>.<CRLF>
C: From: Craig <craig@example.com>
C: To: Priya <priya@greenbox.com.au>
C: Subject: Sprint review notes
C: Date: Mon, 18 May 2027 09:15:00 +0800
C: Message-ID: <abc123@mail.example.com>
C:
C: Hi Priya,
C:
C: Here are the notes from Friday's sprint review.
C: .
S: 250 OK: message queued as 1A2B3C4D
C: QUIT
S: 221 Bye

Notice several things:

EHLO (Extended Hello) identifies the sending server. The original command was HELO; EHLO is the modern version that signals support for SMTP extensions.

STARTTLS upgrades the connection from plaintext to encrypted. This is opportunistic – if the receiving server supports it, the connection is encrypted. If not, the email is sent in the clear. There’s no requirement that the receiving server support TLS, which is one of email’s many security weaknesses. As of 2024, about 93% of outbound Gmail traffic is encrypted in transit, up from 33% in 2013.

MAIL FROM and RCPT TO form the envelope – the routing information used by the SMTP protocol. These are distinct from the From: and To: headers in the message itself. The envelope is like the address on the outside of a physical envelope. The headers are like the letterhead inside. They don’t have to match, and this mismatch is exploited by virtually every phishing email ever sent.

DATA signals the start of the message content. The message ends with a lone period on a line by itself.

250 OK means the receiving server has accepted responsibility for the message. It’s now in the receiving server’s queue, and the sending server can forget about it. If the receiving server can’t deliver it (the recipient doesn’t exist, their mailbox is full), it will generate a bounce message – a new email sent back to the MAIL FROM address explaining the failure.

This is the entire SMTP exchange. It’s roughly the same conversation that was happening in 1982, with the addition of TLS and a few extensions. The protocol has no authentication of the sender. It has no encryption requirement. It has no mechanism for verifying that craig@example.com is who they claim to be. The receiving server accepts the message based on nothing more than the sending server’s say-so.

This is why email security is so hard.

Why email is unauthenticated by default

SMTP was designed in 1982 for a network of a few hundred machines operated by universities and government research labs. The people using it knew each other. The machines were administered by trustworthy people. Authentication was unnecessary because the community was small enough that bad actors could be dealt with socially.

That assumption collapsed as the internet grew. By the mid-1990s, spam had become a serious problem. By the 2000s, phishing – impersonating a trusted sender to steal credentials or install malware – had become a billion-dollar criminal industry.

The core problem is simple: SMTP doesn’t verify the sender. When a server says MAIL FROM:<ceo@yourcompany.com>, the receiving server has no way to know whether the sending server is authorised to send email on behalf of yourcompany.com. Anyone can claim to be anyone. The protocol trusts you implicitly.

Three technologies – SPF, DKIM, and DMARC – were bolted onto email over the next two decades to address this. They’re imperfect, they’re complex, and they’ve been a massive improvement.

SPF: who’s allowed to send?

Sender Policy Framework (SPF), defined in RFC 7208 (2014, though it existed informally from 2003), lets a domain owner publish a DNS record specifying which mail servers are authorised to send email for their domain.

An SPF record is a DNS TXT record that looks like this:

example.com.  IN  TXT  "v=spf1 ip4:203.0.113.0/24 include:_spf.google.com -all"

This says:

• v=spf1 – this is an SPF record
• ip4:203.0.113.0/24 – servers in this IP range are authorised
• include:_spf.google.com – also allow whatever servers Google’s SPF record authorises (because example.com uses Google Workspace for email)
• -all – reject email from any other server

When a receiving server gets an email claiming to be from example.com, it checks the SPF record in DNS. If the sending server’s IP address matches the SPF record, the email passes. If not, the receiving server can reject it, flag it, or let it through depending on policy.

SPF has limitations. It checks the envelope sender (MAIL FROM), not the header From: that the recipient sees. A phishing email can use a different envelope sender while displaying a spoofed From: header. SPF also breaks when email is forwarded – the forwarding server’s IP won’t be in the original domain’s SPF record. And SPF lookups add latency to email delivery, though typically only a few milliseconds for a DNS query.

DKIM: cryptographic proof of origin

DomainKeys Identified Mail (DKIM), defined in RFC 6376 (2011), takes a different approach: instead of checking the sending server’s IP, it uses cryptographic signatures to prove that the email was authorised by the domain owner and hasn’t been tampered with in transit.

The sending server generates a cryptographic signature over selected headers and the message body, using a private key. The corresponding public key is published in DNS as a TXT record:

selector._domainkey.example.com.  IN  TXT  "v=DKIM1; k=rsa; p=MIGfMA0GCSqG..."

The receiving server retrieves the public key from DNS and uses it to verify the signature. If it’s valid, the email provably came from (or was authorised by) the domain owner, and the signed content hasn’t been modified.

DKIM is more robust than SPF in several ways: it survives forwarding (the signature travels with the message), it signs the header From: (not just the envelope), and it provides integrity – a message modified in transit will fail verification. But it’s more complex to set up, and the signature only covers the signed headers and body. Mailing lists that modify the message (adding footers, changing the Subject line) will break the DKIM signature.

DMARC: tying it all together

Domain-based Message Authentication, Reporting, and Conformance (DMARC), defined in RFC 7489 (2015), builds on SPF and DKIM to provide a policy framework and reporting mechanism.

A DMARC record tells receiving servers what to do when an email fails both SPF and DKIM checks:

_dmarc.example.com.  IN  TXT  "v=DMARC1; p=reject; rua=mailto:dmarc@example.com"

The p=reject policy instructs receiving servers to reject emails that fail authentication. Other options are p=quarantine (deliver to spam) and p=none (do nothing, just report). The rua tag specifies where aggregate reports should be sent – XML-formatted reports that tell the domain owner which servers are sending email claiming to be from their domain, and whether those emails passed or failed authentication.

DMARC adds a crucial concept: alignment. For DMARC to pass, either SPF or DKIM must pass, and the domain used in the passing check must align with the domain in the header From: field. This closes the loophole where SPF passes for the envelope sender but the visible From: header shows a different domain.

Together, SPF + DKIM + DMARC provide a layered authentication system:

Notice that all three technologies rely on DNS for publication. Your domain’s email authentication is only as secure as your DNS infrastructure. If an attacker can modify your DNS records (through registrar compromise, DNS cache poisoning, or BGP hijacking), they can undermine all three.

The spam problem

In the early 1990s, a few hundred unsolicited commercial emails were a nuisance. By 2008, spam accounted for over 90% of all email traffic (Symantec Internet Security Threat Report). That number has since fallen – not because spammers have given up, but because spam filters have become remarkably good at their job.

The economics of spam are depressingly simple. Sending an email costs essentially nothing. If even 0.001% of recipients respond – buy a product, click a link, enter credentials – the spam campaign is profitable. The marginal cost of sending the next million emails is near zero. This means spam will exist as long as email exists, because the incentive structure can’t be changed without changing the protocol.

How spam filters work

Modern spam filters use multiple techniques in combination, because no single technique is sufficient.

Bayesian filtering, popularised by Paul Graham’s 2002 essay “A Plan for Spam”, applies Bayes’ theorem to classify messages. The filter maintains a database of word (or token) probabilities: the probability that a given word appears in spam versus legitimate email. When a new message arrives, the filter computes the combined probability that the message is spam based on the words it contains.

If the word “Viagra” appears in 85% of spam and 0.1% of legitimate email, its presence in a message is strong evidence of spam. If the word “meeting” appears in 2% of spam and 40% of legitimate email, its presence is evidence against spam. The filter combines these probabilities across all words in the message using Bayes’ rule and produces an overall spam probability.

Bayesian filters are trainable – they improve as they see more email. They’re also personal: your filter learns from your email, so it adapts to your specific patterns of legitimate and spam messages.

Reputation systems assign scores to sending IP addresses and domains based on their history. A mail server that’s been sending mostly spam gets a low reputation score, and its future emails are more likely to be filtered. Reputation databases like Spamhaus, Barracuda, and Sender Score maintain real-time blocklists of known spam sources.

This is why IP reputation matters so much for legitimate email senders. If your mail server’s IP address gets onto a blocklist – because a user’s account was compromised and used to send spam, or because you’re on a shared IP with a spammer – your legitimate email will be filtered. Email deliverability is a genuine business concern, and companies like Twilio SendGrid, Mailchimp, and Postmark exist partly because managing IP reputation is hard enough to be worth outsourcing.

Content analysis looks for patterns beyond individual words: suspicious URLs, known phishing page structures, image-based spam (text rendered as an image to evade word-based filters), obfuscation techniques (using Unicode lookalike characters, HTML formatting tricks), and attachment types associated with malware.

Machine learning has largely subsumed these individual techniques. Modern spam filters – certainly the ones run by Google, Microsoft, and other major providers – use neural networks trained on billions of messages. They incorporate content, sender reputation, recipient behaviour (do you usually open emails from this sender?), sending patterns (is this sender suddenly emailing millions of people?), and dozens of other signals. The result is remarkably effective – Gmail estimates that less than 0.1% of spam reaches users’ inboxes.

Email headers: reading the envelope

Every email carries a set of headers that record its journey from sender to recipient. Most email clients hide them by default (you can usually see them via “View Source” or “Show Original”), but they contain a wealth of diagnostic information.

Here’s an abbreviated set of headers for a message:

Return-Path: <craig@example.com>
Received: from mail.greenbox.com.au (mail.greenbox.com.au [203.0.113.50])
    by mx.recipient.com (Postfix) with ESMTPS id 1A2B3C4D
    for <priya@greenbox.com.au>; Mon, 18 May 2027 01:15:02 +0000 (UTC)
Received: from mail.example.com (mail.example.com [198.51.100.10])
    by mail.greenbox.com.au (Postfix) with ESMTPS id 5E6F7A8B
    for <priya@greenbox.com.au>; Mon, 18 May 2027 01:15:01 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; d=example.com; s=selector;
    h=from:to:subject:date; bh=abc123...; b=xyz789...
From: Craig <craig@example.com>
To: Priya <priya@greenbox.com.au>
Subject: Sprint review notes
Date: Mon, 18 May 2027 09:15:00 +0800
Message-ID: <abc123@mail.example.com>
Authentication-Results: mx.recipient.com;
    spf=pass (sender IP is 198.51.100.10) smtp.mailfrom=example.com;
    dkim=pass header.d=example.com;
    dmarc=pass (p=reject) header.from=example.com

Received headers are the most diagnostic. Each mail server that handles the message adds a Received: header at the top. You read them bottom to top – the bottom one was added first (by the sending server), and the top one was added last (by the receiving server). They show you the route the message took, the IP addresses involved, the timestamps at each hop, and the protocols used.

If you’re investigating a phishing email, the Received headers are where you start. The bottom-most Received header shows the true origin of the message. A phishing email might claim to be from ceo@yourcompany.com in the From: header, but the Received headers will show it came from a mail server in a different country with an unrelated IP address.

Authentication-Results shows the outcome of SPF, DKIM, and DMARC checks. This header is added by the receiving server and tells you whether the email passed or failed each authentication mechanism.

Message-ID is a globally unique identifier assigned by the sending mail server. It follows the format <unique-string@hostname>. No two messages should have the same Message-ID. It’s used for threading (the In-Reply-To and References headers reference Message-IDs) and for deduplication.

IMAP vs POP3: accessing your mailbox

SMTP handles delivery – getting the message from the sender’s server to the recipient’s server. But how does the recipient read it? That’s where IMAP and POP3 come in.

POP3 (Post Office Protocol version 3, RFC 1939, 1996) is the older, simpler protocol. The client connects to the server, downloads all new messages, and (by default) deletes them from the server. It’s a one-way transfer: messages live on the client’s device. If you read an email on your laptop, your phone doesn’t know about it. If your laptop’s hard drive dies, the emails are gone.

POP3 made sense in the 1990s, when people accessed email from a single computer and server storage was expensive. It makes much less sense now, when people access email from multiple devices and server storage is cheap.

IMAP (Internet Message Access Protocol, RFC 9051, 2021 – though the widely deployed version is RFC 3501 from 2003) keeps messages on the server. The client synchronises with the server, so changes made on one device (reading, deleting, moving to a folder) are reflected on all devices. IMAP is what enables the multi-device email experience that everyone expects today.

It’s worth noting that many modern email services (Gmail, Outlook.com) don’t use IMAP or POP3 internally at all. They use proprietary APIs. But they offer IMAP access as a compatibility layer for third-party email clients.

The mail servers: Sendmail, Postfix, and the modern landscape

The history of email software is, in large part, the history of Sendmail.

Eric Allman wrote the first version of Sendmail in 1983 at UC Berkeley. It became the default mail transfer agent (MTA) on BSD Unix and, by extension, on most Unix systems connected to the internet. For over a decade, Sendmail handled the majority of the internet’s email traffic.

Sendmail was powerful, flexible, and – by nearly universal consensus – almost impossibly difficult to configure. Its configuration file, sendmail.cf, is written in a cryptic language that Bryan Costales needed over 1,000 pages to explain. Eric Allman himself has described the configuration syntax as something he would do differently if starting over. The running joke in the Unix community was that understanding sendmail.cf was a sign of either genius or madness, and it wasn’t always clear which.

Sendmail’s complexity was also a security liability. Its long history of security vulnerabilities – buffer overflows, privilege escalation, remote code execution – made it a favourite target for attackers. The most famous was the 1988 Morris Worm, one of the first internet worms, which exploited a vulnerability in Sendmail’s debug mode to spread across the ARPANET.

Postfix, written by Wietse Venema and first released in 1998, was designed as a Sendmail replacement that prioritised security and simplicity. Its configuration is human-readable (main.cf is mostly key = value pairs). Its architecture is modular – separate processes handle receiving, queueing, and delivery, each running with minimal privileges. Postfix is the default MTA on most modern Linux distributions and handles a large fraction of the internet’s email.

Exim, written by Philip Hazel at the University of Cambridge in 1995, is popular in the UK and among hosting providers. Its configuration is more expressive than Postfix’s, which is either a feature or a footgun depending on your perspective.

On the receiving side, Dovecot is the dominant IMAP/POP3 server, handling the last-mile delivery from the mail server to the user’s mailbox.

In practice, most organisations don’t run their own mail servers any more. Google Workspace, Microsoft 365, and other hosted email services handle email for the majority of businesses. Running your own mail server in 2027 is a labour of love (or stubbornness) – not because the software is hard to set up, but because maintaining deliverability requires constant attention to IP reputation, SPF/DKIM/DMARC configuration, blocklist monitoring, and the ever-shifting rules of the major inbox providers.

The message format

The format of an email message – distinct from the protocol used to transfer it – is defined in RFC 5322 (2008, updated from RFC 2822, which updated RFC 822 from 1982). It’s simple: headers, a blank line, then the body.

For plain text messages, that’s all you need. But modern email is rarely plain text. MIME (Multipurpose Internet Mail Extensions, RFC 2045 through RFC 2049, 1996) extends the format to support:

• Multiple character encodings (not just ASCII)
• Non-text attachments (files, images, PDFs)
• HTML content
• Multipart messages (an HTML version and a plain text version of the same email)
• Inline images
• Nested multipart structures (an email with both text and HTML versions, plus three attachments)

A typical modern email is a multipart MIME message with at least two parts: a text/plain version (for clients that can’t render HTML) and a text/html version (for everyone else). Attachments are base64-encoded and included as additional MIME parts.

The MIME structure is why email is so often used as an attack vector. An HTML email can contain JavaScript (usually blocked by email clients, but not always), invisible tracking pixels, deceptive links that display one URL but point to another, and embedded content that triggers vulnerabilities in rendering engines. The attack surface is vast because the format is so permissive.

Why email is still the backbone of the internet

Every few years, someone predicts the death of email. Slack will replace it. Teams will replace it. Discord will replace it. And every few years, email persists.

The reason is federation. Email is an open, federated protocol. Anyone can run a mail server. Anyone can send email to anyone else. You don’t need to sign up for the same service. You don’t need to be on the same network. A Gmail user can email a Fastmail user can email a self-hosted Postfix user can email an enterprise Exchange user. Nobody owns email.

Every proprietary communication platform is a walled garden. You can’t send a Slack message to someone on Teams. You can’t send a Discord message to someone on Signal. But you can send an email to anyone with an email address. That universality is email’s superpower.

Email is also the identity layer of the internet. When you create an account on almost any service, you verify it with an email address. Password resets go to email. Two-factor authentication codes go to email (or to an authenticator app that was set up with an email address). Email is the root of trust for online identity in a way that no other protocol has achieved.

The irony is that email is terrible at almost everything it’s used for. It’s a poor collaboration tool (long threads are unreadable). It’s a poor task management system (emails get buried). It’s a poor real-time communication channel (delivery is best-effort, not instant). It’s a poor identity verification system (email addresses can be spoofed). It’s a poor file sharing mechanism (attachments have size limits and no version control).

And yet nothing has replaced it, because nothing else is universal, federated, and open. Email’s mediocrity at everything is exactly its strength – it’s good enough at everything, and it works with everyone.

Reading email headers: a practical guide

If you ever need to diagnose an email problem – a message that isn’t arriving, a legitimate email landing in spam, a suspected phishing attempt – headers are your diagnostic tool. Here’s what to look for:

1. Read Received headers bottom to top. The bottom one is the origin. Each subsequent one shows the next hop. Delays between hops indicate where the message was held up.

2. Check Authentication-Results. SPF pass? DKIM pass? DMARC pass? If any fail, that’s your problem. If SPF fails, the sending server’s IP isn’t in the domain’s SPF record. If DKIM fails, the message was modified in transit or the signature is wrong. If DMARC fails, the domain alignment is broken.

3. Look at the sending IP. The IP in the bottom-most Received header (after “from”) is the true origin. Check it against blocklists using tools like MXToolbox. If it’s listed, that’s why emails are being rejected.

4. Check X-Spam-Status or similar. Many mail servers add headers indicating the spam score. SpamAssassin adds X-Spam-Status with a numerical score and a list of the rules that triggered. A score above 5 typically means the message was flagged as spam.

5. Compare Return-Path with From. If the Return-Path (envelope sender) doesn’t match the From header, that’s not necessarily malicious (mailing lists, forwarding), but it’s worth noting when investigating suspicious emails.

The future (probably still email)

Email is fifty years old and counting. It’s been patched, extended, bolted onto, and complained about for decades. SMTP is still the core protocol. DNS still handles routing via MX records. The From header still can’t be trusted without SPF, DKIM, and DMARC, and many domains still don’t configure them properly.

The current trajectory is toward more authentication (Google and Yahoo began requiring DMARC alignment for bulk senders in 2024), more encryption (TLS everywhere, with MTA-STS making it enforceable rather than opportunistic), and more centralisation (a shrinking number of providers handling an increasing share of email, making their spam filtering decisions effectively law).

The protocol that Ray Tomlinson hacked together in 1971 – user@host, send a message, hope it arrives – is still the foundation. Everything else is just trying to make it safe to use on a network that’s nothing like the one it was designed for.

It’s the best kind of engineering legacy: something that works despite everything working against it, used by billions, understood by few, and replaced by nothing.