Amazon S3 turned twenty in March 2026. It stores more than 500 trillion objects, answers over 200 million requests a second, and nearly everything else AWS sells is built on top of it. The API is small enough to learn in an afternoon; the machine behind it is one of the largest distributed systems ever operated, and its internals are unusually well documented if you know where to look.
Where S3 came from
Amazon launched the Simple Storage Service on 14 March 2006, before EC2 existed. The pitch was almost embarrassingly modest: put bytes in over HTTP, get bytes back over HTTP, pay 15 cents per gigabyte per month. No provisioning, no capacity planning, no RAID controller firmware, no purchase order for a storage array that would arrive in eight weeks and be full in eighteen months. In 2006 that was a strange idea. Storage was hardware you bought; Amazon proposed storage as a utility you called.
The original API had a handful of operations: PUT an object, GET an object, DELETE it, LIST a bucket. Twenty years later those four verbs still carry the overwhelming majority of traffic, which tells you something about how well the abstraction was chosen. By 2012 S3 held 1.3 trillion objects. By its twentieth birthday it held more than 500 trillion, spread across hundreds of exabytes in 39 regions and 123 availability zones, serving over a quadrillion requests a year. Nobody has ever migrated off it at that scale because there is nowhere to migrate to.
The service’s history is mostly a history of things added around that stable core: storage classes (2008 onwards), versioning (2010), lifecycle rules (2012), cross-region replication (2015), the strong-consistency rebuild (2020), managed Iceberg tables (2024), and a run of AI-era features through 2025 and 2026 (vectors, metadata tables, file access, annotations). The core object model has not changed. That is the place to start.
There are no folders
An S3 bucket is a namespace. An object is a key, a value, and some metadata. The key is a string of up to 1,024 bytes; the value is a blob of up to 5 TB; the metadata is a small bag of system and user-defined headers. That is the whole model.
The thing everyone learns eventually, usually via a painful listing operation, is that the namespace is flat. There are no directories. When the console shows you a folder called invoices/2027/, it is doing string manipulation on keys that happen to contain slashes. invoices/2027/0042.pdf is one key, stored once, in a flat index. “Renaming a folder” means copying every object under one prefix to another prefix and deleting the originals, which is why it takes forever on a big bucket. The delimiter-and-prefix parameters on ListObjectsV2 exist to let clients fake a hierarchy, and that is all they do. (Directory buckets, which arrived with S3 Express One Zone in 2023, are the exception: they have a genuine hierarchical namespace. More on those later.)
Objects are immutable. You cannot update byte range 4096 to 8191 of an object; you write a whole new object under the same key, and the old one either disappears or becomes a noncurrent version if versioning is on. Everything else depends on this constraint. Immutability is what makes it feasible to erasure-code an object across a dozen machines, cache it at the edge, replicate it across an ocean, and still reason about what “the object” is. Almost every scaling property S3 has flows from refusing to support in-place update. (Express One Zone now supports appending to an object, and it is telling that the feature took seventeen years to arrive and only exists in the single-zone class.)
Bucket names were globally unique across all AWS customers for twenty years, which produced a squatters’ market in good names. In 2026 AWS began rolling out account-scoped namespaces for new general purpose buckets, so your backups bucket no longer collides with every other company’s. It is a small change that removed one of the oldest annoyances in the service.
Control plane, data plane
S3 is not one program. Public engineering talks put it at more than 300 microservices, organised into a few broad tiers, and the tier split follows a distinction worth internalising for every AWS service: the control plane versus the data plane.
The control plane handles the rare, heavyweight operations: creating buckets, attaching policies, configuring replication or lifecycle or notifications. These paths are allowed to be slower and are deliberately kept away from the machinery that serves objects. The data plane handles GET, PUT, LIST, DELETE, billions of times a minute, and it is engineered so that nothing on the control-plane side can take it down. When S3 has a bad day, the question “control plane or data plane?” is the first triage step: bucket creation failing while GETs flow normally is a very different incident from the reverse.
A GET traverses roughly this path. DNS resolves the bucket’s endpoint to a front-end fleet in the region. A request-routing layer authenticates the SigV4 signature, checks the request against IAM policies, bucket policies, and Block Public Access settings, then consults the indexing subsystem: a massive partitioned key-value store that maps bucket-plus-key to the physical locations of the object’s data. The storage fleet, tens of millions of hard drives spread across the region’s availability zones, serves the actual bytes, which are reassembled and streamed back.
One of the more interesting published insights, from Andy Warfield’s engineering write-ups, is that S3’s scale is what makes individual performance good rather than what threatens it. Any single customer’s workload is bursty: idle for hours, then a thousand requests a second. Across millions of customers the bursts decorrelate, so the fleet runs at a smooth aggregate utilisation while any individual burst is absorbed by capacity that someone else is not using that second. Your workload gets spread across a slice of tens of millions of spindles, far more parallelism than you could ever buy for yourself. “Heat management”, spreading hot data so no drive or host becomes a bottleneck, is one of the central ongoing engineering problems, and it works better the bigger the fleet gets.
Eleven nines is arithmetic, then culture
S3 is designed for 99.999999999% annual durability, the famous eleven nines. The arithmetic version of the claim: store 10 million objects and you should expect to lose one, on average, every 10,000 years. It is worth understanding where a number like that can possibly come from, because nobody has run S3 for 10,000 years to check.
The mechanism is erasure coding. Rather than storing three full copies of an object, S3 splits it into shards using a scheme in the Reed-Solomon family: from an object it computes n shards such that any k of them suffice to reconstruct the data. The shards are placed in different failure domains, on different drives, in different racks, in different availability zones, so no single drive failure, host failure, or building-level event touches more than one shard’s worth of redundancy. Erasure coding beats plain replication on both axes at once: it costs less than storing full copies (the overhead is n/k, not 3x) and it survives more simultaneous failures for the same overhead.
The durability number then falls out of a race. Hard drives fail constantly at fleet scale: with tens of millions of drives, multiple drives are dying somewhere in the fleet at any given moment, and that is a normal Tuesday, not an incident. Each failure degrades some set of objects from n surviving shards towards k. Background repair processes notice, reconstruct the missing shards from the survivors, and write them to fresh drives. Durability is the probability that failures never win the race, that no object drops below k shards before repair catches up. You can model that: drive failure rates are measured, repair bandwidth is provisioned, and eleven nines is what the model says when repair capacity comfortably outruns the failure rate. The engineering commitment is keeping the model honest, which means alarming on repair backlog, not just on data loss.
Hardware is the easy part of the threat model, though. The published S3 durability material is refreshingly blunt that the bigger risks are software bugs and operator error, and the defences there are cultural as much as technical. Checksums travel with data everywhere: computed at the client if you ask for it, verified at the front door, stored with every shard, and re-verified continuously by background auditors that scrub the fleet looking for silent corruption. Changes that touch durability-sensitive code go through dedicated durability reviews. And the delete path gets as much paranoia as the write path, because at S3’s scale the most plausible way to lose customer data is a bug that deletes the wrong thing, not a disk that dies.
That last point explains a design choice worth noticing. A tempting shortcut for deletion in an encrypted store is crypto-shredding: encrypt every object under its own key, and “delete” by discarding the key, leaving unreachable ciphertext on disk. S3 does not work that way. A delete removes the index entry and then a careful background pipeline reclaims the physical shards, with the same checking and auditing culture applied to reclamation as to repair. Crypto-shredding concentrates all your durability and deletion guarantees in the key store, and S3’s designers chose not to hang that much on one subsystem. If you want provable cryptographic destruction, you layer it yourself with SSE-KMS and key deletion; the storage engine underneath does real deletion, deliberately slowly and deliberately carefully.
ShardStore: the storage node you can read
You do not have to take the storage layer on faith, because S3’s engineers published it. The SOSP 2021 best-paper winner, “Using Lightweight Formal Methods to Validate a Key-Value Storage Node in Amazon S3”, describes ShardStore, the software that runs on each storage host and durably holds shards.
ShardStore is about 40,000 lines of Rust. Structurally it is a log-structured merge tree with the shard data held outside the tree, which keeps write amplification down: the index is compacted and rewritten, the bulk data is not. If you have read about how databases arrange bytes on disk and survive crashes, ShardStore will feel familiar, because it faces the same problems: sequential writes are cheap, random writes are dear, and a crash can land between any two of them.
The interesting part is the word “validate” in the title. The team wrote an executable reference model, a few hundred lines of straightforward Rust that says what a storage node should do, and then used property-based testing to hammer the real implementation against the model across randomised operation sequences, crash points, and concurrent interleavings. Different correctness properties got different tools: crash consistency checked one way, concurrency another. The approach caught 16 bugs before they reached production, including subtle crash-consistency issues of exactly the kind that turn into data loss at fleet scale, and, because the checks are ordinary tests that run in CI, the validation keeps working as engineers keep changing the code. It is the most practical published example of formal methods in a production storage system, and it is a genuinely readable paper.
The day S3 became strongly consistent
For its first fourteen years, S3 was eventually consistent, and a generation of engineers learned distributed systems the hard way because of it. Overwrite an object and a subsequent GET might return the old version. Delete an object and a LIST might still show it. New-object PUTs got read-after-write consistency in most regions from 2015 or so, with caveats sharp enough to cut yourself on. Whole categories of tooling existed purely to paper over this: EMRFS consistent view, Netflix’s S3mper, Hadoop’s S3Guard, all of them bolting a consistent metadata store (usually DynamoDB) alongside S3 to remember what should be there.
In December 2020 AWS switched the whole service, every bucket, every region, to strong read-after-write consistency, for free, with no performance trade-off. A GET, LIST, or HEAD after a successful PUT or DELETE now reflects that write, full stop. The engineering story, sketched in Werner Vogels’ “Diving Deep on S3 Consistency”, is that the metadata subsystem’s caches were the source of staleness, so the team built a cache-coherence protocol around them, introduced a witness component that tracks in-flight writes so a read can always detect whether its cached view is current, and model-checked the protocol before trusting it at S3 scale. Retrofitting coherence onto a live system holding hundreds of trillions of objects, without a maintenance window, remains one of the great unglamorous achievements in the industry.
Strong consistency became the foundation for something bigger. In August 2024 S3 gained put-if-absent (an If-None-Match: * condition on PUT), and in November 2024, compare-and-swap: a PUT with If-Match on an ETag succeeds only if the object is unchanged since you read it. Conditional copy followed in October 2025, and buckets can now enforce conditional writes so uncoordinated writers cannot clobber each other. This sounds small and is anything but. Compare-and-swap on an object store means S3 can be the coordination point for distributed systems: leader election with nothing but a bucket, single-writer logs, and, most consequentially, table-format commit protocols. Apache Iceberg writers can commit metadata straight to S3 with no locking service in the middle. A decade of DynamoDB-shaped scaffolding around S3 has been deleted since.
How S3 scales to your traffic
S3’s index is partitioned by key range, and the request-rate numbers that matter are per partition: at least 3,500 PUT/COPY/POST/DELETE and 5,500 GET/HEAD requests per second each. A fresh prefix starts on some partition; sustain load against it and S3 splits the partition automatically, again and again, so aggregate throughput scales with how widely your keys spread. There is no ceiling on prefixes, so there is no ceiling on the bucket: spread reads across ten prefixes and 55,000 GETs a second is routine.
The catch is the warm-up. Partition splits happen in response to sustained load, over minutes to tens of minutes, and while S3 is scaling you can receive 503 SlowDown responses. This is documented, expected behaviour, and it pages someone anyway, usually the first time a batch job goes from zero to 20,000 requests a second against keys that all start with 2027-09-22/. Date-first key schemes are the classic own goal: every write lands on the rightmost, newest partition, so the parallelism you paid for never engages. Put the high-cardinality component first (hash prefix, customer ID, shard number) and the load spreads by construction. Since a 2018 re-architecture you no longer need the old ritual of random hex prefixes for steady-state traffic, but bursty workloads still need either gradual ramp-up or keys that spread from the first byte, and every client should retry 503s with exponential backoff because the SDKs’ retry policies exist precisely for this.
LIST deserves its own warning. Listing returns at most 1,000 keys a page and does not scale like GET. Anything that walks a large bucket with ListObjectsV2 as its inner loop is doing it wrong; that is what Inventory and S3 Metadata are for.
Storage classes: one API, many prices
Every object lives in a storage class, and the classes are the same API at different points on a curve that trades the price of the shelf against the price of touching what is on it.
S3 Standard is the default: three-plus availability zones, no retrieval fee, no minimum duration, $0.023/GB-month in us-east-1 for the first 50 TB. Standard-IA stores the same way for $0.0125 but charges $0.01/GB to read and bills a 30-day minimum and a 128 KB minimum object size. One Zone-IA shaves the price further by keeping data in a single availability zone, meaning a zone loss can destroy it; it is for data you can regenerate.
Intelligent-Tiering watches access patterns per object and moves objects between tiers automatically: Frequent Access, then Infrequent Access after 30 days untouched, then Archive Instant Access after 90, all three with Standard-class latency and no retrieval fees. Two deeper opt-in tiers (Archive Access and Deep Archive Access) trade latency for Glacier-level prices. The cost of the automation is a monitoring fee of $0.0025 per 1,000 objects per month; objects under 128 KB are neither monitored nor charged for monitoring. For any dataset whose access pattern you cannot confidently predict, Intelligent-Tiering is the correct default, because it converts a forecasting problem into a small flat fee.
The archive tiers are the descendants of Glacier, folded into S3 proper. Glacier Instant Retrieval ($0.004/GB-month) keeps millisecond access with a $0.03/GB retrieval fee and a 90-day minimum: archive economics for data you still occasionally need right now, like old medical images. Glacier Flexible Retrieval ($0.0036) makes you restore before reading: expedited in minutes, standard in 3 to 5 hours, bulk in 5 to 12 hours, with bulk retrievals free. Glacier Deep Archive ($0.00099, about a dollar per terabyte-month) is the bottom of the curve: 12-hour standard restores, 180-day minimum, tape economics without the tape robot.
S3 Express One Zone goes the other direction: a high-performance class in a single availability zone you choose, so compute can sit next to it. It uses directory buckets, which have a real hierarchical namespace, a session-based authentication model, and scale to 2 million GETs and 200,000 PUTs a second per bucket, with single-digit-millisecond first-byte latency, roughly ten times faster than Standard. It also supports appending to objects. The April 2025 repricing cut storage 31% and GETs 85% (storage now about $0.11/GB-month), which moved it from curiosity to a serious tier for ML training data, log analytics, and anything else that hammers small objects.
The old Reduced Redundancy class is deprecated, and the standalone Glacier service with its “vault” API is a legacy you should not build on. Eight or so live classes remain, and the honest summary is: Standard for hot, Intelligent-Tiering for unknown, Glacier tiers for cold with a calculator in hand, Express One Zone for fast and local.
The data-lake turn: Tables, Metadata, Vectors
The most significant shift in S3’s recent life is that it now manages the structure of analytics data as well as storing it.
S3 Tables, launched at re:Invent 2024 and now in 30-plus regions, is managed Apache Iceberg. A table bucket holds Iceberg tables as first-class resources: S3 runs the compaction, snapshot expiry, and unreferenced-file cleanup that every self-managed Iceberg deployment ends up staffing, and claims materially better query throughput than Iceberg on plain buckets because the maintenance actually happens. Through 2026 Tables grew intelligent-tiering for table data and cross-region, cross-account replication that keeps Iceberg replicas consistent without hand-rolled sync jobs, and picked up Iceberg v3 (deletion vectors, row-level lineage). If your lakehouse strategy in 2024 was “Parquet files, Iceberg metadata, a catalogue, and three Spark maintenance jobs”, most of that is now a bucket type.
S3 Metadata answers the oldest operational question in the service, “what is actually in this bucket?”, with managed Iceberg tables about your objects. A journal table records every change (uploads, deletes, lifecycle transitions) in near real time; an optional live inventory table maintains a current, queryable view of every object and version. You query both with Athena or any Iceberg-capable engine. Since mid-2025 it backfills existing objects rather than only tracking new ones. This obsoletes a whole genre of homegrown systems that fired Lambda functions on ObjectCreated events to keep a DynamoDB table of bucket contents.
S3 Vectors, in preview from July 2025, makes vector embeddings a native type: vector buckets hold up to 10,000 vector indexes, each holding tens of millions of vectors with attached metadata, queryable by similarity through dedicated APIs. The point is economics rather than raw performance: priced like storage instead of like an always-on database cluster, it claims up to 90% cost reduction against conventional vector databases for large, warm-rather-than-hot RAG corpora, and it plugs into Bedrock Knowledge Bases and OpenSearch for the latency-sensitive slice. S3 Annotations (2026) rounds this out by letting you attach mutable, searchable context (classifications, summaries, model outputs) to immutable objects without a side database.
Add the 2026 account-scoped bucket names and the SSE-C lockdown, and the pattern of the last two years is clear: S3 is absorbing the systems people built around S3.
When S3 pretends to be a filesystem
Three official bridges now span the gap between object semantics and file semantics, and knowing which is which saves real pain.
Mountpoint for S3 (2023) is a FUSE client that mounts a bucket as a local filesystem, tuned for high-throughput sequential reads and writes. It is deliberately not POSIX-complete: no file locking, no partial in-place writes, no symlinks. It is the right tool for pointing existing read-heavy tools (training jobs, genomics pipelines, render farms) at a bucket without an S3 SDK, and it supports directory buckets for the Express One Zone latency profile.
S3 Files (April 2026) is the bigger swing: genuine file-system access to S3 data over NFS v4.1/4.2, with file locking, POSIX permissions, and read-after-write semantics, built with EFS technology but with S3 remaining the source of truth. The file system materialises data on demand and writes changes back to the bucket, so the same bytes are simultaneously objects to your data pipeline and files to your legacy application, with no copy-out-copy-back cycle. It launched generally available in 34 regions. For the decades-old pattern of “sync the bucket to an NFS share so the old system can read it”, this is the retirement notice.
Static website hosting is the venerable third bridge: a bucket can serve its objects as a website with index and error documents. The website endpoint speaks only HTTP, so in practice every serious deployment fronts it with CloudFront for TLS and caching; at this point the feature is mostly a historical stepping stone to that pattern.
The machinery around objects
A cluster of features turns the bare object model into something operable, and they interlock more than the documentation lets on.
Versioning keeps every overwrite as a noncurrent version and turns DELETE into the insertion of a delete marker; nothing is destroyed until you delete a specific version ID. It is the undo button for the failure mode eleven nines does not cover, your own code deleting the wrong thing, and it is a prerequisite for replication and Object Lock. Its cost is silent accumulation: every noncurrent version bills at full storage rates until a lifecycle rule expires it.
Lifecycle rules are the janitorial layer: transition objects between classes on age, expire them, expire noncurrent versions, clean up expired delete markers, and, in the rule every bucket should have, abort incomplete multipart uploads after seven days, because abandoned parts bill invisibly forever otherwise.
Replication copies objects to another bucket, same region (SRR) or cross-region (CRR), for DR, latency locality, or account isolation. Both ends need versioning. It applies to new objects only unless you run Batch Replication for the backlog, and Replication Time Control turns best-effort into an SLA: 99.99% of objects replicated within 15 minutes, with metrics you can alarm on.
Multipart upload splits large objects into up to 10,000 parts of 5 MB to 5 GB, uploaded in parallel and completed atomically; it is mandatory above the 5 GB single-PUT limit and sensible far below it for retryability. One trap: a multipart object’s ETag is a hash of part hashes, so it is no longer the MD5 of the content, and checksum-comparison tooling that assumes otherwise breaks.
Presigned URLs delegate a single operation on a single key to whoever holds the URL, for up to seven days with SigV4. They inherit the signer’s permissions as evaluated at request time; a URL signed by a role whose session has expired is a dead URL, which is the most common surprise in a support queue.
Event notifications fire on object changes to Lambda, SQS, or SNS via per-bucket configuration, or to EventBridge, which is the mode to prefer: every event type, richer filtering, and no clashes over notification configuration between teams. Delivery is at-least-once and unordered, so consumers must be idempotent, and design reviews should treat “exactly one event, in order” as the bug it is.
Object Lock provides write-once-read-many retention on versioned buckets: governance mode (privileged users can override) or compliance mode (nobody can, not even root, until the retention date passes), plus legal holds. It exists for regulators, and it doubles as the strongest ransomware backstop in the service; compliance mode is also the easiest way to make an irreversible mistake with a timestamp typo, so automate the retention arithmetic.
Batch Operations runs one operation (copy, tag, restore, invoke a Lambda, set retention, replicate) across billions of objects from a manifest or an Inventory report, with retries, progress tracking, and a completion report. It is how you re-encrypt a decade-old bucket in place. Transfer Acceleration ingests uploads at CloudFront edge locations to ride AWS’s backbone across long distances, for an extra per-GB fee that is worth it roughly in proportion to the ocean between your users and your bucket. Requester pays flips data-transfer and request charges to the caller, which is what makes multi-petabyte open datasets publishable without a bankruptcy plan.
The edges: limits, throttles, and failure modes
The edges are where S3 pages people, and most of the pages rhyme.
The per-partition request rates and their 503 warm-up behaviour, covered above, are the classic. The sneakier cousin is KMS: encrypt with SSE-KMS and every GET and PUT makes a KMS call, and KMS request quotas (tens of thousands per second per region, shared by everything in the account) become your effective S3 throughput ceiling. S3 Bucket Keys fix this by wrapping data keys under a per-bucket key, cutting KMS traffic by up to 99%; there is no good reason for a new bucket to skip them.
Hard limits worth memorising: 5 TB per object, 5 GB per single PUT, 1,024 bytes per key, 10,000 parts per multipart upload. Bucket count per account defaulted to a miserly 100 for eighteen years; since late 2024 the default is 10,000 and the ceiling a million, which legitimised bucket-per-tenant designs that used to require quota-increase grovelling.
Versioning plus automation is a recurring incident pattern: a sync job that repeatedly deletes and rewrites keys in a versioned bucket manufactures millions of delete markers and noncurrent versions, which bloats storage bills and degrades LIST performance until a lifecycle rule cleans house. Archive restores page people twice: once when someone discovers the data they need is 12 hours away, and again when the temporary restored copy expires mid-analysis because nobody read the restore-days parameter.
Replication lag is a silent failure mode unless you alarm on the RTC metrics; without RTC there is no SLA to breach, just an ever-growing gap you discover during the disaster you replicated for. Event delivery can be delayed by minutes under regional stress, so downstream systems need to tolerate late events, not just duplicate ones.
And S3 does go down, rarely and memorably. The 28 February 2017 us-east-1 outage started with an operator debugging the billing subsystem who mistyped a command and removed too much index and placement capacity; the subsystems had not been fully restarted in years and took hours to come back, and half the internet, including AWS’s own status dashboard, turned out to depend on the affected region. The postmortem is a classic of the genre: capacity removal now has guardrails and minimums, and the dashboard no longer lives in the blast radius. The general lesson stands: S3’s regional design means your availability story across regions is yours to build, with CRR and Multi-Region Access Points as the parts bin.
What it costs, and why it costs that way
S3 bills four meters: storage (GB-months, by class), requests (per 1,000, by type and class), retrieval (per GB, on cold classes), and data transfer out. Every confusing line on an S3 bill traces back to one of those four, and the shape is designed: cheap shelf, expensive touch, so that each class is only a bargain for the access pattern it was built for.
The anchor prices in us-east-1: Standard storage $0.023/GB-month; PUT/COPY/POST/LIST $0.005 per 1,000; GET $0.0004 per 1,000; internet egress $0.09/GB for the first 10 TB, after a 100 GB/month free allowance shared across your whole account. Storage is the number everyone quotes and frequently the smallest line on the bill. A terabyte sits for $23.55 a month but costs about $92 to serve to the internet once; a workload of millions of tiny hot objects can spend more on GETs than on storage. Egress is also the moat: it is free to bring data in and $90 a terabyte to walk it out, which is worth remembering whenever multi-cloud comes up in architecture review.
The cold classes have lower shelf prices because of three charges Standard doesn’t have, and each is a trap for the unwary. Retrieval fees: Standard-IA saves $0.0105/GB-month over Standard but charges $0.01/GB to read, so data read on average more than about once a month is cheaper left in Standard. Minimum durations: 30 days for the IA classes, 90 for both Glacier Instant and Flexible, 180 for Deep Archive; delete or transition early and you pay the remainder anyway. Minimum sizes and overheads: IA classes bill at least 128 KB per object, and the Flexible/Deep Archive tiers add roughly 40 KB of index and metadata overhead per object, 8 KB of it at Standard rates.
The compound trap is small objects in deep archive. Transitions are billed requests ($0.01 to $0.05 per 1,000 depending on destination), so lifecycle-transitioning 10 million 50 KB objects to Deep Archive costs about $500 in transition requests, adds 400 GB of overhead, and saves almost nothing on 500 GB of actual data, before you have paid a retrieval fee. The fix is aggregation: tar or Parquet small objects into large ones before archiving, or let Intelligent-Tiering’s no-fee tiers handle them and accept the monitoring charge as the cost of not doing arithmetic.
Two myths round this out. The S3 free tier was never generous: 5 GB and some requests for twelve months on legacy accounts, and since mid-2025 new accounts get a general credit allowance instead of per-service free usage, so “S3 is free for small stuff” is now simply false. And “Glacier is a fraction of a cent, archive everything” ignores that the classes are priced so that AWS wins whichever way you guess wrong; the only defence is knowing your access pattern or paying Intelligent-Tiering to learn it for you.
Running it in anger: security
S3’s security model has spent a decade being simplified by better defaults, and the current state is genuinely good, provided you know which era your buckets were born in.
Encryption at rest is universal: since January 2023 every new object is encrypted with SSE-S3 (AES-256 under S3-managed keys) unless you specify otherwise. The step up is SSE-KMS, which puts the keys in KMS where you control policy, rotation, and audit; pair it with Bucket Keys for the throughput reasons above. DSSE-KMS applies two independent encryption layers for the small set of compliance regimes that demand it. SSE-C, where the client supplies the raw key with every request, was always a niche, and after ransomware crews discovered they could re-encrypt victims’ objects under keys only the attacker held, AWS disabled SSE-C by default on new buckets from April 2026; enabling it now requires an explicit opt-in that almost nobody should exercise. Client-side encryption remains the option when S3 must never see plaintext. The mechanics of all of these are the standard envelope pattern, covered in how encryption actually works: a data key encrypts the object, a master key encrypts the data key.
Access control converged on two mechanisms: IAM policies on principals and bucket policies on resources, evaluated together with an explicit deny beating everything. ACLs, the original 2006 mechanism, have been disabled by default since April 2023 (Object Ownership set to bucket-owner-enforced), and Block Public Access has been on by default just as long; a modern bucket cannot be made public by accident, only by deliberate, multi-step choice. Access points give each application its own named endpoint and policy on a shared bucket instead of one thousand-line bucket policy; Multi-Region Access Points add a global endpoint with routing and failover across replicated buckets; Object Lambda puts a Lambda function in the GET path to redact or transform responses per caller without duplicating data. In VPCs, gateway endpoints keep S3 traffic off the internet at no charge, and conditions like aws:SourceVpce and s3:ResourceAccount in policies close off exfiltration paths to attacker-controlled buckets.
Running it in anger: seeing it and operating it
Observability is a choice between two audit trails and a set of dashboards. Server access logs are free (you pay only their storage), delivered on a best-effort basis with hours of delay, in a flat format from 2006. CloudTrail data events are structured, near-real-time, and integrated with everything, but billed per event, which on a busy bucket becomes real money; the standard compromise is CloudTrail data events on sensitive buckets, access logs or nothing elsewhere. CloudWatch request metrics are opt-in per bucket or prefix and are what you alarm on for 4xx/5xx rates and first-byte latency.
For fleet-level questions, Storage Lens gives account-wide and organisation-wide dashboards, a free default tier and a paid advanced tier with prefix-level granularity, and it is where cost anomalies (an incomplete-multipart pile-up, a runaway version count) surface first. For content-level questions, S3 Inventory delivers daily or weekly manifests, and S3 Metadata’s live tables are the modern, queryable replacement for most Inventory jobs.
Operationally, the resilient-bucket posture stacks four features: versioning (undo), lifecycle (cost hygiene and version cleanup), replication to another region or account (blast-radius isolation, ideally into an account the primary workload cannot write to), and Object Lock where the data warrants immutability. Batch Operations is the remediation tool when policy changes after the fact: re-encrypting, re-tagging, or re-tiering billions of existing objects. And restores should be rehearsed, because an untested archive strategy is a hypothesis, and 12-hour retrieval latency is a bad time to test hypotheses.
What this means for you
Design keys for parallelism from the start: high-cardinality prefix first, dates last, because re-keying a large bucket later is a migration project. Treat 503s as a normal signal to back off, not an outage. Turn on versioning and write the lifecycle rules (noncurrent expiry, multipart abort) on day one, when they are two minutes of work instead of a cleanup project.
Default unknown access patterns to Intelligent-Tiering, and never send small objects to a Glacier tier without doing the arithmetic on transition fees, per-object overhead, and minimum durations. Watch egress and request counts as closely as storage, because storage is usually the line item that matters least.
Use conditional writes before reaching for a lock service; compare-and-swap on an ETag now covers leader election, config publication, and commit protocols that used to need DynamoDB. Look at S3 Tables and S3 Metadata before building Iceberg maintenance or bucket-catalogue plumbing, because AWS has been systematically absorbing that layer since 2024.
Finally, keep the division of responsibility straight. Amazon’s eleven nines protect you from their hardware and, thanks to an unusual engineering culture, from most of their software. Nothing in that number protects you from your own DeleteObject, your own lifecycle rule, or your own compromised credentials; versioning, replication into a separate account, and Object Lock are how you buy nines against yourself. S3 has spent twenty years being the most reliable component in almost every architecture that uses it. The failure modes that remain are nearly all on your side of the API.