System Design Interview: Designing Distributed Object Storage Like Amazon S3
3 views
"Design an object store like S3" is a demanding system design question because it combines huge scale, strong durability promises and a simple API. Here is a structure that works within 45 minutes.
1. Requirements
Functional: create buckets, PUT, GET and DELETE objects by key, list objects by prefix, support objects from a few bytes to several terabytes. Non-functional: eleven nines of durability as a target, high availability for reads, read-after-write consistency for new objects, and cost efficiency for cold data.
2. High-level architecture
Separate the metadata path from the data path.
- API/front-end service: authentication, request routing, multipart upload coordination.
- Metadata service: maps bucket + key to object version, size, checksum and the list of data chunk locations. Store it in a sharded, replicated key-value store, partitioned by a hash of bucket and key prefix.
- Data nodes: store immutable chunks on local disks, grouped into placement groups across racks and availability zones.
3. Write path
The client uploads to a front-end node, which splits large objects into chunks (for example 64 MB) and writes each chunk to data nodes. Only after the required replicas or erasure-coded fragments are durably acknowledged does the metadata service commit the new version. Committing metadata last is what gives clean read-after-write semantics: a reader never sees a key pointing to incomplete data.
Large files use multipart upload: parts are uploaded in parallel and independently retried, then a final "complete" call stitches them together in metadata.
4. Durability: replication vs. erasure coding
Three-way replication is simple and fast to repair but costs 200% overhead. Erasure coding, such as 10 data + 4 parity fragments, survives four failures with only 40% overhead, at the cost of more CPU and slower reconstruction. A common answer: replicate hot, recently written data and convert it to erasure coding as it cools.
5. Keeping data healthy
Background scrubbers read chunks and verify checksums to catch silent corruption. When a disk or node dies, a repair service re-creates missing fragments elsewhere, prioritizing objects that have lost the most redundancy.
6. Listing and hot spots
Listing by prefix needs the metadata store to support ordered range scans within a bucket. Sequential key names (timestamps) can create hot partitions, so mention automatic partition splitting.
7. Deletes, versioning and lifecycle
Deletes write a tombstone in metadata; garbage collection reclaims chunks later. Lifecycle rules move old objects to cheaper storage tiers.
Close by restating the trade-offs: metadata committed last, erasure coding for cost, scrubbing for durability. During real system design rounds, Linkjob AI (https://linkjob.ai) provides real-time help in live technical interviews, which can remind you of a missing piece such as the repair path when the conversation moves quickly.