Phase 3 of 9 · 9 lessons
Data: databases, storage and scaling data
Choose the right database, keep data correct with transactions, make queries fast with indexes, and scale data with replication and sharding.
App servers are easy to scale; data is not. Most deep dives in interviews end up here: which store, which key, how many copies, how to split it, and what consistency you promise.
Lessons, in order
-
3.1
SQL vs NoSQL: choosing a database
SQL (relational) databases store data in tables with a fixed schema and support joins and transactions. NoSQL is a family of other models: key-value, document, wide-column and graph.
-
3.2
ACID transactions and isolation levels
A transaction groups several operations so they succeed or fail together. ACID stands for Atomic, Consistent, Isolated, Durable.
-
3.3
Database indexing
An index is an extra data structure that lets the database find rows without scanning the whole table.
-
3.4
Replication
Replication keeps copies of the same data on several machines.
-
3.5
Partitioning and sharding
Sharding splits a large dataset across several databases (shards), each holding part of the data.
-
3.6
Consistent hashing
Consistent hashing assigns keys to servers so that adding or removing a server moves only a small share of the keys.
-
3.7
Object storage (S3) for files and media
Object storage keeps files ('objects') such as images, videos and backups as a key pointing to bytes plus metadata, in a cheap, very durable, almost unlimited store like Amazon S3.
-
3.8
Search: inverted indexes and Elasticsearch
Full-text search finds the documents that contain some words and ranks them by relevance. It's built on an inverted index: a map from each word to the documents containing it.
-
3.9
Unique ID generation
Generating IDs that are unique across many servers, without one shared database counter.