Phase 2 of 9 · 9 lessons
Core concepts: scale, reliability and speed
The ten ideas every high-level design uses: scalability, availability, fault tolerance, load balancing, caching, consistency and CAP, rate limiting, asynchronous processing, and observability.
Almost every design answer is a combination of these ideas: copy things for availability, spread load for scale, cache for speed, queue slow work, and watch everything. Learn each one well enough to say when to use it and what it costs.
Lessons, in order
-
2.1
Scalability: vertical vs horizontal
Scalability is a system's ability to handle more load by adding resources.
-
2.2
Availability: nines, SLAs and redundancy
Availability is the share of time a system works and responds correctly, usually written in 'nines', like 99.9%.
-
2.3
Fault tolerance: timeouts, retries, circuit breakers
Fault tolerance means the system keeps working, perhaps in a reduced way, when parts of it fail.
-
2.4
Load balancing
A load balancer spreads incoming requests across several servers and stops sending traffic to unhealthy ones.
-
2.5
Caching
A cache keeps a copy of frequently used data in fast storage, usually memory, so it can be served without repeating slow work.
-
2.6
Consistency, CAP and PACELC
Consistency (in distributed systems) means every read sees the latest write. The CAP theorem says that during a network partition, a system must choose between consistency and availability.
-
2.7
Rate limiting and backpressure
Rate limiting caps how many requests a client can make in a time window. Backpressure is a system telling the senders upstream to slow down when it's overloaded.
-
2.8
Asynchronous processing and message queues (Kafka, SQS)
Asynchronous processing means doing slow work later, outside the user's request. A message queue holds the work, or events, between the service that produces them and the services that consume them.
-
2.9
Observability: logs, metrics and traces
Observability is how well you can understand what a system is doing from the outside, using logs, metrics and traces.