Step 1 Understand it
The problem it solves. One can only handle so much, and if it dies, everyone using it is affected. You need to spread across many servers and route around the broken ones.
New words in this topic (tap one to see what it means)
Watch it happen
Read the animation as text
- Clients only ever see the load balancer's address. It makes many servers look like one.
- Round robin: request 1 goes to the next server in turn — server 1.
- Round robin: request 2 goes to the next server in turn — server 2.
- Round robin: request 3 goes to the next server in turn — server 3.
- Every few seconds it calls each server's /health endpoint. Server 2 fails a few checks in a row, so it is taken out of the rotation.
- Request 4 → server 3. Traffic goes only to healthy servers; server 2 is added back when it recovers.
- Request 5 → server 1. Traffic goes only to healthy servers; server 2 is added back when it recovers.
Picture it
flowchart LR C["Clients"] --> LB["Load balancer"] LB -->|"request 1"| S1["Server 1 ✓"] LB -->|"request 2"| S2["Server 2 ✓"] LB -.->|"failed health check:<br/>no traffic"| S3["Server 3 ✗"]
The idea
It makes many servers look like one, enables and hides failures. balancers route by and (fast). Layer 7 balancers read the request, so they can route by path, header or . Common algorithms: , , and (the same user always goes to the same server).
How it works, step by step
- connect to the 's address, never directly to a server.
- For each request or connection, it picks a healthy server using an algorithm: the next in turn (round robin), or the one with the fewest active connections.
- Every few seconds it calls each server's health endpoint. After a few failures in a row it stops sending traffic there, and adds the server back when it recovers.
- A layer 7 balancer can read the URL and send /api to servers and /images to image servers.
- So the balancer itself isn't a , run two of them (one active, one , sharing an address) or use a managed service.
Analogy
A supermarket sending shoppers to the shortest checkout , and closing a till when it breaks.
Where the analogy breaks
Shoppers are fairly similar. Requests aren't: one may take 5 and another 5 seconds, so 'the same number of requests' isn't 'the same amount of work'. Least connections helps.
Example
An AWS Application Load Balancer (layer 7) sends /api/* to API servers and /images/* to a media service, and checks each server's /health endpoint every few seconds.
Must-know points
- Algorithms: round robin, weighted round robin, least connections, IP or .
- Health checks remove failed servers automatically.
- Layer 4: fast and protocol-agnostic. Layer 7: smarter routing and handling, but more work per request.
- tie a user to one server; avoid them by keeping servers .
- The load balancer itself must be (a pair, or a managed service).
- Load balancers are also used between internal services, not only at .
Common mistakes
- Sticky sessions that tie one user to a server forever, which breaks when that server dies.
- Health checks that only test 'is the process running?', not 'can it reach its ?'.
- Forgetting that the load balancer needs redundancy too.
Step 2 Go further (free, optional)
Optional. The notes above are written to be enough on their own. These free resources give a second explanation and extra practice.
-
Watch
ByteByteGo: load balancing algorithms
ByteByteGo · YouTube
-
Read
Cloudflare Learning Center, free
-
Read
Donne Martin · GitHub, free
Step 3 Practise
Try each exercise yourself first: on paper, out loud or in code. Then open the answer and compare it with yours.
-
ExplainSay it in plain words in under a minute, with no notes.
Explain layer 4 and layer 7 load balancing in two sentences each.
Show answer
Layer 4: the load balancer looks only at the network level, the IP addresses and ports, and forwards whole connections without reading what is inside them. It is very fast and works for any protocol, but it can't route by URL or user.
Layer 7: the load balancer reads the HTTP request itself, its path, headers and cookies, so it can send /api to API servers and /images to an image service, and handle TLS. It is smarter, but does more work per request, and it must decrypt the traffic to read it.
-
DrawSketch the diagram (or class diagram) from memory.
Draw a load balancer with three servers, one of them failing its health check.
Show answer
The load balancer calls each server's /health endpoint every few seconds. Server 3 failed several checks in a row, so it gets no traffic until it passes again; requests are shared between servers 1 and 2. A good health check also tests what the server needs (can it reach its database?), not just that the process is running.
textClients │ Load balancer ┌────────────┼ ─ ─ ─ ─ ─ ─ ┐ │ │ ╎ no traffic ▼ ▼ ▼ Server 1 Server 2 Server 3 /health ✓ /health ✓ /health ✗ ✗ ✗ requests requests (back in when 1, 3, 5… 2, 4, 6… it passes again) -
ApplyUse it in a real design question and say exactly where it fits.
Pick an algorithm for: identical short API calls; long video uploads; a cluster. Justify each.
Show answer
• Identical short API calls: round robin. Every request costs about the same, so taking turns spreads the work evenly, and it is the simplest algorithm. (Least connections works too.)
• Long video uploads: least connections. An upload can last seconds or an hour, so "the same number of requests" is not "the same amount of work". Round robin could stack several long uploads on one server while another sits idle; least connections sends each new upload to the server with the fewest open ones.
• A cache cluster: consistent hashing on the cache key. The same key must always reach the same , or every request misses (the data is in another node's memory). Consistent hashing also means that adding or removing a node only moves about 1 in n keys, instead of reshuffling almost all of them. -
Trade-offsSay what it costs, when not to use it, and the alternative.
Why are sticky sessions a problem when you scale out or a server dies?
Show answer
Sticky sessions pin each user to the server that served them first. That causes problems:
• out doesn't help straight away: new servers only get new users, while the existing users stay on the overloaded old servers.
• Uneven load: a few heavy users stuck on one server can overload it while others idle, and the balancer isn't allowed to move them.
• A server dies: everything stored in its memory for its users is lost. They are out, lose their basket or their half-filled form.
• Deploys and scaling in are slow: you have to wait for to drain before you can remove a server.The fix is to make servers stateless: keep sessions in a shared store like (or in a signed on the client), so any server can serve any request and the balancer is free to spread the load.
Step 4 Interview-ready, then prove it
Your interview answer
About 20 to 40 seconds, said out loud. Practise it until it sounds like you, not like notes.
I'd put a layer 7 load balancer in front of the stateless , with health checks and least-connections routing. It runs as a redundant pair or a managed service, so it isn't a single point of failure.
Follow-up questions
How does a load balancer know a server is down?
Health checks: it calls an regularly and removes the server after several failures in a row.
When would you route by hashing?
When requests for the same key should reach the same server, for example a cache , so the data already in its memory gets reused.
Tick each check when you can do it without notes
A topic is finished when you pass all four checks, not when you finish a video. Be strict: if you needed your notes, it is not ticked yet.
- Explain Say it in plain words in under a minute, with no notes.
- Draw Sketch the diagram (or class diagram) from memory.
- Apply Use it in a real design question and say exactly where it fits.
- Trade-offs Say what it costs, when not to use it, and the alternative.
0 of 4 checks ticked.
Saved in this browser. Back it up or move it to another device.