Load balancers, proxies and CDNs

Your Next.js app runs on one server with next start.

Then a sale starts, and ten times more people arrive. Or the server's disk fails at 3 AM. One server cannot carry every user, and when it stops, the whole site stops.

Big sites put three helpers in front of their app: a load balancer, a reverse proxy and a CDN. In this lesson you learn the job of each, and how one request passes through all three.

Why one server is not enough

One server has two limits:

It has a ceiling. It has a fixed amount of CPU and memory. Past that, every page gets slow.
It is a single point of failure. A crash, a bad deploy or a reboot takes the whole site down.

The fix is to run several copies of the app on several servers. Each copy is a . Now a dead server is a small problem, not an outage.

But users type one address, shop.com. Something must sit at that address and pass each request to one of the replicas. That is the load balancer.

The load balancer: many servers, one address

A spreads the requests over your servers.

┌────────────────────────────────────────────────┐
│                                                │
│             ┌───────────────┐                  │
│             │ load balancer │                  │
│             └───────┬───────┘                  │
│        ┌────────────┼────────────┐             │
│        ▼            ▼            ▼             │
│   ┌─────────┐  ┌─────────┐  ┌─────────┐        │
│   │ app 1   │  │ app 2   │  │ app 3   │        │
│   │ 1  3  5 │  │ 2  4  6 │  │ × down  │        │
│   └─────────┘  └─────────┘  └─────────┘        │
│    numbers: requests in order · ×: taken out   │
│                                                │
└────────────────────────────────────────────────┘

Round robin: requests take turns. App 3 failed its health check, so it gets nothing.

How it picks a server

The rule it uses is the . Two are common:

Algorithm	Rule	Good when
Round robin	Each server in turn	Requests cost about the same
Least connections	Server with fewest open requests	Some requests are slow

How it finds a dead server

The load balancer sends a small test request to each server again and again. This is a .

AWS Application Load Balancer sends one every 30 seconds by default. After 2 failures in a row, it stops sending users to that server.
After 5 passes in a row, it sends traffic there again.
nginx can also watch real requests. When a server returns errors, nginx skips it for a short time.

Docker in Action shows the same idea inside Docker. A service with several replicas gets one virtual IP address. Docker's built-in load balancer sends each connection on that address to one of the replicas.

The reverse proxy: a front desk for your app

A sits between the internet and your app. The user talks to the proxy, and the proxy talks to next start.

The Next.js docs recommend one when you host the app yourself. It takes the boring jobs off your app:

TLS. It holds the certificate and does the HTTPS work. Your app speaks plain HTTP on port 3000.
Compression. It shrinks text responses with gzip before they leave.
Routing by path. /api/ goes to one service, everything else goes to the Next.js app.
Protection. It can reject huge uploads, very slow clients and too many requests from one address.

nginx is the most common reverse proxy. Here it also balances two Next.js servers:

/etc/nginx/conf.d/shop.conf

upstream shop {
  server 10.0.0.11:3000;
  server 10.0.0.12:3000;
}

server {
  listen 443 ssl;
  server_name shop.com;
  ssl_certificate     /etc/ssl/shop.crt;
  ssl_certificate_key /etc/ssl/shop.key;
  gzip on;

  location /api/ { proxy_pass http://10.0.0.21:4000; }
  location /     { proxy_pass http://shop; }
}

upstream shop lists the two Next.js servers. nginx uses round robin over them.
listen 443 ssl and the two ssl_ lines do the HTTPS work.
The two location lines route by path: API calls go to the API server, pages go to shop.

A load balancer and a reverse proxy overlap a lot. nginx can be both, as above. The words describe the job, not the product.

The CDN: copies near your users

Your servers may sit in one city, for example Virginia in the US. A user in Mumbai waits for every byte to cross the world.

A (content delivery network) fixes this. It keeps copies of your files in many cities, and each user gets the nearest copy.

┌─────────────────────────────────────────────────┐
│                                                 │
│    MISS: the first request                      │
│   ┌──────┐     ┌───────┐     ┌────────┐         │
│   │ user │ ──▶ │  CDN  │ ──▶ │ origin │         │
│   └──────┘     └───────┘     └────────┘         │
│                 keeps a copy                    │
│                                                 │
│    HIT: every request after it                  │
│   ┌──────┐     ┌───────┐                        │
│   │ user │ ──▶ │  CDN  │     origin not asked   │
│   └──────┘     └───────┘                        │
│                                                 │
└─────────────────────────────────────────────────┘

The first request fills the cache. Later requests stop at the CDN.

A cache miss means the CDN has no copy. It asks your server, the , and keeps the answer.
A cache hit means the CDN answers alone. It is fast for the user and free for your server.

You already set the rules

The CDN reads the Cache-Control header to know what it may keep, and for how long:

Header value	Meaning
`public, max-age=31536000, immutable`	Keep for a year. Next.js sets it on hashed JS and CSS files.
`s-maxage=60`	CDNs keep it 60 seconds. Browsers ignore it.
`private, no-store`	Never keep it in a shared cache. Use it for pages with user data.

Next.js sets most of these for you. Static pages get a long cache. A page that reads cookies gets private, so one user's cart never goes to another user.

See it yourself

Vercel runs its own CDN in front of every deployment. This is a real run of curl -sI https://vercel.com on 8 November 2026, with five of the lines kept:

Terminal

$ curl -sI https://vercel.com
HTTP/2 200
age: 2923
server: Vercel
x-vercel-cache: HIT
x-vercel-id: bom1::v9ggz-1794105453930-0ef8d52013db

x-vercel-cache: HIT means the CDN answered from its copy. A MISS means it asked the app.
age: 2923 means the copy was about 49 minutes old.
bom1 in x-vercel-id is Vercel's Mumbai region, the one nearest to the machine that ran the command.

Cloudflare is the other name you will hear. Many teams put Cloudflare's CDN in front of servers they run themselves.

How they stack in one request

Here is one request to a self-hosted Next.js shop, from the browser to the app:

The CDN answers what it can. Most images, scripts and static pages stop here.
On a miss, the load balancer picks a healthy server.
nginx on that server handles TLS and compression, then passes the request to next start.

On Vercel, you see none of these boxes. Vercel's CDN does the caching, the routing and the TLS. It also starts more copies of your functions when traffic grows.

The jobs are the same. Someone else runs them.

Remember4 points

01
One server has a ceiling and is a single point of failure. Big sites run several replicas.
02
A load balancer gives the replicas one address, spreads requests with rules like round robin, and uses health checks to skip dead servers.
03
A reverse proxy such as nginx sits in front of next start and handles TLS, compression and routing by path.
04
A CDN keeps copies near users. Cache-Control decides what it keeps, and x-vercel-cache shows a HIT or a MISS.

Using nginx as HTTP load balancer from nginx. Round robin, least connections and health checks, with config.
Vercel CDN Cache from Vercel. What Vercel caches and how s-maxage works on it.
Cache-Control on MDN. Every directive, with examples.
Docker in Action, 2nd edition, chapter 11: read 11.4 on how Docker load balances a service over its replicas.
Summary of June 8 outage from Fastly, the story above.

The next lesson is "What is the cloud?", the first lesson of the cloud chapter.