Beginner System Architecture

Horizontal Scaling

Horizontal scaling, or “scaling out”, means adding more machines instead of making one bigger. Work is spread across many servers that run the same application. This is how large systems handle huge traffic without relying on a single, ever-larger box.

The idea

            +------------------+
 Clients -->|  Load Balancer   |
            +--------+---------+
                     |
        +------------+------------+
        |            |            |
     Server 1     Server 2     Server 3
   (same app)    (same app)   (same app)

A load balancer sits in front and distributes incoming requests across the servers.

The load balancer

The load balancer decides which server handles each request, using strategies like round-robin or least-connections. If one server dies, it stops sending traffic there.

GET /orders HTTP/1.1
Host: shop.example.com
# The load balancer routes this to whichever server is healthy

The hard part: shared state

If a user logs in on Server 1, Server 2 must also know they are logged in. The fix is to keep servers stateless and store shared state somewhere external.

Bad:  session stored in each server's memory  -> breaks on next request
Good: session stored in shared store (Redis, DB) -> any server can serve

Stateless servers are the key to horizontal scaling: any server can handle any request.

Advantages

  • Near-unlimited growth: keep adding machines.
  • High availability: if one node fails, others continue.
  • Use cheaper commodity machines instead of one expensive giant.
  • Scale up and down with demand.

Disadvantages

  • More complex: load balancing, service discovery, coordination.
  • State must be externalized (sessions, cache, files).
  • Network calls between nodes add latency and failure modes.
  • Harder to test and debug a distributed system.

When to use it

Scale out when traffic outgrows a single machine, when you need fault tolerance, or when load varies and you want elasticity. Design for it early by keeping your servers stateless, even if you start with just one.