Health checks in system design

A health check is a small test used to decide whether a service instance is healthy enough to run, receive traffic, or stay in rotation.

Imagine a load balancer sending traffic to three app servers.

Health checks help infrastructure stop sending traffic to instances that are crashed, stuck, overloaded, still starting, or unable to serve requests correctly.

This distinction matters a lot in interviews and production systems.

Example: an app may be alive but still warming up caches, applying migrations, overloaded, or unable to reach a required dependency. That is a readiness problem, not necessarily a liveness problem.

A very simple health endpoint may look like this:

import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class HealthController {

    @GetMapping("/health")
    public String health() {
        return "OK";
    }
}

This only proves the web server can respond. It does not prove the whole application is ready to serve real user traffic.

A readiness check may verify important dependencies needed to serve traffic.

import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class ReadinessController {
    private final DatabaseClient databaseClient;

    public ReadinessController(DatabaseClient databaseClient) {
        this.databaseClient = databaseClient;
    }

    @GetMapping("/ready")
    public ResponseEntity<String> ready() {
        if (!databaseClient.canConnect()) {
            return ResponseEntity
                    .status(503)
                    .body("Database unavailable");
        }

        return ResponseEntity.ok("READY");
    }

    interface DatabaseClient {
        boolean canConnect();
    }
}

If this returns 503, the load balancer or orchestrator can stop routing traffic to that instance until it becomes ready again.

Health checks should be useful, but they should not overload your own system.

Load balancers use health checks to decide which instances should receive traffic.

Load Balancer
    checks /ready on App 1 → 200 OK
    checks /ready on App 2 → 503
    checks /ready on App 3 → 200 OK

Traffic goes to App 1 and App 3 only.

This is a major reason health checks improve availability: they help route around bad instances automatically.

Health checks are also important during deployments.

Deploy new version
    ↓
New instance starts
    ↓
Startup/readiness checks run
    ↓
Only after passing readiness does it receive traffic

This helps avoid sending users to a version that has started but is not actually ready yet.

What you check depends on the system, but common checks include:

• process is alive
• HTTP server can respond
• database connection pool is usable
• required cache or queue is reachable
• disk is not full
• critical config loaded correctly
• instance is not overloaded
• startup/warmup has completed

A health check should not become a full integration test for the entire company.

• Do not call dozens of downstream services every few seconds.
• Do not perform expensive database queries.
• Do not mutate production data.
• Do not make health checks depend on optional features.
• Do not expose sensitive internal details publicly.

Health checks and monitoring are related, but they are not the same.

A service can pass its health check but still have problems visible in monitoring, such as high p99 latency, rising error rate, or slow database queries.

In interviews, it helps to ask:

• Is this check for liveness, readiness, or startup?
• Who consumes the result: load balancer, Kubernetes, monitor, or human?
• What action happens when it fails?
• Should this instance be restarted or only removed from traffic?
• Which dependencies are critical for this endpoint?
• Could this check cause cascading failure?
• How often will it run?
• What timeout should the health check use?