Health checks need smaller promises

Most backend apps eventually get a /health route, and mine usually starts with something embarrassingly small:

func health(w http.ResponseWriter, r *http.Request) {
    w.WriteHeader(http.StatusOK)
    w.Write([]byte("ok"))
}

That handler is perfectly honest. The process accepted a request, wrote a response, and stayed alive long enough to finish. The name is where I start lying to myself. I see a green health check and quietly upgrade it into “the app is fine”, even while login is broken, Redis is timing out, a migration has done something rude, or the cache is serving data from yesterday. I have stared at a green endpoint while the site behaved badly. The ridiculous claim was mine: I was asking two bytes of ok to explain the whole system.

Kubernetes separates liveness from readiness, and I like that distinction even on small systems that have never seen a Kubernetes manifest. Liveness answers whether restarting the process might help. Readiness answers whether this instance should receive traffic right now. Those questions feel similar until a dependency has a bad five seconds. Restarting every web process because the database blinked usually adds more noise, while leaving a struggling instance in rotation sends users straight into the mess. The check should match the action attached to it: keep restart checks local and let traffic checks cover the dependencies normal requests need.

A database check often begins with this:

SELECT 1;

It proves the database accepted one tiny query. The important query may still fail, the pool may run out under load, and the migration can be wrong in the exact table a user needs. I still use SELECT 1 because that small fact is useful. I just label it correctly in my head. It is a quick look into one corner of the room, not an inspection certificate for the building.

For a small app I usually want endpoints with three different jobs:

• /live says the process is running.
• /ready says the instance can receive normal traffic.
• /smoke checks one important user-ish path.

Running /smoke every second from every region would turn a helpful check into a tiny denial of service generator, which is a fairly embarrassing way to create my own outage. I save it for deploys and slower scheduled checks. If a deployment claims the app works, asking it to complete one small user-shaped workflow seems reasonable.

Failure output needs the same restraint. This is fine for a public endpoint:

{ "ok": false }

An internal readiness endpoint can give me a little more shape:

{
  "status": "not_ready",
  "checks": {
    "database": "ok",
    "redis": "timeout",
    "migrations": "ok"
  }
}

That saves whoever is on call from guessing which dependency is upset, but it also contains information I do not want to hand to random visitors. Public health routes should keep hostnames, versions, stack traces, secrets, and dependency details to themselves. Otherwise a debugging helper becomes a small gift basket for strangers on the internet.

The route names now remind me how little each check promises. /live knows about the process, /ready knows enough to make a traffic decision, and /smoke tries one important workflow. The code can remain boring. I mainly want to stop reading ok and imagining it checked work that it never even attempted.