The invisible bottleneck called JSON

After the Go framework benchmark, I kept thinking about the missing part.

The benchmark was fair for what it was. Every server had one route. Every route returned pong. That makes router overhead easy to compare.

But most APIs return more than pong.

They parse JSON, validate it, call something, then return more JSON. Sometimes a lot of it.

So if someone sees a framework benchmark and thinks their whole app will be 2x faster, that is probably too much confidence.

The router is usually the cheap part.

the fake app#

This is closer to a normal endpoint:

type User struct {
    ID        int      `json:"id"`
    Name      string   `json:"name"`
    Email     string   `json:"email"`
    Roles     []string `json:"roles"`
    CreatedAt string  `json:"created_at"`
}

func handler(w http.ResponseWriter, r *http.Request) {
    users := make([]User, 0, 1000)
    for i := 0; i < 1000; i++ {
        users = append(users, User{
            ID:        i,
            Name:      "Rajat",
            Email:     "[email protected]",
            Roles:     []string{"admin", "writer"},
            CreatedAt: "2026-02-14T01:10:00+05:30",
        })
    }

    w.Header().Set("Content-Type", "application/json")
    json.NewEncoder(w).Encode(users)
}

This is still fake, ofc, but closer to a normal handler than pong.

The router finds /users, and then the app has to build structs, encode fields, escape strings, write all those repeated keys, and send a bigger response.

At that point, 10 microseconds of router overhead may not matter much.

small code can hide real cost#

JSON feels cheap because the code is one line:

json.NewEncoder(w).Encode(users)

But the work behind that line can be large. Code size and runtime cost can diverge.

A rough shape can be:

router match      maybe microseconds
small db query    maybe milliseconds
json encode       depends on payload size
network           depends on payload size too

If the response is small, no issue. If the response is a huge list with nested objects and repeated field names, JSON can start showing up in CPU, memory, and latency.

And it rarely announces itself as “JSON is slow”. It shows up as higher CPU, weird p99, memory spikes, or gzip suddenly mattering.

Then everyone checks the database first. Many times the database is the issue.

But sometimes the app is spending a lot of time turning objects into text.

what I would benchmark next#

If I redo that benchmark, I would keep /ping, but add a few more routes:

/small   returns one JSON object
/list    returns 1000 JSON objects
/parse   accepts JSON and returns JSON

That would show where framework differences stop being visible.

My guess is that /ping shows router overhead, /small still shows runtime differences a little, and /list becomes mostly JSON plus memory pressure.

I may be wrong. That is why it should be measured.

shape matters too#

This response is easy to read:

[
  { "id": 1, "name": "A", "role": "admin" },
  { "id": 2, "name": "B", "role": "admin" }
]

It also repeats keys for every row.

For 2 rows, who cares. For 20,000 rows, the response can become a lot of repeated text. gzip helps, but gzip also uses CPU. This is normal engineering sadness: fixing one cost can move cost somewhere else.

I still like JSON. I have zero plans to invent a binary protocol for a normal API and then pretend it was wisdom.

JSON is fine, with real cost at larger sizes.

what I would check first#

Before changing libraries, I would measure response size and encoding time.

Something simple:

route=/users status=200 bytes=482931 duration_ms=88

And maybe in code:

started := time.Now()
payload, err := json.Marshal(users)
encodeDuration := time.Since(started)

I would only keep this long enough to learn where the time is going.

If encoding is visible, I would first try returning fewer fields, paginating properly, avoiding repeated nested objects, or streaming if memory is the issue.

The best fix might be deleting data from the response. Annoying, but often true.

So yeah, framework benchmarks are useful. I still like them.

When a real endpoint is slow, I would check the response shape before blaming the router.