Trimming 100 KB API Responses to the Five Fields Your Client Uses, at the Proxy
Take a mobile screen that lists orders: a customer name, a status, a total, a date. The endpoint behind it returns 100 KB for one page, because the vendor’s schema carries about sixty fields per order, plus nested addresses, hypermedia links and an audit trail. The phone downloads all of it over whatever connection it has, parses all of it, and keeps five fields. You can’t change the vendor’s API. You can change what reaches the phone.
A reducer is a small proxy between the app and an API that returns a subset of each response, following rules in a file. It’s a backend-for-frontend without the backend: no new service to design, only projections. It earns its place for APIs you don’t control, and for teams that want a BFF’s effect without writing and operating one.
If you control the API, over-fetching already has good answers. GraphQL lets the client name its fields (the trade-offs are in the REST, GraphQL and gRPC comparison). JSON:API has sparse fieldsets (fields[articles]=title,body), Google’s APIs take a fields parameter, OData has $select, and the backend-for-frontend pattern gives each client a thin server of its own. Every one needs the API’s owner to cooperate or you to build a server. Gateways and edge runtimes can already rewrite JSON, and a Worker that does it fits in a page of JavaScript, much like caching a third-party response in a Pages Function with a transform added. Anyone can write the transform in an afternoon. The rest takes longer: validators, cache keys, drift alerts, and a way to see what was removed. That’s the part worth packaging.
Subset, Don’t Reshape
Rules are per route, and the path language is the smallest thing that works: dotted paths, [] for each array element, * for any key. They look like JMESPath but mean less: keep this key with its ancestors, drop the rest, preserve the structure. JMESPath can build new objects, while JSONPath only selects nodes, so a rename or a flatten needs a second step. A projection that keeps the original shape gives you a document any client written for the full response can still read, so the proxy stays removable: point the base URL back at the vendor and the app works. Once rules rename and flatten, the proxy defines a new API, and you own a contract. Here’s a rules file (invented syntax, for a tool that doesn’t exist yet) with a sample run underneath.
upstream: https://api.vendor.example
routes:
- match: GET /v2/orders
keep:
- items[].id
- items[].status
- items[].total.amount
- items[].placed_at
- items[].customer.name
pass: [next_cursor, links.next]
# reducer test GET /v2/orders --sample orders.json
# in 102,311 B out 3,874 B kept 7 of 61 paths missing 0
The numbers in that run are made up: a page of 25 orders at about 4 KB each, cut to five short fields apiece plus the cursor. Compression already hides much of the repetition on the wire, because field names repeat and gzip or brotli shrink JSON well, so measure after compression before promising anyone a ratio. What survives compression is the work on the device: the phone parses and allocates 4 KB instead of 100. An allow-list (keep) keeps output bounded when the vendor adds fields; a deny-list (drop) suits the case where one huge subtree is the whole problem. Default to keep; a client that starts reading a new field finds out in development, where it’s cheap.
Validators, Caches and Who Is Asking
The proxy touches only 2xx JSON from routes that have rules. Errors, redirects, other content types, unknown routes and bodies over a size limit pass through untouched, along with headers such as Link where an API paginates there. Content-Length, ETag and Content-Encoding get recomputed.
A transformed body isn’t the upstream’s representation, so it needs its own validator. Derive it from the upstream tag plus a hash of the projection, and keep the weakness of the original: a weak upstream tag gives a weak derived tag. When the client sends If-None-Match, the proxy sends its stored upstream validator to the vendor as a conditional request of its own. A 304 from the vendor becomes a 304 to the client without transforming anything; a 200 gets projected and cached. Edit the projection and every client’s tag changes, which is right, because the bytes changed.
The cache key is the URL, the request headers the upstream names in Vary, and the projection version, with the upstream validator stored alongside. Authorization needs care: if the upstream says Cache-Control: private, or varies on Authorization, the cache is per credential or absent. A reducer must never turn a private response into a shared one. Re-encoding to CBOR or MessagePack comes later, only for clients that ask via Accept, with Vary: Accept on the result.
At the edge the budget is tight, with limited CPU and memory per request. Parse once, project in a single pass, and pass through anything over a configured size. Whether bytes are your problem at all is a measurement question: diagnosing a slow API from one request splits time into DNS, connect, TLS, server wait and transfer, and if server wait dominates, a reducer buys little. On the device side, payload size is one of the numbers in the mobile performance metrics that matter.
The Proxy Can’t See What the Client Reads
The original pitch said a reducer could watch access patterns and recommend smaller schemas. The proxy sees responses. It never sees which fields the app reads afterward, so on its own it can’t recommend anything. Three sources can. First, the client’s own types: a Swift Codable struct, a Kotlin data class or a TypeScript interface already lists the fields the app decodes, and a small script can turn them into keep paths. That’s static analysis at its cheapest, and it’s often right because the decoder was written to those types.
Second, a debug-build wrapper that records which paths the app touches (in JavaScript, wrap the parsed JSON in a Proxy that logs reads) and reports a sample. It over-reports when code iterates over every key, and under-reports for screens nobody opened during the window. A path unread for a week is a candidate for removal, never a verdict: the settings screen that reads it opens once a month. Third, shadow mode: compute the projection, serve the full body, report what would have been removed, then send a canary cohort or the test suite through the reduced version before flipping it for everyone.
The proxy can see one thing the client can’t: whether the keep paths still match. If the vendor renames placed_at, a subset projection quietly returns documents with no date, which an app may render as a blank. Track a match rate per path and alert on a drop against the trailing baseline, since some fields are legitimately absent from a third of responses and a fixed threshold would nag forever.
Old App Versions Own Your Projection
Support will ask what the vendor really sent, so every reduced response carries headers with the projection version and the bytes in and out, and allowed callers can bypass to the original. Ownership matters more. Keep the rules in the client’s repo and deploy them with the client, so the projection changes in the same pull request as the code that reads the field. Mobile complicates that: you can’t recall app versions in the wild, and the build that shipped in March still reads a field you removed in June. The proxy has to serve each live version its own projection (keyed on an app-version header), and a path can leave the rules only when no live version reads it. Where several clients share a route, each gets its own projection, and the cache key includes it so they never collide.
Version 0.1 is an HTTP proxy for JSON with keep and drop rules, recomputed validators, a revalidating cache, drift counters, debug headers and the test command that prints before and after. It refuses re-encoding, a client SDK until the static extractor works, and a UI. Buyers are mobile teams with heavy third-party payloads, edge platforms that want partial responses as a feature, and API providers who want them without rebuilding. The same idea applies to the biggest payloads agents see, MCP tool results, and the case for reducing data near the source is why it belongs in a small tool next to it.
Pick the one route that hurts and start there.