An agent edits the wrong config file, then spends forty minutes trying to repair its own repair. The user files a bug with a screenshot of the last message. The maintainer asks for logs, and the logs live in four places: the provider’s usage page, the tool server’s stdout, the framework’s debug output, and a terminal scrollback that closed with the window. Nobody can say what the agent saw on turn six.

A repository has 300 integration tests that each send a prompt to a model. The suite runs on every push to every branch, then again in the merge queue. Most of those runs don’t touch a prompt (the diff was a stylesheet or a migration), so the model receives the same 300 requests it saw an hour earlier, writes roughly the same 300 answers, and the provider bills every one. Nobody chose that. It’s what happens when a test calls a live API.

An agent calls a code-search tool and gets back a hundred hits. Each hit carries dozens of fields: node IDs, a URL for every related resource, avatar links, permission flags. The model needed three of them, a repo, a path and a snippet. The rest now sits in the context window for the remainder of the session, and the model reads past it on every later turn.

Tool definitions get most of the attention in agent token costs, and they’ve earned it. A server that exposes 150 tools puts 150 schemas in front of the model on every turn. Results are the other half of the bill, and they have fewer standard answers. An ordinary API client ignores the fields it doesn’t use, and ignoring is free. A model pays to read every token it’s handed.

LLM routing usually starts as if statements. One service picks the cheap model for ticket summaries. Another hardcodes the strong model because it needs tool calling. A third has a comment reading “never send this to the cloud” directly above a fallback that sends it to the cloud when the local server times out. Changing a model, a price or a provider means editing all three.

The router worth building is a small one: ordered rules over properties you can compute before the request leaves, written like an nginx config in a single file, with no database and no UI. Input size, whether tools or images are present, a data classification tag, a cost ceiling for the single request, which upstreams are healthy. One binary on a cheap VPS reads the file, accepts the OpenAI request shape and picks where each request goes. The rule that justifies the exercise is that privacy routing fails closed.

An agent works through a ticket and asks a docs-search tool the same question on turn 3, again on turn 14, and again the next morning in a fresh session. Each repeat costs a charge against the upstream’s rate limit and a wait with the model idle, for an answer that never changed. Agents retry after errors and start every session by looking up what the last one already knew.

Every enterprise racing to deploy AI hits the same wall eventually: the outputs are technically impressive but commercially useless. The model knows everything about everything and nothing about your business. That gap — between general capability and contextual intelligence — is a data problem, and it’s the problem KeyAPI.ai is built to solve.

The Generic AI Problem Is a Data Problem

General large language models are trained on public datasets. That makes them broadly knowledgeable and entirely generic. Ask one to help with e-commerce user preference analysis, social media content strategy, or brand marketing targeting, and it will produce polished, plausible, competely undifferentiated output. It has no idea what your customers actually buy, what your community actually says, or what your competitors are actually doing.