Zoltan FodorZoltan Fodor

MCP was terrible

Early MCP felt terrible because of a context bill, not because of the protocol. Clients fixed their half with progressive discovery and code mode. Workflow tools and scoping are the half you still owe the agent.

·9 min read·
On this page
  1. Why early MCP felt so bad
  2. How the ecosystem fixed the client half
  3. Progressive / layered tool discovery
  4. Programmatic tool calling (code mode)
  5. Neon as a concrete case study
  6. Practical design rules (Clausis checklist)
  7. Soft close
  8. Further reading

If you wired up an MCP server in the first wave, you probably remember the feeling. You connected the client, the handshake succeeded, and then nothing useful happened for a suspiciously long time. The model was not thinking about your product. It was chewing through a phone book of tool schemas you had just shoved into the prompt.

That was the context window problem, and it was not subtle.

Every tool name, description, and JSON Schema sat in the context on every turn. A "complete" API surface meant hundreds of near-duplicate tools. Agents picked the wrong sibling endpoint. Turns got slower and more expensive because intermediate results bounced through the model instead of staying in a runtime. You paid tokens to discover capability, then paid again to misuse it, then paid a third time to recover.

I ship agents and MCP integrations for a living (Clausis, Budapest). The painful part is how predictable this failure mode was, and how long we treated "expose more tools" as a feature.

One clarification before the rant, because the title invites it: the protocol itself was never the problem. tools/list and tools/call look today roughly like they did on day one. What was broken was the default behaviour on both ends of the wire. Hosts dumped everything into the prompt. Servers gave them far too much to dump.

Where the context window went
Where the context window went
Where the context window went
Where the context window went: ~150,000 tokens of definitions before vs ~2,000 after.
Where the context window went: ~150,000 tokens of definitions before vs ~2,000 after.
Where the context window went: ~150,000 tokens of definitions before vs ~2,000 after.

Why early MCP felt so bad

Three design myths stacked on top of each other.

Myth 1: MCP is a transport for your existing OpenAPI. Codegen from OpenAPI to MCP is a fine bootstrap. It is a terrible finished product. Endpoint-shaped tools inherit endpoint-shaped granularity: list X, get X, patch X, delete X, plus every admin corner case you forgot to prune. Speakeasy wrote this up after generating 50+ production servers: descriptions written for humans become token sludge, a 200-endpoint spec becomes 200 tools, and in their experience descriptions start to overlap and confuse selection at around 30 tools, with selection close to guaranteed to fail past 100.

Myth 2: More tools equal more power. For humans browsing a reference site, maybe. For a model choosing among peers in one forward pass, catalog size is load. Similar names (create_branch, create_endpoint, create_project) compete. The model does not get a free index. You gave it a haystack and called it an API.

Myth 3: The honest thing is to put the full catalog in every prompt. Early clients did exactly that: tools/list, then dump everything into the model. Before the user finished typing, context was already taxed. The official MCP client guide now says the quiet part out loud: loading every definition upfront "wastes tokens, increases latency, and degrades model performance", and with dozens of servers the definitions alone "can consume the majority of the context window before the model has even read the user's message."

Anthropic put a number on it in November 2025: one workflow went from 150,000 tokens to 2,000 once definitions were loaded on demand. That is 98.7% of the bill spent on reading the menu.

So yes, early MCP was often "REST with extra steps," plus a context bill.

How the ecosystem fixed the client half

Two patterns moved the bottleneck. Both are now written up as MCP client best practices (the 2026-07-28 client guide on modelcontextprotocol.io).

Progressive / layered tool discovery

Stop injecting every schema up front. The host still calls tools/list, but keeps the result to itself and gives the model a small discovery surface instead:

  1. Catalog: something like search_tools returns names and one-line descriptions.
  2. Inspect: get_tool_details fetches the full input (and output) schema for one candidate.
  3. Execute: call only what you loaded.

Clients choose retrieval: keyword, embeddings, a cheap subagent, or hybrid. The guide's rule of thumb is to switch once definitions pass roughly 1 to 5% of the context window. Providers ship native tool search now (both OpenAI and Anthropic). Codex made tool search for MCP tools unconditional wherever the model supports it; before that it only kicked in behind a flag or past 100 tools. Claude Code defers every MCP tool by default: a fresh session sees tool names and server instructions, and schemas load on demand.

Search, inspect, execute. The host keeps the phone book; the model gets an index card.
Search, inspect, execute. The host keeps the phone book; the model gets an index card.
Search, inspect, execute. The host keeps the phone book; the model gets an index card.

Important split of responsibility: servers used to implement search/inspect/execute as their own three tools. That helped when hosts were naive. Now serious hosts do progressive discovery themselves. Your job is no longer to reimplement a tool search engine inside every server. Your job is to publish a catalog worth searching.

One detail most posts skip: discovery fights prompt caching. Providers cache the prompt prefix including the tools array, so adding and removing definitions mid-conversation can cost more than the definitions you saved. Good hosts append new schemas after the cache breakpoint, or route everything through one stable call_tool({name, args}) meta-tool. As a server author you cannot fix that, but it is the reason a small, stable always-on set still matters.

Programmatic tool calling (code mode)

Discovery answers "which tool." Composition answers "how many round trips bleed into context."

In direct tool calling, each hop returns into the model. Large JSON, log dumps, and row sets become prompt furniture. In code mode (Anthropic's engineering writeup calls it code execution with MCP; Cloudflare coined "Code Mode"), the host turns MCP tools into a typed API inside a sandbox. The model writes a short script. The script calls tools, filters and joins results in-process, and returns a summary. Intermediate payloads stay in the sandbox. Credentials stay on the host, and the sandbox has no network of its own.

The client guide's own illustration: 100K+ tokens of intermediates in direct calling, versus a roughly 200-token script and a 15-token summary.

Direct calling vs code mode.
Direct calling vs code mode.
Direct calling vs code mode.

Progressive discovery plus code mode is the combo shot: fewer definitions in context, fewer intermediates in context.

That still does not mean you should ship a raw OpenAPI mirror and walk away.

Neon as a concrete case study

Neon's @neon/tools launch (blog post, 3 September 2026, plus a video walkthrough) makes the remaining server-side job concrete.

The arc matches what many of us lived:

  1. First instinct: one tool per API endpoint.
  2. Pain: hundreds of tools in context; confused selection.
  3. Server-side layered tools as a patch.
  4. Then clients absorb progressive discovery and code mode.
  5. Temptation: "great, go back to 1:1 mappings."
  6. Corrective: not quite. Agents can chain raw endpoints, but common workflows still waste tokens if every agent rediscovers the same three-call dance.

Neon's example is branch provisioning. In their words, creating a branch over REST is three calls: create the branch, attach compute, then fetch a connection string. An SDK exists so humans do not relearn that sequence. MCP tools should get the same ergonomic shortcuts.

In @neon/sdk, that shows up as two layers in one package: raw, the generated 1:1 OpenAPI surface for full coverage, plus an ergonomic client with workflows such as projects.createAndConnect and branches.createAndConnect. Those workflows poll until the operations finish and hand back something an agent can actually use.

Four model turns and a polling loop vs one workflow tool.
Four model turns and a polling loop vs one workflow tool.
Four model turns and a polling loop vs one workflow tool.

Then @neon/tools turns SDK methods into agent tools: selectors are SDK paths, published IDs are verb-first snake_case (projects.list becomes list_projects), and the set includes both endpoint-level operations and composed workflows. The same package adapts to MCP, Eve, and Mastra. Every non-read operation is conservatively marked as needing approval, and so are reads that return connection credentials. A few SDK methods are deliberately not tools at all, credentials.reveal among them.

The hosted Neon MCP server builds on that surface and adds the product controls agents actually need: category filters (?category=querying&category=schema), project scoping (?projectId=), read-only mode (?readonly=true), and hand-written tools for SQL, migrations, diagnostics, docs and search.

The number I find most telling: the hosted server now exposes 104 tools (85 Management API tools plus 19 hand-written). In 2025 that count would have been a bug report. With client-side discovery it is just coverage, and the 19 are where the craft went.

The design slogan is the right one: the server defines which tools exist; the client decides how to discover and execute them.

Who owns what.
Who owns what.
Who owns what.

That is not a retreat from server craft. It is a clearer boundary. Clients own context scheduling. Servers own capability shape.

Practical design rules (Clausis checklist)

When I design an MCP server now, I run this list before I celebrate "we exposed the API."

  1. Lead with jobs, not verbs from OpenAPI. List the five workflows users actually ask agents to do. Those become first-class tools. Raw endpoints are fallback, not the homepage.
  2. Budget the always-on catalog. Not every host defers. If one loads you eagerly, keep the default set small. Use categories or install-time filters (Neon's category query param is a good product pattern). Do not make every consumer import storage, auth admin, and observability on day one.
  3. Assume progressive discovery, still write for it. Clear names, first-sentence descriptions, discriminable siblings. The cheapest discovery strategy is keyword matching over names and descriptions, so that text is now your search index. Search cannot save a catalog of do_thing, do_thing_v2, and do_thing_alt.
  4. Prefer workflow tools for multi-step provisioning. If humans needed an SDK helper, agents need a tool. Bundle waits and "ready enough to use" into the tool contract instead of forcing the model to poll.
  5. Publish outputSchema where you can. Code mode stubs get sharper; clients can type results instead of guessing from prose. Without it the host falls back to any, or pays a small model to guess your types. The client guide is blunt: "the real fix is for server authors to provide outputSchema."
  6. Design for code mode, not against it. Return structured data that is easy to filter in a sandbox. Avoid forcing the model to re-echo huge blobs into the next call. Remember that errors arrive as isError: true and get turned into exceptions in generated wrappers, so make error payloads something a script can branch on.
  7. Separate read, write, and secret-bearing reads. Connection strings and password reveals are not the same risk class as list_projects. Mark approval needs explicitly. Neon's requiresApproval tool metadata is a useful precedent. Note that it is advisory: the host enforces it, the protocol does not. Flag anyway.
  8. Deprecate the OpenAPI dump as a product. Codegen is an internal accelerator. The public MCP surface should be curated, versioned, and scoped like any other developer API.
  9. Document the client boundary in your README. Tell integrators: we define tools; your host should search and compose. Stop shipping custom search/inspect/execute triples unless you are targeting hosts that still dump the full list.
  10. Eval tool selection under load. Add distractor tools. Measure wrong-tool rate and tokens before first useful call. If quality collapses past N tools, your catalog is lying about being "complete."
Jobs on top, metal at the bottom, blast radius on the side.
Jobs on top, metal at the bottom, blast radius on the side.
Jobs on top, metal at the bottom, blast radius on the side.

Soft close

MCP matured when we stopped pretending the prompt was a free hard disk.

The early servers that hurt were not evil. They were honest mirrors of REST at the exact moment clients were too young to schedule context. Clients grew up: progressive discovery, code mode, deferred catalogs. That is necessary and still insufficient. Someone still has to decide what a good tool is.

If you build MCP for a real product, borrow the boring wisdom of SDK design. Cover the metal. Bundle the workflows. Scope the blast radius. Let the client search and script. Keep the server opinionated about jobs-to-be-done.

That is how you stop paying a context tax for the privilege of confusing your own agent.

Zoltan Fodor builds AI products around agents, MCP, evals, and production constraints. Founder @ Clausis, Budapest.

Further reading