handling tool bloat - hundreds of tools #2036

A pattern that scales better than putting 300-400 schemas in the prompt is capability discovery + activation.

What I would do:

1. split into domain-scoped MCP servers (crm, billing, identity, etc.)
2. keep a tiny always-on broker surface such as search_capabilities / activate_capabilities
3. when a domain is activated, expose only that slice and fire notifications/tools/list_changed
4. keep examples / long docs outside inputSchema so the active toolset stays cheap

A couple of nuances:

• tools/list pagination helps transport size, but it does not solve model prompt bloat if the client still injects every tool schema.
• I would keep a separate tool catalog with metadata like risk level, scopes, mutating vs read-only, and use that in the activation step. Otherwise a prompt-injected model can "discover" a powerful tool just because the description sounds relevant.
• Pin the active toolset per task/session and expire it when the task ends. That reduces both latency and accidental cross-domain tool use.

So yes: MCP can support your dynamic-tooling direction, but the win comes from just-in-time exposure, not from one huge server with every concrete tool always visible.

honestly we dealt with this exact thing building relay (https://github.com/valtors/relay) which has like 40 tools and it got messy fast

the biggest thing that helped was just grouping tools into categories and letting users enable/disable them per session. like if you're not doing image stuff, why even expose those tools to the model. we prefix tool names with the category too (fileread, webfetch etc) and weirdly that alone made the model call the right tool more often

also dont sleep on MCP prompts. if you got 100+ tools the model is gonna guess wrong half the time. prompts let you wrap common workflows into one call so the model doesnt need to figure out which 5 tools to chain together

another thing - if a tool needs an api key or a binary that isnt there, just dont register it server side. no point showing it to the client if it cant work

if you really have hundreds of tools i'd probably just split into multiple smaller focused servers tbh. one for db stuff, one for web, one for files. only load what you need for that session

The token cost problem is real and gets worse fast. We have 40+ tools on our own MCP server (Relay) and saw the same issue.

One approach that works without changing your server architecture: put a proxy between the client and the MCP server that can filter tools before they reach the model. We built Observer (github.com/valtors/observer) as a transparent stdio proxy that wraps any MCP server command. It has OBSERVER_FILTER (hide specific tools from the client) and OBSERVER_MAX_TOOLS (cap the total tools exposed) to reduce the schema tokens sent to the model.

The proxy also logs every tool call to SQLite, so you can see which of your 300 tools are actually being used and which are dead weight. That data helps you decide what to prune or group.

It's not a full solution to dynamic tool discovery (which the spec needs to address), but it's a practical layer you can drop in today without changing your server code.

A useful way to structure this is to separate the MCP catalog from the model-visible working set.

tools/list pagination and notifications/tools/list_changed keep the client's catalog synchronized, but they do not by themselves solve the model-context cost. If the host eventually places all 400 schemas in the model request, pagination only changed the wire/discovery step. MCP deliberately does not prescribe how a host exposes tools to its model.

For the stale-history problem, I would make the working set monotonic within a conversation:

catalog = all MCP tools
visible = {discover_tools}
pinned = {}

on discover(ids):
    visible |= policy_filter(ids)

on tool_call(name):
    pinned.add(name)

before each model turn:
    visible = baseline | pinned | top_k(relevant_tools, remaining_budget)

on new conversation or safe compaction:
    pinned.clear()

In practice, pin a tool as soon as it is called or becomes part of an unresolved plan. Do not evict it merely because the relevance ranking changed on the next turn. Only reset the pinned set at a real conversation boundary or after compaction has preserved the required tool state explicitly.

If a pinned capability truly becomes unavailable because of auth, deployment, or configuration changes, keep a small tombstone adapter under the same model-visible name. It can return a structured unavailable result with the recovery action (discover_tools, re-authenticate, etc.). That is much easier for the model to recover from than a tool name that exists in its history but has disappeared from the current tool set.

I would therefore keep these layers separate:

1. Canonical catalog: MCP tools/list, refreshed on list_changed.
2. Policy layer: authorization, risk, tenant, and human-approval constraints.
3. Conversation working set: baseline discovery tool + pinned tools + a budgeted relevant top-K.

If you use a generic invoke_capability fallback, validate its capability ID and arguments against the original schema, apply the original authorization/risk policy, and log the original tool identity. Otherwise the compact wrapper can accidentally erase the safety benefits of typed tool definitions.

One forward-compatibility note: the current draft direction moves away from connection/session-specific mutation of list operations and toward explicit server-minted handles for cross-call state. So I would implement per-conversation selection in the host/gateway, not by mutating the server's canonical tools/list for each conversation.

References:

• Stable tools specification: https://modelcontextprotocol.io/specification/2025-11-25/server/tools
• Draft changelog: https://modelcontextprotocol.io/specification/draft/changelog
• Related protocol change (SEP-2567): SEP-2567: Sessionless MCP via Explicit State Handles #2567

On the specific worry — "if we do it with MCPs, we lose the [dynamic tooling] flexibility" — you don't have to. The move is to decouple the catalog (every tool your servers expose) from the model-visible working set (what goes into the request), and do that decoupling in a gateway/proxy that sits between the client and your MCP servers. All servers stay connected; the gateway just decides which tools appear in the tools/list it returns to the model, and uses notifications/tools/list_changed to swap that set live. Nothing in the MCP spec forces the model-facing list to equal the union of everything connected — that's a client/gateway policy choice, so MCP interop and your dynamic-working-set idea aren't mutually exclusive.

Two things worth separating in this thread, since they get conflated:

• Latency/orchestration scales mostly with server count and how you shard domains — the listChanged + domain-scoping points others made cover this.
• Token cost scales with the tool count in each request, which is purely a function of what the working set contains — independent of how many servers are connected behind the gateway.

Tradeoff vs. your briefs + fetch_tool approach: a deterministic, policy-driven filter in the gateway (scope by domain / recent-use / task tag) costs zero extra model round-trips, but it can't have the model self-select from a brief. fetch_tool lets the model reach into the long tail, at the price of an extra turn. In practice they compose well — policy-filter the obvious 90% down to a small always-visible set, keep a fetch_tool-style escape hatch for the rest.

The tools/list-filtering-in-a-gateway piece is small to build (I have a stdlib-Go reference that does per-policy tools/list response filtering + audit, zero server changes: https://github.com/bharat3645/mcp-gateway-lite) — the harder, unsolved part is the policy that picks the working set, which is where the domain knowledge in this thread actually matters.

@tzookb

We want to build an MCP for each service integration, but this can easily go to 300-400 tools.
The other approach we have is dynamic tooling... If we do it with mcps, we are losing this flexibility.

You do not lose that flexibility on MCP. Earlier replies already cover listChanged, a gateway that filters the model-visible set, and splitting by domain. One extra pattern that does not need a proxy:

Ship a static small surface, and only enable the rest behind an explicit switch.

The server still has every tool in code. tools/list simply does not advertise the heavy set unless the operator asked for it. No listChanged dance, no extra process. The model never sees 300 schemas.

On patchloom we do this with PATCHLOOM_MCP_SURFACE=core|full:

• core is the default. About a dozen tools (read/replace/search/batch/plan, plus list_files). Enough for an agent to edit a workspace.
• full adds the long tail (AST, markdown section ops, tidy, schema, …). Same binary, same stdio/HTTP server.
• Invalid surface values are rejected (error_kind: invalid_input) instead of silently falling back to “everything.”

That is complementary to the gateway pattern, not a replacement. Use a gateway when many third-party servers are already connected and you cannot change them. Use a core/full (or domain) switch when you own the server and can keep 80% of calls on a short list.

Practical split that has worked for us:

1. Always-on: discover, read, exact replace, search, multi-file plan.
2. Opt-in: AST rename, formatters, rare mutators.
3. Never dump examples into inputSchema. Put those in tool descriptions or a help / agent-rules resource.

tools/list pagination only shrinks the wire. If the host still injects every schema into the model, token cost stays high. The host or your server must keep the advertised list small.

One empirical data point for the catalog-vs-working-set pattern, and specifically for the stale-history problem mentioned above:

I implemented the host/gateway variant in DeepSeek Harness. The model-facing names never change: it always sees mcp_search and mcp_call. Search returns the chosen remote tool's exact inputSchema; call routes an explicit server/tool plus arguments. Because the concrete MCP tool name is data rather than a temporarily registered model-facing function, an old conversation cannot keep calling a function name that disappeared after listChanged.

I tested the two paths with the same Harness 0.1.0-rc.6, DeepSeek V4 Flash, a 1,000-tool stdio fixture, and three tool-use tasks:

The cost uses a dated 2026-08-15 DeepSeek pricing snapshot. This is only a three-task functional pilot, not a general quality or latency benchmark. The gateway adds a search call, and output tokens were higher in this run (491 → 794).

The important safety tradeoff with a generic call interface is that it can erase the policy boundary of typed tools if implemented naively. In this implementation, search and call share the same default-deny allow/deny policy; the selected exact schema is returned before execution; the original server/tool identity stays in routing and logs; and catalog, discovery, cursor, metadata, timeout, and HTTP-response sizes are bounded.

Source, method, and raw aggregate fields are public here: https://github.com/labmimors/dsh-mcp-lens/blob/main/docs/LIVE_DEEPSEEK_PILOT.md

Disclosure: I am the plugin author. The design is host-specific today, but the stale-name tradeoff and measurement method should transfer to other MCP hosts.