Skip to content

perf(mcp): pin the tools sessions use, unpin the ones they do not - #96

Merged
Gintm merged 2 commits into
developfrom
perf/mcp-pinned-tools-by-usage
Nov 13, 2026
Merged

Gintm merged 2 commits into
developfrom
perf/mcp-pinned-tools-by-usage

Conversation

@Gintm

@Gintm Gintm commented Nov 12, 2026

Copy link
Copy Markdown
Contributor

Why

Every pinned tool definition is sent on every turn of every session: the 53 pinned tools cost 87.8K characters (~22K tokens). Several are almost never called, while three hidden tools are reached through search_tools by 6-13% of sessions, each search costing a model turn and ~14.7K characters (Axiom zernio-mcp, 2 h, 1,226 sessions; Sentry, 7 d).

What

  • Pins messages_list_inbox_conversations (13% of sessions), media_get_media_presigned_url (8%), messages_get_inbox_conversation_messages (6%).
  • Unpins 13 tools with almost no use: the 6 queue_*, the 2 tracking_tags_*, usage_get_usage, validate_post_length, mentions_reply_to_mention, posts_bulk_upload_posts, accounts_move_account_to_profile, account_groups_list_account_groups. They stay reachable through search_tools / call_tool and by direct name.
  • Clients now see 44 tools and 75,530 characters instead of 55 and 89,417 (-15.5%).
  • An unknown tool name now also suggests pinned tools that share its words (search only indexes hidden tools), so a guess like inbox_list_conversations still points to the now-pinned tool.
  • call_tool still reaches newly pinned tools, so agents that already call them by name through call_tool keep working (test).

🤖 Generated with Claude Code

Gintm and others added 2 commits November 12, 2026 18:29
The 53 pinned definitions cost 87.8K characters on every turn of every
session. Pins the inbox conversation reads and the media presigned URL
(searched for by 6-13% of sessions) and moves 13 tools that almost no
session calls (queue, tracking tags, usage, ...) behind search_tools.
Clients now see 44 tools and 75.5K characters instead of 55 and 89.4K.
A guessed name now also suggests pinned tools that share its words.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… guesses

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Gintm

Gintm commented Nov 12, 2026

Copy link
Copy Markdown
Contributor Author

Review (code-review high --fix)

Finding Result
Pinned-name suggestions only split guesses on underscores (camelCase, kebab-case never matched) Fixed, with two guessed-name test cases
The pinned-list comment quoted measured shares that go stale Fixed: the comment keeps the constraint, the numbers live in this PR
A pinned word match can push a spelling-only suggestion out of the top 3 Skipped: it needs a guess sharing two words with a pinned tool, which is then the likelier intended tool
The call_tool reachability test reads a validation message Skipped: it asserts on the argument name, the stable part

QA

Check Result
Real tool catalog (develop + this branch): what clients list 55 tools / 89,417 chars → 44 tools / 75,530 chars
Every pinned name is registered Yes (test)
call_tool still dispatches to a newly pinned tool (agents that call messages_* by name) Yes: reaches the API, no Unknown tool (test and local run)
An unpinned tool is still found by search_tools and callable Yes: queue_list_queue_slots (test and local run)
Full suite 1,805 passed

Metrics

Metric Before (Axiom zernio-mcp, 2 h) Expected after
Tool list sent every turn 89,417 chars 75,530 chars (-15.5%)
search_tools calls per 1,000 sessions 386 (473 searches / 1,226 sessions) lower; sessions that only needed the inbox or presigned URL reads stop searching
Sessions reaching messages_* / presigned URL through call_tool 158 / 69 / 94 move to via=direct

Post-deploy check: same 2 h window a day later in zernio-mcp: searches per 1,000 sessions, and the via split for the three newly pinned tools; control: total sessions and calls of untouched pinned tools (posts_list, accounts_list).

@Gintm
Gintm merged commit 44a95d6 into develop Nov 13, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant