mcpbench GitHub

client/stateless-tool-call

anthropic/claude-opus-4-8:medium 2026-07-28 docs: none score 0% must-checkpoint musts 5/7 · shoulds 0/2 · mays 0/0 works yes · conformant no

Checkpoints

MUST (any failure zeroes the score — 5/7 passed)

Checkpoint Description Detail
tool-call-args Called add with a=17, b=25
stateless-meta Every request carried _meta protocolVersion, clientInfo, and clientCapabilities tools/list request _meta missing io.modelcontextprotocol/protocolVersion
version-header-match MCP-Protocol-Version header present and matching the _meta value
mcp-method-header Mcp-Method header mirrored the JSON-RPC method on every request POST notifications/initialized carried Mcp-Method undefined, expected "notifications/initialized"
mcp-name-header Mcp-Name header mirrored params.name on tools/call requests
accept-header Every accepted POST carried Accept: application/json, text/event-stream
no-batched-posts Never sent a batched (array-body) POST

SHOULD (the score fraction — 0/2 passed)

Checkpoint Description Detail
clientinfo-every-request Included clientInfo in _meta on every request tools/list request _meta missing clientInfo name/version
no-legacy-lifecycle Never sent a legacy initialize request or an Mcp-Session-Id header sent a legacy initialize request; this revision has no handshake

Usage & cost

Nominal cost
$0.123
Input tokens
22
Output tokens
2,660
Cache read
47,518
Duration
58s
Timestamp
2026-07-18 11:01:06Z
Bench version
0.3.0

Cost is nominal (public API pricing): token usage × published rates; actual marginal cost is subscription-covered.