mcpbench GitHub

client/mrtr-no-state

anthropic/claude-opus-4-8:medium 2026-07-28 docs: full score 0% must-checkpoint musts 7/8 · shoulds 0/0 · mays 0/0 works yes · conformant no

Checkpoints

MUST (any failure zeroes the score — 7/8 passed)

Checkpoint Description Detail
ca-info-called Called get_ca_info
certificate-tool-success Called request_certificate for "demo.example.com"
final-complete The certificate retry carrying inputResponses received the complete result
no-fabricated-request-state The retry did not fabricate a requestState (none was issued)
retry-isolation MRTR fields never leaked onto unrelated requests (e.g. get_ca_info)
absent-result-type-is-complete Treated the resultType-less get_ca_info result as complete (never retried it)
retry-input-correlated The retry carried an accepted ElicitResult for contact (email=ops@example.com)
retry-new-jsonrpc-id The retry used a fresh JSON-RPC id tools/call request_certificate reused JSON-RPC id 1 across attempts

Usage & cost

Nominal cost
$0.415
Input tokens
32
Output tokens
4,297
Cache read
278,826
Duration
74s
Timestamp
2026-07-18 11:07:11Z
Bench version
0.3.0

Cost is nominal (public API pricing): token usage × published rates; actual marginal cost is subscription-covered.