Scenarios

Each scenario runs per (spec version × docs condition). Client scenarios: the agent must call an instrumented reference server conformantly. Server scenarios: the agent builds a server that survives our conformance probes. Checkpoints are asserted from runner-side observation only.

client/bad-version-response

client 2025-11-25

sent-initialize MUST — Sent an initialize request
disconnect-on-mismatch SHOULD — After the server negotiated an unsupported protocol version, sent nothing further

client/composite

client 2025-11-25

publish-report MUST — Called publish_report with the summary derived from the prompt format and the changelog values
prompt-fetched MUST — Retrieved the release_report prompt via prompts/get
resource-read MUST — Read the changelog resource via resources/read with params.uri
accept-header MUST — Every accepted POST carried Accept: application/json, text/event-stream

client/discover-instructions

client 2026-07-28

discover-called MUST — Issued server/discover to obtain the server's guidance
tool-call-passphrase MUST — Called unlock_vault with the passphrase from the instructions
stateless-meta MUST — Every request carried the required _meta fields
clientinfo-every-request SHOULD — Included clientInfo in _meta on every request
version-header-match MUST — MCP-Protocol-Version header present and matching the _meta value
mcp-method-header MUST — Mcp-Method header mirrored the JSON-RPC method on every request
no-legacy-lifecycle SHOULD — Never sent a legacy initialize request or an Mcp-Session-Id header

client/drop-stream-mid-call

client 2025-11-25

tool-call MUST — Called generate_report
reconnect-get MUST — After the dropped stream, reconnected via GET and received an event stream
last-event-id SHOULD — The reconnect GET carried Last-Event-ID with the last received event id
retry-delay MUST — Waited at least the server's SSE retry interval before reconnecting

client/elicit-form

client 2025-11-25

capability-declared MUST — Declared the elicitation capability during initialization
tool-call-args MUST — Called book_meeting_room with room "Aurora"
elicit-accept-valid MUST — Sent a schema-valid accept (attendees=6, catering=coffee)
elicit-response-correlated MUST — The elicitation response id matched the server-issued request id

client/elicit-url

client 2025-11-25

capability-url-declared MUST — Declared elicitation.url in the initialize capabilities
elicit-url-accept MUST — Returned an accept response for the url elicitation
accept-content-omitted SHOULD — The url-mode accept omitted the content field
url-visited-after-accept MUST — Fetched the elicitation URL after accepting
web-flow-completed MUST — Completed the hosted web flow with the correct linking code
secret-never-in-band SHOULD — The linking code never appeared in any MCP request body

client/ema-id-jag-flow

client 2025-11-252026-07-28

id-jag-token-exchange MUST — Performed an RFC 8693 token exchange at the IdP with subject_token(+type), requested_token_type=id-jag, audience, and resource
jwt-bearer-grant MUST — Redeemed the IdP-issued ID-JAG at the AS via the jwt-bearer grant with client_secret_basic
no-authorize-endpoint MUST — Never requested the AS authorization endpoint
exchange-before-grant MUST — The successful IdP token exchange preceded the successful AS grant
capability-declared SHOULD — Declared the enterprise-managed-authorization extension capability
bearer-token MUST — All post-grant accepted MCP requests carried Authorization: Bearer; the token never appeared in a URL
authorized-tool-call MUST — Successfully called get_org_report once authorized

client/get-stream

client 2025-11-25

get-stream-accept MUST — Opened a GET stream with Accept: text/event-stream
report-word MUST — Called report_word with the announced word ("papaya")

client/header-mirroring

client 2026-07-28

read-resource MUST — Read the config resource
tool-call MUST — Called traducteur-météo with the word and language from the resource
prompt-get MUST — Fetched the greeting prompt
mcp-method-header MUST — Mcp-Method header mirrored the JSON-RPC method on every request
mcp-name-all-primitives MUST — Mcp-Name mirrored params.name/params.uri on tools, resources, and prompts
mcp-name-base64-sentinel MUST — The non-ASCII tool name was carried via the =?base64?...?= sentinel
stateless-meta MUST — Every request carried the required _meta fields
version-header-match MUST — MCP-Protocol-Version header present and matching the _meta value
accept-header MUST — Every accepted POST carried Accept: application/json, text/event-stream

client/initialize-handshake

client 2025-11-25

initialize-first MUST — The first non-ping JSON-RPC request was initialize
initialize-request MUST — Sent initialize with protocolVersion, capabilities, and clientInfo
initialize-version-latest SHOULD — initialize protocolVersion was the targeted spec version
accept-header MUST — Every accepted POST carried Accept: application/json, text/event-stream
initialized-notification MUST — Sent notifications/initialized after the initialize response
tools-list MUST — Called tools/list after completing the handshake
no-premature-requests SHOULD — No non-ping request before the initialize response completed

client/mrtr-basic

client 2026-07-28

tool-call-args MUST — Called create_deploy_key with repo "mcpbench/demo"
final-complete MUST — A retry carrying inputResponses received the complete result
retry-is-original-request MUST — The retry re-issued the original request (same tool name and arguments)
input-responses-correlated MUST — The retry's inputResponses carried a valid accepted ElicitResult for gh_user (name=octocat)
request-state-echoed-verbatim MUST — The retry echoed the issued requestState byte-for-byte
retry-new-jsonrpc-id MUST — The retry used a fresh JSON-RPC id
stateless-meta MUST — Every request carried the required _meta fields

client/mrtr-elicit-url

client 2026-07-28

capability-url-declared MUST — Every accepted tools/call declared elicitation.url in _meta clientCapabilities
input-response-accept MUST — The retry's inputResponses carried an accept for the url request
accept-content-omitted SHOULD — The url-mode accept omitted the content field
state-echoed-verbatim MUST — Retries echoed the issued requestState byte-for-byte
url-visited MUST — Fetched the elicitation URL after the url input request
web-flow-completed MUST — Completed the hosted web flow with the correct linking code
final-complete MUST — A retry carrying inputResponses received the complete result
secret-never-in-band SHOULD — The linking code never appeared in any MCP request body

client/mrtr-multi-round

client 2026-07-28

tool-call-args MUST — Called provision_account with plan "team"
final-complete MUST — A retry carrying inputResponses received the complete result
round1-input-correlated MUST — First retry carried an accepted ElicitResult for org (org=acme)
round2-input-correlated MUST — Second retry carried an accepted ElicitResult for region (region=eu-west)
state-rotation-echo MUST — Each retry echoed the most recently issued requestState byte-for-byte
retry-args-stable-across-rounds MUST — Both retries re-issued the original request (same name and arguments)
all-ids-distinct MUST — The initial call and both retries used pairwise-distinct JSON-RPC ids
stateless-meta MUST — Every request carried the required _meta fields

client/mrtr-no-state

client 2026-07-28

ca-info-called MUST — Called get_ca_info
certificate-tool-success MUST — Called request_certificate for "demo.example.com"
final-complete MUST — The certificate retry carrying inputResponses received the complete result
no-fabricated-request-state MUST — The retry did not fabricate a requestState (none was issued)
retry-isolation MUST — MRTR fields never leaked onto unrelated requests (e.g. get_ca_info)
absent-result-type-is-complete MUST — Treated the resultType-less get_ca_info result as complete (never retried it)
retry-input-correlated MUST — The retry carried an accepted ElicitResult for contact (email=ops@example.com)
retry-new-jsonrpc-id MUST — The retry used a fresh JSON-RPC id

client/no-batching

client 2025-11-25

square-4 MUST — Called square with n=4
square-7 MUST — Called square with n=7
square-12 MUST — Called square with n=12
no-batched-posts MUST — Never sent a batched (array-body) POST

client/oauth-cimd

client 2025-11-252026-07-28

metadata-discovery MUST — Fetched the protected-resource and authorization-server metadata
cimd-url-client-id SHOULD — The exchanged authorization used a URL-formatted client_id
cimd-document-hosted MUST — The AS fetched the hosted client metadata document successfully
cimd-document-shape MUST — The document carried client_id, client_name, and redirect_uris
cimd-client-id-match MUST — The document's client_id exactly matched its URL
redirect-uri-registered MUST — The presented redirect_uri was listed in the document (localhost or HTTPS)
pkce MUST — S256 code_challenge on authorize; matching code_verifier at token
resource-indicator MUST — Included RFC 8707 resource=<canonical server URI> in both requests
token-request-client-id SHOULD — The token request carried the same URL client_id as the authorize request
bearer-token MUST — All post-grant accepted MCP requests carried Authorization: Bearer; the token never appeared in a URL
authorized-tool-call MUST — Successfully called get_secret_number once authorized

client/oauth-cimd-over-dcr

client 2025-11-252026-07-28

cimd-preferred SHOULD — The exchanged authorization used a URL client_id, not a DCR-issued id
cimd-document-hosted MUST — If the CIMD path was taken, the AS fetched the hosted document successfully
cimd-document-shape MUST — If the CIMD path was taken, the document carried the required properties
cimd-client-id-match MUST — If the CIMD path was taken, the document's client_id matched its URL exactly
pkce MUST — S256 code_challenge on authorize; matching code_verifier at token
resource-indicator MUST — Included RFC 8707 resource=<canonical server URI> in both requests
bearer-token MUST — All post-grant accepted MCP requests carried Authorization: Bearer; the token never appeared in a URL
authorized-tool-call MUST — Successfully called get_secret_number once authorized

client/oauth-dcr-application-type

client 2026-07-28

dcr-application-type-present MUST — The exchanged registration declared application_type native|web
dcr-application-type-native SHOULD — The registration used application_type "native" (localhost redirect)
registration-failure-recovered MAY — If a registration was rejected, a later registration succeeded
pkce MUST — Authorization request carried code_challenge (S256) and token exchange the matching secret
resource-indicator MUST — Included RFC 8707 resource=<canonical server URI> in authorization and token requests
authorized-tool-call MUST — Successfully called get_secret_number once authorized

client/oauth-iss-mismatch

client 2026-07-28

metadata-discovery MUST — Fetched the authorization-server metadata (and protected-resource metadata)
authorize-attempted MUST — A well-formed authorization request was served its redirect
iss-rejected-no-code-exchange MUST — Rejected the wrong-iss authorization response: the code never reached /token
no-poisoned-bearer MUST — Never sent a Bearer token from this run's token endpoint to the MCP server
pkce MUST — The authorization request carried an S256 code_challenge

client/oauth-pkce

client 2025-11-252026-07-28

unauthorized-first MAY — Received the initial 401 unauthenticated and did not retry unauthenticated indefinitely
metadata-discovery MUST — Performed per-version authorization metadata discovery (default well-known paths on 03-26; WWW-Authenticate → PRM → RFC 8414 on 06-18; PRM well-known fallback allowed on 11-25)
dynamic-registration MAY — Registered a client via RFC 7591 dynamic client registration
pkce MUST — Authorization request carried code_challenge (S256) and token exchange carried the matching code_verifier
resource-indicator MUST — Included RFC 8707 resource=<canonical server URI> in authorization and token requests
state-param SHOULD — The exchanged authorization request carried a state parameter
redirect-uri-valid MUST — Used a registered localhost-or-HTTPS redirect_uri
bearer-token MUST — All post-grant accepted MCP requests carried Authorization: Bearer; the token value never appeared in any URL
authorized-tool-call MUST — Successfully called get_secret_number once authorized (Bearer request, <400 response)
stateless-meta MUST — Every accepted request carried _meta protocolVersion and clientCapabilities
version-header-match MUST — MCP-Protocol-Version header present and matching the _meta value
mcp-method-header MUST — Mcp-Method header mirrored the JSON-RPC method on every accepted request
mcp-name-header MUST — Mcp-Name header mirrored params.name on the authorized tools/call
no-legacy-lifecycle SHOULD — Never sent a legacy initialize request or an Mcp-Session-Id header

client/oauth-refresh-token

client 2026-07-28

grant-types-refresh SHOULD — Registration grant_types included refresh_token
refresh-token-confidential MUST — Refresh token values only ever appeared in POST bodies to the token endpoint
bearer-token MUST — All post-grant accepted MCP requests carried Authorization: Bearer; no token in any URL
recovered-after-expiry MUST — A tool call succeeded with a token issued after the first grant
finish-job-success MUST — Successfully called finish_job while authorized
begin-job-success MUST — Successfully called begin_job while authorized

client/param-headers

client 2026-07-28

tool-call MUST — Called execute_sql with the query, region, and comment
param-headers-mirrored MUST — Mirrored annotated parameters into Mcp-Param-* headers
param-header-base64 MUST — The non-ASCII comment value used the =?base64?...?= sentinel
param-header-omitted MUST — Never sent Mcp-Param-Trace-Id (the parameter was unset)
invalid-tool-never-called MUST — Never called the tool whose x-mcp-header annotation is invalid
mcp-method-header MUST — Mcp-Method header mirrored the JSON-RPC method on every request
mcp-name-header MUST — Mcp-Name header mirrored params.name on tools/call requests
stateless-meta MUST — Every request carried the required _meta fields

client/pkce-unsupported-as

client 2025-11-252026-07-28

metadata-discovery MUST — Fetched the authorization-server metadata (and protected-resource metadata)
refused-authorization MUST — Never sent an authorization request to an AS that does not advertise PKCE support

client/prompts-get

client 2025-11-25

report-content MUST — Called report with the exact prompt message text
prompts-get-args MUST — Sent prompts/get with name release_notes and arguments {version: "2.4.1"}
prompts-list-first MUST — A successful prompts/list preceded the successful prompts/get
session-header MUST — Echoed the issued Mcp-Session-Id on every subsequent request
protocol-version-header MUST — Sent MCP-Protocol-Version on every post-initialize request
accept-header MUST — Every accepted POST carried Accept: application/json, text/event-stream

client/protocol-version-header

client 2025-11-25

tool-call MUST — Called lookup with term "idempotent"
version-header-post-init MUST — Sent a valid MCP-Protocol-Version header on every accepted request after initialize
version-header-negotiated SHOULD — The header value matched the negotiated spec version on every request
version-header-value SHOULD — The tools/call header value matched the negotiated spec version exactly

client/resources-read

client 2025-11-25

report-content MUST — Called report with the full text of the Deployment Runbook resource
read-correct-uri MUST — Read the target resource via resources/read with params.uri
pagination-followed MUST — Requested the second resources/list page using the issued nextCursor
cursor-opaque-roundtrip MUST — Every sent cursor was a server-issued token echoed byte-for-byte
stop-at-end SHOULD — Stopped paginating after the response without a nextCursor
session-header MUST — Echoed the issued Mcp-Session-Id on every subsequent request
protocol-version-header MUST — Sent MCP-Protocol-Version on every post-initialize request
accept-header MUST — Every accepted POST carried Accept: application/json, text/event-stream

client/session-echo

client 2025-11-25

first-call MUST — Called echo with message "alpha"
second-call MUST — Called echo with message "bravo"
session-echoed MUST — Echoed the issued Mcp-Session-Id on every post-initialize request
session-delete-on-done SHOULD — Sent DELETE carrying the issued session id when finished
no-session-on-initialize MAY — Did not invent a session id on the initial initialize request

client/session-recovery

client 2025-11-25

first-fetch MUST — Called fetch_record with id "r-100"
reinitialize-after-404 MUST — On session 404, started a fresh initialize without any session id attached
second-fetch MUST — Called fetch_record with id "r-200" after recovering
new-session-echoed MUST — Echoed the newly issued session id after recovery

client/sse-response

client 2025-11-25

accept-event-stream MUST — tools/call POST declared text/event-stream in Accept
tool-call MUST — Called slow_word
sse-handled MUST — Consumed the event-stream response to completion (no premature client abort)
report-word MUST — Called report_word with the word slow_word returned ("hippopotamus")

client/stalled-tool

client 2025-11-25

started-processing MUST — Called process_data
recovered-with-status MUST — Called get_status despite process_data never answering
cancelled-notification SHOULD — Sent notifications/cancelled carrying the stalled request's id

client/stateless-composite

client 2026-07-28

publish-report MUST — Called publish_report with the summary derived from the prompt format and the changelog values
prompt-fetched MUST — Retrieved the release_report prompt via prompts/get
resource-read MUST — Read the changelog resource via resources/read with params.uri
stateless-meta MUST — Every request carried _meta protocolVersion and clientCapabilities
accept-header MUST — Every accepted POST carried Accept: application/json, text/event-stream

client/stateless-prompts

client 2026-07-28

report-content MUST — Called report with the exact prompt message text
prompts-get-args MUST — Sent prompts/get with name release_notes and arguments {version: "2.4.1"}
prompts-list-first MUST — A successful prompts/list preceded the successful prompts/get
stateless-meta MUST — Every request carried _meta protocolVersion and clientCapabilities
clientinfo-every-request SHOULD — Included clientInfo in _meta on every request
mcp-method-header MUST — Mcp-Method header mirrored the JSON-RPC method on every request
mcp-name-header-prompt MUST — Mcp-Name header mirrored params.name on prompts/get requests
version-header-match MUST — MCP-Protocol-Version header present and matching the _meta value
no-legacy-lifecycle SHOULD — Never sent a legacy initialize request or an Mcp-Session-Id header
accept-header MUST — Every accepted POST carried Accept: application/json, text/event-stream

client/stateless-resources

client 2026-07-28

report-content MUST — Called report with the full text of the Deployment Runbook resource
read-correct-uri MUST — Read the target resource via resources/read with params.uri
pagination-followed MUST — Requested the second resources/list page using the issued nextCursor
cursor-opaque-roundtrip MUST — Every sent cursor was a server-issued token echoed byte-for-byte
stateless-meta MUST — Every request carried _meta protocolVersion and clientCapabilities
clientinfo-every-request SHOULD — Included clientInfo in _meta on every request
mcp-method-header MUST — Mcp-Method header mirrored the JSON-RPC method on every request
mcp-name-header-uri MUST — Mcp-Name header mirrored params.uri on resources/read requests
version-header-match MUST — MCP-Protocol-Version header present and matching the _meta value
no-legacy-lifecycle SHOULD — Never sent a legacy initialize request or an Mcp-Session-Id header
accept-header MUST — Every accepted POST carried Accept: application/json, text/event-stream

client/stateless-sse-response

client 2026-07-28

tool-call MUST — Called slow_word
sse-handled MUST — Consumed the event-stream response to completion (no premature client abort)
report-word MUST — Called report_word with the word slow_word returned ("hippopotamus")
stateless-meta MUST — Every request carried _meta protocolVersion, clientInfo, and clientCapabilities
clientinfo-every-request SHOULD — Included clientInfo in _meta on every request
mcp-name-header MUST — Mcp-Name header mirrored params.name on tools/call requests

client/stateless-tool-call

client 2026-07-28

tool-call-args MUST — Called add with a=17, b=25
stateless-meta MUST — Every request carried _meta protocolVersion, clientInfo, and clientCapabilities
clientinfo-every-request SHOULD — Included clientInfo in _meta on every request
version-header-match MUST — MCP-Protocol-Version header present and matching the _meta value
mcp-method-header MUST — Mcp-Method header mirrored the JSON-RPC method on every request
mcp-name-header MUST — Mcp-Name header mirrored params.name on tools/call requests
accept-header MUST — Every accepted POST carried Accept: application/json, text/event-stream
no-batched-posts MUST — Never sent a batched (array-body) POST
no-legacy-lifecycle SHOULD — Never sent a legacy initialize request or an Mcp-Session-Id header

client/task-cancel

client 2026-07-28

tasks-capability-declared MUST — Declared the io.modelcontextprotocol/tasks capability on the tool call
tool-call-args MUST — Called sync_dataset for "telemetry-2026"
cancel-issued MUST — Sent tasks/cancel carrying the issued taskId
no-notifications-cancelled MUST — Never used notifications/cancelled for task cancellation
polled-before-cancel SHOULD — Polled tasks/get at least once before cancelling
poll-interval-respected SHOULD — Consecutive polls respected the server's pollIntervalMs
mcp-name-taskid MUST — tasks/get and tasks/cancel carried Mcp-Name equal to the taskId
no-post-cancel-polling MAY — Did not keep polling tasks/get after cancelling
stateless-meta MUST — Every request carried the required _meta fields

client/task-polling

client 2026-07-28

tasks-capability-declared MUST — Declared the io.modelcontextprotocol/tasks capability on the tool call
tool-call-args MUST — Called generate_report with region "emea" and 4 sections
polled-to-completion MUST — Polled tasks/get on the issued taskId until the completed result
poll-interval-respected SHOULD — Consecutive polls respected the server's pollIntervalMs
mcp-name-taskid MUST — tasks/get requests carried Mcp-Name equal to the taskId
mcp-method-header MUST — Mcp-Method header mirrored the JSON-RPC method on every request
stateless-meta MUST — Every request carried the required _meta fields

client/tool-call-basic

client 2025-11-25

initialize-first MUST — The first non-ping JSON-RPC request was initialize
handshake MUST — Completed initialize + notifications/initialized before the tool call
tool-call-args MUST — Called add with a=17, b=25
accept-header MUST — Every accepted POST carried Accept: application/json, text/event-stream