Skip to content

Add a official MCP conformance testsuite, based/referencing the specification #1990

@bmerkle

Description

@bmerkle

Is your feature request related to a problem? Please describe.

  • Currently there seems to be no official MCP conformance test suite.
  • There are some internal tests for different implementations, e.g. fastmcp but no official testsuite.
  • It would be cool if we offer a automated (ci based) MCP conformance testsuite.
  • Ideally the tests should refer to the chapters/requirements in the specification.

Describe the solution you'd like
We do not have to start a point zero because there are already:

  • MCP Inspector / Explorer (reference client)
  • internal testsuites of different MCP implementations e.g. fastmcp
    based on these testsuites, it could be possible to implement a official testsuite.

Describe alternatives you've considered
na

Additional context

  • the MCP community can work together, using the existing testsuites and
  • assemble a official MCP conformance testsuite

Possible test:

  • Server startup
  • Capability discovery
  • Tool schema correctness
  • Tool invocation round-trips
  • Resource listing & fetching
  • Prompt exposure
  • JSON-RPC framing
  • Error propagation
  • Streaming responses
  • Cancellation handling

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions