Skip to content

feat: embedded content framework for MCP instructions, prompts and resources - #697

Merged
jeffreyaven merged 20 commits into
mainfrom
mcp-embedded-content
Jul 31, 2026
Merged

jeffreyaven merged 20 commits into
mainfrom
mcp-embedded-content

Conversation

@jeffreyaven

@jeffreyaven jeffreyaven commented Jul 23, 2026 •

Copy link
Copy Markdown
Member

Implements #696. MCP server instructions, prompts and resources are now authored as markdown under pkg/mcp_server/content/ and embedded into the binary with go:embed; adding or changing published content is a markdown-only edit.

  • Instructions: content/instructions/*.md concatenated in lexical order and surfaced in the initialize result; suppressible via the new top-level disable_instructions config flag.
  • Prompts: YAML frontmatter (name, description, arguments) + markdown body with {{argument}} substitution; published via prompts/list and prompts/get, gated by the existing enabled_prompts allowlist. The write_safe_select prompt is migrated with byte-identical output (pinned by unit test against the legacy constant).
  • Resources: frontmatter with optional uri (default stackql://docs/) and mime_type (default text/markdown); published via resources/list and resources/read, gated by the new enabled_resources allowlist. The resources capability is declared only when at least one resource is published.

Loader validates frontmatter, duplicate names and unresolved placeholders; unit tests fail the build on malformed content. Tool surface unchanged. Binary size delta: +44,112 bytes (~43 KiB) on a ~91 MB linux/amd64 build (+0.05%).

Test plan: unit tests in embedded_content_test.go exercise the loader and the published surface end-to-end through the SDK in-memory client (initialize instructions, prompts/get byte-identity, resources list/read, allowlist gating). No robot scenario: the bundled stackql_mcp_client speaks only tool calls, and extending it would contradict the minimal-touch constraint; existing mcp.robot list_tools scenarios verify the tool surface is unchanged.

Addendum: tool annotations derived from the policy gate

This PR also uplifts the tool surface with MCP behavioural annotations
(readOnlyHint, destructiveHint, idempotentHint), advertised in
tools/list for elicitation-capable clients (Claude Desktop, Cursor) to
shape approval UX.

Design: annotations are defaulted at the addToolWithGate chokepoint from
the same policy.QueryClass that drives mode enforcement, so the advertised
hint and the enforced behaviour share one source of truth and cannot drift:

  • Statically select-classified tools (metadata/discovery tools plus
    validate_select_query and list_registry) claim readOnlyHint: true.
  • SQL-carrying tools (run_select_query included) make no read-only
    claim: their effect depends on the submitted statement, and enforcement
    stays with the policy gate.
  • run_mutation_query / run_lifecycle_operation carry an explicit
    destructiveHint: true. Their previously duplicated registrations are
    factored into registerExecQueryTool (also resolves a dupl lint hit).
  • pull_provider and reload_credentials override the select-class
    default: not read-only (they write local cache / process env), idempotent,
    non-destructive.

Annotations are advisory per the MCP spec; the policy gate remains the
enforcement point. Wire change is additive fields only - tool names, input
and output schemas, and behaviour are unchanged (robot tools/list
assertions are substring-based and unaffected). Covered by
TestTools_AnnotationsDerivedFromGate through the SDK in-memory client.

Addendum: MCP query library tools (query_library_search / query_library_get)

Implements the MCP server side of the query library work order "as if" the published library
at https://stackql.io/docs/query-library exists (site publication happens separately, from
its own repo). Retrieval of a known-good template replaces the model guessing dialect
nuances. The library adds no execution path and no credential path: execution still runs
only through the existing gated query tools.

New tools

  • query_library_search - locate a template by natural-language intent, with optional
    provider / service / tags filters, include_mutations (default false) and limit
    (default 5, max 20). Lexical ranking over the cached catalogue (weighted matching across
    title, description, intent_keywords, keywords, tags, services); steady-state search makes
    no network call. Miss path: when no hit clears the relevance threshold, the response
    carries the nearest neighbours plus a compact dialect guide (the embedded dialect
    instructions), so the model degrades to authoring with the right rules, not guessing.
    Mutation entries are excluded in read_only mode regardless of include_mutations,
    since run_mutation_query is gated off for the session.
  • query_library_get - retrieve one entry by id. Without params: the teaching surface -
    raw template with {{placeholder}}s intact, param declarations, notes, doc URL. With
    params: server-side validation and rendering. Unknown params are rejected; missing
    required params come back structurally with type, description and example; identifier
    params must match ^[A-Za-z_][A-Za-z0-9_-]*$; enum params must match a declared value
    exactly; string values are escaped for the literal position they occupy. The response
    carries the mutation flag and next_tool (run_select_query or run_mutation_query).
    Rendering happens in the server; the model never performs substitution. Values are
    interpolated (with escaping) rather than bound: the execution tools carry SQL text only,
    so there is no bind-parameter transport from this surface (settles the work order's open
    question 1).

Both tools are select-classified through the existing policy gate, carry readOnlyHint,
and are audited like every other tool.

Fetch, cache and fallback

Consumes the URL contract (manifest.json, index.json, queries/<id>.json). The
manifest build_id is the cache key: a changed build discards cached documents. Manifest
TTL defaults to 300s. Fallback ordering: primary site -> GitHub raw mirror -> snapshot
embedded in the binary (pkg/mcp_server/content/query_library/, six seed entries whose
SQL is robot-suite ground truth). Stale content beats no content: on total fetch failure
the server serves from cache or snapshot and flags the response stale instead of erroring.

Configuration (config wins over env, env over defaults): query_library.base_url /
STACKQL_QUERY_LIBRARY_BASE_URL, query_library.offline / STACKQL_QUERY_LIBRARY_OFFLINE
(forces the bundled snapshot), query_library.ttl_seconds / STACKQL_QUERY_LIBRARY_TTL,
plus query_library.fallback_url for enterprise mirrors.

Changes to existing tools

  • run_select_query and run_mutation_query descriptions now direct the model to consult
    query_library_search before composing SQL from scratch.
  • Both accept an optional source parameter carrying the library id when the SQL
    originated from a library entry; recorded in the audit log for attribution without
    correlating event streams.

Testing

  • Go unit suite (pkg/mcp_server/query_library_test.go): an httptest fixture implements
    the full URL contract. Covers ranking, filters, mutation gating, miss path, escaping,
    every validation failure class, steady-state no-network behaviour, build-change cache
    invalidation, fallback tier ordering, stale snapshot serving, offline end-to-end against
    the embedded seed, and path-traversal-shaped id rejection.
  • Mock server for live testing: test/python/stackql_test_tooling/flask/query_library/app.py
    serves the URL contract standalone (usage in its docstring) so the tools can be driven
    against a real HTTP endpoint before the site exists.
  • Robot scenarios (mcp.robot): a dedicated MCP server on port 9917 started with
    STACKQL_QUERY_LIBRARY_OFFLINE=true; black-box search and rendered-get through the real
    binary and MCP client. The exact flow (same binary, client, env and args) was verified
    locally end to end before raising: search returns aws/ec2/regions-enabled as top hit,
    get renders WHERE region = 'us-west-2' and routes to run_select_query.

Deliberately out of scope

  • Telemetry events and dashboard (work order Phase 3).
  • Wiring the flask mock into Start All Mock Servers (that path forks into
    docker-compose-testing.yml on docker-equipped runners; robot coverage uses the snapshot
    tier instead, which exercises the identical tool surface).
  • Embedding-based retrieval, multi-query recipes, view emission (deferred per work order).

…sources

Implements #696. MCP server instructions, prompts and resources are now
authored as markdown under pkg/mcp_server/content/ and embedded into the
binary with go:embed; adding or changing published content is a
markdown-only edit.

- Instructions: content/instructions/*.md concatenated in lexical order
  and surfaced in the initialize result; suppressible via the new
  top-level disable_instructions config flag.
- Prompts: YAML frontmatter (name, description, arguments) + markdown
  body with {{argument}} substitution; published via prompts/list and
  prompts/get, gated by the existing enabled_prompts allowlist. The
  write_safe_select prompt is migrated with byte-identical output
  (pinned by unit test against the legacy constant).
- Resources: frontmatter with optional uri (default
  stackql://docs/<filename>) and mime_type (default text/markdown);
  published via resources/list and resources/read, gated by the new
  enabled_resources allowlist. The resources capability is declared
  only when at least one resource is published.

Loader validates frontmatter, duplicate names and unresolved
placeholders; unit tests fail the build on malformed content. Tool
surface unchanged. Binary size delta: +44,112 bytes (~43 KiB) on a
~91 MB linux/amd64 build (+0.05%).

Test plan: unit tests in embedded_content_test.go exercise the loader
and the published surface end-to-end through the SDK in-memory client
(initialize instructions, prompts/get byte-identity, resources
list/read, allowlist gating). No robot scenario: the bundled
stackql_mcp_client speaks only tool calls, and extending it would
contradict the minimal-touch constraint; existing mcp.robot list_tools
scenarios verify the tool surface is unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@jeffreyaven jeffreyaven self-assigned this Jul 23, 2026
…r Hub login)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
jeffreyaven and others added 18 commits July 24, 2026 08:27
MCP behavioural hints (readOnlyHint, destructiveHint, idempotentHint) are
now advertised in tools/list, defaulted at the addToolWithGate chokepoint
from the same policy.QueryClass that drives mode enforcement, so hint and
enforcement share one source of truth.

- Statically select-classified tools (no SQL input) claim readOnlyHint.
- SQL-carrying tools make no read-only claim: their effect depends on the
  submitted statement; enforcement stays with the policy gate.
- run_mutation_query / run_lifecycle_operation carry an explicit
  destructiveHint (registration deduplicated into registerExecQueryTool,
  which also resolves a dupl lint finding).
- pull_provider and reload_credentials override the select-class default:
  not read-only (local cache / process env writes), idempotent,
  non-destructive.

Annotations are advisory per the MCP spec; the policy gate remains the
enforcement point. Additive wire fields only; tool names, schemas and
behaviour unchanged. Covered by TestTools_AnnotationsDerivedFromGate via
the SDK in-memory client.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Publishes the docker audit quickstart (docs/audit.md) as an MCP prompt
via the embedded content framework: markdown-only addition, no Go
changes. Repo-relative links are flattened to plain paths / absolute
URLs since the prompt is served out of repository context.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replaces the docker-compose quickstart transcription with an
agent-driven audit: the analyst configures least-privilege read-only
credentials, invokes the prompt from an MCP client (eg Claude Desktop),
and the agent runs scope discovery (AWS region sweep + S3 scan, GCP org
descent, Azure management-group descent, Entra identity checks) and the
security/FinOps checks entirely through the MCP tools, returning an
executive summary, findings table and coverage appendix. Optional
clouds/focus arguments exercise the {{argument}} substitution path.
Read-only contract stated in the prompt body; registry provider names
(aws, google, azure, entra_id) verified against the live registry.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Aligns queries/<id>.json with the work-order front matter so the agent can
warn on fan-out cost pre-execution, see output columns without a
describe_resource round trip, verify credentials via the auth env var list,
pivot via related ids, and route EXEC templates to run_lifecycle_operation
(verb wins over the mutation bool). Additive, non-breaking.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Search cites the catalogue origin (source_url), get cites the document
origin - live URL, fallback URL or embedded:// snapshot path - in both the
structured DTO and rendered text. Default fallback now points at the
stackql.io repo raw path (library mastered there; committed build artifacts).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… dialect guide

The dialect rules already ship in the initialize instructions; embedding
them in every search miss cost ~1k redundant tokens. The miss response now
carries a one-line guidance field directing the model to the discovery
workflow and dialect rules, with a size guard in the unit test.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
stackql://docs/instructions serves the exact initialize instructions bytes
as a resource, giving instruction-blind clients a pull-on-demand channel;
the query library miss guidance now cites it. Authored once, exposed twice.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Makes clear the embedded content is a vendored survival-ration copy of the
library (mastered in the stackql.io repo), serving only the offline and
total-fetch-failure tiers - not the master and not ground truth.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… actions

Answers 'am I authorized', not just 'can I authenticate': queries/<id>.json
may declare provider-native permissions (eg ec2:DescribeRegions) which pass
through query_library_get, enabling precise 403 remediation and
least-privilege provisioning. Additive; seeded on the safest entries.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
scope_discovery becomes a strategy resource (per-cloud enumeration approach
plus library entry ids, no inline SQL); the audit-family prompts fetch
enumeration queries via query_library_search/get. The data__ prefix is
reframed in the dialect doc as a minority case (eg generic aws.cloud_control),
not a general body-param pattern.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ry.stackql.io repo

The library is now mastered in stackql/query-library.stackql.io (own repo,
own site, served at stackql.io/docs/query-library via proxy rewrite). The
primary URL is unchanged; only the raw-GitHub fallback tier and the
provenance docs move to the new repo.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…nly commits

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ssive-concurrency variants

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Interactive read-only tour for new users: mental model, credential and
provider discovery, discover/validate/query loop, troubleshooting habits,
pointers to the audit prompts and resources. Optional provider argument,
credential-free github fallback. Refreshes the stale docs/mcp.md prompt
register (write_safe_select was retired) to defer to the pkg README.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- deploy_stack_authoring resource: project layout, manifest anatomy,
  .iql anchors, execution strategy, live-query-testing workflow via the
  MCP tools, CLI commands, StackQL GitHub Actions for CI
- create_deploy_stack prompt: guided stack generation with every
  exists/statecheck/exports query live-tested before it lands; mutation
  anchors rendered for stackql-deploy --dry-run only, never executed
- getting_started wrap-up introduces stackql-deploy and the marketplace
  actions; README + docs/mcp.md registers updated (mcp.md resources
  bullet was stale)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@jeffreyaven
jeffreyaven merged commit c926a33 into main Jul 31, 2026
19 checks passed
@jeffreyaven
jeffreyaven deleted the mcp-embedded-content branch July 31, 2026 05:39
social4hyq pushed a commit to social4hyq/homebrew-core that referenced this pull request Sep 20, 2026
stackql 0.10.582

Created-by: HarmonybrewBot
Commit-by: HarmonybrewBot
Merged-by: HarmonybrewBot
Description: Created by `brew bump`

---

Created with `brew bump-formula-pr`.<details>
  <summary>release notes</summary>
  <pre>## What's Changed
* Install doc updates by @jeffreyaven in stackql/stackql#693
* Mcpb env file arg by @jeffreyaven in stackql/stackql#694
* working-self-audit by @general-kroll-4-life in stackql/stackql#695
* feat: embedded content framework for MCP instructions, prompts and resources by @jeffreyaven in stackql/stackql#697


**Full Changelog**: https://github.com/stackql/stackql/compare/v0.10.559...v0.10.582</pre>
  <p>View the full release notes at <a href="https://github.com/stackql/stackql/releases/tag/v0.10.582">https://github.com/stackql/stackql/releases/tag/v0.10.582</a>.</p>
</details>
<hr>

See merge request: Harmonybrew/homebrew-core!15611
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants