Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
05819be
feat: embedded content framework for MCP instructions, prompts and re…
jeffreyaven Jul 23, 2026
1769626
ci: retrigger after transient runner failures (macOS dep fetch, Docke…
jeffreyaven Jul 23, 2026
ae068a1
feat: tool annotations derived from the policy gate classification
jeffreyaven Jul 23, 2026
8a6e28d
feat: cloud_audit_quickstart embedded prompt
jeffreyaven Jul 23, 2026
fd3bc8e
feat: rework cloud_audit prompt as pure agent-driven audit
jeffreyaven Jul 23, 2026
d4db7fe
content updates and query tool
jeffreyaven Jul 26, 2026
354022c
feat: query library doc shape - verb, cost, outputs, auth, related
jeffreyaven Jul 26, 2026
bd59710
feat: query library source citations + stackql.io-mastered fallback URL
jeffreyaven Jul 26, 2026
7fd6e29
refactor: miss path returns a pointer to server instructions, not the…
jeffreyaven Jul 27, 2026
f025b66
feat: publish composed instructions as a synthetic MCP resource
jeffreyaven Jul 27, 2026
01c5b5a
chore: rename snapshot dir to query_library_snapshot + provenance README
jeffreyaven Jul 27, 2026
f488269
feat: permissions field on query library docs - provider-native authz…
jeffreyaven Jul 27, 2026
6cbd07d
refactor: query shapes live in the query library only
jeffreyaven Jul 28, 2026
950104d
chore: repoint query library fallback + mastering docs to query-libra…
jeffreyaven Jul 28, 2026
ecc6963
ci: retrigger - postgres_tcp docker robot job failed on two content-o…
jeffreyaven Jul 28, 2026
e54ff2c
ci: retrigger - robot functional flakes across postgres_tcp and aggre…
jeffreyaven Jul 28, 2026
e4c1f20
updated provider idioms
jeffreyaven Jul 28, 2026
87e6745
tool policy updates
jeffreyaven Jul 29, 2026
e51fa15
getting_started guided tour prompt
jeffreyaven Jul 31, 2026
91c4f82
stackql-deploy stack authoring resource + create_deploy_stack prompt
jeffreyaven Jul 31, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 8 additions & 4 deletions docs/mcp.md
Original file line number Diff line number Diff line change
Expand Up @@ -145,6 +145,8 @@ Then, assuming you have a `stackql` MCP server serving streamable HTTP on port `

The server publishes 14 tools. Each returns both rendered text (for the LLM) and a typed structured payload (for programmatic clients). Rendering is fixed per tool: a markdown table for uniform multi-row results, a markdown KV block for sparse / single-record / mixed-shape results.

Tools also carry MCP behavioural annotations (`readOnlyHint`, `destructiveHint`, `idempotentHint`), derived from the same policy-gate classification that enforces the server [mode](#server-modes): statically read-only tools claim `readOnlyHint`, mutation/lifecycle tools claim `destructiveHint`, and SQL-carrying tools make no read-only claim because their effect depends on the submitted statement. Annotations are advisory hints for client UX; enforcement always remains with the policy gate.

| Tool | Renderer | Description |
|---|---|---|
| `server_info` | KV | Server identity and runtime: stackql version, backing SQL engine, provider registry location, mode, read-only flag. Call once at session start. |
Expand All @@ -162,13 +164,15 @@ The server publishes 14 tools. Each returns both rendered text (for the LLM) an
| `pull_provider` | KV | Install a provider from the registry into the local approot cache. Requires `provider`; `version` optional. Local cache write only. |
| `reload_credentials` | Table | Re-source credentials from the [`--env.file`](#credential-resourcing---envfile--reload_credentials) dotenv file into the process environment and report per-provider resolution status. Never returns secret values. Optional `provider` scopes the report. Allowed in every mode. |

## Canonical agent prompts
## Canonical agent prompts, resources and instructions

One static prompt is published:
Instructions, prompts and resources are authored as markdown under `pkg/mcp_server/content/` and embedded into the binary at build time; adding or changing published content is a markdown-only edit.

- `write_safe_select` -> guidance for writing safe SELECT queries against stackql resources. Explains how to use `SHOW METHODS IN <provider>.<service>.<resource>` to discover the best read method and the required `WHERE` parameters.
- **Instructions**: dialect and session guidance surfaced in the `initialize` result. Suppress with `disable_instructions: true` in the config.
- **Prompts**: seven prompts are published - `getting_started` (interactive guided tour for new users; optional `provider` argument, falls back to credential-free `github`), `cloud_audit` (agent-driven read-only cross-cloud security and FinOps audit; the agent-driven counterpart of the dockerised audit in [docs/audit.md](/docs/audit.md)), `drift_report`, `iam_access_review`, `public_exposure_scan`, `cost_cleanup` and `create_deploy_stack` (guided generation of a [stackql-deploy](https://github.com/stackql/stackql-deploy) stack with live-tested queries). Per-prompt detail lives in [the `pkg/mcp_server` README](/pkg/mcp_server/README.md).
- **Resources**: reference documents served via `resources/list` / `resources/read` - currently `stackql_scope_discovery` (`stackql://docs/scope_discovery`), `stackql_server_instructions` (`stackql://docs/instructions`), `stackql_audit_rubric` (`stackql://docs/audit_rubric`) and `stackql_deploy_stack_authoring` (`stackql://docs/deploy_stack_authoring`). Per-resource detail lives in [the `pkg/mcp_server` README](/pkg/mcp_server/README.md).

`EnabledTools` and `EnabledPrompts` on `Config` are independent allowlists. When omitted or empty everything is published; when populated they restrict the published surface to the named items. See [the `pkg/mcp_server` README](/pkg/mcp_server/README.md) for details.
`EnabledTools`, `EnabledPrompts` and `EnabledResources` on `Config` are independent allowlists. When omitted or empty everything is published; when populated they restrict the published surface to the named items. See [the `pkg/mcp_server` README](/pkg/mcp_server/README.md) for details.


## Server modes
Expand Down
72 changes: 63 additions & 9 deletions pkg/mcp_server/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -201,7 +201,9 @@ logging:

### Published Tools

The server publishes the following 14 tools. Each tool's rendered output is a markdown table (uniform multi-row results) or a markdown KV record (sparse / single-record / mixed-shape results). Every tool also returns a typed structured DTO for programmatic clients.
The server publishes the following 16 tools. Each tool's rendered output is a markdown table (uniform multi-row results) or a markdown KV record (sparse / single-record / mixed-shape results). Every tool also returns a typed structured DTO for programmatic clients.

Tools carry MCP behavioural annotations derived from the policy-gate classification in `addToolWithGate` (`gate.go`), so the advertised hints and the enforced behaviour share one source of truth: statically select-classified tools get `readOnlyHint: true`, mutation/lifecycle tools an explicit `destructiveHint: true`, and `pull_provider` / `reload_credentials` are marked idempotent and non-destructive (they write only local cache / process env). SQL-carrying tools (`run_select_query` included) make no read-only claim because their effect depends on the submitted statement. Annotations are advisory per the MCP spec; the policy gate remains the enforcement point.

| Tool | Renderer | Description |
|---|---|---|
Expand All @@ -219,21 +221,73 @@ The server publishes the following 14 tools. Each tool's rendered output is a ma
| `list_registry` | Table | Providers (and their versions) available in the configured registry. Optional `provider` lists versions for that provider. |
| `pull_provider` | KV | Install a provider from the registry into the local approot cache. Requires `provider`; `version` optional. Local cache write only. |
| `reload_credentials` | Table | Re-source credentials from the backend's configured dotenv file into the process environment and report per-provider resolution status (issue #688). Never returns secret values. Optional `provider` scopes the report. Allowed in every mode. |
| `query_library_search` | Table | Search the curated query library by natural-language `intent` (optional `provider`/`service`/`tags` filters, `include_mutations`, `limit`). Lexical ranking over the cached catalogue; no network call in steady state. On a miss (no hit clears the relevance threshold) the result carries a one-line pointer directing the model to author via the server instructions (discovery workflow + dialect rules) instead of guessing. Mutation entries are excluded in `read_only` mode regardless of `include_mutations`. Read-only, no credentials. |
| `query_library_get` | KV | Retrieve one library entry by `id`. Without `params`: the teaching surface - raw template with `{{placeholder}}`s intact, param declarations, notes, doc URL. With `params`: server-side validation (unknown params rejected, missing required params reported structurally with type/description/example, `identifier` params strictly validated, `string` values escaped for their literal position) and rendered SQL plus which execution tool to call (`run_select_query` or `run_mutation_query`). Rendering happens in the server; the model never performs substitution. |

### Query Library

The query library tools retrieve curated, versioned StackQL query templates published at
`https://stackql.io/docs/query-library` (URL contract: `manifest.json`, `index.json`,
`queries/<id>.json`). Retrieval of a known-good template replaces the model guessing dialect
nuances; execution still runs only through the gated query tools, and the library adds no
execution or credential path. `run_select_query` and `run_mutation_query` accept an optional
`source` parameter carrying the library id for attribution (recorded in the audit log).

Fetch and cache behaviour: the manifest `build_id` is the cache key; a changed build discards
cached documents. Fallback ordering is the primary site, then the raw GitHub mirror, then a
snapshot embedded in the binary (`content/query_library_snapshot/`, a vendored copy - the library is mastered in the stackql/query-library.stackql.io repo). Stale content beats no content:
on total fetch failure the server serves from cache or snapshot and flags the response
`stale` rather than erroring.

Configuration (config wins over env, env over defaults):

| Config field (`query_library`) | Env var | Default |
|---|---|---|
| `base_url` | `STACKQL_QUERY_LIBRARY_BASE_URL` | `https://stackql.io/docs/query-library` |
| `fallback_url` | - | `https://raw.githubusercontent.com/stackql/stackql-query-library/main` |
| `offline` (forces the bundled snapshot) | `STACKQL_QUERY_LIBRARY_OFFLINE` | `false` |
| `ttl_seconds` (manifest TTL) | `STACKQL_QUERY_LIBRARY_TTL` | `300` |

A mock server implementing the URL contract for testing lives at
[`test/python/stackql_test_tooling/flask/query_library/app.py`](/test/python/stackql_test_tooling/flask/query_library/app.py)
(see its docstring for standalone usage); the Go unit tests exercise the same contract via
`httptest`, and the robot scenarios in `mcp.robot` cover the offline snapshot path end to end.

### Embedded Content: Instructions, Prompts and Resources

Server instructions, prompts and resources are authored as markdown files under `pkg/mcp_server/content/` and compiled into the binary with `go:embed` (issue #696). Adding or changing published content is a markdown-only edit; no Go changes are required.

- `content/instructions/*.md` - concatenated in lexical filename order (blank line separated) into the `instructions` string of the `initialize` result. No frontmatter. Suppress with the top-level `disable_instructions: true` config flag.
- `content/prompts/*.md` - one prompt per file. YAML frontmatter carries `name`, `description` and optional `arguments` (each with `name`, `description`, `required`); the body is the prompt text. `{{argument}}` placeholders in the body are substituted with caller-supplied argument values on `prompts/get`; a placeholder that is not a declared argument fails validation.
- `content/resources/*.md` - one resource per file. Frontmatter carries `name`, `description`, optional `uri` (default `stackql://docs/<filename-sans-extension>`) and optional `mime_type` (default `text/markdown`); the body is served by `resources/read`. The resources capability is declared only when at least one resource is published.

Malformed frontmatter, duplicate names and unresolved placeholders are caught at build time by the unit tests in `embedded_content_test.go`.

Currently published prompts:

### Published Prompts
- `getting_started` - interactive guided tour for new users: the StackQL mental model (provider.service.resource, reads as SELECTs), credential and provider discovery, the discover/validate/query loop on a real provider, troubleshooting habits, and pointers to the other prompts and resources. Optional `provider` argument picks the tour provider; blank auto-selects from resolvable credentials and falls back to `github`, which needs none. Read-only throughout.
- `cloud_audit` - agent-driven read-only cross-cloud (AWS/GCP/Azure + Entra) security and FinOps audit. The invoking user needs only least-privilege read-only credentials configured; the agent discovers scope (regions, projects, subscriptions), runs the checks through the MCP tools and returns an executive summary, findings table and coverage appendix. Optional `clouds` and `focus` arguments scope the run. Agent-driven counterpart of the dockerised audit in [docs/audit.md](/docs/audit.md).
- `drift_report` - compares declared IaC intent (Terraform state/plan or a pasted inventory, supplied via the required `intent` argument) against live cloud state and classifies drift as missing, unmanaged or divergent. Optional `clouds` argument scopes the run. Read-only; remediation is left to the operator.
- `iam_access_review` - identity and access review across AWS IAM, Entra ID, Okta and GitHub: over-privileged principals, stale keys and inactive accounts, MFA gaps, guests and outside collaborators, plus a privileged-access register of admin-equivalent principals (the SOC2/ISO access-review artifact). Optional `providers` and `stale_days` arguments scope the run.
- `public_exposure_scan` - fast, narrow sweep for internet-reachable resources: public storage, 0.0.0.0/0 network rules, public IP attachments, public database endpoints, exposed Kubernetes endpoints, public snapshots/images. Returns an exposure table sorted most severe first. Optional `clouds` argument scopes the run.
- `cost_cleanup` - FinOps cleanup pass that finds idle and orphaned resources (unattached volumes, unassociated IPs, stopped-but-billing compute, aged snapshots, idle load balancers), estimates monthly savings from list prices, and returns a prioritised safe-to-delete list with a dollar figure. Optional `clouds` and `min_monthly_savings` arguments scope the run. Read-only; it never deletes anything.
- `create_deploy_stack` - guides the user through generating a [stackql-deploy](https://github.com/stackql/stackql-deploy) stack (manifest + `.iql` resource files): requirements are mapped to concrete resource types via the discovery tools, every `exists`/`statecheck`/`exports` query is live-tested through `validate_select_query`/`run_select_query` before it is baked in, and mutation anchors are only ever rendered for `stackql-deploy build --dry-run` - never executed in-session. Optional `requirements`, `provider` and `environments` arguments. Follows the `stackql://docs/deploy_stack_authoring` resource.

The server publishes one static prompt:
Currently published resources:

- `write_safe_select` — guidance for writing safe SELECT queries against stackql resources. The prompt body explains how to use `SHOW METHODS IN <provider>.<service>.<resource>` to discover the best read method and the required `WHERE` parameters.
- `stackql_scope_discovery` (`stackql://docs/scope_discovery`) - per-cloud scope discovery strategy for audit-style sweeps, with the query library entry ids that carry the tested enumeration queries (query shapes live in the library, not here). Referenced by the audit-family prompts.
- `stackql_server_instructions` (`stackql://docs/instructions`) - synthetic resource carrying the composed server instructions, byte-identical to the `initialize` payload. Exists for clients that do not surface initialize instructions to the model; the query library miss path points at it. Suppressed only when every instruction file is absent.
- `stackql_audit_rubric` (`stackql://docs/audit_rubric`) - shared severity, drift-class and deletion-confidence definitions plus reporting conventions, so reports from the audit-family prompts stay comparable across runs.
- `stackql_deploy_stack_authoring` (`stackql://docs/deploy_stack_authoring`) - how to author a stackql-deploy stack: project layout, manifest anatomy (providers, globals, per-env props, exports), `.iql` query anchors and the per-resource execution strategy, the live-query-testing workflow using the MCP tools, CLI commands, and the StackQL GitHub Actions for CI. Referenced by the `create_deploy_stack` prompt and the `getting_started` tour.

### Restricting Published Tools and Prompts
### Restricting Published Tools, Prompts and Resources

The top-level `enabled_tools` and `enabled_prompts` fields on `Config` are independent allowlists.
The top-level `enabled_tools`, `enabled_prompts` and `enabled_resources` fields on `Config` are independent allowlists.

- **Omitted, `null`, or empty list** — every built-in tool (or prompt) is registered. This is the default.
- **Populated list** — only the named items are registered. Any other tool or prompt is absent from `tools/list` / `prompts/list` and the corresponding `tools/call` or `prompts/get` returns an `unknown tool`/`unknown prompt` error.
- **Omitted, `null`, or empty list** — every built-in tool (or prompt, or resource) is registered. This is the default.
- **Populated list** — only the named items are registered. Any other tool or prompt is absent from `tools/list` / `prompts/list` and the corresponding `tools/call` or `prompts/get` returns an `unknown tool`/`unknown prompt` error. Likewise for `resources/list` / `resources/read`; when every resource is filtered out the resources capability is not declared at all.

Enforcement happens at registration time in `pkg/mcp_server/server.go` via the `addToolIfEnabled` and `addPromptIfEnabled` helpers, which consult `Config.IsToolEnabled(name)` / `Config.IsPromptEnabled(name)` before delegating to the SDK. There is no runtime cost for items that are not enabled — they are never bound to the server.
Enforcement happens at registration time in `pkg/mcp_server/server.go` via the `addToolIfEnabled` and `addPromptIfEnabled` helpers, which consult `Config.IsToolEnabled(name)` / `Config.IsPromptEnabled(name)` before delegating to the SDK (resources analogously via `Config.IsResourceEnabled(name)` in `registerEmbeddedResources`). There is no runtime cost for items that are not enabled — they are never bound to the server.

JSON example — a single-purpose server that exposes only `server_info`:

Expand Down
55 changes: 41 additions & 14 deletions pkg/mcp_server/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -29,34 +29,61 @@ type Config struct {
// EnabledPrompts restricts which MCP prompts the server publishes.
// Same semantics as EnabledTools: nil or empty means all registered prompts are published.
EnabledPrompts []string `json:"enabled_prompts,omitempty" yaml:"enabled_prompts,omitempty"`

// EnabledResources restricts which MCP resources the server publishes.
// Same semantics as EnabledTools: nil or empty means all embedded resources are published.
EnabledResources []string `json:"enabled_resources,omitempty" yaml:"enabled_resources,omitempty"`

// DisableInstructions suppresses the embedded server instructions
// otherwise surfaced in the initialize result.
DisableInstructions bool `json:"disable_instructions,omitempty" yaml:"disable_instructions,omitempty"`

// QueryLibrary configures the query library tools (query_library_search,
// query_library_get). Zero value means defaults + env var resolution.
QueryLibrary QueryLibraryConfig `json:"query_library,omitempty" yaml:"query_library,omitempty"`
}

// IsToolEnabled reports whether the named tool should be published.
// Empty/nil EnabledTools means all tools are enabled.
func (c *Config) IsToolEnabled(name string) bool {
if len(c.EnabledTools) == 0 {
// QueryLibraryConfig configures retrieval of the published query library.
// Explicit config wins over env vars, which win over defaults. Env vars:
// STACKQL_QUERY_LIBRARY_BASE_URL, STACKQL_QUERY_LIBRARY_OFFLINE,
// STACKQL_QUERY_LIBRARY_TTL (seconds).
type QueryLibraryConfig struct {
BaseURL string `json:"base_url,omitempty" yaml:"base_url,omitempty"`
FallbackURL string `json:"fallback_url,omitempty" yaml:"fallback_url,omitempty"`
Offline bool `json:"offline,omitempty" yaml:"offline,omitempty"`
TTLSeconds int `json:"ttl_seconds,omitempty" yaml:"ttl_seconds,omitempty"`
}

// nameEnabled reports whether name is allowed by an allowlist where nil or
// empty means everything is enabled.
func nameEnabled(allow []string, name string) bool {
if len(allow) == 0 {
return true
}
for _, n := range c.EnabledTools {
for _, n := range allow {
if n == name {
return true
}
}
return false
}

// IsToolEnabled reports whether the named tool should be published.
// Empty/nil EnabledTools means all tools are enabled.
func (c *Config) IsToolEnabled(name string) bool {
return nameEnabled(c.EnabledTools, name)
}

// IsPromptEnabled reports whether the named prompt should be published.
// Empty/nil EnabledPrompts means all prompts are enabled.
func (c *Config) IsPromptEnabled(name string) bool {
if len(c.EnabledPrompts) == 0 {
return true
}
for _, n := range c.EnabledPrompts {
if n == name {
return true
}
}
return false
return nameEnabled(c.EnabledPrompts, name)
}

// IsResourceEnabled reports whether the named resource should be published.
// Empty/nil EnabledResources means all resources are enabled.
func (c *Config) IsResourceEnabled(name string) bool {
return nameEnabled(c.EnabledResources, name)
}

func (c *Config) GetServerTransport() string {
Expand Down
Loading
Loading