The transport and auth tests never reached the extraction helpers on a
successful call, so the mapping from tool kwargs into execute_workflow was
executed by nothing — a renamed kwarg would have passed every test and failed
on the first real extraction.

Adding that coverage surfaced three real problems:

- Tool descriptions promised documents could be passed as any reachable URL.
  The execution serializer accepts *only* S3 pre-signed URLs, so an agent
  following the description would have failed every call. Descriptions, JSON
  schema and README now state the S3 requirement.
- The tags schema advertised an unbounded array; the serializer caps it at
  one. Both limits are now sourced from the serializers rather than restated,
  so the advertised schema tracks what is actually enforced.
- Documents were downloaded before the rate-limit slot was taken, so a call
  about to be rejected still pulled every document over the network. The slot
  is now acquired first, with the fetch inside the try block so a failed fetch
  still releases it.

Also converts a RateLimitExceeded raised from deeper in the stack into an
agent-readable message instead of letting it reach the generic
"failed unexpectedly" branch.

Tests: 34 passing (was 20).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…ment one

The deployment MCP server can only ever expose one workflow: its credential is
an API deployment key, which resolves to no user and grants nothing beyond that
deployment. Reaching the rest of the platform needs a different credential —
and Unstract already has one.

PlatformApiKey (backend/platform_api/) is an org-scoped bearer key that
CustomAuthMiddleware resolves to a service-account User, validating the key's
organization against the URL and rejecting unknown, inactive and wrong-org keys
before any view runs. So the new server does not authenticate at all; it is
mounted where that middleware can do it.

That placement is the security-critical detail. WHITELISTED_PATHS is matched
with startswith, so the deployment server's /mcp/ prefix is exempt from the
middleware while the tenant path is not:

    POST /mcp/<org>/<api_name>/          deployment key, view authenticates
    POST /api/v1/unstract/<org>/mcp/     platform key, middleware authenticates

Moving the latter under the whitelisted prefix would silently remove all
authentication, so a test asserts it is not whitelisted.

Because that auth lives in middleware, the platform auth tests go through
django.test.Client and the real URL. APIRequestFactory — correct for the
deployment server, which owns its auth — bypasses middleware entirely and would
have passed against a completely open endpoint.

Shared JSON-RPC transport extracted to transport.py so the two servers cannot
drift on protocol behaviour; the deployment server's URL, behaviour and 34
tests are unchanged.

The platform tools are read-only by design (readMeFirst, whoami,
listApiDeployments, listWorkflows, listPromptStudioProjects). Listings go
through each model's for_user manager so platform sharing rules apply, and are
capped with an explicit truncation note rather than silently cut.

Two constraints are documented rather than worked around:
- A read-tier key cannot use the server at all: the middleware gates tiers on
  HTTP method and every MCP call is a POST. Special-casing MCP paths in shared
  middleware is a maintainer decision, not this app's to make.
- Service accounts bypass per-user sharing (for_user returns self.all()), so
  listings cover the whole org. whoami and readMeFirst both say so, since an
  agent assuming otherwise would misread a listing.

check_tool_allowed refuses write tools below read_write, and a test pins the
registry as read-only so adding the first write tool is a deliberate act.

Tests: 50 in this app (was 34); full backend unit tier 310 passing.
Registers the mcp-platform-auth critical path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

for more information, see https://pre-commit.ci

…deferred read tools

Four changes.

1. Documents the platform server's exclusions in its README — the
   credential-bearing surfaces and destructive operations no tool wraps, and
   why. Framed as what this server does not do rather than as an inventory of
   which endpoints return secrets, since this repository is public.

2. Adds the billable Prompt Studio operations: indexDocument, fetchResponse,
   bulkFetchResponse and singlePassExtraction. Each delegates to
   PromptStudioCoreView rather than reimplementing run-id generation, profile
   resolution, lookup gating and Celery dispatch — a parallel implementation
   would drift from the UI on the first change to either.

3. Adds a spend guard for them. There is no OSS token budget to gate on
   (usage_v2 reports after the fact; subscription enforcement is enterprise),
   so this is a per-organization *call* budget in Redis. Enforced at the
   transport via a `billable` flag, so a new costly tool cannot be added
   without being budgeted. Three deliberate behaviours: budget is consumed on
   invocation and never refunded — the opposite of the rate-limit slot in
   tools/execution.py, because that models concurrency while this models money
   already spent, and refunding would let an agent spend without limit by
   failing in a loop; exhaustion returns a retryable isError result rather than
   a permission error, so the agent waits instead of giving up; and it fails
   open, because bounding runaway loops is not worth taking the server offline
   when Redis blips. whoami reports remaining budget so an agent can pace
   itself.

4. Adds the read tools previously skipped for no good reason: listExecutions,
   getExecutionDetail, getUsageSummary, getWorkflowEndpoints, listToolInstances,
   listTags and listPipelines. Execution history in particular closes the gap
   where an agent could trigger work but not see how it went.

Two things surfaced while building this, both now fixed:

* A registry-wide test seeds an org with fake secrets, invokes every read tool
  and fails if any appears in the output. It caught getExecutionDetail being
  uncovered, then caught a real leak — an execution's error_message can embed
  the connection string a failed connector tried, which listExecutions returned
  verbatim. Error text is now passed through redact_secrets.
* spend_guard.peek() did not fail open, so whoami broke when the cache was
  unavailable. Now matches consume().

getWorkflowEndpoints is the case to look at for the pattern: a workflow
endpoint references a connector instance whose metadata decrypts on access, so
the tool returns the shape of the connection and never the configuration. All
responses here are built field-by-field from named attributes; never
serializer.data.

Tests: 82 in this app (was 59); full backend unit tier 331 passing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

for more information, see https://pre-commit.ci

…r GET per spec

The four billable Prompt Studio tools each require a document_id, and
fetchResponse also a prompt_id, but nothing in the registry returned either.
An MCP client sees only tools/list — it has no database and cannot invent a
UUID — so the most expensive tools on the server were listed and uncallable.

Adds listPromptStudioDocuments and listPrompts as the producers. Both resolve
through the project rather than by a bare id lookup: the platform key
authenticates as a service account, for which for_user() returns everything,
so the project join is the scoping that actually holds. listPrompts builds its
response field by field instead of via a serializer, because ToolStudioPrompt
carries a profile_manager FK (the LLM, embedding and vector-store adapters)
and a webhook URL that may embed a token — both of which the README promises
never reach an agent, and neither of which was pinned by a test until now.

The durable fix is test_registry_reachability: every id a tool requires must
have a declared producer. The bug class was invisible per tool and only wrong
in aggregate, so the invariant is asserted over the registry.

Also, per the Streamable HTTP spec (rev 2025-06-18), GET opens a
server-to-client SSE stream and a server offering none must answer 405. This
server pushes nothing, so GET now returns 405 with Allow: POST, keeping the
identity body for uptime probes. The "every MCP call is a POST" comments were
imprecise in the same way and now say that every JSON-RPC *message* arrives as
a POST — which is what makes required_method necessary.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

… docs

test_no_credential_leak drives every read tool through a hardcoded argument
map and raises if a tool requires an argument the map lacks. Both new tools
require project_id, so without an entry the sweep would have failed outright —
and the whole reason listPrompts builds its response by hand is that
ToolStudioPrompt carries credential-bearing fields. Seeds a project, a
document and a prompt whose webhook URL embeds a canary, so the sweep now
actually exercises the tool whose leak risk motivated the hand-built dict.

Also updates the README's GET description, which still promised a 200 identity
response, and the last "every MCP call is a POST" comment.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…de to the registry

readMeFirst's tool listing is hand-maintained prose, so it drifted the moment
listPromptStudioDocuments and listPrompts were added to the registry without
being added to the guide. That is a softer form of the unreachability bug those
two tools were built to fix: an agent trusting the guide would find the four
billable tools and no way to obtain the document_id and prompt_id they require.

The reachability invariant did not catch it because it checks the registry's
schemas, not this prose. Extends it to call both readMeFirst handlers and
compare their listings against the registry — every registered tool must be
named, no phantom tools may be advertised, and the billable grouping must match
the registry's billable flags exactly, since that grouping is the guide's main
signal about which calls cost money.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Requiring a deployment key to extract bought no safety: the platform server
already spends money through executePipeline and the Prompt Studio tools, and
it deliberately cannot hand over a deployment key (key retrieval is excluded),
so an agent was told to go get a credential it had no way to obtain. Both
servers now extract, and the caller picks the blast radius — one deployment, or
one credential across the organization.

The work is delegated to tools.execution rather than reimplemented, so both
servers share one path through validation, rate limiting and the execution
helper. Only the target lookup differs: there it arrives with the credential,
here it is resolved from api_name through for_user.

Two divergences, both consequences of the credential:

llm_profile_id is not offered. It is validated against the API key's owner, and
a platform key resolves to a service account with no meaningful owner. Passing
some deployment's key to satisfy that check would assert a principal the caller
is not. Omitting the field is also what keeps the serializer from ever reading
the api_key this context lacks — an invisible coupling, so it is pinned.

getExecutionStatus re-checks the execution against the organization first.
DeploymentHelper.get_execution_status does a bare lookup by id with no tenant
filter, and TENANT_APPS is empty so there is no per-tenant schema to fall back
on. The deployment server's API key made this moot; nothing does here.

Also corrects the _is_service_account docstring, which still claimed DELETE was
blocked for all API keys — untrue since full_access was added, and it
contradicted platform_api/permissions.py. The conclusion it draws was right;
the stated reason was not.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

… sweep off Redis

The org-scope check in get_platform_execution_status is the one genuinely new
security boundary this change creates — on the deployment server the API key
made it moot — and nothing exercised it. Adds the refusal cases: another
tenant's execution, an unknown id, a pipeline run with no deployment behind it,
and extraction naming a deployment outside the org. The cross-tenant test also
asserts the refusal does not name the owning organization, since confirming an
id exists elsewhere is itself a disclosure.

The leak sweep now invokes getExecutionStatus, and the seeded execution has
status ERROR — which ExecutionStatus counts as terminal, so the tool takes its
is_completed branch and reads results through a raw Redis client that
override_settings(CACHES=...) does not reach. Stubbed the same way
_handle_execution_cache already is, so the sweep stays in the unit tier's reach
rather than failing in CI for a reason unrelated to credentials.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

The budget is consumed on invocation and never refunded — right for a call that
may have spent tokens upstream, wrong for one refused on an argument the server
could check for free. extractDocument made this visible: api_name is
caller-supplied prose rather than an id copied from a listing, so an agent
guessing at deployment names could exhaust the organization's window without
ever reaching an LLM.

Adds an optional preflight to MCPTool, run after the permission check and
before the budget is claimed, and gives every billable tool one that resolves
what it names. The handlers already did these lookups; they just did them on the
far side of the counter. executePipeline was found by the new invariant rather
than by inspection — it had the same shape and no preflight — so its resolution
is now shared between the preflight and the handler.

Pinned four ways: a failed preflight leaves the counter at zero, repeated bad
names never exhaust the window, a passing preflight still consumes budget (the
hook must not become a way around the guard), and an exhausted budget still
reports the bad name rather than masking it — that last one is what
distinguishes running preflight *before* the claim from merely reporting after
it. Reversing the two in transport.py fails three of them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

The server advertised protocol revision 2024-11-05 while the transport followed
2025-06-18 rules — Streamable HTTP, and answering GET with 405 rather than an
SSE stream. The version string was describing a server this is not.

Bumping the constant alone would have been worse than the inconsistency: the
initialize handler ignored the client's requested version entirely and always
returned the server's own, so every client pinned to 2024-11-05 would have been
told to disconnect. The spec's rule is an echo, not an announcement — respond
with the requested version when supported, and offer one of ours otherwise —
so that is now implemented.

2024-11-05 stays in the supported set deliberately. The subset implemented here
(initialize, tools/list, tools/call, ping) is identical across the two
revisions; what differs is server-initiated streaming and session ids, neither
of which this server does. Accepting the older revision therefore costs nothing
and keeps working clients working.

Pinned through the view as well as the pure function, so the handler is proven
wired to the negotiation rather than to the constant: removing the echo fails
three tests.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

The catch-all in extract_document returned `redact_secrets(str(error))` to the
MCP client. Greptile flagged this as the reason for its 4/5 confidence:
"pattern redaction removes recognized secrets without removing other internal
details".

That is right, and understated. The handler catches *any* failure from a
connector, a provider client or the execution stack, so the text can carry
internal hostnames, filesystem paths, row ids and stack detail — none of which
redaction targets, and none of which an MCP client should see. Worse,
"recognized" is doing more work than it can bear: redact_secrets currently
misses every credential name with a prefix (`access_token=`, `client_secret=`)
because its pattern is anchored with `\b`, so it does not reliably remove even
the recognised ones. That is raised separately in review.

The failure is already logged in full one line above, with the api name, so an
operator can correlate it by name and timestamp. The client now gets a fixed
message instead.

Expected, agent-actionable failures are unaffected: validation errors and
MCPToolError still carry their specific text, because those are written for the
agent to act on. Only the unexpected catch-all is generalised.

… check liveness in preflight

Five review findings, all independent of the open sanitize.py question.

transport.py — `method` and `params.name` were checked for truthiness but not
type. JSON permits any type there, so a non-string `method` reached
`method.startswith(...)` as an AttributeError, and an unhashable `name` (a JSON
object or array) reached `registry.get()` — a dict lookup — as
`TypeError: unhashable type`. Both escaped as a Django 500 with no JSON-RPC
envelope, the one failure shape a conformant client cannot parse. The `params`
check immediately below already documented this exact class; it simply was not
applied to the other two.

registry.py — `getExecutionStatus` advertised `readOnlyHint: true`. It sets
neither `writes` nor `billable`, so the derivation made it read-only, but the
result store is one-shot: reading a COMPLETED execution acknowledges it and the
payload cannot be fetched again. A client is told it is safe to poll freely,
which is how a result gets lost. Adds a `consumes_result` flag that
participates in the derivation, rather than letting a tool declare a hint its
own flags contradict — the property that method exists to preserve.

observability.py, platform.py — three tools filtered a UUID column with a raw
client-supplied string. A malformed id raises Django's ValidationError, which
surfaces as "Tool execution failed" instead of the actionable message
`valid_uuid` produces. `list_executions` is deliberately untouched: its
membership check against visible ids runs first and already rejects cleanly.

platform_execution.py, platform.py — the preflights resolved their target but
skipped the liveness check the handler makes, so calling either against an
inactive deployment or a paused pipeline claimed a non-refundable budget slot
for a call that reached no LLM. That is the precise failure preflight exists to
prevent, and it compounds: `setApiDeploymentActive` and `setPipelineActive` are
registered alongside, so an agent can deactivate a target and then exhaust
MCP_BILLABLE_CALL_LIMIT retrying against it, locking the organization out of
every billable tool.

`test_read_tools_are_marked_read_only` and its inverse classified tools with
`tool.writes or tool.billable` — the previous derivation, duplicated in the
test rather than sourced from it — so adding `consumes_result` left
getExecutionStatus in the "read tool" bucket and the assertion failed.

Both filters now include `consumes_result`, and a named test pins the actual
intent: getExecutionStatus reads, but reading acknowledges a one-shot result,
so it must not be advertised as read-only. That test is explicit rather than
left to the loops because it is the only tool in this category on either
server — a refactor dropping the flag would otherwise just make one subtest
quietly stop running instead of failing.

Deliberately NOT changed: the three `writes or billable` filters in
test_no_credential_leak.py. They use the same expression for a different
question — which tools return data worth sweeping for credentials — and
adding `consumes_result` there would exclude getExecutionStatus from the leak
sweep, which is the opposite of what that suite is for.

Not verified locally: `uv run pytest` cannot install in this environment
(django-celery-beat==2.5.0 ships a wheel with duplicate ZIP entries), so CI is
the check.

Only the deployment-scoped MCP server is required for the initial release. The
organization-scoped one exposes 23 tools that reach the whole organization, so
it now ships off behind MCP_PLATFORM_SERVER_ENABLED (default false) rather than
being removed — the code is reviewed and tested, and turning it on later is a
settings change instead of a revert.

The deployment server is unaffected: it mounts unconditionally from
api_v2/execution_urls.py and shares only the transport.

Enabled in settings/test.py, deliberately. Several of its tests drive requests
through the full URL stack precisely so CustomAuthMiddleware runs, and an
unmounted route would 404 them — turning a suite that asserts a bad credential
is *rejected* into one that passes because nothing is there. Shipping-off and
untested are different things.

Also resolves the merge with main, which added the global-api-deployment mount
on the same line the platform MCP server previously occupied.

Worth noting this defers rather than answers the tier question raised in
review: a `read`-tier platform key cannot use that server at all, because every
MCP call is an HTTP POST and the middleware gates tiers on method. Nothing is
exposed while it is off.

getExecutionStatus returned "-32002 Tool execution failed" for any well-formed
execution_id that matched no execution. Reproduced against a live deployment:
polling a random UUID gives an opaque protocol error rather than anything an
agent can act on.

ExecutionQuerySerializer.validate_execution_id raises two different types —
ValidationError for a malformed UUID, and ExecutionDoesNotExistError (an
APIException, not a ValidationError) for a well-formed id that does not exist.
Only the first was caught, so the second escaped to the transport catch-all.

The deployment-scoped not-found message further down the function was therefore
unreachable: the serializer runs three lines earlier and raises first. The new
arm answers with that same wording, so whether an id belongs to another
deployment or to no execution at all is indistinguishable to the caller — which
is the existing scoping guard's intent, and the next step is the same either
way.

Found by calling the deployed endpoint, not by reading: the app's own tests
drive the view through APIRequestFactory and never exercise this path.

The key=value redaction pattern anchored on `\b`, which reads as "whole word
only" but does not behave that way: underscore is a word character, so there is
no boundary inside `aws_secret_access_key` for `\b` to find and the match never
started. Every prefixed spelling was therefore unredacted — `aws_secret_access_key`,
`db_password`, `x-api-key`, `client_secret`, `my_token` — while only the bare
names were ever covered. Those prefixed forms are the shapes credentials
actually travel under in an env dump or an SDK error, i.e. the common case,
not the exotic one.

Replaced with a negative lookbehind that keeps the part of `\b`'s intent that
was paying for itself — a match still cannot begin mid-word after an
alphanumeric, so `hastoken` is not a hit — while allowing a `_`/`-` separated
prefix. Guarding against bare-word false positives was never `\b`'s job here:
`token_count=5` has always been safe because `token` is not followed by a
delimiter, and it still is.

Also scopes the Bearer pattern's case-insensitivity to the keyword. Under a
pattern-wide `(?i)` the token class `[A-Za-z0-9._-]` was the same class written
twice, which reads as deliberate but is redundant — SonarCloud python:S5869.

The literal pre-filter needs no change: `secret`, `access_key` and the rest are
already anchors, so the newly-matching strings all reach the regex pass.

The previous form used `(?i:bearer)` to keep the token class ASCII-explicit
while folding only the keyword's case. SonarCloud reads the scoped-flag group
as a pointless wrapper (python:S6395), so this reverts to a pattern-wide `(?i)`
and writes the token class in one case, which is what `(?i)` already means.

`\w` would have been shorter still, and wrong: it is Unicode by default, so it
would quietly widen what counts as a token character. The explicit ASCII class
keeps the previous behaviour exactly.

No behaviour change — `test_bearer_is_matched_case_insensitively` pins all four
spellings.

… one

`_SECRET_KEY_NAMES`'s comment claims it shares a vocabulary with the key=value
pattern. It did not. `secret_key`, `private_key` and `access_key_id` were
listed there as credentials and were not matchable in free text, so the same
name was redacted when it arrived as a dict key and passed through untouched
when it arrived in an error message — the same secret reaching an agent by a
different route.

`private_key` needed a second fix to be reachable at all. `redact_secrets`
short-circuits on a literal pre-filter before running any regex, and no anchor
contained "private", so the alternative would have been dead code even once
added. All three spellings the pattern accepts are now anchors.

Deliberately NOT added: a `credentials?` plural. It matches inside
`has_credentials: true` — an ordinary boolean field — because the lookbehind
permits the `has_` prefix and only the trailing `s` currently keeps the
delimiter from lining up. This pattern also runs over extracted document output
via `redact_structure`, where a false positive destroys real data rather than
being merely noisy, so the plural stays with `_is_secret_key`, which can demand
a whole-key match. Pinned as a test so it is not "fixed" later by accident.

Still no bare `key` alternative: `prompt_key` and `foreign_key` are real field
names on these paths.

Verified over 42 strings — 24 that must redact, 18 that must survive verbatim
(`token_count`, `monkey`, `keyboard`, `public_key_algorithm`, `primary_key`).

All three descriptions a client reads — the tool description, the
`document_urls` schema, and the readMeFirst guide — said that only S3
pre-signed URLs are accepted and that an ordinary public link is rejected. The
validator only ever checked the *host*: HTTPS, on an `*.amazonaws.com` S3
endpoint. It has never looked for a signature, so a publicly-readable S3 object
is accepted and extracts normally.

This was not a cosmetic inaccuracy. Found by attaching the server to a real MCP
client and giving a fresh agent a plain-English task: it read that sentence,
concluded the call would be refused, and declined to make it — citing the
tool's own warning not to call speculatively. A second, independent run reached
the same conclusion. For an MCP tool the description *is* the interface, and
this one talked a working call out of existing. Confirmed against the live
deployment afterwards: the same unsigned public URL extracted successfully.

The wording now states the restriction that is real (S3 host, HTTPS) and names
the pre-signed URL as the usual way to reach a private object rather than as
the only accepted form.

Pinned by tests that run the real serializer over the accept/reject matrix, so
the prose and the validator cannot drift apart again silently. The guard checks
all three description sites, because the claim was wrong in all three and
fixing one would have left an agent reading either of the others. It asserts
the absence of the specific false sentences rather than word co-occurrence —
correct wording legitimately contains both "publicly readable" and "rejected"
(of a non-S3 host). Verified to fail against the old text.