Skip to content

QA run · api-backend (8 of 14 items) · e4e5c6e3 · 2026-08-17 · 4 PASS / 4 PARTIAL / 0 FAIL #9333

Description

@os-zhuang

R3-C round of the #9296 wave. Scope is 8 of the area's 14 items — this is NOT a full-area run; the 6 unreached items are listed as not-run at the bottom rather than quietly omitted.

Headline: zero product defects found. Two would-be failures were disproven before being written down (both are recorded below as checklist-accuracy findings, which is where the real value of this round sits), and two fixture gaps are recorded as blocked rather than scored.

Environment fingerprint

reading value
framework sha (subject, unmoved) e4e5c6e3c608b1b807c83a0d5b734f213eb1a1dd
.objectui-sha pin 665661ab093263f39f2e660a295ea615dbcee35a
console dist stamp 665661ab0932pnpm check:console-sha ✓ matches pin
checklist spec origin/main post-#9309 — validator: 15 areas, 190 items (190 active); 30 kinds mapped, 0 waived
app / port / db showcase · 3401 · file:/tmp/qa-r3c/api-backend-filter.db
seed --seed-adminadmin@objectos.ai; com.example.showcase 130 rows
driver SqlDriver(better-sqlite3), tenancy single, 48 plugins
clock / TZ 2026-08-17T14:00Z · UTC

Subject pin vs spec: the checklist was taken forward with git checkout origin/main -- docs/qa/platform-checklist/ only. git status showed only docs/qa/ paths modified and HEAD remained e4e5c6e3…. Nothing committed.

Setup correction (affects the other R3 sessions)

The dispatched setup recipe is incomplete. pnpm exec turbo run build --filter=@objectstack/client... + pnpm objectui:build do not build @objectstack/cli, so packages/cli/dist stays empty and the boot smoke dies with Error: command dev not found. R0 never hit this because turbo run test builds the whole graph incidentally. Fix: a full pnpm exec turbo run build (3m12s warm here) before the first boot. The #9307 workaround itself behaved exactly as documented — stage 1 then objectui:build, both green, stamp matching.

Scope — selector and per-item revision

node scripts/checklist-select.mjs area:api-backend --json → 14 runnable, 0 blocked.

# item rev pri surface verdict
1 filter-comparand-conformance 1 P1 api pass
2 date-range-preset-matrix 1 P2 api pass
3 retired-def-refusal 1 P1 build partial
4 query-contract-matrix 1 P0 api partial
5 bulk-write-contract 1 P1 api pass
6 declarative-endpoint-execution 1 P1 api pass
7 batch-transactional-discovery 2 P1 api partial
8 error-envelope-ledger 1 P1 api partial

Per-clause verdicts

1. filter-comparand-conformance (rev 1) — pass · 5/5

Matrix pinned to ACCEPTED_FILTER_COMPARAND_TYPES, 6 members, all exercised (not sampled).

clause verdict oracle evidence (text)
0 all 6 accepted members accepted + known answer pass REST door: string/number/boolean/null natively + Date-as-ISO and bigint-as-number surrogates each 200 with set-equality against a locally computed baseline. JSON cannot carry a native bigint/Date, so those two members were additionally proved at the contract door: isAcceptedFilterComparand → true for all six native JS values
1 non-accepted comparand refused, never dropped pass object / array-for-scalar-op / nested-array / object-in-$in → all 400, message locates the path (where.name.$eq) and quotes the accepted-set sentence. No probe ever returned 200 over the full table — the dangerous direction never occurred
2 bigint exact-integer limit at the boundary pass 2^52 accepted (narrowed to number), 2^53 at limit accepted, 2^53+1 refused, 2^70 refused — message names the magnitude and the no-exact-JS-number reason
3 dotted head classified + handled per class pass live: relation (f_lookup.name, f_master_detail.name), virtual (f_formula.x), scalar (name.x, f_number.x) each 400 INVALID_FIELD with class-specific prescriptive text. classifyDottedFilterHead cross-checked directly: relation for lookup/master_detail/user/tree, virtual for formula, scalar for text/number/boolean/date/select, null for all 7 structured-JSON types + multiple:true
4 refusal is contract-level, not a transport quirk pass same bad comparand through POST /data/:o/query body and GET ?$filter=byte-identical message; dotted-head refusal carries code: INVALID_FIELD on both

Fixture note: the item's declared knownGap (virtual head blocked if no formula field exists) does not apply — showcase_field_zoo carries f_formula (virtual), f_master_detail (relation) and plain scalars, so all three classes were reachable.

Hypothesis raised and disproven (RUNNER rule 5): all 7 structured-JSON dotted heads (f_json.foo, f_location.lat, f_address.city, f_composite.a, f_repeater.a, f_vector.a, showcase_account.hq.lat) are refused 400, even though classifyDottedFilterHead deliberately returns null (unjudged) for them "so refusing it for symmetry would [not] delete a working capability on two of three backends". This looked like the classifier's abstention being defeated. It is not a defect: the refusal comes from a different, separately declared gate — semantic migration driver-sql-unresolvable-where-column-refused (#8790, ruled 2026-08-15), which refuses an unresolvable where-column with INVALID_FILTER/400 on driver-sql specifically. Both contracts hold at once; the capability survives on memory/mongodb. Recorded so the next sweep does not re-derive it.

2. date-range-preset-matrix (rev 1) — pass · 4/4

Matrix pinned to DATE_RANGE_PRESETS, 13 members, all exercised on two objects (26 rows). Literal windows were resolved with the platform's own resolveFilterToken, never a hand-guessed week/quarter convention.

clause verdict oracle evidence (text)
0 preset window === explicit literal window pass 26/26 agreement. e.g. this_week2026-08-17..2026-08-23, macro 2 rows === literal 2 rows; last_quarter2026-04-01..2026-06-30, 2 === 2 on showcase_account
1 exhaustive over the pinned members pass one row per member × 2 objects; none sampled away
2 bare preset name refused with a prescription pass all 13 bare names → 400 on showcase_task.due_date. Message is located (where.due_date.$eq) and names the failure mode outright: "would … return 200 with an empty result — indistinguishable from 'there is no data'. The filter was NOT applied"
3 unknown preset refused, not unbounded pass last_60_days, Last 7 Days, nonesuch_window400 each; no 200 over the full table

Discrimination honesty (the item's own knownGap): across both objects every preset was answer-discriminating on at least one, except yesterday, which was empty on both — it proved only that the window resolved, not that it selected. Windows are genuinely computed, not aliased: this_quarter/this_year/last_90_days return the whole showcase_task table but differ sharply on showcase_account (1 / 6 / 3).

Checklist note (clause 2's verify names an unobservable oracle): the clause says the message "matches bareDateRangePresetComparandMessage". At the api oracle it cannot — that function serves the authoring doors (data/filter.zod.ts schema door and the @objectstack/lint rule). The live data-API refusal is the engine-door wording (#8690/#8808). The clause's intent is fully met; its stated verify is unreachable from the oracle it declares.

3. retired-def-refusal (rev 1) — partial · 3 pass, 1 blocked

Registry: 54 entries, prefixes 17 and 18. Sampled 5 spanning both prefixes and 4 domains — 18.identity__ApiKey, 17.data__ExternalDataSource, 17.data__ExternalLookup, 17.system__MessageQueueConfig, 17.automation__ETLPipeline (+ the ui/ widget+i18n family via its gate test).

clause verdict oracle evidence (text)
0 refused at os build blocked(fixture) not expressible — see below
1 distinguishable from generic where a prescription was authored pass every sampled def took Route 3 (confirmed in-file for ExternalDataSource, MessageQueueConfig, ApiKey: "no tombstone, no D2 conversion — this table plus the D3 semantic entry ARE the declaration"). Per the item's own knownGap, the generic refusal is the CORRECT answer for Route 3, and that is exactly what was observed: byte-identical to the invented-type control apart from the type name
2 runtime write-door refuses the same types as build pass PUT /api/v1/meta/<type>/… → all 5 retired defs 400 INVALID_REQUEST; controls NotARealTypeXyz/frobnicator identical; sanity control view422 INVALID_METADATA, proving the door distinguishes unknown-type from known-type-bad-body. No two-door disagreement
3 sample spans both version prefixes pass 18 (identity) + 17 (data, system, automation, ui)

Build-tier evidence for the retirement mechanism: the four dedicated gate tests pass 16/16 (api-key-retirement, external-lookup-retirement, message-queue-retirement, widget-i18n-retirement) — "every retired name has ZERO holders on any public entry", "runtime namespace agrees with the compiler view", "the module is gone from disk".

Why clause 0 is blocked, not failed: the step says "author a stack that declares metadata of each sampled retired type and run os build". Retired defs are schema-definition names (identity/ApiKey, data/ExternalDataSource), not authorable metadata types. GET /api/v1/meta/types returns 34 live types and the overlap with the sampled retired defs is zero, so there is no expressible stack that declares one. The step needs rewriting against the real mechanism (the #4725 manifest-deletion gate over RETIRED_DEFS_BY_MAJOR).

4. query-contract-matrix (rev 1, P0) — partial · 8 pass, 1 blocked

clause verdict oracle evidence (text)
0 operator matrix, both spellings, set equality pass 11 operators × 2 spellings = 22 runs, all MATCH on id-set equality vs a locally computed baseline: $eq(9) $ne(5) $gt(9) $gte(11) $lt(3) $in(12) $nin(12) $between(10) $contains(1) $startsWith(1) $null(12); AST twins (=,!=,>,>=,<,in,not_in,between,contains,starts_with) identical
1 pagination windowed and stable pass GET ?$orderby=name&$top=3&$skip=2 returned exactly rows 3–5 of the locally sorted baseline; POST {orderBy:[{field,order}],top,skip} returned the identical id triple
2 projection narrows the payload pass $select=name,status → response keys exactly name,status; POST fields:['name','status'] identical. No unrequested business field leaked
3 sort orders the FULL result both ways + INVALID_SORT pass asc and desc on annual_revenue both full-order-match the locally sorted baseline (not just first row); $orderby=not_a_field400 INVALID_SORT
4 expand enforces target RLS/FLS for the contributor blocked(fixture) no credentialed contributor persona exists — see fixture gaps
5 scalar comparand on a collection operator, both doors pass $nin:'done'400 INVALID_FILTER ("requires an ARRAY of values. Received string"); AST ['status','not_in','done']400 INVALID_FILTER. Never 500 DATABASE_ERROR (#5869 holds on both doors)
6 unknown inputs get their ledgered code pass ?$pageSize=5400 UNSUPPORTED_QUERY_PARAM (lists supported params); ?not_a_field=x400 INVALID_FIELD; unknown where-field → 400 INVALID_FIELD. None answered 200 with rows (#4134 holds)
7 empty-list boundary pass $in: [] → 200 / 0 rows; $nin: [] → 200 / 14 rows = the full visible set
8 every variant carries a recorded verdict pass all 11 operator variants tabulated above; none silently omitted

Load-bearing negative (#3899 silent widening) — did NOT reproduce. Four malformed bodies probed; the catastrophic direction (degrading to an unfiltered 200 full read) never occurred: {wibble:{…}} → 400 INVALID_FIELD; {where:"a string"} → 400 VALIDATION_FAILED; bare {where:{status:'active'}} → 200 with 9 correctly filtered rows; {filter:{status:'active'}} → 200 with 9 rows (baseline 14) — i.e. correctly filtered, because filter IS an accepted alias for where in this build. The clause's chosen example key is stale (see findings).

5. bulk-write-contract (rev 1) — pass (automated pin, RUNNER rule 6)

packages/qa/dogfood/test/owner-anchor-and-bulk-writes.dogfood.test.ts9/9 passed: member cannot forge another owner on INSERT; may INSERT with self-owner; cannot TRANSFER; cannot DISOWN (owner_id:null); unchanged no-op form echo tolerated; privileged caller with modifyAllRecords CAN reassign; a bulk update whose change-set carries owner_id fails closed for a member; member bulk UPDATE and bulk DELETE each touch only their own rows, never a peer's. Both sides of the gate are covered by the pin itself.

6. declarative-endpoint-execution (rev 1) — pass (automated pin, RUNNER rule 6)

showcase-declarative-endpoints.dogfood.test.ts + declarative-endpoint-policy.dogfood.test.ts25/25 passed. Covers both gate sides: omitted authRequired denies anonymous (schema default true) while serving an authenticated caller; explicit authRequired:false serves anonymous and is metered (429 with Retry-After, never a cache directive on the 429, budget per-endpoint not global); object_operation endpoint answers byte-identically to the built-in /data route and 401s anonymous; flow endpoint invokes the real flow over HTTP and 404s on the wrong method; undeclared paths keep the transport's bare 404; /openapi.json documents both declared endpoints and does not document one the runtime would not serve; an ungated direct write never becomes a live route.

7. batch-transactional-discovery (rev 2) — partial · 4 pass, 1 partial

clause verdict oracle evidence (text)
0 discovery bit agrees with live behavior pass GET /api/v1/discoverycapabilities.transactionalBatch.enabled = true; live POST /api/v1/batch with atomic:true created rows in two different objects in one call (task 10→11, account 14→15)
1 atomic:false refused BATCH_NOT_ATOMIC pass top-level atomic:false400 BATCH_NOT_ATOMIC, reproduced twice, wrote nothing (count unchanged)
2 failing middle op rolls back with #4793 per-row codes partial rollback proven — 3-op batch with an invalid middle op → 400, showcase_task count unchanged across two runs. Per-row codes not exercised: the induced failure is a pre-execution validation error that short-circuits with a flat VALIDATION_FAILED envelope and no per-operation results array, so ROLLED_BACK/NOT_ATTEMPTED never had a chance to be emitted. Reaching them needs a middle op that passes validation and fails at write time
3 oversize batch refused up front, creates nothing pass 201 ops → 400 BATCH_TOO_LARGE, body {count:201,max:200}; count unchanged
4 per-object door defaults to non-atomic per-row outcomes pass POST /api/v1/data/showcase_task/batch with one valid + one invalid record → 200 {success:false,total:2,succeeded:1,failed:1,results:[…]}; valid row persisted (13→14). ADR-0119 D4 holds

8. error-envelope-ledger (rev 1) — partial · 3 pass, 3 partial

clause verdict oracle evidence (text)
0 every sampled code is ledgered partial 11 observed codes all resolve in errors.zod.tserror-code-ledger.zod.ts: VALIDATION_FAILED, OBJECT_NOT_FOUND, RECORD_NOT_FOUND, UNSUPPORTED_QUERY_PARAM, BATCH_TOO_LARGE, BATCH_NOT_ATOMIC, INVALID_FILTER, INVALID_FIELD, INVALID_SORT, INVALID_REQUEST, INVALID_METADATA. The 401 anonymous sample carries no code at all (body keys error,message), so its ledger membership is unverifiable
1 validation sample carries field-level array under fields pass POST /data/showcase_invoice {} → 400, fields:[{field:"name",code:"required",message:"Invoice Number is required",label:"Invoice Number"},{field:"account",code:"required",…}] — lowercase-snake field codes per ADR-0114 D2
2 retired fieldErrors key never appears pass zero occurrences across all 6 sampled bodies — ADR-0114 D4 tombstone holds
3 status matches the code's declared mapping partial 400 for validation/filter/param codes ✓, 404 for both not-found flavors ✓; 401 unverifiable (no code); 403 not sampled (no member persona)
4 the two 404 flavors are distinguishable pass unknown OBJECT → OBJECT_NOT_FOUND "Object 'not_a_real_object' is not registered"; unknown RECORD → RECORD_NOT_FOUND "Record nonexistent-id-0000 not found in showcase_task". Different codes, different bodies — a client can tell schema drift from data absence
5 one sample per family variant partial 5 of 6 families sampled; the authorization/403 family could not be induced (fixture)

Findings — two checklist-accuracy defects, no product defects

Both of these would have produced a false P1 fail for a runner following the steps literally. Neither is a product bug; both need a checklist edit (revise + bump revision + append history), which this run does not make because nothing from it lands in the repo.

F1 — batch-transactional-discovery steps 2 and 3 prescribe the wrong request shape. The steps say options {"atomic": true} / options {"atomic": false}. The guard reads top-level atomic (packages/rest/src/rest-server.ts:9869: if (parsed.data.atomic === false)). Reproduction of the divergence, on a fresh boot:

  • POST /api/v1/batch body {"operations":[{"operation":"create","object":"showcase_task","data":{"title":"x","project":"<id>","status":"todo"}}],"atomic":false}400 BATCH_NOT_ATOMIC, writes nothing. (correct)
  • Same body with "options":{"atomic":false} instead → 200, and the row is created. (the checklist's spelling; the flag is silently ignored by CrossObjectBatchRequestSchema)

Expected-vs-actual for the item as written: expected 400 BATCH_NOT_ATOMIC, actual 200 + write. Following the step literally reports a P1 contract violation that does not exist. Secondary, milder observation for maintainers: an unrecognised options.atomic is dropped rather than refused, so an author who writes the nested spelling gets atomic semantics silently instead of being told the key is wrong — safe in outcome (the endpoint is always atomic) but it is how the wrong spelling survives in docs.

F2 — query-contract-matrix's load-bearing negative uses a key that is not unrecognised. The negative says a malformed body "(e.g. {"filter": {…}} — not a QueryAST key) must answer 400 VALIDATION_FAILED". Observed: POST /api/v1/data/showcase_account/query with {"filter":{"status":"active"},"top":500}200 with 9 rows against a 14-row baseline — i.e. filter is an accepted alias for where and was honoured correctly. The negative's premise is stale. The property it exists to protect is intact and was verified with a genuinely unrecognised key: {"wibble":{"status":"active"}}400 INVALID_FIELD. Suggested edit: swap the example key, or assert that filter is a documented alias.

F3 (minor, provenance) — date-range-preset-matrix cites a source path that does not exist. packages/spec/src/data/temporal-comparand.ts is absent at the subject pin and on current main. The real modules are packages/core/src/utils/temporal-comparand.ts (also cited, correctly) and packages/spec/src/data/temporal-conformance.ts. Provenance only — it does not block the item.

Fixture gaps

  1. No credentialed non-admin persona in stock showcase seeds. sys_user holds exactly admin@objectos.ai, auditor.demo@example.com, phone.demo@example.com. The seed comments reference ada@example.com under the showcase_contributor permission set, but no such user exists with a usable password (4 candidate passwords all 401). This blocks query-contract-matrix clause 4 (contributor expand RLS/FLS) and error-envelope-ledger's 403 family, and will block every "both sides of the gate" clause in this area that needs an unentitled caller. Worth folding into Land the showcase seed fixtures that unblock the checklist's stock-fixture gaps (loginable 2nd member, publicSharing object, writable summary field, readable:false FLS grant) #9308.
  2. showcase_private_note returns 0 rows to the seeded admin, so it is not usable as a batch fixture target.
  3. yesterday has no seeded row in any probed object — that preset proved resolution only, not selection.

Not run — 6 of 14 items

Recorded rather than omitted, per the guardrail against faking coverage. None of these were started, so none carries a verdict:

item rev pri surface why not reached
formula-gates 2 P1 build round budget
enforce-or-remove-authoring-gates 2 P1 build round budget
server-timing-admin-gated 2 P2 api needs two extra server boots (OS_SERVER_TIMING on/off) and a non-admin persona (fixture gap 1)
route-ledger-live-parity 2 P2 api 9 clauses sweeping 4 route ledgers; round budget
package-rest-lifecycle 1 P2 api round budget
api-console-discovery-execute 1 P2 browser round budget (console dist is built and ready)

Reproduction rules

No fail verdicts were recorded, so no defect reproduction rule is owed. The two checklist divergences (F1, F2) carry their exact request/response pairs inline above so either can be re-hit on a fresh boot without this run's artifacts.

Parent: #9296.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions