proof: knowledge-layer validation programme (stacked on #18) - #19
Closed
NetDevAutomate wants to merge 69 commits into
Closed
NetDevAutomate wants to merge 69 commits into
NetDevAutomate wants to merge 69 commits into
Conversation
Freeze the SessionWeaver Phase 2 retrofit's spec and design ahead of any Phase B code, per EXECUTION-ERRATA.md correction #3 and council ruling R2 ("acceptance is not spec-check alone"). The design document normatively fixes the cross-machine standing order for replicated concept lifecycle events and its two-copy test matrix, the v48/v49 migration contracts and their rollback strategy, the non-fatal ontology refresh-failure seam, seed sanitization, the byte-for-byte memory_recall contract, the fresh-install scope diagnostic, and the ConceptService compatibility seam -- so B1-B6 can implement against a reviewed contract instead of inventing one under implementation pressure. Delta specs add Gherkin-scenario requirements to six existing capabilities (harness-session-memory, data-store-and-sync, mcp-server, session-export, health-and-diagnostics, configuration-and-secrets); no capability is newly created. openspec validate --specs --all: 25 passed, 0 failed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ange - proposal.md: the request_scope() call-site list has eight entries; the source plan mislabels it "seven" (design.md and tasks.md already said eight). - design.md: B3 must test whether the reference allocator's table-wide MAX(logical_time) already satisfies two-copy matrix item 4 before adding an explicit advance-on-import step. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…and diagnostics Lift SessionWeaver's frozen tier-1 ontology extraction/rebuild/status module (extraction version tier1-v2-canonical-messages, same logical-hash algorithm, so A2's baseline stays comparable). Migration v48 installs the six schema objects additively; install_schema is tolerant of a pre-existing ad hoc ontology schema (found on the real sessions.db) so a real upgrade converges instead of crashing -- the first rebuild replaces it regardless of its prior shape. Wire an incremental ontology refresh into export_sessions._run_export, behind a named, monkeypatchable seam (refresh_ontology_after_export) called after the capture commit: a refresh failure is caught, logged as a structured ontology_refresh_failed warning, surfaced in the export summary, and never rolls back captured sessions (EXECUTION-ERRATA.md #7). Add `session-maint ontology-rebuild [--incremental]` and `ontology-status` for manual/maintenance recovery. Add the sync boundary: ontology tables never join SYNC_TABLES or GLOBAL_SYNC_TABLES (positive-control tested), and sync._seed_remote_db strips every ontology row from a seed snapshot before scp so a freshly seeded remote derives its own ontology rather than inheriting the source's. Add a report-only "harness" doctor check (present/coverage/freshness/ extraction-version), never fatal, with a fixture-inserted red-path test. Add the ontology_live safety harness (Online Backup only, real DB never touched, new opt-in live_ontology pytest marker) and R7 migration-safety tests: fresh creation, interrupted-migration recovery, downgrade, and idempotent re-migration. Retains real acceptance receipts (docs/data/) from a live Online Backup run: coverage 100%, zero integrity violations, identical hash across two full rebuilds and one incremental no-op, cold rebuild 3.47s. Fixes a v47-pinned assertion in test_replica_coordinator.py that this version bump would otherwise break, and regenerates .secrets.baseline (detect-secrets 1.5.0) for the two evidence files' SHA-256 hashes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ed scope
A virgin install crashed or leaked ad-hoc errors the moment any scope-dependent
path ran: `studyloop study` propagated a bare ScopeError as a generic exit-1
ClickException, each of the seven unguarded request_scope() MCP tool sites let
FastMCP wrap it in ad-hoc text, and session-db-mcp's open_context()/
_get_connection() let sqlite3's "unable to open database file" leak through a
distinct, unhelpful error shape on a database that simply doesn't exist yet.
Add ScopeUnconfiguredError (a ScopeError subclass identifying exactly the
"nothing configured" case) and scope_setup_diagnostic(), one shared
{code: "scope_unconfigured", message, remediation} shape. Every entry point
that can hit it now reports that same payload instead of a traceback: the
CLI exits 2, both MCP servers' tool-call boundaries return isError with the
JSON payload embedded in the text content, and a missing database is treated
identically to a missing scope rather than as its own error class.
Both packages' config writers (generate_default_config(), ensure_config_dir())
now write memory.default_scope: unclassified explicitly for a brand-new
config.yaml, so a fresh install is immediately usable. The runtime default for
an absent file or a file that omits the key stays unset (errata #9) -- this
changes only what a *generated* file contains, not what an absent setting means.
Tests: packages/studyloop/tests/test_fresh_install_scope.py drives a virgin
HOME (no STUDYLOOP_CONFIG, no SESSION_CONTEXT_SCOPE) as a real subprocess
against the source tree -- the CLI, all seven MCP tool sites, and
session-db-mcp's session_search/memory_search -- so this suite's own
config-isolation fixtures cannot hide the defect the way the existing fixture
already did. test_fresh_install_scope_installed.py (marked integration)
repeats every check against a real `uv build` wheel install per plan ruling
R10. Unit tests cover scope_setup_diagnostic, ScopeUnconfiguredError, the two
config writers' round trip, and open_context()/_get_connection() on a missing
database.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Add a "Tier-1 ontology (derived, never synced)" section to docs/session-memory.md covering what it captures, why it never syncs, the automatic export-time refresh, and session-maint ontology-rebuild/ontology-status for manual recovery. Record the feature under CHANGELOG.md's [Unreleased] Added. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Mark tasks.md's B2 tasks 2.1 (migration v48 + lift), 2.2 (refresh hook + session-maint ontology-rebuild), and 2.3 (sync boundary + seed sanitization) complete. Leave 2.4 unchecked with an inline note: its tests/hash/timing/count verification and a green preflight are done, but code/build-ontology.py's retirement is explicitly B6's (task-B2-brief.md) and the independent review is the reviewer's step. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Task B1 of openspec/changes/sessionweaver-phase2-retrofit: generated config writes memory.default_scope: unclassified; one structured ScopeError diagnostic at the CLI and every MCP tool in both servers; missing-database path covered; virgin-HOME tests on source-tree and installed-package paths. Review: .superpowers/sdd/sessionweaver-phase2/task-B1-review.md (0 findings). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…d agents CI run 34167164262 (jobs test (3.12) and test (3.13)) failed test_fresh_install_scope.py::test_studyloop_study_exits_2_with_the_diagnostic_on_a_virgin_home because on the GitHub runner `studyloop study 'Test Topic'` exits 1 with "No AI agent found. Install one of: Kiro CLI, Codex, Claude Code, OpenCode, or pi" before the scope check ever runs. session/start.py calls detect_agents() (studyloop.agent_launcher) before start_study_session() raises ScopeUnconfiguredError, so the expected exit 2 + structured scope diagnostic is unreachable unless some agent binary is resolvable via shutil.which on the subprocess's PATH. Locally the test only passed because this machine has agent CLIs installed. Fix the test, not production code: add a `_fake_agent_bin()` helper that writes a no-op executable named `claude` (matching studyloop.adapters.claude.ADAPTER.binary) into a throwaway bin dir and prepends it to the subprocess PATH via `_usable_path()`/`_virgin_env()`'s new `agent_bin` parameter. detect_agents() only calls shutil.which(), so the script is never actually executed -- start_study_session() still raises ScopeUnconfiguredError immediately after agent selection, well before any launch command is built. Applied to both test_studyloop_study_exits_2_with_the_diagnostic_on_a_virgin_home and its installed-wheel sibling in test_fresh_install_scope_installed.py, which had the same dependency. Reproduced RED locally by running both files with a PATH stripped of every real agent CLI (venv bin + tmux + /usr/bin:/bin only), then confirmed GREEN with that same stripped PATH after the fix. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Replace permission-bit failure injection with the backup helper's real None failure contract. This keeps the abort-before-stream invariant reliable under both ordinary users and root CI containers.
Preserve direct coverage that a nonzero SSH/sqlite command result is translated into None after the push-contract test moved to the orchestration seam.
Bring the independently reviewed migration v48, deterministic ontology evidence, and CI-parity sync test correction into the SessionWeaver Phase 2 integration branch. # Conflicts: # .secrets.baseline # docs/session-memory.md
Lift SessionWeaver's concept_schema.py unchanged (SCHEMA_VERSION = 2, SCHEMA_FINGERPRINT byte-identical to the reference, af95685e6e39e166148006519862bee3be1a15219d76772236a82890fe11011d) and add migrate_v49, which installs exactly that DDL: context_concepts, context_concept_events, context_concept_clock, context_concept_fts and context_concept_schema, plus their indexes/triggers and the two context_citations guard triggers. UPSTREAM_SCHEMA_VERSION is pinned to 49, the migration number that installs the sidecar. Why: tasks.md 3.1 -- B3 owns v49 (v48 stays B2's); the sidecar must be additive-only, with context_assertions.proposed_state left as execution state (errata #3), and adopt a byte-identical pre-existing PoC sidecar while refusing drift, because sidecar rows are authored data. Migration-safety per B2's R7 pattern: fresh create to v49, additive-only shape check on context_assertions, exact-adopt, drift refusal, interrupted-migration recovery, repeated-open idempotence, downgrade contract, and a real Online Backup v47->v49 upgrade retained as docs/data/concept-sidecar-migration-v49-receipt.json (aggregates only; source sentinels asserted unchanged; live DB still v47, 5813 sessions). The detect-secrets baseline is regenerated for the receipt's hex-hash false positives, matching B2's precedent. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ection Lift SessionWeaver v0.2.0's concepts.py, winddown.py, okf.py (as okf_import.py), projection.py (A3b2's fixed Publisher with its identity-guarded rollback), authorization.py and safe_fs.py into agent_session_tools.context, adapting only import paths to the package (relative imports; okf -> okf_import). All frozen contracts are kept byte-identical: the sessionweaver://session/ source-URI prefix, the session_weaver_projection frontmatter key, the projection marker and manifest filenames, and ConceptService's public surface (project, winddown, transition, bind_legacy, import_okf). Why: tasks.md 3.1 -- B4 consumes ConceptService as a caller, never a second implementation, and B6 deletes the SessionWeaver copies, so the lifted behaviour must be the reference behaviour, provable by the reference's own tests. Tests: the reference suites lifted with the same import rewrites (223 tests: schema exactness/adoption, wind-down parsing, lifecycle transitions, bind closure, OKF scan/classification precedence, projection publish/rollback, adversarial SQL integrity). Two adaptations, both consequences of v49 installing the sidecar up front: the upstream-version pin asserts 49, and a rejected import call now asserts zero sidecar rows instead of no sidecar table. One targeted nosec B506 documents that _StrictSafeLoader narrows yaml.SafeLoader. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Concept roots and their complete append-only event history join the context replication protocol: snapshot selection follows the authorization seam's two visibility shapes (bound roots travel with their assertion's citation closure, legacy roots with their claimed session), the apply path inserts only rows it does not already have, and the receiving side's own schema triggers re-validate every root. The local clock, the FTS read model and the schema marker never travel. Standing is exactly the frozen order (design.md, ruling R2): standing = max(events, key=(lamport, machine_id, event_id)) with machine_id = context_access_state.instance and lamport = logical_time. The three read-model sites drop the reference's standing-kind precedence, which contradicted the frozen pure-triple order under replication ties while being locally unobservable (local lamports are strictly increasing); the lifted total-order test is adapted to the frozen order with the reasoning recorded inline. Duplicate machine_id is diagnosed and refused, never merged: negotiate() already refuses two live peers with one instance, and the new ConceptReplicaIdentityError refuses incoming events that claim the receiver's own instance for history it never wrote, or that bind one (origin_instance, origin_seq) slot to two different events. Lamport advance on import (design.md's B3 verification note): matrix item 4 was run against the unmodified reference allocator first, and passes -- _allocate's table-wide MAX(logical_time) already advances the next local event past every imported lamport, so no explicit advance-on-import step was added. Also wired: v49 adds the three replica-content-generation triggers for the two replicated tables (v46 froze its own list); staged transport indexes both tables by id; the legacy-sync guard treats the sidecar's seeded singletons and FTS shadow tables as bookkeeping, not populated context; tiering's archive-completeness proof excludes the per-database clock and concept-FTS shadow tables while keeping roots/events in the retention proof; compaction seeds a fresh clock/marker and copies roots after their citation closure so the sidecar triggers re-validate. Tests: the normative two-copy matrix (7 scenarios) on fixtures in both replication orders, replay idempotence, post-convergence Lamport advance, read-model hash equivalence, concurrent accept/retire resolved to the computed winner in either order, causal accept-then-retire, and three duplicate-identity refusals; plus the same matrix once on two real Online Backup copies (live_concepts marker, copy B honestly re-identified before any concept history exists, source sentinels asserted unchanged). Full package suite: 1999 passed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…MCP tool session-context gains winddown --session ID --from FILE|--stdin and concept accept|retire ID --reason, concept bind LEGACY_ID --from FILE --reason, concept import-okf DIR [--dry-run] [--report PATH], and concept project --out DIR [--project ID] [--json], all thin wrappers over the frozen ConceptService seam with the reference CLI's bounded input reading (wind-down byte limit, descriptor-anchored non-symlink files), atomic report writes, deterministic JSON payloads, and the reference exit-code contract (2 validation, 1 runtime/write failure). mcp_server.py gains memory_winddown with the same validation: a field-level error list raises a structured ToolError and writes nothing; a valid batch commits atomically. Why: tasks.md 3.2 -- one seam, one validation surface, no second implementation of concept transitions; context_assertions keeps its execution-state vocabulary (verified in the round-trip test). Also frozen (tasks.md 3.4): ConceptService's public API is pinned by an exact-signature regression test (method set, every signature string, and the four result shapes), so B4 consumes a checked contract. This commit also carries the live_concepts harness extensions and their opt-in tests (they type-check as one unit with the CLI): the live two-copy matrix on two real Online Backup copies, and the full 2,033-record legacy OKF import runner whose sanitized report feeds the OpenSpec evidence attachment. Tests: lossless round trip, stdin input, field-level failures for malformed documents/invalid JSON/symlinked input, lifecycle verbs with retired-terminal diagnostics, legacy bind, OKF dry-run/write/report, projection output, MCP happy path and loud structured failure with zero partial writes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…evidence docs/context-memory.md gains the wind-down/concept command reference, the two trust labels (model-proposed vs legacy-unbound, never blended), the frozen cross-machine standing order in one paragraph, and the frozen ConceptService surface table. CHANGELOG records the concept sidecar under [Unreleased] Added. OpenSpec tasks 3.1-3.5 are checked with their evidence pointers (3.5's independent review remains the reviewer's step). The evidence file is added with -f: openspec/ is gitignored for new files, but this change's artifacts are tracked and the brief requires the report attached to the change directory. Evidence: openspec/changes/sessionweaver-phase2-retrofit/evidence/ legacy-okf-import-report.json -- the full legacy OKF import on a real Online Backup (live_concepts marker): 2,035 scanned, 2,033 parseable records imported legacy-unbound with zero write failures and the exact A3b1 baseline FTS content hash (0c036eae...); the two extra files are post-baseline non-records classified invalid_schema (reported, never dropped); dry-run/write classification identical; idempotent re-import already_present=2,033 with zero writes; OKF tree sentinel and source database sentinels unchanged. The visibility sub-classification split differs from the baseline because this run retains every project unclassified on a newer corpus (5,813 sessions vs 5,678); the report retains the split (no_exact_match 1559, no_visible_evidence 429, oversized_evidence 45) for the reviewer. The detect-secrets baseline is regenerated for the evidence hashes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ence Completes the previous docs commit: the gitignore advice for openspec/ aborted the add chain, so the evidence attachment missed it. openspec/ ignores only new untracked files -- this change's artifacts are tracked, and the brief requires the sanitized legacy OKF import report attached to the change directory, so the evidence file is added with -f. The detect-secrets baseline is regenerated for the report's hex hashes, matching the established false-positive handling. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
pytest resolves rootdir (and its configfile) from the paths it is given, so any package-scoped invocation reads this package's pyproject.toml and never the workspace root's -m exclusions. Round 0 shipped the package config without them, so a plain 'pytest packages/agent-session-tools/...' silently ran the live_concepts suites: two 1 GB SQLite Online Backups of the owner's real sessions.db, a 140-second OKF import, and two committed evidence files rewritten -- the same trap fired during the independent review (B3 review round 1, Important #2). Add the -m exclusion for every opt-in marker this package declares (integration, live_ontology, live_concepts), matching the workspace-root convention and packages/studyloop's MUST-duplicate precedent, and pin it with a subprocess regression that proves package-scoped collection deselects every live-marked test by default while an explicit -m opt-in still selects them and ordinary tests stay selected. Co-Authored-By: Kiro <noreply@kiro.dev>
Round 0's committed migration receipt was captured at commit 1, before migrate_v49 gained its six replica_content_context_concept* triggers, so its whole-database schema_sha256 (5c2f2c56...) no longer described the shipped migration -- and nothing deterministic could catch that, because that hash covers every sqlite_master row and is therefore a property of the source corpus, not the migration (a fresh schema.sql install yields 81f3310a..., the real backup 6dfb4027...) (B3 review round 1, Important #1). Give the receipt a source-independent fingerprint instead: sidecar_migration_fingerprint() hashes the DDL of exactly the objects migrate_v49 installs (context_concept* objects, the two context_citations_bound_* guard triggers, the six replica_content_* triggers; FTS5 shadow tables excluded -- library-generated from the fingerprinted virtual-table declaration), emitted as sidecar_objects_sha256 (evidence_version 1 -> 2). Two always-on fixture regressions pin it: the name selection equals the complete non-shadow v49 delta (so a future migration edit cannot slip an object past the receipt check), and a fresh install of the shipped migration must reproduce the committed receipt's hash byte-for-byte. Receipt regenerated at HEAD on a fresh SQLite Online Backup (live DB untouched: v47, 5,813 sessions); its schema_sha256 is now 6dfb4027..., byte-identical to the review's independently reproduced value. Secrets baseline regenerated for the receipt's changed hex strings only. Co-Authored-By: Kiro <noreply@kiro.dev>
Bring the independently reviewed migration v49, lifted concept lifecycle/wind-down/import/projection, replication proofs, CLI/MCP surface, and full-corpus import evidence into the SessionWeaver Phase 2 integration branch.
Capture the existing public search contract before introducing the shared AND-to-OR planner. The fixture freezes row keys, deterministic ordering, null/default values, phrase/operator behavior, and the 300-character preview boundary.
Reuse the released SessionWeaver planner behind session_search while keeping its public row schema, SQL ranking, filters, and 300-character previews unchanged. Black-box tests prove the frozen cases remain identical and only the previously empty implicit-AND query widens through OR fallback.
Port SessionWeaver v0.2.0 concept-first recall onto StudyLoop's B3 authorization seam and shared query planner. The public MCP tool validates strict bounds, preserves the frozen report schema, deduplicates concept source sessions, and proves through behavioral traces that embeddings and ontology are not consulted.
Have studyloop install agents merge session-db and studyloop into Claude Code, Kiro, and Codex configuration without touching the real HOME. Format-preserving temp-HOME tests prove unrelated entries survive, repeated installation is byte-identical, missing configs are created, and doctor reports registration without mutation.
Repair an incorrect StudyLoop-owned JSON MCP entry by replacing only that value object, then append any missing owned entry. This avoids reserializing unrelated MCP entries while retaining byte-idempotence on subsequent installs.
Retain the exact 25-question corpus, aggregate-only v47 Online Backup evidence, and a reproducible released-reference acceptance runner. Document the frozen recall contract and MCP registration, add positive work/personal scope coverage, and mark OpenSpec B4 complete after a green 5,964-test preflight.
Route explicit phrase and boolean queries through the historical FTS escaping path so the shared recall planner remains frozen. Restrict AND-to-OR widening to empty implicit plain-text searches and pin compatibility with negative controls.
Replace owned JSON values and TOML subtrees by parser-validated byte spans rather than canonical-only regexes. This preserves unrelated configuration, prevents duplicate keys, handles CRLF and nested tables, and converges byte-identically on rerun.
Track TOML lexical state before recognizing candidate MCP table headers. This prevents owned-looking text inside multiline values from truncating unrelated config while retaining byte-preserving subtree repair.
… a derived artefact Before any model writes a claim: claims-writer-spec-v1.md fixes the model (claude-sonnet-5), the prompt (writer_prompt_v1.md, sha on every claim and receipt), the strict output schema, the insertion contract (≥1 overlap-aware unambiguous quote per claim, cross-session citations refused, no partial inserts, post-run substr() re-check proving unbound writes = 0), budgets (pilot 40, population ≤302, hard cap 400), and the G2 audit: blinded (statement, quote)-only entailment by a second family on 100 random claims, seed fixed, with a fixed failure taxonomy. What v1 deliberately omits is listed. The ruler binds G2 to "the 348 PoC sessions" with no committed list. Found its definition (RESULTS-final.md: updated ≥ 2026-08-01, ≥10 messages) and the run's own frozen corpus snapshot, whose sha matches the run's SHA256SUMS. pin_poc_set.py reproduces the rule against that snapshot: 345 (the snapshot was cleaned of empty rows AFTER the 348 was counted; 3 fell below threshold). All 345 exist today; 342 are in the store. Under typed events "≥10 messages" means prose events: 200 sessions. ruler-amendment-003 records both denominators and declares n_prose_ge10 = 200 primary for the G2 clause, without editing the ruler. Population is hash-ordered by session id. 26 of 60 DEV gold sessions lie inside the 342 by design of the gold split (verified, not assumed); the writer is gold-blind by construction (no gold file readable from its environment, asserted by test), not by instruction.
…, substr re-check, receipts scripts/knowledge_proof/claims_writer.py: the deterministic half of the writer. It calls no model. population (342 ingested PoC sessions, hash-ordered, prose≥10 subset 200), packet (per-event citable evidence + derive-v1 flags, ≤48 KiB with truncation recorded, prompt and packet sha256), render (byte-deterministic prompt text), ingest (strict JSON; every schema and citation refusal recorded with reason; never a partial insert; post-run SQLite substr() re-check of every inserted citation; a writer label whose prompt sha does not match the packet's fails the run), summarise (yield on both G2 denominators, refusals by reason, recheck total, runs used). Gold-blind by construction: the module contains no gold or sealed reference (test-asserted); packets are built from the store alone. Orchestrator smoke on a scratch copy: a fenced response with one grounded claim, one fabricated quote and one citation-less claim → inserted 1, refused 2 (citation_unbound, no_citations), recheck_mismatches 0. The real store still holds zero claims. 52 harness tests; ruff + pyright clean. pyproject testpaths gains scripts/knowledge_proof/tests (the one permitted line outside the lane). Note: the builder saw 16 failures in packages/studyloop/tests locally; CI is green on every pushed commit, and the one reproduced (test_cli_brain expecting exit 1) assumes no second brain is configured — a pre-existing test-isolation issue on main, out of scope.
…d and entailment below gate Pilot: 40 sessions in hash order, 40 claude-sonnet-5 writer runs, packet-only context. 183 claims proposed, 162 inserted (104 Finding, 25 Decision, 19 Procedure, 12 Problem, 2 Preference), 180 citations, UNBOUND WRITES 0 by SQLite substr() re-check of every citation. 21 refusals: 13 paraphrases, 8 from one session where the writer cited row numbers instead of evidence ids on a 141-row packet — the harness held on every one. Yield on the pre-registered primary denominator: 24/29 = 82.8% (gate 90%). Decomposed, not argued: 4 of the 5 misses are sessions inside the denominator with no learner voice (no learner turn; council-judge briefs; a pasted AGENTS.md); 1 is the id-format defect. On sessions with learner voice: 12/13. Blinded entailment audit (deepseek-3.2, 100 claims, statement+quotes only, key outside the repo): 82 yes / 18 partial / 0 no (gate 95%). Taxonomy: hallucinated-detail 13, over-claim 4, preference-inferred 1. Decomposed: for 14 of the 18 partials the statement's extra details ARE in the session's evidence — the writer read them and cited only one sentence (86/100 claims carry a single citation). 4/100 contain material absent from the session. Transcript fidelity ~96%; citation completeness 82%. G2 is defined on completeness, correctly. G2 NOT PASSED on the pilot. Per the ruler: investigate the writer, never relax the trigger. Next: writer-v2 spec (every factual element covered by a quote; prefer 2+ citations; cite by 64-hex evidence_id only) and a fresh blinded audit. The no-learner-voice denominator finding is recorded for the ruler owner, not changed here. Harness: cap now truncates to 8 and records dropped_over_cap (spec deviation, documented on the receipt).
… before any v2 run Each prompt change is tied to a measured failure from the pilot audit: element-by-element quote coverage (14 under-citation partials), prefer 2–3 citations (86 % single-citation), verbatim 64-hex evidence_id (the 8-claim row-number refusal), statement ≤ 300 chars, no stating implication as fact. Held fixed: model, the same 40 sessions and byte-identical packets, harness, insertion contract, auditor family and blinding; fresh audit seed over v2 claims only. Pass/improve/stop readings declared in advance; two prompt rounds without passing stops Stage E.
…oid the reading, pin the brief The v2 blinded audit returned 17/100 yes, but the brief had been rewritten from memory (stricter wording) and the auditor ruled 3 of 4 known-answer items wrong (statements verbatim from their quotes marked 'partial'). The reading is void as a gate measurement, kept as the record of the fault. The v1 brief is extracted verbatim into a committed template so every future audit is byte-reproducible; the remedy (re-audit both samples with the pinned brief, measure auditor noise on the already-scored v1 sample) is declared before it runs.
…ms: 82 → 16 yes Noise control on the already-scored v1 sample: per-item agreement 34/100, all 66 flips yes→partial. No single-seat entailment reading (v1's 82 included) is a valid G2 measurement. Protocol declared before running: majority-of-three within family plus a gpt-5.6 cross-family seat on both samples, gate reading = the lower of the two, and an explicit 'not measurable' outcome if the families disagree by >10 points.
…; G2 not established: instrument Writer-v2 on the same 40 sessions and byte-identical packets: 145 claims, 0 unbound writes, citations/claim 1.11→1.55, single-citation 89%→54%, the row-number defect gone, yield 23/29 (v1 24/29) — five agent-brief sessions now yield nothing, two warm-ups gained one claim. Audit under the pinned brief, seven seats total. deepseek on the identical v1 items: 82 → 16 → 86 yes (per-item agreement 34/100 between the first two; unanimous across three on 27/100). gpt-5.6: v1 58, v2 77. deepseek valid seats on v2: 91, 96. Every clean seat ranks v2 above v1 (deepseek +12, gpt +19). Families disagree by 21 (v1) and 14 (v2) points — over the pre-declared 10-point bound — so G2 is recorded NOT ESTABLISHED — INSTRUMENT rather than passed or failed on a number nobody can reproduce. Two findings for the ruler owner, recorded not actioned: the audit clause needs a reliability floor before it can be met or failed; the primary yield denominator contains sessions with no learner turn.
…before the last DEV look Pilot claims cover 3 of 60 DEV gold sessions (6 of 91 questions, no paraphrase items); a claims arm over them would be flat by construction and would fire the stop rule for a reason unrelated to the architecture. The full population reaches 26 gold sessions / 39 questions across all strata, inside the ruler's 400-run writer budget (382). Ruler unchanged; spec-v2 deviation stated.
…ms declared before look 3 Two arms, declared with their coverage bound (19/60 DEV gold sessions carry a writer-v2 claim; a claims-only arm can reach 30/91 questions) before any look-3 run. recall_claims: in-memory FTS5 over writer-v2 claims (title, statement, tags), same planner as B1_clean, bm25 then claim rowid, first K distinct sessions. B1_clean_plus_claims: reciprocal rank fusion (k=60, 1-based ranks, weight 1) of the two full deduped rankings, tie-break best prose rank then session id. B1_clean's ranking is factored into a shared helper so the fused arm consumes the same list B1_clean cuts; its output is unchanged (tested). score.py now emits comparisons between every ordered pair of feature arms, so the attribution reading B1_clean_plus_claims_vs_B1_clean is produced by the committed script. Statistics unchanged. 63 harness tests pass.
…bound writes, 19/60 DEV gold sessions covered 302 sessions attempted in gold-blind hash order; 301 responded; one (index 318, not a gold session) was refused at the agent layer on both the first run and the single permitted retry and is recorded as not attempted. Yield on the primary denominator 149/200; refusals are paraphrases the harness held. Writer runs 384/400. Coverage bound for look 3 recorded here and in fusion-spec-v2 before the look.
…_clean); G1 looks end B1_clean_plus_claims 0.151 vs B1_clean 0.291, Δ −0.140 CI95 [−0.245, −0.041]: not established and significantly worse. Two flat looks → the two-flat-looks stop rule fires; no look 4. B1_clean +0.184 over B1 reproduced exactly. recall_claims alone: 0.130 (R 0.207 > B1 0.103; P 0.000) within its 30/91 coverage bound. Mechanism from the receipt: fused lost 17 questions B1_clean had (gold at prose rank 1–2 in 12/17, absent from the claims list in 16/17, fused rank 7–39) and gained 4. Equal-weight RRF over a ~127-session OR-planner claims list promotes any weakly matching claimed session above the best prose hit. ADR-0011's retrieval benefit is NOT ESTABLISHED on DEV under G1; recorded, not re-tried.
…h, established; G1 bar not met SEALED (84 questions, 56 clusters, sha 90ef67ad byte-verified, mode 0400 before and after, fresh store copy deleted after use): B1_clean 0.283 vs B1 0.115, Δ +0.168 CI95 [+0.076, +0.268], non-inferior on K/P/R — the DEV result (+0.184) replicates on held-out data. Claims fusion replicates its harm (−0.154 vs B1_clean, CI95 [−0.252, −0.065]). G1 NOT ESTABLISHED: the clause's 0.64 macro bar for a fused arm is not approached by any arm built here. The composite claim may not be written. The shippable finding stands: the shipped AND-first planner is the defect; a phrase-token OR planner over prose-only FTS is a +0.17 recall fix.
…CHANGES; dispositions recorded The gpt seat's BLOCKER (receipts void on digest mismatch) is refuted on the artefacts: every result receipt's digest equals amendment-002's committed per-split reference (DEV 9aa2b495 at looks 2 and 3, SEALED 965f5b1e); the field the seat compared against carries the superseded, non-reproducible original digest — a record-keeping defect, accepted. Also accepted: the digest omits retrieval configuration (provenance gap); the stop-rule wording is restated in the ruler's lower-bound terms (+0.092 → +0.092/+0.060 → −0.054: two looks without a rise); the mechanism evidence is now retained by a committed script (look3_mechanism.py → stage-f-look3-mechanism.json: lost 17, gained 4, gold at prose rank ≤2 in 12/17, absent from claims list in 16/17); and B1_clean's lift is relabelled a prose-control planner finding, not a G1 or knowledge-layer result. Gate outcomes unchanged.
… finding; deviations; follow-ons Status → measured, not established (v1.2). G1: fused arm 0.129 on SEALED against a 0.64 bar, worse than the prose control (−0.154). G2: binding held (0 unbound over 2,057 citations); entailment not measurable with a single-model blinded audit. Composite claim not writable. Established separately: the pre-declared prose control beats the shipped path by +0.168 on SEALED — a planner fix for agent-session-tools. Three deviations from the ADR/ruler text recorded with their mitigations.
… showcase) Minimal truthful revision of the Stage A architecture diagram: the retrieval path that was established on SEALED (prose_fts, OR planner, +0.168) is the emphasised edge; the claims fusion edge carries its measured −0.154; the three cards state what is established, what is not, and what held. archify validate --quality showcase: ok; deliver: ok (specification sha a65b0878…, artifact sha 5a79003e…). visual-check could not run in this environment (Chrome sandbox init refused; the Stage A artefact fails identically here), so browser evidence is not claimed.
…ary gap, 35% ranking Deterministic, read-only census over every learner turn in the 2,361 human-driven sessions (8,414 turns; 1,144 pastes over 200 words and 2,693 sub-3-token turns excluded; 4,577 measured). Two proxies per question: stemmed content-token overlap with the REST of its own session, and whether the committed prose FTS + OR planner recovers the question's own session in the top 5 from the other turns alone. Median overlap 0.67; only 7.4% of questions share no content word with their transcript. Self-retrieval: hit 57.3%, miss-by-vocabulary-gap 7.4%, miss-by- RANKING 35.3%. Ranking failures outnumber vocabulary failures five to one, and the vocabulary figure is inflated by unanswered sessions and two adapter artefacts (kilocode 48%, gemini 22% vs kiro 8%, codex 1%, claude_code 0%). Own-session comparison is a lower bound on cross-session drift; stated in the artefact. Decision input for the embeddings question, not a gate.
Andy's 2026-09-10 ruling fixes the supported session sources at claude_code, codex, grok, kiro_cli, opencode, pi plus first-party study_mentor; main hides the 1,279 sessions under seven retired labels at every read path and never deletes them. The archive adapter is the corpus for the paraphrase census and the knowledge-proof arms, so it needs the same allow-list or census v2 would measure a corpus the product no longer serves. ArchiveAdapter.open(path, sources=SUPPORTED_SOURCES) filters every session enumeration -- discover, session_ids, message digests, and all three lineage views -- with `source IN (...)` (bound, sorted). parse/parse_id by explicit id stay unfiltered so deliberate access works. lineage_map now requires both endpoints in scope; an in-scope sub-agent whose parent is under a retired label is reported by the new out_of_scope_lineage() instead of parking in lineage_pending forever. sources=None emits SQL byte-identical to before, so `ingest-archive --include-retired-sources` is a true escape hatch, and the ruler's pinned _load_corpus_digest is untouched so existing receipts stay comparable. The ingest manifest records the scope and hidden counts. One pre-existing fixture inserted its duplicates session as source="repoprompt", which the allow-list would have silently excluded while four assertions kept passing over an absent row; it is now codex. 299 -> 314 tests.
…rawing v1's retired-label rows v1 measured retrieval over a store ingested from all 14 sessions.db labels. The 2026-09-10 ruling fixed the supported set at six harnesses plus the first-party study_mentor source, and hid the 1,279 sessions under the seven retired labels from every read path. A census has to be measured over the corpus the retriever will actually serve, so v2 re-ingests through the scoped ArchiveAdapter into a separate store (v1's store keeps its recorded sha256) and re-runs the same script with the same phrase-token OR planner. Headline: 3,299 questions (was 4,577), self-retrieval@5 61.1% (was 57.3%), vocabulary-gap 5.7% (was 7.4%), ranking 33.2% (was 35.3%). The sidecar decomposes the +3.8pt honestly: every supported harness's question set is byte-identical to v1 (same n, same vocabulary-gap counts), so most of the shift is composition from dropping low-scoring retired labels; the genuine retrieval effect is 23 ranking misses (22 kiro_cli, 1 claude_code) that had lost their top-5 slot to a retired-label session and now rank, with zero new misses anywhere. Ranking remains the dominant miss class and the 70% target is not met on the six-source corpus, so the ranking pass stays open. The sidecar withdraws v1's aider/kilocode/repoprompt/litellm-proxy/gemini_cli rows as product findings without editing the v1 receipt, and records that Stage 4's planner is not a confound: both censuses queried the shipped planner. Every figure in the sidecar was re-derived from the two JSON receipts by script before commit (one miscount, 9 -> 10 withdrawn examples, was caught). The .secrets.baseline gains three entries: the sha256 digests in the two new JSON receipts, the same hex-high-entropy false positive the v1 receipts carry.
…v2 aggregates only implied The single-seat Stage 2 review (gpt-6-astra) made two MAJOR points about the v2 sidecar that were right about evidence class: identical per-harness n and identical vocabulary-gap counts make "same question set, 23 recoveries, zero regressions" an inference, and equal archive counts make "corpus unchanged" an inference. Both stores still exist, so these two scripts measure instead. paraphrase_census_pair.py keys every eligible question by session|sha256(text)|occurrence and runs the census's own decision (same eligibility, tokeniser, self-exclusion, planner, K) in both stores. Result: membership identical (3,299 = 3,299); 41 questions changed class, 32 miss->hit and 9 hit->miss, net +23 -- so the aggregate hid 9 regressions and 9 extra recoveries, including a 4-in/4-out swap inside codex's "unchanged" row. Of the 32 recoveries, 23 had a retired-label session in v1's top-5 (displacement) and 9 had none (bm25/IDF shift from the smaller index); all 9 regressions are IDF-shift. Decomposition: composition +3.09 pt, retrieval +0.70 pt. Content: 4,579/4,580 sessions byte-identical prose; one live codex rollout gained 4 prose events between ingests (learner turns unchanged, none of the 41 changes are its questions). v2's 20 rejects are a subset of v1's 41. paraphrase_census_duplicates.py exists because the v1 aider row (1/204) turned out to be 203 byte-identical copies of one fixture prompt: identical text in N sessions is a structural self-retrieval miss no ranker can fix, and the census counts it as "ranking". On the six-source corpus 475/3,299 (14.4%) eligible questions have an identical sibling in another session and 221 (6.7%) have five or more -- claude_code 78/205 -- which bounds the "ranking misses dominate" reading in the sidecar. Both scripts are read-only on both stores. The baseline gains the sha256 digests in the two JSON receipts, the same false positive as every receipt.
… six corrections measured Single-seat review per the plan council's cadence. The seat found the sidecar's arithmetic sound and its interpretation inferred from aggregates that cannot distinguish net from gross. Rather than hedge the wording, the paired and duplicate-text censuses (dacbe46) measured each point: 32 recoveries / 9 regressions (net +23, 23 displacement + 9 IDF shift, all regressions IDF shift); composition +3.09 pt vs retrieval +0.70 pt; 4,579 of 4,580 sessions byte-identical (one live codex rollout re-exported, no question affected); the v1 aider row was 203 copies of one fixture prompt, not a prose or adapter finding; 221 of the 1,096 "ranking" misses are identical-text duplicates no ranker can win, so ranking is 26.5-33.2%, not 33.2%; the 21 under-threshold questions are opencode 16 + pi 5, study_mentor has 0 eligible learner turns. The record lists the six sidecar sentences it supersedes (receipts are never edited) and what changes for Stage 4 and the later ranking pass.
…s pre-planner and planner code alike score.py builds the shipped arm from mcp_server._session_search_queries, which only exists after PR #18's 4fe2e4c; a B0 pinned at today's main has no such helper. This wrapper mirrors whichever query shape the imported package ships (single escape_fts_query AND query, or the planner's sequence) and purges the module cache on EVERY arm build -- the harness only purged when a pin was given, so the second arm silently reused the first arm's modules and both arms scored one code (caught when both arms printed the same shape). Used for the Stage 4 receipt on main.
…ecar -- code, tests, migrations, registrations Owner ruling 2026-09-10 (ADR-0011 on main): OKF is not part of the solution and is removed entirely, leaving no remnant, in preparation for a semantic layer. Evidence in receipts/okf-removal-inventory-2026-09-10.md: no serving path read any of it, 0/2,033 imported concepts citation-bound, the ontology moved fused recall by +0.00, value gates never reached. agent-session-tools: 12 modules deleted (concept_cli, concept_live, concept_schema, concepts, okf_import, ontology, ontology_live, plus the sidecar-only authorization, projection, safe_fs, winddown and recall seams) and 18 test files; migrations v48/v49 and their registration removed, CURRENT_VERSION back to 47 (what main ships); the memory_winddown and memory_recall MCP tools removed so the tool set equals main's; live_ontology/live_concepts markers removed from both pyproject files. Ten files whose whole branch diff was OKF were restored to the MERGE BASE (not main's current version -- restoring to main would have imported main's newer work; caught and corrected during the lane's own audit). recall.py was memory_recall's engine (PR #18 drop-half, d573133): deleted rather than kept as a second session_search. studyloop: check_ontology_freshness and its doctor registration removed (_doctor.py and doctor/harness.py now match main); test_doctor_ontology.py deleted. scripts/b4_recall_acceptance.py deleted (existed only to run the v49 migration and import_okf); pin_poc_set.py docstring retargeted. Kept, because they are not OKF: the learning-memory package and its claims/evidence store; studyloop's learning tier (history/concepts.py, learning/concept_quality.py, get_concept_context -- identical on main). Gates on the worktree: ruff check clean, 727 files formatted, pyright 0/0/0; agent-session-tools 1,691 passed / 0 failed; studyloop reds equal the pinned environment set plus two expected on a branch that predates main's Stage 3 fix. Finish line: git grep for okf|ontolog|concept_sidecar|context_concept|memory_winddown|memory_recall over packages/ returns nothing.
…ED; amend the frozen ruler; move the evidence receipts ADR 0011 (claim-centric learning memory) keeps its text: a dated "Superseded sections" note at the top marks every OKF-import, tier-1 ontology and concept-sidecar section RETIRED 2026-09-10 and cites the removal inventory; the claim-centric learning-memory decision itself stands. The validation ruler is frozen, so it gains an amendment recording that G3a/G3b scored layers that no longer exist, rather than a rewrite. docs/session-memory.md, context-memory.md and mcp.md no longer describe the removed layers; CHANGELOG gains a Removed entry naming them and why. The four OKF evidence receipts under docs/data (migration v48/v49 receipts, tier-1 baseline, B4 live evidence; ~7 KB) move unchanged into receipts/retired-okf-evidence/ -- they are the measurements the retirement rests on, so they live with the other receipts instead of beside the product's data files. Nothing under docs/ describes OKF in the present tense; the finish-line grep over the branch hits only receipts, the RETIRED-marked ADR sections, the learner-topic string, openspec (retired in Stage 8) and the CHANGELOG entry.
…of (byte-identical to main)
NetDevAutomate
added a commit
that referenced
this pull request
Sep 15, 2026
Council D-13: ADR-0011 is amended, not rewritten. The header gains `Amended: 2026-09-15`; the two statements that no longer hold are kept verbatim and marked superseded in place; a dated "Disposition after semantic-layer completion" section records what actually happened. Why an amendment and not a rewrite: the ADR is the record of what was decided on 2026-09-10 with the evidence then in hand. Two of its claims were overtaken -- the branch ADR's claim-centric learning-memory decision "stands and will be renumbered when merged" (it never merged; PR #19 is closed, tip 464a8cd archived), and the learning-memory store was "the semantic layer's prerequisites" (the programme sealed on 2026-09-15 without it, and the branch's own Stage F measured the fused claims arm at -0.140 recall). Overwriting them would erase the reasoning a reader needs to judge the decision; marking them superseded keeps both the history and the correction visible. The section also separates the one portable hypothesis the branch left -- plan_prose_query's OR widen -- from the retired storage architecture and points at its pre-registered verdict (receipts/lexical/ or-fallback-dev-2026-09-15.md: adopt false, clause 1), and states that a never-merged ADR is not renumbered. No tag is created and GitHub is not touched by this commit; the tag and PR state are recorded as D-13 decided them.
NetDevAutomate
added a commit
that referenced
this pull request
Sep 15, 2026
The council review of the 2026-09-15 amendment to ADR-0011 (GPT §5, with DeepSeek and Grok concurring on decision-versus-execution) found the disposition section stating things as done that had only been decided, and drawing three conclusions wider than the receipts they cite. Each is narrowed to what the evidence establishes; every original 2026-09-10 sentence and both dated supersession markers are kept intact. - Decision, not execution: "PR #19 is closed", "its tip is tagged" and "remain reachable via that tag" were present-tense completion claims. Verified before this edit: `gh pr view 19` reports OPEN, no `archive/feat-knowledge-proof-2026-09-15` tag exists, and 464a8cd is the branch tip on both local and origin. The header annotation and item 4 now read "Decision: close PR #19 and tag tip 464a8cd … Execution pending; recorded here when command output establishes it", and item 4 names the two commands whose output will establish it. Item 1's "its pull request is closed" becomes "is to be closed" for the same reason. An ADR records what is true, not what the next script will do. - The prerequisite conclusion is bounded to this shipped programme. Sealing without the claims/evidence store refutes its necessity here; it does not show claim-centric memory has no other useful role. - The measured loss is attributed to the tested fused-claims retrieval arm, not to "the store": Stage F compared retrieval configurations, not storage in isolation. - "did not carry" becomes "was not established for the pre-registered narrow widen placement on this DEV corpus against the Stage 2 planner", and the two OR-only arms are stated as neither adopted nor disproved — the receipt tested one placement, not the hypothesis in every form. - "The one retrieval win the branch produced" becomes "the historical retrieval improvement cited here": the supplied evidence does not support an exhaustive claim. Nothing here executes the closure or the tag; that remains the owner's action.
NetDevAutomate
added a commit
that referenced
this pull request
Sep 15, 2026
…ected; ADR-0011 amended The last loose end from the retired knowledge-proof programme (PR #19): its one established retrieval lift, plan_prose_query (+0.142 DEV / +0.168 SEALED against the pre-Stage-2 planner), re-measured on main under a pre-registered protocol. Narrow OR-widen candidate: recall@5 0.1599 vs shipped 0.1700, delta -0.010, CI95 [-0.050, +0.028] -> adopt: false (clause 1). The historical lift was mostly the crash fix main already has. Lands: the prose_or_query helper and planner-variant eval arms as measured code (no serving change; pre-planner golden byte-identical), the lexical-verdict door that fails closed, precision@K + value bootstrap, the pre-registration and measurement receipts, and the ADR-0011 amendment recording the disposition with decision separated from execution. Council: planning D-12/D-13 (GPT Astra, Grok 4.6, Kimi K2); receipt review (GPT Astra, Grok 4.6, DeepSeek R1) unanimous on the rejection. Records under docs/architecture/plan-integration/council/ on the seam branch.
Owner
Author
|
Closing without merging. Its base (#18) closed on 2026-09-10 and Disposition (owner ruling 2026-09-15, recorded in ADR-0011 §"Disposition after semantic-layer completion"):
Nothing becomes unreachable: tip Council record: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #18 (
feat/sessionweaver-phase2-retrofit); do not merge before it.Purpose of this PR is the pre-registered proof programme for the concept sidecar, tier-1 ontology and embeddings. It exists as a draft so GitHub CI runs on every push — the authoring sandbox cannot run Chrome or
ps, so CI is the zero-failure oracle named in the ruler.docs/architecture/session-memory/validation-ruler.mddocs/architecture/session-memory/receipts/council-ruler-review.mddocs/architecture/session-memory/receipts/as stages complete.Merge decision stays human. Changing the ruler is listed as never-automatic.