Tracking: Verified Completions - OpenAI-compatible endpoint with consistency checking #89

What we are building

An OpenAI-compatible chat endpoint inside InputLayer. Clients change base_url and nothing else. On every request the gateway (a) forwards the conversation to Claude for the completion and (b) in parallel, turns the conversation into small factual claims inside a per-conversation knowledge graph, where a fixed rule pack finds contradictions. Findings come back attached to the normal response, each with the quoted sentences that clash and the proof tree that connects them.

Example: message 1 says "flying out of Geneva on August 14th", message 3 says "since we leave on the 12th". The response carries a finding that quotes both spans and shows the rule that fired. Not a second LLM opinion; a deterministic check with receipts.

Why this feature earns its place: the engine's incremental evaluation keeps per-turn checking at milliseconds no matter how long the conversation gets, correct retraction handles clients that edit history, and every finding is grounded in quoted text. It also dogfoods the whole engine (facts, rules, recursion, retraction, proof trees) and gives the repo a thirty-second demo.

Full design: docs/internals/verified-completions/docs/rfc-verified-completions.md (plus REVIEW-GUIDE.md and HANDOFF.md next to it).

Ground rules baked into the design

• The LLM authors data only, never rules. Rules are human-written and frozen at load
• Only claims marked as plainly stated (or plainly denied) can trigger findings. Hedges, questions, opinions, and conditionals are recorded but stay inert, so uncertain language never produces a false alarm
• Every claim must carry a verbatim quote from the source message, or it is dropped
• A correction ("actually, make that the 12th") retracts and replaces; it is not a contradiction
• False alarms are the existential risk: a checker that cries wolf gets turned off. CI gates on the false-alarm rate

Work items in order

• Verified Completions Phase 1: run the consistency rule pack on the real engine #81 Phase 1: run the rule pack on the real engine (nothing has been executed yet; expected outputs are hand-derived)
• Verified Completions: split the repo into a Cargo workspace (engine + gateway) #82 Split the repo into a Cargo workspace (can run in parallel with Verified Completions Phase 1: run the consistency rule pack on the real engine #81)
• Verified Completions M0: POST /v1/verify - extract facts, return consistency findings with proofs #83 M0: POST /v1/verify - the verify-only endpoint (needs Verified Completions Phase 1: run the consistency rule pack on the real engine #81, Verified Completions: split the repo into a Cargo workspace (engine + gateway) #82)
• Verified Completions M1: OpenAI-compatible /v1/chat/completions proxy (annotate + enforce) #84 M1: POST /v1/chat/completions proxy with annotate and enforce modes (needs Verified Completions M0: POST /v1/verify - extract facts, return consistency findings with proofs #83)
• Verified Completions M2: incremental sessions, edited-history retraction, streaming #85 M2: incremental sessions, edited-history retraction, streaming (needs Verified Completions M1: OpenAI-compatible /v1/chat/completions proxy (annotate + enforce) #84)
• Verified Completions M3: verify model output against system prompt constraints, repair mode #86 M3: verify model output against system prompt constraints, repair mode (needs Verified Completions M2: incremental sessions, edited-history retraction, streaming #85)
• Verified Completions M4: hardening - entailment guard, pluggable extractor, tenant limits #87 M4: hardening - entailment guard, pluggable extractor, tenant limits (needs Verified Completions M3: verify model output against system prompt constraints, repair mode #86)
• Verified Completions: benchmark suite with CI gate on false alarms #88 Benchmark suite with CI gate on false alarms (engine-only part needs Verified Completions Phase 1: run the consistency rule pack on the real engine #81, end-to-end part needs Verified Completions M0: POST /v1/verify - extract facts, return consistency findings with proofs #83)

Open questions to settle along the way

• Rule pack loading: copy into each session KG (safe default) vs one shared read-only KG, depends on cross-KG read support (Verified Completions M0: POST /v1/verify - extract facts, return consistency findings with proofs #83)
• May a user turn retract a claim the assistant made earlier? (Verified Completions M3: verify model output against system prompt constraints, repair mode #86)
• Detecting references to entities whose defining claim was retracted (Verified Completions M3: verify model output against system prompt constraints, repair mode #86 or later)