Verified Completions M2: incremental sessions, edited-history retraction, streaming #85

Why

The OpenAI API is stateless: every request carries the full message list. Re-extracting the whole conversation on every turn would make cost grow with history length. The engine is incremental, so the gateway should be too: recognize a conversation it has seen before, extract only the new messages, and let the engine update findings in milliseconds. This is where the engine's incremental evaluation and correct retraction stop being marketing words and become the reason per-turn cost stays flat.

What to do

• Session matching: compute a chained hash over messages (h_i = H(h_prev + role + content)). If a stored session's hash chain is a prefix of the incoming one, reuse that session KG and extract only the new suffix (usually one user turn)
• New conversation: create a KG named _conv_<hash prefix>, load the rule pack, extract everything once
• Edited history: if the client changed or truncated messages at index k, retract all facts whose source message index is >= k, then extract the new suffix. Findings that depended on removed turns must disappear; findings still supported by surviving turns must stay. No recompute from scratch
• Session pinning: optional il_conversation_id body field or X-IL-Conversation header, for clients whose frameworks rewrite the message prefix (sliding-window truncation etc.)
• Eviction: TTL plus LRU, limits from the [verify] config. An evicted conversation that comes back is just re-extracted in full; correctness unaffected, only cost
• Streaming: pass stream: true through as SSE. Verification runs alongside generation; findings are emitted as one extra SSE chunk right before [DONE], on an otherwise empty delta
• il verify CLI: thin wrapper around /v1/verify, so a system prompt can be linted in CI before it ships
• Enable the temporal cycle rules (O1) end to end if anything was deferred in M0

Done when

A 40-turn conversation shows flat per-turn verification cost (one small extraction call plus millisecond rule evaluation, regardless of history length). Editing history removes exactly the findings that depended on the removed turns. Streaming clients get findings in the final chunk without breaking.

Depends on

• /v1/chat/completions proxy (M1)