Skip to content

Verified Completions Phase 1: run the consistency rule pack on the real engine #81

Description

@jsam

Why

The Verified Completions rule pack (docs/internals/verified-completions/rules/consistency-core.iql) and its test fixture were written in a design session that had no Rust toolchain. They are syntax-checked and linted, but have never run on the real engine. The expected query results in the fixture were derived by hand. Before we build any gateway code on top of this, we need to prove the rules actually work.

What to do

  • Build the engine (cargo build --release), start the server, open the CLI
  • Run docs/internals/verified-completions/quickstart/hello-consistency.iql. Expected: one conflict reported in both orders (a,b) and (b,a), plus a .why proof tree
  • .load docs/internals/verified-completions/rules/consistency-core.iql, then run docs/internals/verified-completions/tests/consistency-core-fixture.iql
  • Compare actual query results against the expected rows written in the fixture comments. Fix the pack or fix the expectations, whichever is wrong
  • Check the two rules most likely to fail at load time:
    • areach (recursion where the attribute is a variable). If the engine rejects it, generate one specialized rule per attribute at load time instead
    • member_tally (an aggregation result used by another rule). Same fallback: restructure if the engine objects
  • Generate a real .iql.out snapshot using the existing harness (tests/example_verification.rs conventions)
  • Re-run docs/internals/verified-completions/tools/lint_iql.py after every edit to the pack

Done when

The fixture runs green against the real engine and a .iql.out snapshot is checked in. The retraction check at the end of the fixture must also pass: after deleting the m4 fact, the three "cycle" findings disappear and every other finding survives.

Notes for whoever picks this up

Two engine facts were already verified by reading the source, do not rediscover them the hard way:

  • Var != Var works on any type, including strings
  • Var < Var only works on ints and floats. On strings it silently evaluates to false, no error. That is why every ordered comparison in the pack runs on integer mirror columns (claim_num, constraint_num), with dates encoded as YYYYMMDD integers

Full context: docs/internals/verified-completions/HANDOFF.md and docs/internals/verified-completions/docs/REVIEW-GUIDE.md.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    backendBackend/server relatedverified-completionsVerified Completions: OpenAI-compatible endpoint with consistency checking

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions