Shipped a small MLflow plugin today and figured this was the right place to surface it.

What

mlflow-falsify — an MLflow Run Context Provider that auto-tags every mlflow.start_run() with the SHA-256 hash of a PRML manifest found alongside your code.

pip install mlflow-falsify

If a .prml.yaml exists in your CWD or any parent directory, the plugin adds these tags automatically — no code change in your MLflow workflow:

prml.manifest_hash    <sha256>
prml.manifest_path    .prml.yaml
prml.version          prml/0.1
prml.metric           accuracy
prml.comparator       ">="
prml.threshold        0.85
prml.dataset_id       imagenet-val-2012

Why

PRML — Pre-Registered ML Manifest is an open spec (CC BY 4.0) for committing an evaluation claim to a SHA-256 hash before the run. The hash binds metric, comparator, threshold, dataset hash, and seed, so silent threshold drift between v1 and v2 of a report becomes a CI failure, not a footnote. Four reference implementations (Python, JS, Go, Rust) are byte-equivalent on 20 conformance vectors.

Layered on MLflow, this gives you a tamper-evident receipt inside the run record itself — useful for EU AI Act Article 12 logging evidence, reproducibility audits, and detecting "we changed the threshold and pretended we didn't" between experiments.

Details

• Discovery: standard mlflow.run_context_provider entry point
• Dependencies: mlflow>=2.0, pyyaml>=5.1 (no dependency on the falsify package — canonicalize is vendored byte-for-byte from falsify v0.1.4)
• Tests: 5 unittests including a TV-001 conformance assertion against the v0.1 normative vector
• License: MIT
• Zenodo DOI for the spec itself: 10.5281/zenodo.20177839
• Source: studio-11-co/mlflow-falsify

Feedback welcome

If this fits an existing MLflow integration pattern I missed, or if there's a better entry-point category for this kind of "verification layer on top of run logging" plugin, would value the pointer. Also happy to evolve the spec — v0.2 RFC comment window closes 2026-05-22.