Replace MySQL with SQLite for the benchmark runtime - #68

## What

Stacked on #68, targeting `refactor/sqlite-runtime`. Merge #68 first.

Give every candidate a fresh Docker container and enable isolated
concurrent execution. Database-only resets leave plugins, uploads,
temporary files, logs, and processes shared; a candidate can enumerate
another test's plugin files. This PR makes serial and concurrent runs
use the same private runtime boundary.

- Prepare one trusted WordPress install from a resolved image ID. Keep
its SQLite snapshot and configuration on the host, and remove the
preparation container before grading.
- Copy pristine WordPress files into a private bounded tmpfs and restore
the database for each candidate. Existing Compose containers and their
data are never used or reset by isolated runs.
- Disable networking, run as `www-data`, drop all Linux capabilities,
prevent privilege escalation, and make the image filesystem read-only.
Each container has CPU, memory, filesystem, and process limits.
- Remove the entire container after verification, timeout, crash, or
abort, including background processes. Cancel queued futures on errors
and Ctrl-C, including interrupts while waiting in `as_completed`.
Coordinate cleanup with in-flight container creation.
- Support `run.execution_concurrency` under `reset_per_test` and add
`--execution-concurrency`. Reference checks, exploit audits, and
model/skill variants use the private runtime path.
- Disable WordPress's automatic update-check hooks in the offline
verifier. Without this, the admin settings reference triggers a
WordPress.org request and fails with networking disabled. Dataset
assertions and reference solutions are unchanged.
- Record the isolation boundary, requested concurrency, resolved image
ID, and grader limits in result metadata. Document the execution profile
required for published scores and cleanup after forced termination.

New grader settings default to `container_cpus: 1.0`,
`container_memory_mb: 512`, `container_filesystem_mb: 256`, and
`container_pids_limit: 64`. Swap is disabled and `/tmp` is limited to 64
MiB. Default concurrency remains 1. The existing shared runtime and
external CLI grader remain diagnostic modes with `execution_isolation:
none`.

## Why

SQLite makes a private database practical, but database separation alone
does not prevent one candidate from observing or modifying
filesystem/process state belonging to another test. Fresh containers
provide the same state isolation at every concurrency level and discard
mutations between serial tests too.

Containers still share the host kernel and physical resources. CPU
quotas are limits, not reservations; published runs must use a fixed
documented profile and sufficient host capacity. This PR does not
promise identical timeout behavior under arbitrary host load.
Already-running model requests remain subject to their provider
timeout/retry policy when aborting.

## Verification

- [x] `ruff check python` — clean.
- [x] `mypy python` — clean across 46 source files.
- [x] `pytest python -q` — 255 passed; 6 opt-in Docker checks skipped in
this command.
- [x] `WP_BENCH_TEST_IMAGE=wp-bench-grader:isolated-sqlite pytest
python/tests/test_container_runtime_integration.py -q` — all 6 passed.
Covers filesystem/database/core mutations, concurrent plugin visibility,
offline maintenance hooks, enforced limits, and background-process
cleanup on timeout/exit.
- [x] Runtime image build, all seven PHP syntax checks, entrypoint `bash
-n`, and `git diff --check` — passed.
- [x] All 350 reference solutions pass at concurrency 1, 4, and 8, with
identical per-test score dictionaries across runs.
- [x] Replayed 16 saved model completions, including a plugin artifact
and both passing/failing answers: 9 pass and 7 fail identically at
concurrency 1, 4, and 8. No model calls were made.
- [x] Database/security exploit audit at concurrency 4 — 78 incorrect
candidates rejected, 0/9 auditable tests exploitable.
- [x] Live Ctrl-C after six streamed results — exit 130 in 0.475
seconds, partial results retained, no disposable containers left.
- [x] Every completed benchmark/audit left zero disposable containers;
the pre-existing local Compose runtime remained running and untouched.

Local full-suite timings on Docker Desktop (16 CPUs, 8.22 GB available
memory, Apple Silicon), including CLI startup, baseline preparation,
container creation, restoration, verification, and cleanup:

| Concurrent candidates | References passing | Wall time | Speedup over
isolated serial |
| --- | --- | --- | --- |
| 1 | 350/350 | approximately 259 s | 1× |
| 4 | 350/350 | 72.3 s | approximately 3.6× |
| 8 | 350/350 | 54.7 s | approximately 4.7× |

The serial timing is estimated from the run's start/end file timestamps;
parallel timings use a monotonic clock. These compare the isolated
execution path across concurrency levels. Fresh containers add
startup/copy/cleanup overhead relative to a database-only reset.

CI builds the image, runs the six live isolation checks, and executes
SQLite reference checks and the exploit audit with concurrency 4.

## Score impact

- [ ] This PR does **not** change how any test is scored.
- [x] This PR changes runtime behavior — bump runtime to **2.1.0** and
record the container isolation boundary and execution profile.

The scoring formula, suite fixtures, assertions, and reference solutions
are unchanged. Serial/concurrent parity is verified for all references
and the saved model sample. Filesystem permissions, offline networking,
and resource limits can change outcomes compared with the previous
database-only runtime, so published scores must identify the runtime
image and profile rather than assume cross-runtime comparability.

---------

Co-authored-by: Miguel Lezama <lezama@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Matthew Batchelder <borkweb@gmail.com>

## What

Add `wp-cli-v1`: **27 model capability tasks** that execute plain
command or Bash script strings through real WP-CLI in private WordPress
7.1 / SQLite containers. The suite covers content operations, taxonomy
and metadata, users and capabilities, options and transients, reporting,
maintenance, and migration. Thirteen workflows run twice to check
convergence and empty selections.

Tasks combine discovery, selection, changes, and preservation. Advanced
workflows include prefix-scoped URL migration with serialized values,
serialized option-key migration, classic menu repair, ordered widget
relocation, autoload cleanup, enclosure deduplication, incident
credential revocation, and CSV onboarding that preserves existing
accounts. Each advanced task records its research basis and links to
official documentation, upstream reports, installed WP-CLI
implementations, and WordPress 7.1 source.

The verifier captures stdout, stderr, exit status, elapsed time, and
repeat runs; applies shared timeout and output limits; refreshes
WordPress state before trusted assertions; and runs teardown. Bash and
jq are included in the runtime. Suite filtering works for both local
datasets and multi-suite Hugging Face exports, with the selected suite
also used as the result label.

Native WordPress operations are required. Runtime WP-CLI configuration
disables common PHP/SQL execution shortcuts; the exported `wp` function
rejects PHP preloads; file snapshots and loaded-file tracking reject
planted PHP/config files. Read-only `wp db prefix` and `wp db tables`
remain available. These are benchmark policy checks, not a shell
security boundary; Docker provides containment. Network downloads,
MySQL-only operations, and multisite are outside this suite.

## Why

The existing PHP suite does not measure how models operate WordPress
through WP-CLI. These tasks grade realistic workflows and preservation
constraints using the SQLite runtime from #68 and per-candidate
isolation/concurrency from #69. Equivalent native command workflows are
accepted; the grader does not require the reference script's exact
spelling.

Five overlapping tasks that all four sampled models passed were removed:
`e-cli-content-001`, `e-cli-taxonomy-meta-003`, `e-cli-users-001`,
`e-cli-options-002`, and `e-cli-reporting-003`. Retained IDs are
unchanged. Repeated model runs to assess saturation are separate
follow-up work; four successful answers alone do not establish that a
test is permanently saturated.

## Review follow-ups

Preserve and verify all four contributor commits:

- Runtime enforcement catches quoted `eval`, PHP preload flags, and
temporary mu-plugin shortcuts, including files removed after WordPress
loads them and fixture rewrites in the same second.
- Reserve five seconds of the grader timeout so slow scripts can report
their captured output before the outer timeout.
- Default Hugging Face filtering to the selected run suite. A further
regression found during review is also fixed: an explicit
`dataset.suite` now sets the result-suite label, with `--suite`
overriding both.
- Tear down fixture plugins and their activation entries. Remove the
integration case for the deleted plugin-status reporting task, while
retaining cleanup checks for the cron and bootstrap-recovery fixtures.

The proposed shared preservation helper is deferred to a subsequent PR.
It is reusable test infrastructure, and changing every preservation
assertion is a separate refactor from adding CLI support.

## Verification

- [x] `ruff check python` — passed
- [x] `mypy python` — passed, 48 files
- [x] `WP_BENCH_TEST_IMAGE=wp-bench-grader:cli-review pytest python -q`
— **307 passed**, including all Docker integration checks
- [x] Built the updated runtime image; all **9 runtime PHP files**
passed `php -l`
- [x] Real SQLite reference run with eight workers — **27/27 CLI
references passed**
- [x] CLI exploit audit with eight workers — **127 generic/authored
scripts rejected; 0/27 tasks exploitable** by this battery
- [x] `python datasets/export_dataset.py` — **377 rows** exported,
preserving shell contracts and excluding authored exploit candidates
- [x] `git diff --check` — clean

The timeout regression checks retained stdout/stderr through the real
harness timeout path. Policy integration checks cover quoted commands,
config overrides, PHP preload flags, and planted or rewritten PHP files.
Hugging Face regressions cover default selection, explicit filters,
conflicting run labels, and CLI overrides.

## Score impact

- [x] This PR changes scoring/dataset/runtime behavior — adds
**`wp-cli-v1`** and bumps the runtime/plugin from **2.1.0 to 2.2.0**.

A CLI pass requires every behavioral assertion, the expected exit status
on every repeat, completion within execution/output limits, and no
authored forbidden error pattern. Existing PHP contracts and scoring
version are unchanged. Compare CLI scores only for the same task set and
execution profile; the current 27-task suite is not directly comparable
to the earlier 24- or 32-task runs. Publishing the generated export to
the Hub is separate from this PR.

Historical model results for the eight advanced additions (October 12,
before the latest runtime-policy and timeout changes; one answer per
model/test, no skills):

| Model | Passed | Execution pass rate |
| --- | ---: | ---: |
| Claude Opus 5.5 | 7/8 | 87.5% |
| Claude Sonnet 5.5 | 6/8 | 75% |
| GPT-6.1 Sol | 6/8 | 75% |
| GPT-6 Astra | 6/8 | 75% |

All seven failed outputs were reproduced against unchanged assertions;
targeted maintainer corrections passed. Those corrected scripts are
diagnostic only and excluded from model scores. These historical scores
are not a new model run of the current 27-task suite.

---------

Co-authored-by: Miguel Lezama <lezama@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>