Repository navigation
refactor: share execution preservation snapshots - #77
Merged
Merged
Conversation
|
The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message. To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook. Props Bot commandsYou can trigger Props Bot actions by commenting on this PR:
|
An empty prefix matches every metadata key, so it would silently allow any meta change on that record. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017dieY2m63AHakMQc6MZEM1
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The CLI fixtures repeated large before/after preservation blocks with generated no-op branches, making their grading boundaries difficult to review. This follow-up to #71 replaces those blocks with shared trusted snapshots and explicit per-record allowlists, addressing the implementation deferred in #71 (comment).
WPBench\Runtime\Preservation::capture_posts()captures every existing non-revision post;capture_users()captures every account. Snapshots retain private before-state in the verifier process and exposepreserved($allow)pluscount(). Allowlists use captured IDs and declarefields, exactmetakeys, literalmeta_prefixes,terms(category/post_tag), ordeleted. Comparison retains strict stored metadata/type/multiplicity checks and detects unrelated modifications, deletions, and relationship changes. The helper is loaded for both PHP and CLI verification.Migrates 25 content comparisons and 26 account comparisons across 26 CLI tests, removing about 59 KB of repeated JSON. All 27 retained IDs, model-facing prompts/requirements, reference scripts, authored exploits, assertion descriptions/weights, and task-specific checks are unchanged. Menu assertions keep their raw rows, record counts, and
nav_menuchecks; URL migration retains its specialized serialization/GUID/table checks. The core suite is untouched.Allowances were carried over from the actual assertions after inspecting WordPress
origin/7.1source (dcc660c01a76f8fe46faed6850e5f8e22c88eb80), includingWP_Post::to_array(),wp_insert_post(),wp_update_post(),_publish_post_hook(),WP_User::set_role(), andwp_delete_user(). Derivedpost_category/tags_inputfields are checked via term IDs. There are no blanket timestamp or publishing-metadata exemptions; existing date/field/meta allowances stay scoped to their declared records. Regression cases exercise modified dates, draft date refreshes,_pingme/_encloseme, roles, reassignment/deletion, nested metadata types, duplicate values, metadata prefixes, relationship changes, and unrelated records through both PHP and CLI execution.Validation uses real WordPress 7.1 + SQLite private containers (
reset_per_test, CLI concurrency 4), without model API calls:Additional checks: full Python suite with
WP_BENCH_TEST_IMAGE=wp-bench-grader:dev(345 passed, including 38 helper regressions and the existing Docker isolation/CLI integration tests);ruff check python;mypy python; PHP syntax for all 10 runtime files and 116 CLI setup/assertion/teardown blocks; CLI dry run (27 tasks); dataset export (377 rows);git diff --check.The helper only checks captured records: task assertions still verify intended new values and creation/count rules, and custom taxonomies/raw database contracts remain specialized. It is test infrastructure, not a PHP security boundary. Existing SQLite support, cache refresh, shell policy, and private-container isolation are unchanged. This does not include repeated model runs or saturation analysis. The original review thread was intentionally left open when this implementation was deferred.