Skip to content

Repair wp/7.1 branch Actions: Remove the broken flaky-test-to-GitHub-issues reporting job - #150

Closed
desrosj wants to merge 25 commits into
fix-actions-7.1/pin-wp-env-core-versionfrom
fix-actions-7.1/disable-flaky-test-issue-reporting
Closed

desrosj wants to merge 25 commits into
fix-actions-7.1/pin-wp-env-core-versionfrom
fix-actions-7.1/disable-flaky-test-issue-reporting

Conversation

@desrosj

@desrosj desrosj commented Sep 1, 2026 •

Copy link
Copy Markdown
Owner

What?

Removes end2end-test.yml's report-to-issues job.

Why?

This job checks out live trunk (not this branch) before building and running ./packages/report-flaky-tests, which means it always runs against whatever trunk's tooling looks like today rather than a fixed target

On wp/7.1 specifically, trunk's packages/report-flaky-tests/action.yml now requires an output-path input this job never supplies (confirmed directly against a real, recent CI run showing the job already drifting — an earlier "Unexpected input(s) 'label'" warning as trunk's action shape changed out from under it). Removing the job only drops an automated notification; no actual test-correctness signal depends on it.

Use of AI Tools

This PR was created by Claude Code under my supervision. All code should be treated as AI-produced and not yet reviewed by a human until this PR is marked Ready for Review.

@desrosj
desrosj force-pushed the fix-actions-7.1/pin-wp-env-core-version branch from 73c992c to b410aee Compare September 1, 2026 13:46
@desrosj
desrosj force-pushed the fix-actions-7.1/disable-flaky-test-issue-reporting branch from 972170b to 797aef3 Compare September 1, 2026 13:46
@desrosj
desrosj force-pushed the fix-actions-7.1/pin-wp-env-core-version branch from b410aee to c64d0db Compare September 1, 2026 14:01
@desrosj
desrosj force-pushed the fix-actions-7.1/disable-flaky-test-issue-reporting branch from 797aef3 to c789f80 Compare September 1, 2026 14:01
@desrosj
desrosj force-pushed the fix-actions-7.1/pin-wp-env-core-version branch from c64d0db to b27b8e8 Compare September 10, 2026 18:18
@desrosj
desrosj force-pushed the fix-actions-7.1/disable-flaky-test-issue-reporting branch from c789f80 to 06ca35c Compare September 10, 2026 18:18
@desrosj
desrosj force-pushed the fix-actions-7.1/pin-wp-env-core-version branch from b27b8e8 to 502b7ef Compare September 10, 2026 20:11
@desrosj
desrosj force-pushed the fix-actions-7.1/disable-flaky-test-issue-reporting branch from 06ca35c to 13fb241 Compare September 10, 2026 20:11
@desrosj
desrosj force-pushed the fix-actions-7.1/pin-wp-env-core-version branch from 502b7ef to ed3a397 Compare September 10, 2026 23:59
@desrosj
desrosj force-pushed the fix-actions-7.1/disable-flaky-test-issue-reporting branch from 13fb241 to 7c4b713 Compare September 11, 2026 00:00
@desrosj
desrosj force-pushed the fix-actions-7.1/pin-wp-env-core-version branch from ed3a397 to ad9508c Compare September 11, 2026 00:29
@desrosj
desrosj force-pushed the fix-actions-7.1/disable-flaky-test-issue-reporting branch from 7c4b713 to 8aa0a4d Compare September 11, 2026 00:29
@desrosj
desrosj marked this pull request as ready for review September 11, 2026 00:46
@desrosj
desrosj force-pushed the fix-actions-7.1/pin-wp-env-core-version branch from ad9508c to 94bedb1 Compare September 11, 2026 00:56
@desrosj
desrosj force-pushed the fix-actions-7.1/disable-flaky-test-issue-reporting branch from 8aa0a4d to ebb3234 Compare September 11, 2026 00:56
@github-actions

Copy link
Copy Markdown

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: desrosj <desrosj@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

@desrosj
desrosj force-pushed the fix-actions-7.1/pin-wp-env-core-version branch from 94bedb1 to 30c9fc8 Compare September 11, 2026 02:42
@desrosj
desrosj force-pushed the fix-actions-7.1/disable-flaky-test-issue-reporting branch from ebb3234 to 2846f9e Compare September 11, 2026 02:42
`storybook-pages.yml` deploys the single canonical Storybook GitHub
Pages site and only triggers on pushes to `trunk`; there is one
Storybook site for the whole project, not one per branch, so this
branch's copy can never run.
`cherry-pick-wp-release.yml` only triggers on pushes to `trunk` or
labeled pull requests targeting `trunk`. `wp/X.Y` branches, including
this one, are its destination, never its trigger source, so it can
never run here.

`publish-npm-packages.yml` is kept on this branch per updated project
policy.
…oad workflows.

`sync-assets-to-plugin-repo.yml` triggers only on `push: branches:
[trunk]` events that touch `assets/**`, so it can never run against
this branch. `upload-release-to-plugin-repo.yml` is removed too: it
triggers on `release: types: [published]`, which isn't literally
trunk-restricted, but no release events actually happen in the
context of these numbered `wp/X.Y` branches — releases are cut from
separate `release/X.Y` branches instead — so it's functionally dead
weight here despite the trigger being technically reachable.
`stale-issue-gardening.yml` is schedule-triggered, and GitHub only
ever runs a scheduled workflow using the copy on the repository's
default branch, regardless of any copy present elsewhere — this
branch's copy can never execute.

`sync-backport-changelog.yml` triggers on `push: branches: [trunk]`
or `issues: types: [labeled]`. The push trigger is trunk-only, and
the issue-label trigger isn't tied to any git ref at all — like the
schedule case above, GitHub always runs it using the workflow file
from the repository's default branch, so this branch's copy is never
the one that actually executes for that event either.
`check-backport-changelog.yml` and `check-package-changelogs.yml`
both trigger on `pull_request: branches: [trunk]` only, so neither
can ever run against a pull request targeting this branch.
Removes the local `reusable-workflow-lint.yml` and points
`workflow-lint.yml` at
`WordPress/gutenberg/.github/workflows/reusable-workflow-lint.yml@trunk`
instead, so this branch always lints against trunk's current lint
logic rather than a stale local copy.

Also widens `workflow-lint.yml`'s own `push`/`pull_request` branch
filters from `trunk` only to also include `release/[0-9]+.[0-9]`,
`wp/[0-9]+.[0-9]`, `wp/latest`, and `wp/next` — porting
`WordPress#82744`'s fix for this file. Without this, pushes
or pull requests targeting this branch itself never triggered linting
at all, since `trunk` was the only branch the filter recognized.
`actions/checkout` was pinned to v7.0.0 across every workflow file
that uses it. v7.0.1 is the latest release.
`.github/setup-node/action.yml` (the composite action most workflows
call into for Node.js setup) still pinned `actions/setup-node` to
v6.4.0, older than the v7.0.0 pin some workflow files call the action
with directly. v7.0.0 is the latest release.
`docker/login-action` was pinned to v4.4.0 in `build-plugin-zip.yml`.
v4.6.0 is the latest release.
`oras-project/setup-oras` was pinned to v2.0.0 in
`build-plugin-zip.yml`. v2.0.1 is the latest release.
`actions/labeler` was pinned to v6.1.0 in
`pr-labeling-automation.yml`. v7.0.0 is the latest release.
`mheap/github-action-required-labels` was pinned to v5.5.2 in
`enforce-pr-labels.yml`. v5.6.0 is the latest release.

Trunk has since replaced this action with a custom `tools/pr-meta`-based
implementation, so there is no trunk pin to follow here; the action is
bumped to its own latest upstream release instead. Porting trunk's
replacement is a larger design change (a different label-check
mechanism entirely), not a version bump, and is out of scope for this
CI repair.
`publish-npm-packages.yml` was restored to this branch after this
layer's original `actions/checkout` bump had already been applied to
every other workflow file, so its three `actions/checkout` calls were
left on the old v7.0.0 pin. This follow-up brings it in line with the
rest of the layer's final state, pinning it to v7.0.1 like every other
workflow file in the tree.
Brings the remaining direct `actions/setup-node` calls (in `bundle-size.yml`, `build-plugin-zip.yml`, `static-checks.yml`, and `publish-npm-packages.yml`) in line with what the composite `.github/setup-node/action.yml` and every other workflow file already use.
Corrects five pinned-SHA version comments that no longer matched what
the SHA actually resolves to:

- `preactjs/compressed-size-action`: `v2.10.0` -> `2.10.0` (the real
  tag has no leading "v").
- `shivammathur/setup-php` (x2): `v2.37.2` -> `2.37.2`. The SHA is also
  the current tip of the `master` branch, which zizmor's own fix
  offered as an alternative, but a branch name is a moving target and
  would go stale the moment `master` advances further -- the version
  tag stays accurate indefinitely and matches this repo's existing
  convention for every other pin.
- `ramsey/composer-install` (x2): `v4.0.0` -> `4.0.0` (same "v" prefix
  issue as `compressed-size-action`).

No pinned SHA was changed, only the trailing comments.
@desrosj
desrosj force-pushed the fix-actions-7.1/pin-wp-env-core-version branch from 30c9fc8 to 80f7bb2 Compare September 11, 2026 16:59
@desrosj
desrosj force-pushed the fix-actions-7.1/disable-flaky-test-issue-reporting branch from 2846f9e to 4325529 Compare September 11, 2026 16:59
This file didn't exist yet when this layer originally bumped every
other workflow's `actions/checkout` pin, since `storybook-check.yml`
was removed by the `remove-rn-workflows` layer and only restored
afterward. Now that it's back, bring its pin in line with the same
`v7.0.1` target every other file in this stack already uses.
Every `actions/setup-node` call in the tree passed `check-latest: true`,
which makes the action re-check the toolchain cache against whatever
Node.js version is currently "latest" upstream on every run. This
branch is frozen to whatever Node.js major `.nvmrc` pins and will
never move to a newer one, so the extra check only adds latency
without ever changing which version actually gets installed.
The `unit-js` and `unit-js-date` jobs ran on Node.js 20, 22, and 24,
with a PR-only `exclude:` restricting PRs to Node 20. This branch is
frozen to whatever Node.js major `.nvmrc` pins and will never run
under a different Node.js version in practice, so testing against
three separate majors on every push doesn't catch anything a single
run against the pinned version wouldn't.

`node-version` is no longer passed to `./.github/setup-node`, so it
falls back to `.nvmrc`. The `FK_ENV_NODEJS` flakiness-reporting
dimension is removed along with it, since there is no longer more
than one Node.js version to distinguish. The `event` matrix
dimension is left in place even though it's now a no-op for the
`unit-js-date` job (nothing left excludes on it) — removing that
PR-restriction pattern entirely is separate, deferred work.
The `checks` job ran on Node.js 20, 22, and 24 across three
operating systems, with a PR-only `exclude:` restricting PRs to
Node 20 on Ubuntu. This branch is frozen to whatever Node.js major
`.nvmrc` pins and will never run under a different Node.js version
in practice, so testing against three separate majors on every push
doesn't catch anything a single run against the pinned version
wouldn't.

`node-version` is no longer passed to `./.github/setup-node`, so it
falls back to `.nvmrc`. The OS-restriction excludes are preserved
unchanged — they're an orthogonal concern to the Node.js version.
The `event` matrix dimension is left in place even though only the
OS-restriction still uses it — removing that PR-restriction pattern
entirely is separate, deferred work.
This workflow was missed by the earlier commits in this branch that
eliminated the `node` matrix dimension from `unit-test.yml` and
`create-block.yml` — it kept testing three Node versions (`20`, `22`,
`24`) via `actions/setup-node`'s `node-version: ${{ matrix.node }}`,
the same wasteful multi-version testing this branch will never need
since it's pinned to `.nvmrc`'s single version.

Removes the `node` matrix dimension and the `exclude` entry that
existed solely to restrict it on pull requests, and switches
`actions/setup-node` to `node-version-file: '.nvmrc'`. The `include`
entry that generates lint annotations once is adjusted to key off
`os` alone, since there's no longer a Node dimension to combine it
with, but keeps the same effect: annotations are still generated
exactly once, on the canonical `ubuntu-24.04` run. The job's
display-name template and remaining `os`-only exclude are adjusted to
match, preserving the existing behavior of showing a bare "All" for
the canonical run and an OS suffix for the others.
`publish-npm-packages.yml` was restored to this branch after this
layer's earlier commit had already removed `check-latest: true` from
every `actions/setup-node` call in the tree, so its two calls were
left with the flag still set. This follow-up brings it in line with
the rest of the layer's final state: this branch is frozen to whatever
Node.js major `.nvmrc` pins and will never move to a newer one, so the
extra check only adds latency without ever changing which version
actually gets installed.
Both `.wp-env.json` and `.wp-env.test.json` pinned `core` to the bare
`WordPress/WordPress` source string, which `@wordpress/env` resolves
to Core's `master` branch — i.e. Core trunk — unless overridden. On a
frozen `wp/7.1` maintenance branch that means every local environment
and every CI run silently tracks whatever Core trunk currently is,
rather than the actual WordPress version this branch is maintained
against.

Pinning to `WordPress/WordPress#7.1-branch` makes both environments
install the Core version this Gutenberg branch actually targets,
matching the pattern already used for a specific WordPress release.
The `test-php` job ran every PHP version against both the current
and the previous major WordPress version, computed at runtime by the
`compute-previous-wordpress-version` job querying the WordPress.org
stable-check API and overriding `WP_ENV_CORE` accordingly. That
machinery exists to catch trunk-tracking regressions against Core's
moving target, which doesn't apply on a frozen `wp/7.1` maintenance
branch — `.wp-env.json`/`.wp-env.test.json` now pin `core` to the
`7.1-branch` directly (previous commit in this stack), so there is
no longer a "previous major version" to compute or test against.

The `compute-previous-wordpress-version` job, the `wordpress` matrix
dimension and its associated `exclude:` entries, and the
`WP_ENV_CORE` env override are all removed. `test-php`'s `needs:`
drops the now-deleted job, keeping the array brackets for the single
remaining entry. The `event` matrix dimension and its PHP-version/
multisite PR-restriction excludes are left in place — removing that
PR-restriction pattern entirely is separate, deferred work.
@desrosj
desrosj force-pushed the fix-actions-7.1/pin-wp-env-core-version branch from 80f7bb2 to c391e0d Compare September 11, 2026 17:03
`report-to-issues` checks out `ref: trunk` and then runs
`./packages/report-flaky-tests` from that checkout — always today's
live trunk copy of the action, never this frozen branch's own. Trunk
has since redesigned this action into a PR-comment-only report
(no more GitHub issues), with a different, incompatible set of
inputs (`output-path` instead of `repo-token`/`label`). A real run
from 2026-08-17 already shows GitHub Actions warning
`Unexpected input(s) 'label'` for the mismatch at that point in
trunk's history; current trunk's `action.yml` requires `output-path`
with no default, which this branch's job step never supplies, so any
future run where `has-flaky-test-report` is true will fail outright
with "Input required and not supplied: output-path". The job has
been `skipped` in every recent run on this branch (no flaky tests to
report), so the failure hasn't shown up live yet, but the break is
structural — trunk will keep evolving this action out from under a
job that was never meant to track it.

Retrofitting trunk's current design was considered and rejected: it's
not a straight port (trunk's version predates a CJS→ESM migration
this package has also gone through) and it's also a real product
policy change — issues become PR comments only — that's out of scope
for a CI repair on a frozen, security-maintenance-only branch. This
only drops an automated notification; no test-correctness signal
depends on it.

The `has-flaky-test-report` output on `merge-artifacts`, which
existed solely to gate this job, is removed along with it. The
`merge-flaky-tests-reports` step itself is left in place, since it
still has independent value consolidating the artifact for anyone
who wants to download it manually.

`packages/report-flaky-tests` itself is left in place, now unused by
any workflow — removing the package is optional cleanup out of scope
for this fix.
@desrosj
desrosj force-pushed the fix-actions-7.1/disable-flaky-test-issue-reporting branch from 4325529 to b72110e Compare September 11, 2026 17:04
@desrosj
desrosj force-pushed the fix-actions-7.1/pin-wp-env-core-version branch from c391e0d to 4144e3f Compare September 11, 2026 18:26
@desrosj
desrosj removed this pull request from stack #177 September 11, 2026 18:31
@desrosj

desrosj commented Sep 11, 2026

Copy link
Copy Markdown
Owner Author

Closing — this layer is now fully redundant.

Real wp/7.1 merged WordPress/gutenberg#82249 ("CI: Unify the automation comments into a single PR comment") since this fork's wp/7.1 mirror was last synced. That PR replaced the report-to-issues job this PR removes with a completely different, working pr-meta job (PR-comment-based reporting via tools/pr-meta, not GitHub issues) — the exact class of fix this PR was going for, done natively upstream instead.

Nothing left for this PR to do. Closing without merging; the branch will be deleted.

@desrosj desrosj closed this Sep 11, 2026
@desrosj
desrosj deleted the fix-actions-7.1/disable-flaky-test-issue-reporting branch September 11, 2026 18:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant