Skip to content

Performance tests tooling : Compare performance with correct branch - #66196

Merged
cbravobernal merged 2 commits into
trunkfrom
fix/compare-performance-with-correct-branch
Oct 17, 2024
Merged

cbravobernal merged 2 commits into
trunkfrom
fix/compare-performance-with-correct-branch

Conversation

@cbravobernal

Copy link
Copy Markdown
Contributor

What?

Target Performance tests to base branch instead of just Trunk.

Why?

If trunk differs from another base branch, like wp/6.7, changing for example selectors. Performance tests will fail on all cherry picks to releases branches.

Testing Instructions

I'll make this PR to 6.7 to see if performance is being executed there.

@cbravobernal cbravobernal added [Type] Bug An existing feature does not function as intended [Type] Performance Related to performance efforts labels Oct 17, 2024
@cbravobernal
cbravobernal requested a review from desrosj as a code owner October 17, 2024 09:56
@github-actions

github-actions Bot commented Oct 17, 2024 •

Copy link
Copy Markdown

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: cbravobernal <cbravobernal@git.wordpress.org>
Co-authored-by: youknowriad <youknowriad@git.wordpress.org>
Co-authored-by: SantosGuillamot <santosguillamot@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

@cbravobernal cbravobernal added the [Type] Automated Testing Testing infrastructure changes impacting the execution of end-to-end (E2E) and/or unit tests. label Oct 17, 2024
Comment thread .github/workflows/performance.yml Outdated
@youknowriad youknowriad removed [Type] Bug An existing feature does not function as intended [Type] Automated Testing Testing infrastructure changes impacting the execution of end-to-end (E2E) and/or unit tests. labels Oct 17, 2024
@cbravobernal
cbravobernal enabled auto-merge (squash) October 17, 2024 10:23
@SantosGuillamot

Copy link
Copy Markdown
Contributor

Thanks for the fix 🙂 I wasn't sure why the performance tests were failing in manual cherry picks like this one: link.

@cbravobernal
cbravobernal merged commit ab46764 into trunk Oct 17, 2024
@cbravobernal
cbravobernal deleted the fix/compare-performance-with-correct-branch branch October 17, 2024 10:51
@github-actions github-actions Bot added this to the Gutenberg 19.6 milestone Oct 17, 2024
@github-actions

Copy link
Copy Markdown

I just cherry-picked this PR to the wp/6.7 branch to get it included in the next release: f98cdf7

gutenbergplugin pushed a commit that referenced this pull request Oct 17, 2024
…6196)

* Target base branch instead of just trunk

* Update description

Co-authored-by: cbravobernal <cbravobernal@git.wordpress.org>
Co-authored-by: youknowriad <youknowriad@git.wordpress.org>
@github-actions github-actions Bot added the Backported to WP Core Pull request that has been successfully merged into WP Core label Oct 17, 2024
@cbravobernal cbravobernal added the [Type] Bug An existing feature does not function as intended label Oct 30, 2024
@SantosGuillamot SantosGuillamot removed the [Type] Performance Related to performance efforts label Oct 30, 2024
@cbravobernal cbravobernal changed the title Performance tests : - Fix/compare performance with correct branch Performance tests tooling : Compare performance with correct branch Oct 30, 2024
@cbravobernal cbravobernal added the [Type] Build Tooling Issues or PRs related to build tooling label Oct 30, 2024
@github-actions

Copy link
Copy Markdown

Warning: Type of PR label mismatch

To merge this PR, it requires exactly 1 label indicating the type of PR. Other labels are optional and not being checked here.

  • Type-related labels to choose from: [Type] Automated Testing, [Type] Breaking Change, [Type] Bug, [Type] Build Tooling, [Type] Code Quality, [Type] Copy, [Type] Developer Documentation, [Type] Enhancement, [Type] Experimental, [Type] Feature, [Type] New API, [Type] Task, [Type] Technical Prototype, [Type] Performance, [Type] Project Management, [Type] Regression, [Type] Security, [Type] WP Core Ticket, Backport from WordPress Core, Gutenberg Plugin.
  • Labels found: [Type] Bug, [Type] Build Tooling, Backported to WP Core.

Read more about Type labels in Gutenberg. Don't worry if you don't have the required permissions to add labels; the PR reviewer should be able to help with the task.

desrosj added a commit to desrosj/gutenberg that referenced this pull request Sep 21, 2026
The `pull_request`-triggered step in `performance.yml` hardcoded `trunk` as
the comparison branch. On a frozen branch like `wp/5.9` that is always the
wrong target: every pull request opened against `wp/5.9` was compared
against live upstream `trunk` rather than against its own base, so the job
had to build and install current `trunk`'s Gutenberg code using this
branch's pinned Node 14.21.3 toolchain. Current `trunk` requires Node
`>=24.18.0`, so the comparison branch could not even be installed and the
step failed outright before any measurement was taken.

Using the base branch instead of a hardcoded one is the fix upstream
applied in `WordPress#66196` (`ab467645855eef40b1570e5bd0a8db03aa62dad6`),
which renamed the step to `Compare performance with base branch` and
replaced `trunk` with the base ref. That commit does not apply cleanly to
this branch, whose `run:` line predates the `--ci` flag's introduction
elsewhere in the file and has different surrounding steps, so the same
change is made by hand here.

The base ref is read through the `GITHUB_BASE_REF` environment variable
rather than the `${{ github.base_ref }}` expression. Workflow expressions
are substituted into the script text before the shell runs, so a branch
name is interpolated as code; the environment variable form passes it as
data instead. The existing `$GITHUB_SHA` references are quoted at the same
time so every argument on the line is passed the same way, matching the
`release`-triggered step directly below it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DtSAAJkQ9P6vPtDumqStg8
desrosj added a commit to desrosj/gutenberg that referenced this pull request Sep 21, 2026
The `pull_request` leg of `performance.yml` compared every pull request
against a hardcoded `trunk`. On a frozen `wp/X.Y` branch that is always
the wrong comparison point: `trunk` is Gutenberg's live development
branch, whose toolchain and dependency tree have moved on by years from
what this branch's Node 14 / npm lockfile can install, so the second
comparison branch cannot even be built. Comparing against the pull
request's own base branch measures the change the pull request actually
makes.

Ported from `WordPress/gutenberg` commit
`ab467645855eef40b1570e5bd0a8db03aa62dad6`, originally
"Performance tests : -  Fix/compare performance with correct branch"
(`WordPress#66196`). The upstream change does not apply
cleanly to this branch, so it was applied by hand and adapted: the
branch is passed as the `GITHUB_BASE_REF` environment variable rather
than upstream's `${{ github.base_ref }}` expression interpolation, which
keeps workflow-context data out of the generated shell script, and all
three arguments are quoted. This branch's pre-existing `--ci` flag is
preserved.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DtSAAJkQ9P6vPtDumqStg8
desrosj added a commit to desrosj/gutenberg that referenced this pull request Sep 21, 2026
…rdPress#66196)

The `pull_request`-triggered step in `.github/workflows/performance.yml`
hardcoded `trunk` as the branch to compare against. On a frozen `wp/6.4`
branch that is always the wrong reference: Gutenberg's `trunk` has moved
on by several years of tooling and runtime requirements, so it can no
longer even be built with this branch's own pinned Node version, let
alone produce a meaningful performance comparison.

Compare against the pull request's own base branch instead, and rename
the step to match. Upstream's version of this change interpolates
`${{ github.base_ref }}` directly into the `run:` script; this port uses
the equivalent `$GITHUB_BASE_REF` environment variable, and quotes the
existing `$GITHUB_SHA` references alongside it, so no workflow context
value is expanded into a shell command line.

(cherry picked from commit ab46764,
the merge commit for WordPress#66196)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DtSAAJkQ9P6vPtDumqStg8
desrosj added a commit to desrosj/gutenberg that referenced this pull request Sep 21, 2026
…rdPress#66196)

Cherry-picks ab46764
(WordPress#66196) into the `wp/6.3` branch.

`performance.yml`'s `pull_request` step hardcoded `trunk` as the second
comparison branch, so every pull request opened against this branch was
measured against Gutenberg's own live `trunk` rather than against the branch
it actually targets. That comparison cannot even be built here: the perf CLI
checks out and runs `npm ci` against each comparison branch, and this frozen
branch's toolchain is incompatible with `trunk`'s current dependency graph
and lockfile format, so the step fails outright before any measurement runs.

Using the pull request's own base ref compares like with like and keeps the
step working on a branch that will never move back to `trunk`.

Adapted from the upstream commit: the base ref is read from the
`GITHUB_BASE_REF` environment variable rather than interpolated into the
`run:` script as `${{ github.base_ref }}`, matching the environment-variable
form used for `GITHUB_SHA` on the same line and avoiding expression
interpolation directly into a shell command.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DtSAAJkQ9P6vPtDumqStg8
desrosj added a commit to desrosj/gutenberg that referenced this pull request Sep 21, 2026
…rdPress#66196)

The `pull_request` performance comparison compared every pull request
against one hardcoded branch. Pull requests are not always opened against
that branch, so any pull request with a different base was measured against
an unrelated branch and produced a meaningless comparison.

The branch to compare against is now taken from the base branch of the pull
request being tested, and the step is renamed to reflect that it no longer
targets a single fixed branch.

On this branch the hardcoded value is `master`, not `trunk`: it predates the
`master`-to-`trunk` rename, so the upstream change does not apply as-is and
the equivalent edit was made by hand. The base branch is read from the
`GITHUB_BASE_REF` environment variable GitHub Actions already provides
rather than the `${{ github.base_ref }}` workflow expression, and the
remaining arguments are quoted to match. The pre-existing `--ci` flag is
kept.

Ported from WordPress/gutenberg@ab46764
(WordPress#66196).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DtSAAJkQ9P6vPtDumqStg8
desrosj added a commit to desrosj/gutenberg that referenced this pull request Sep 21, 2026
…rdPress#66196)

`.github/workflows/performance.yml` hardcoded `master` as the branch to
compare against. Gutenberg renamed its default branch from `master` to
`trunk`, and `refs/heads/master` no longer exists on
`WordPress/gutenberg` at all, so the comparison can never be built: the
performance CLI fetches each comparison branch from that repository
(`bin/plugin/config.js` hardcodes `WordPress` as the repository owner)
and `git fetch origin master` fails outright before any measurement
runs.

Compare against the pull request's own base branch instead. That is the
branch a pull request is actually proposing to change, so it is the only
meaningful reference for a frozen branch that will never move back to a
shared default branch.

Adapted from the upstream commit, which does not apply literally here:

- Upstream renames a step already called `Compare performance with
  trunk`. On this branch that name sits on the *job* (`Compare
  performance with master`), while the step that actually runs the
  comparison is called `Run the performance tests` — the two labels are
  swapped relative to every neighbouring branch. They are swapped back:
  the job takes the name `Run performance tests`, matching `wp/5.7`,
  `wp/5.9`, `wp/6.0` and `wp/6.2` verbatim, and the step takes
  upstream's `Compare performance with base branch`.
- Upstream adds no trigger condition because this workflow has only ever
  had a single `pull_request` trigger, so the step needs no
  `if: github.event_name == 'pull_request'` guard.
- The base ref is read from the `GITHUB_BASE_REF` environment variable
  rather than interpolated into the `run:` script as
  `${{ github.base_ref }}`, matching the environment-variable form
  already used for `GITHUB_SHA` on the same line and keeping workflow
  context values out of a shell command line.

The `--ci` flag this branch passes is preserved.

(cherry picked from commit ab46764,
the merge commit for WordPress#66196)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DtSAAJkQ9P6vPtDumqStg8
desrosj added a commit to desrosj/gutenberg that referenced this pull request Sep 21, 2026
…rdPress#66196)

Cherry-picks ab46764
(WordPress#66196) into the `wp/6.3` branch.

`performance.yml`'s `pull_request` step hardcoded `trunk` as the second
comparison branch, so every pull request opened against this branch was
measured against Gutenberg's own live `trunk` rather than against the branch
it actually targets. That comparison cannot even be built here: the perf CLI
checks out and runs `npm ci` against each comparison branch, and this frozen
branch's toolchain is incompatible with `trunk`'s current dependency graph
and lockfile format, so the step fails outright before any measurement runs.

Using the pull request's own base ref compares like with like and keeps the
step working on a branch that will never move back to `trunk`.

Adapted from the upstream commit: the base ref is read from the
`GITHUB_BASE_REF` environment variable rather than interpolated into the
`run:` script as `${{ github.base_ref }}`, matching the environment-variable
form used for `GITHUB_SHA` on the same line and avoiding expression
interpolation directly into a shell command.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DtSAAJkQ9P6vPtDumqStg8
desrosj added a commit to desrosj/gutenberg that referenced this pull request Sep 21, 2026
…rdPress#66196)

The `pull_request`-triggered step in `.github/workflows/performance.yml`
hardcoded `trunk` as the branch to compare against. On a frozen `wp/6.4`
branch that is always the wrong reference: Gutenberg's `trunk` has moved
on by several years of tooling and runtime requirements, so it can no
longer even be built with this branch's own pinned Node version, let
alone produce a meaningful performance comparison.

Compare against the pull request's own base branch instead, and rename
the step to match. Upstream's version of this change interpolates
`${{ github.base_ref }}` directly into the `run:` script; this port uses
the equivalent `$GITHUB_BASE_REF` environment variable, and quotes the
existing `$GITHUB_SHA` references alongside it, so no workflow context
value is expanded into a shell command line.

(cherry picked from commit ab46764,
the merge commit for WordPress#66196)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DtSAAJkQ9P6vPtDumqStg8
desrosj added a commit to desrosj/gutenberg that referenced this pull request Sep 21, 2026
…rdPress#66196)

Cherry-picks ab46764
(WordPress#66196) into the `wp/6.3` branch.

`performance.yml`'s `pull_request` step hardcoded `trunk` as the second
comparison branch, so every pull request opened against this branch was
measured against Gutenberg's own live `trunk` rather than against the branch
it actually targets. That comparison cannot even be built here: the perf CLI
checks out and runs `npm ci` against each comparison branch, and this frozen
branch's toolchain is incompatible with `trunk`'s current dependency graph
and lockfile format, so the step fails outright before any measurement runs.

Using the pull request's own base ref compares like with like and keeps the
step working on a branch that will never move back to `trunk`.

Adapted from the upstream commit: the base ref is read from the
`GITHUB_BASE_REF` environment variable rather than interpolated into the
`run:` script as `${{ github.base_ref }}`, matching the environment-variable
form used for `GITHUB_SHA` on the same line and avoiding expression
interpolation directly into a shell command.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DtSAAJkQ9P6vPtDumqStg8
desrosj added a commit to desrosj/gutenberg that referenced this pull request Sep 21, 2026
…rdPress#66196)

The `pull_request` performance comparison compared every pull request
against one hardcoded branch. Pull requests are not always opened against
that branch, so any pull request with a different base was measured against
an unrelated branch and produced a meaningless comparison.

The branch to compare against is now taken from the base branch of the pull
request being tested, and the step is renamed to reflect that it no longer
targets a single fixed branch.

On this branch the hardcoded value is `master`, not `trunk`: it predates the
`master`-to-`trunk` rename, so the upstream change does not apply as-is and
the equivalent edit was made by hand. The base branch is read from the
`GITHUB_BASE_REF` environment variable GitHub Actions already provides
rather than the `${{ github.base_ref }}` workflow expression, and the
remaining arguments are quoted to match. The pre-existing `--ci` flag is
kept.

Ported from WordPress/gutenberg@ab46764
(WordPress#66196).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DtSAAJkQ9P6vPtDumqStg8
desrosj added a commit to desrosj/gutenberg that referenced this pull request Sep 21, 2026
…rdPress#66196)

The `pull_request`-triggered step in `.github/workflows/performance.yml`
hardcoded `trunk` as the branch to compare against. On a frozen `wp/6.4`
branch that is always the wrong reference: Gutenberg's `trunk` has moved
on by several years of tooling and runtime requirements, so it can no
longer even be built with this branch's own pinned Node version, let
alone produce a meaningful performance comparison.

Compare against the pull request's own base branch instead, and rename
the step to match. Upstream's version of this change interpolates
`${{ github.base_ref }}` directly into the `run:` script; this port uses
the equivalent `$GITHUB_BASE_REF` environment variable, and quotes the
existing `$GITHUB_SHA` references alongside it, so no workflow context
value is expanded into a shell command line.

(cherry picked from commit ab46764,
the merge commit for WordPress#66196)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DtSAAJkQ9P6vPtDumqStg8
desrosj added a commit to desrosj/gutenberg that referenced this pull request Sep 21, 2026
…rdPress#66196)

The `pull_request`-triggered step in `.github/workflows/performance.yml`
hardcoded `trunk` as the branch to compare against. On a frozen `wp/6.4`
branch that is always the wrong reference: Gutenberg's `trunk` has moved
on by several years of tooling and runtime requirements, so it can no
longer even be built with this branch's own pinned Node version, let
alone produce a meaningful performance comparison.

Compare against the pull request's own base branch instead, and rename
the step to match. Upstream's version of this change interpolates
`${{ github.base_ref }}` directly into the `run:` script; this port uses
the equivalent `$GITHUB_BASE_REF` environment variable, and quotes the
existing `$GITHUB_SHA` references alongside it, so no workflow context
value is expanded into a shell command line.

(cherry picked from commit ab46764,
the merge commit for WordPress#66196)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DtSAAJkQ9P6vPtDumqStg8
desrosj added a commit to desrosj/gutenberg that referenced this pull request Sep 21, 2026
…rdPress#66196)

Cherry-picks ab46764
(WordPress#66196) into the `wp/6.3` branch.

`performance.yml`'s `pull_request` step hardcoded `trunk` as the second
comparison branch, so every pull request opened against this branch was
measured against Gutenberg's own live `trunk` rather than against the branch
it actually targets. That comparison cannot even be built here: the perf CLI
checks out and runs `npm ci` against each comparison branch, and this frozen
branch's toolchain is incompatible with `trunk`'s current dependency graph
and lockfile format, so the step fails outright before any measurement runs.

Using the pull request's own base ref compares like with like and keeps the
step working on a branch that will never move back to `trunk`.

Adapted from the upstream commit: the base ref is read from the
`GITHUB_BASE_REF` environment variable rather than interpolated into the
`run:` script as `${{ github.base_ref }}`, matching the environment-variable
form used for `GITHUB_SHA` on the same line and avoiding expression
interpolation directly into a shell command.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DtSAAJkQ9P6vPtDumqStg8
desrosj added a commit to desrosj/gutenberg that referenced this pull request Sep 21, 2026
…rdPress#66196)

The `pull_request`-triggered step in `.github/workflows/performance.yml`
hardcoded `trunk` as the branch to compare against. On a frozen `wp/6.4`
branch that is always the wrong reference: Gutenberg's `trunk` has moved
on by several years of tooling and runtime requirements, so it can no
longer even be built with this branch's own pinned Node version, let
alone produce a meaningful performance comparison.

Compare against the pull request's own base branch instead, and rename
the step to match. Upstream's version of this change interpolates
`${{ github.base_ref }}` directly into the `run:` script; this port uses
the equivalent `$GITHUB_BASE_REF` environment variable, and quotes the
existing `$GITHUB_SHA` references alongside it, so no workflow context
value is expanded into a shell command line.

(cherry picked from commit ab46764,
the merge commit for WordPress#66196)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DtSAAJkQ9P6vPtDumqStg8
desrosj added a commit to desrosj/gutenberg that referenced this pull request Sep 22, 2026
…rdPress#66196)

The `pull_request` performance comparison compared every pull request
against one hardcoded branch. Pull requests are not always opened against
that branch, so any pull request with a different base was measured against
an unrelated branch and produced a meaningless comparison.

The branch to compare against is now taken from the base branch of the pull
request being tested, and the step is renamed to reflect that it no longer
targets a single fixed branch.

On this branch the hardcoded value is `master`, not `trunk`: it predates the
`master`-to-`trunk` rename, so the upstream change does not apply as-is and
the equivalent edit was made by hand. The base branch is read from the
`GITHUB_BASE_REF` environment variable GitHub Actions already provides
rather than the `${{ github.base_ref }}` workflow expression, and the
remaining arguments are quoted to match. The pre-existing `--ci` flag is
kept.

Ported from WordPress/gutenberg@ab46764
(WordPress#66196).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DtSAAJkQ9P6vPtDumqStg8
desrosj added a commit to desrosj/gutenberg that referenced this pull request Sep 22, 2026
…rdPress#66196)

The `pull_request`-triggered step in `.github/workflows/performance.yml`
hardcoded `trunk` as the branch to compare against. On a frozen `wp/6.4`
branch that is always the wrong reference: Gutenberg's `trunk` has moved
on by several years of tooling and runtime requirements, so it can no
longer even be built with this branch's own pinned Node version, let
alone produce a meaningful performance comparison.

Compare against the pull request's own base branch instead, and rename
the step to match. Upstream's version of this change interpolates
`${{ github.base_ref }}` directly into the `run:` script; this port uses
the equivalent `$GITHUB_BASE_REF` environment variable, and quotes the
existing `$GITHUB_SHA` references alongside it, so no workflow context
value is expanded into a shell command line.

(cherry picked from commit ab46764,
the merge commit for WordPress#66196)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DtSAAJkQ9P6vPtDumqStg8
desrosj added a commit to desrosj/gutenberg that referenced this pull request Sep 22, 2026
…rdPress#66196)

Cherry-picks ab46764
(WordPress#66196) into the `wp/6.3` branch.

`performance.yml`'s `pull_request` step hardcoded `trunk` as the second
comparison branch, so every pull request opened against this branch was
measured against Gutenberg's own live `trunk` rather than against the branch
it actually targets. That comparison cannot even be built here: the perf CLI
checks out and runs `npm ci` against each comparison branch, and this frozen
branch's toolchain is incompatible with `trunk`'s current dependency graph
and lockfile format, so the step fails outright before any measurement runs.

Using the pull request's own base ref compares like with like and keeps the
step working on a branch that will never move back to `trunk`.

Adapted from the upstream commit: the base ref is read from the
`GITHUB_BASE_REF` environment variable rather than interpolated into the
`run:` script as `${{ github.base_ref }}`, matching the environment-variable
form used for `GITHUB_SHA` on the same line and avoiding expression
interpolation directly into a shell command.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DtSAAJkQ9P6vPtDumqStg8
desrosj added a commit to desrosj/gutenberg that referenced this pull request Sep 22, 2026
The `pull_request`-triggered step in `performance.yml` hardcoded `trunk` as
the comparison branch. On a frozen branch like `wp/5.9` that is always the
wrong target: every pull request opened against `wp/5.9` was compared
against live upstream `trunk` rather than against its own base, so the job
had to build and install current `trunk`'s Gutenberg code using this
branch's pinned Node 14.21.3 toolchain. Current `trunk` requires Node
`>=24.18.0`, so the comparison branch could not even be installed and the
step failed outright before any measurement was taken.

Using the base branch instead of a hardcoded one is the fix upstream
applied in `WordPress#66196` (`ab467645855eef40b1570e5bd0a8db03aa62dad6`),
which renamed the step to `Compare performance with base branch` and
replaced `trunk` with the base ref. That commit does not apply cleanly to
this branch, whose `run:` line predates the `--ci` flag's introduction
elsewhere in the file and has different surrounding steps, so the same
change is made by hand here.

The base ref is read through the `GITHUB_BASE_REF` environment variable
rather than the `${{ github.base_ref }}` expression. Workflow expressions
are substituted into the script text before the shell runs, so a branch
name is interpolated as code; the environment variable form passes it as
data instead. The existing `$GITHUB_SHA` references are quoted at the same
time so every argument on the line is passed the same way, matching the
`release`-triggered step directly below it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DtSAAJkQ9P6vPtDumqStg8
desrosj added a commit to desrosj/gutenberg that referenced this pull request Sep 22, 2026
The `pull_request` leg of `performance.yml` compared every pull request
against a hardcoded `trunk`. On a frozen `wp/X.Y` branch that is always
the wrong comparison point: `trunk` is Gutenberg's live development
branch, whose toolchain and dependency tree have moved on by years from
what this branch's Node 14 / npm lockfile can install, so the second
comparison branch cannot even be built. Comparing against the pull
request's own base branch measures the change the pull request actually
makes.

Ported from `WordPress/gutenberg` commit
`ab467645855eef40b1570e5bd0a8db03aa62dad6`, originally
"Performance tests : -  Fix/compare performance with correct branch"
(`WordPress#66196`). The upstream change does not apply
cleanly to this branch, so it was applied by hand and adapted: the
branch is passed as the `GITHUB_BASE_REF` environment variable rather
than upstream's `${{ github.base_ref }}` expression interpolation, which
keeps workflow-context data out of the generated shell script, and all
three arguments are quoted. This branch's pre-existing `--ci` flag is
preserved.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DtSAAJkQ9P6vPtDumqStg8
desrosj added a commit to desrosj/gutenberg that referenced this pull request Sep 22, 2026
…rdPress#66196)

The `pull_request` performance comparison compared every pull request
against one hardcoded branch. Pull requests are not always opened against
that branch, so any pull request with a different base was measured against
an unrelated branch and produced a meaningless comparison.

The branch to compare against is now taken from the base branch of the pull
request being tested, and the step is renamed to reflect that it no longer
targets a single fixed branch.

On this branch the hardcoded value is `master`, not `trunk`: it predates the
`master`-to-`trunk` rename, so the upstream change does not apply as-is and
the equivalent edit was made by hand. The base branch is read from the
`GITHUB_BASE_REF` environment variable GitHub Actions already provides
rather than the `${{ github.base_ref }}` workflow expression, and the
remaining arguments are quoted to match. The pre-existing `--ci` flag is
kept.

Ported from WordPress/gutenberg@ab46764
(WordPress#66196).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DtSAAJkQ9P6vPtDumqStg8
desrosj added a commit to desrosj/gutenberg that referenced this pull request Sep 22, 2026
…rdPress#66196)

`.github/workflows/performance.yml` hardcoded `master` as the branch to
compare against. Gutenberg renamed its default branch from `master` to
`trunk`, and `refs/heads/master` no longer exists on
`WordPress/gutenberg` at all, so the comparison can never be built: the
performance CLI fetches each comparison branch from that repository
(`bin/plugin/config.js` hardcodes `WordPress` as the repository owner)
and `git fetch origin master` fails outright before any measurement
runs.

Compare against the pull request's own base branch instead. That is the
branch a pull request is actually proposing to change, so it is the only
meaningful reference for a frozen branch that will never move back to a
shared default branch.

Adapted from the upstream commit, which does not apply literally here:

- Upstream renames a step already called `Compare performance with
  trunk`. On this branch that name sits on the *job* (`Compare
  performance with master`), while the step that actually runs the
  comparison is called `Run the performance tests` — the two labels are
  swapped relative to every neighbouring branch. They are swapped back:
  the job takes the name `Run performance tests`, matching `wp/5.7`,
  `wp/5.9`, `wp/6.0` and `wp/6.2` verbatim, and the step takes
  upstream's `Compare performance with base branch`.
- Upstream adds no trigger condition because this workflow has only ever
  had a single `pull_request` trigger, so the step needs no
  `if: github.event_name == 'pull_request'` guard.
- The base ref is read from the `GITHUB_BASE_REF` environment variable
  rather than interpolated into the `run:` script as
  `${{ github.base_ref }}`, matching the environment-variable form
  already used for `GITHUB_SHA` on the same line and keeping workflow
  context values out of a shell command line.

The `--ci` flag this branch passes is preserved.

(cherry picked from commit ab46764,
the merge commit for WordPress#66196)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DtSAAJkQ9P6vPtDumqStg8
desrosj added a commit to desrosj/gutenberg that referenced this pull request Sep 22, 2026
…rdPress#66196)

Cherry-picks ab46764
(WordPress#66196) into the `wp/6.3` branch.

`performance.yml`'s `pull_request` step hardcoded `trunk` as the second
comparison branch, so every pull request opened against this branch was
measured against Gutenberg's own live `trunk` rather than against the branch
it actually targets. That comparison cannot even be built here: the perf CLI
checks out and runs `npm ci` against each comparison branch, and this frozen
branch's toolchain is incompatible with `trunk`'s current dependency graph
and lockfile format, so the step fails outright before any measurement runs.

Using the pull request's own base ref compares like with like and keeps the
step working on a branch that will never move back to `trunk`.

Adapted from the upstream commit: the base ref is read from the
`GITHUB_BASE_REF` environment variable rather than interpolated into the
`run:` script as `${{ github.base_ref }}`, matching the environment-variable
form used for `GITHUB_SHA` on the same line and avoiding expression
interpolation directly into a shell command.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DtSAAJkQ9P6vPtDumqStg8
desrosj added a commit to desrosj/gutenberg that referenced this pull request Sep 22, 2026
…rdPress#66196)

The `pull_request` performance comparison compared every pull request
against one hardcoded branch. Pull requests are not always opened against
that branch, so any pull request with a different base was measured against
an unrelated branch and produced a meaningless comparison.

The branch to compare against is now taken from the base branch of the pull
request being tested, and the step is renamed to reflect that it no longer
targets a single fixed branch.

On this branch the hardcoded value is `master`, not `trunk`: it predates the
`master`-to-`trunk` rename, so the upstream change does not apply as-is and
the equivalent edit was made by hand. The base branch is read from the
`GITHUB_BASE_REF` environment variable GitHub Actions already provides
rather than the `${{ github.base_ref }}` workflow expression, and the
remaining arguments are quoted to match. The pre-existing `--ci` flag is
kept.

Ported from WordPress/gutenberg@ab46764
(WordPress#66196).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DtSAAJkQ9P6vPtDumqStg8
desrosj added a commit to desrosj/gutenberg that referenced this pull request Sep 22, 2026
…rdPress#66196)

The `pull_request`-triggered step in `.github/workflows/performance.yml`
hardcoded `trunk` as the branch to compare against. On a frozen `wp/6.4`
branch that is always the wrong reference: Gutenberg's `trunk` has moved
on by several years of tooling and runtime requirements, so it can no
longer even be built with this branch's own pinned Node version, let
alone produce a meaningful performance comparison.

Compare against the pull request's own base branch instead, and rename
the step to match. Upstream's version of this change interpolates
`${{ github.base_ref }}` directly into the `run:` script; this port uses
the equivalent `$GITHUB_BASE_REF` environment variable, and quotes the
existing `$GITHUB_SHA` references alongside it, so no workflow context
value is expanded into a shell command line.

(cherry picked from commit ab46764,
the merge commit for WordPress#66196)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DtSAAJkQ9P6vPtDumqStg8
desrosj added a commit to desrosj/gutenberg that referenced this pull request Sep 22, 2026
The `pull_request`-triggered step in `performance.yml` hardcoded `trunk` as
the comparison branch. On a frozen branch like `wp/5.9` that is always the
wrong target: every pull request opened against `wp/5.9` was compared
against live upstream `trunk` rather than against its own base, so the job
had to build and install current `trunk`'s Gutenberg code using this
branch's pinned Node 14.21.3 toolchain. Current `trunk` requires Node
`>=24.18.0`, so the comparison branch could not even be installed and the
step failed outright before any measurement was taken.

Using the base branch instead of a hardcoded one is the fix upstream
applied in `WordPress#66196` (`ab467645855eef40b1570e5bd0a8db03aa62dad6`),
which renamed the step to `Compare performance with base branch` and
replaced `trunk` with the base ref. That commit does not apply cleanly to
this branch, whose `run:` line predates the `--ci` flag's introduction
elsewhere in the file and has different surrounding steps, so the same
change is made by hand here.

The base ref is read through the `GITHUB_BASE_REF` environment variable
rather than the `${{ github.base_ref }}` expression. Workflow expressions
are substituted into the script text before the shell runs, so a branch
name is interpolated as code; the environment variable form passes it as
data instead. The existing `$GITHUB_SHA` references are quoted at the same
time so every argument on the line is passed the same way, matching the
`release`-triggered step directly below it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DtSAAJkQ9P6vPtDumqStg8
desrosj added a commit to desrosj/gutenberg that referenced this pull request Sep 22, 2026
The `pull_request` leg of `performance.yml` compared every pull request
against a hardcoded `trunk`. On a frozen `wp/X.Y` branch that is always
the wrong comparison point: `trunk` is Gutenberg's live development
branch, whose toolchain and dependency tree have moved on by years from
what this branch's Node 14 / npm lockfile can install, so the second
comparison branch cannot even be built. Comparing against the pull
request's own base branch measures the change the pull request actually
makes.

Ported from `WordPress/gutenberg` commit
`ab467645855eef40b1570e5bd0a8db03aa62dad6`, originally
"Performance tests : -  Fix/compare performance with correct branch"
(`WordPress#66196`). The upstream change does not apply
cleanly to this branch, so it was applied by hand and adapted: the
branch is passed as the `GITHUB_BASE_REF` environment variable rather
than upstream's `${{ github.base_ref }}` expression interpolation, which
keeps workflow-context data out of the generated shell script, and all
three arguments are quoted. This branch's pre-existing `--ci` flag is
preserved.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DtSAAJkQ9P6vPtDumqStg8
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Backported to WP Core Pull request that has been successfully merged into WP Core [Type] Bug An existing feature does not function as intended [Type] Build Tooling Issues or PRs related to build tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants