Skip to content

Commit a2e9438

Browse files
manzoorwanijkciampo
manzoorwanijk
and
ciampo
authored
CI: Unify the automation comments into a single PR comment (#82249)
* CI: Add a shared action for a single PR automation comment Every automation posts its own comment today, so a pull request collects three to five bot comments before a human says anything. Add tools/pr-meta, an action that maintains one comment split into marker-delimited sections. Each producer owns one section and rewrites only its own, so the comment reads the same however the workflows interleave. Writers hold a shared concurrency group rather than verifying their own write, which cannot detect a lost update: two writers reading the same body both succeed, and the first section is gone. queue: max is required there, since the default cancels a pending writer once a third arrives. actionlint does not know the queue key. Its last release predates the feature and rhysd/actionlint#654 is open, so suppress that one message. The action imports nothing: depending on @actions/core would mean a full npm install in every writer job just to edit a comment. * CI: Report the bundle size in the shared PR comment compressed-size-action sets its rendered markdown as an output before it decides whether to comment, so use-check diverts the comment into a check run and the body still reaches the shared comment. use-check is conditional on the pull request not being from a fork. A fork's token cannot create a check, and unlike its comment path, its check path does not catch that failure and would fail the job. * Report Flaky Tests: Render the report instead of commenting The action now writes markdown to output-path and no longer talks to GitHub, so the report joins the shared PR comment. Its repo-token input and its GitHub API client go with it. The workflow only invoked the action when a report artifact existed, so a clean run could never clear a stale report. The writer job now runs on every pull request and treats the missing artifact as an empty section. * CI: Post the performance results on the pull request The results were only ever written to the job summary, buried in the logs of a workflow run. Limiting the number of bot comments was the stated reason for not posting them, which the shared comment removes. Each shard already renders a summary, so upload those and merge them in the results job, keeping the heading and preamble they share once. * CI: Check the type label in the workflow itself github-action-required-labels cannot report through the shared comment: its failure path exposes only a status, never the rendered message or the labels it matched. Turning its comment off also disables the branch that deletes its own warning once the labels are fixed, which would strand a warning on every pull request that ever had one. The rule is one label matching ^\[Type\], so read the labels directly. Read them at the point of writing rather than from the event payload, since runs queue behind each other and an older one can arrive after a newer one already fixed them. Also fire on opened and reopened. A pull request opened without a type label got no warning until some unrelated event happened. * Project Automation: Expose the contributor prompts as outputs Both first-time contributor tasks posted their own comment: a welcome on the pull request, and a prompt to link a WordPress.org profile once it merged. They now set outputs and the workflow posts them, so both join the shared PR comment. The account link task runs on push, whose payload carries no pull request number, so it exposes the one it resolved alongside the prompt. Its writer needs that for the comment and for the concurrency group both. * Changelogs: Reference the pull request * PR meta: Add publishConfig and emoji section headings Every other tools/* workspace carries publishConfig despite being private, and the repo-wide lint rule requires it. Lead each heading with an emoji so a reader can pick their section out of a comment holding seven of them without reading any of the others. * PR meta: Wrap the comment title in emoji Marks the comment as automation at a glance, matching the section headings below it. * PR meta: Collapse the long sections behind a fold Performance tables and flaky test traces would otherwise push the rest of the comment out of view. The fold sits outside the section delimiters, alongside the heading and the footer. Everything between them is read back as the body, so a fold kept inside would nest into itself on every write. * PR meta: Do not let a failed check clear a section Three ways a section was cleared by something other than a result: Every write re-renders every section, footers included, so a writer for a PR-state section needs the head too. Without it, updating Labels dropped the 'not the current head' marker from a stale Bundle size, presenting an old result as current. The label reporter ran even when the check step failed, where a missing body file means 'labels are fine', so a failed API call erased a warning nothing had disproved. The flaky renderer treated every readdir failure as a clean run. Only a missing directory means clean; anything else now surfaces. * PR meta: Skip the write when the head cannot be read Swallowing that failure re-rendered every section without a head, which strips the 'not the current head' footer from stale results and presents them as current. Letting it reach the outer handler skips the write instead, and adds the first tests covering the action's entry point. Also drop the two dependencies report-flaky-tests no longer uses, consolidate its unreleased changelog entries now that it renders rather than comments, and document the automation action's new outputs. * PR meta: Truncate between blocks, and fail a broken flaky run Truncation cut at a fixed offset, so a cut landing inside a stack trace left the fence open and rendered the note and its link as code. It now cuts at a paragraph break and closes whatever fences and disclosures the cut left open. The flaky renderer logged its errors and left the step green, which looks exactly like a clean run, so the writer downstream cleared a report that nothing had disproved. It now fails the step, and the writer runs only when the job producing the report succeeded. Also update the README, package description and action description, which still described the action as posting a comment. Co-authored-by: manzoorwanijk <manzoorwanijk@git.wordpress.org> Co-authored-by: ciampo <mciampini@git.wordpress.org>
1 parent 87dfece commit a2e9438

40 files changed

Lines changed: 2131 additions & 306 deletions

‎.github/actionlint.yml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,10 @@ paths:
77
ignore:
88
# [SC2129](https://www.shellcheck.net/wiki/SC2129) is ignored because it is a stylistic issue.
99
- 'shellcheck reported issue in this script: SC2129:.+'
10+
# `queue` is a real concurrency key, but actionlint does not know it
11+
# yet, and its last release predates the feature. Tracked upstream in
12+
# https://github.com/rhysd/actionlint/issues/654.
13+
- 'unexpected key "queue" for "concurrency" section.+'
1014
.github/workflows/end2end-test.yml:
1115
ignore:
1216
# This file gets created in the step prior.

‎.github/workflows/bundle-size.yml‎

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,10 @@ jobs:
4646
runs-on: 'ubuntu-24.04'
4747
permissions:
4848
contents: read
49+
# Only used on a fork, where the action falls back to commenting and
50+
# its own failure handling takes over. See `use-check` below.
4951
pull-requests: write
52+
checks: write
5053
timeout-minutes: 15
5154

5255
steps:
@@ -67,9 +70,73 @@ jobs:
6770
uses: ./.github/setup-npm
6871

6972
- uses: preactjs/compressed-size-action@f322c295dde06a1cb7ccaef105732dd8a726d1d9 # v2.10.0
73+
id: size
7074
with:
7175
omit-unchanged: true
7276
repo-token: '${{ secrets.GITHUB_TOKEN }}'
7377
pattern: '{build/scripts/**/*.min.js,build/styles/**/*.css,build/modules/**/*.min.js}'
7478
clean-script: 'distclean'
7579
build-script: 'build'
80+
# Report a check rather than a comment, leaving the size report
81+
# to the unified comment. Not on a fork: its token cannot create
82+
# a check, and the check path, unlike the comment path, does not
83+
# catch that failure.
84+
use-check: ${{ github.event.pull_request.head.repo.full_name == github.repository }}
85+
86+
- name: Save the size report
87+
if: ${{ !cancelled() && steps.size.outputs.comment-body }}
88+
env:
89+
# Through the environment, never interpolated into the script.
90+
BODY: ${{ steps.size.outputs.comment-body }}
91+
run: |
92+
mkdir -p "${RUNNER_TEMP}/pr-meta"
93+
# Drop the action's own footer, which the unified comment replaces.
94+
printf '%s' "$BODY" | sed '/compressed-size-action<\/sub>/d' > "${RUNNER_TEMP}/pr-meta/body.md"
95+
96+
- name: Upload the size report
97+
if: ${{ !cancelled() && steps.size.outputs.comment-body }}
98+
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
99+
with:
100+
name: pr-meta--bundle-size
101+
path: ${{ runner.temp }}/pr-meta
102+
if-no-files-found: ignore
103+
104+
pr-meta:
105+
name: Update PR meta comment
106+
needs: build
107+
# Runs even when the build failed, so a stale size report is cleared.
108+
# Skipped on a fork, whose token cannot write a comment.
109+
if: ${{ !cancelled() && github.event.pull_request.head.repo.full_name == github.repository }}
110+
runs-on: 'ubuntu-24.04'
111+
permissions:
112+
contents: read
113+
pull-requests: write
114+
timeout-minutes: 5
115+
# Every writer of this comment shares one lock, across workflows. The
116+
# default `queue: single` would cancel a pending writer once a third
117+
# arrived, silently dropping its section.
118+
concurrency:
119+
group: pr-meta-${{ github.event.number }}
120+
cancel-in-progress: false
121+
queue: max
122+
123+
steps:
124+
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
125+
with:
126+
sparse-checkout: tools/pr-meta
127+
show-progress: ${{ runner.debug == '1' && 'true' || 'false' }}
128+
persist-credentials: false
129+
130+
- uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
131+
continue-on-error: true
132+
with:
133+
name: pr-meta--bundle-size
134+
path: pr-meta
135+
136+
- uses: ./tools/pr-meta
137+
with:
138+
repo-token: ${{ secrets.GITHUB_TOKEN }}
139+
section: bundle-size
140+
body-path: pr-meta/body.md
141+
commit-sha: ${{ github.event.pull_request.head.sha }}
142+
run-url: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}

‎.github/workflows/end2end-test.yml‎

Lines changed: 51 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -216,8 +216,6 @@ jobs:
216216
runs-on: 'ubuntu-24.04'
217217
permissions:
218218
contents: read
219-
# Needed to comment the flaky tests report on the pull request.
220-
pull-requests: write
221219
timeout-minutes: 5
222220
steps:
223221
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
@@ -275,9 +273,58 @@ jobs:
275273
name: flaky-tests-report
276274
path: flaky-tests
277275

278-
- name: Report flaky tests
276+
- name: Render the flaky tests report
279277
if: ${{ steps.download-flaky-tests-report.outcome == 'success' }}
280278
uses: ./packages/report-flaky-tests
281279
with:
282-
repo-token: '${{ secrets.GITHUB_TOKEN }}'
283280
artifact-path: flaky-tests
281+
output-path: pr-meta/body.md
282+
283+
- name: Upload the flaky tests report
284+
if: ${{ steps.download-flaky-tests-report.outcome == 'success' }}
285+
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
286+
with:
287+
name: pr-meta--flaky-tests
288+
path: pr-meta
289+
if-no-files-found: ignore
290+
291+
pr-meta:
292+
name: Update PR meta comment
293+
needs: merge-artifacts
294+
# Runs on every pull request, including the clean ones: a run with no
295+
# flaky test uploads no report, and the empty section clears a stale one.
296+
# Only when the job above succeeded, though, since a missing report then
297+
# means the run was clean rather than that rendering it failed.
298+
# Skipped on a fork, whose token cannot write a comment.
299+
if: ${{ needs.merge-artifacts.result == 'success' && github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository }}
300+
runs-on: 'ubuntu-24.04'
301+
permissions:
302+
contents: read
303+
pull-requests: write
304+
timeout-minutes: 5
305+
# Shared with every other writer of this comment. See bundle-size.yml.
306+
concurrency:
307+
group: pr-meta-${{ github.event.number }}
308+
cancel-in-progress: false
309+
queue: max
310+
311+
steps:
312+
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
313+
with:
314+
sparse-checkout: tools/pr-meta
315+
show-progress: ${{ runner.debug == '1' && 'true' || 'false' }}
316+
persist-credentials: false
317+
318+
- uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
319+
continue-on-error: true
320+
with:
321+
name: pr-meta--flaky-tests
322+
path: pr-meta
323+
324+
- uses: ./tools/pr-meta
325+
with:
326+
repo-token: ${{ secrets.GITHUB_TOKEN }}
327+
section: flaky-tests
328+
body-path: pr-meta/body.md
329+
commit-sha: ${{ github.event.pull_request.head.sha }}
330+
run-url: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}

‎.github/workflows/enforce-pr-labels.yml‎

Lines changed: 73 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,25 +1,89 @@
11
name: Enforce labels on Pull Request
22
on:
33
pull_request_target:
4-
types: [labeled, unlabeled, ready_for_review, review_requested]
4+
types:
5+
[
6+
opened,
7+
reopened,
8+
labeled,
9+
unlabeled,
10+
ready_for_review,
11+
review_requested,
12+
]
513

614
# Disable permissions for all available scopes by default.
715
# Any needed permissions should be configured at the job level.
816
permissions: {}
917

1018
jobs:
1119
type-related-labels:
20+
name: Check the type label
1221
runs-on: 'ubuntu-24.04'
1322
if: ${{ github.event.pull_request.state == 'open' }}
1423
permissions:
24+
contents: read
1525
pull-requests: write
26+
timeout-minutes: 5
27+
# Shared with every other writer of this comment. See bundle-size.yml.
28+
concurrency:
29+
group: pr-meta-${{ github.event.number }}
30+
cancel-in-progress: false
31+
queue: max
32+
1633
steps:
17-
- uses: mheap/github-action-required-labels@23e10fde7e062233401931a0eece796cd9bf3177 # v5.6.0
34+
# A `pull_request_target` job has a writable token, so it checks out
35+
# the base branch it was triggered on, never the pull request's head.
36+
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
1837
with:
19-
mode: exactly
20-
count: 1
21-
labels: '^\[Type\]'
22-
use_regex: true
23-
add_comment: true
24-
message: "**Warning: Type of PR label mismatch**\n\n To merge this PR, it requires {{ errorString }} {{ count }} label indicating the type of PR. Other labels are optional and not being checked here. \n- **Required label**: Any label starting with `[Type]`.\n- **Labels found**: {{ applied }}.\n\nRead more about [Type labels in Gutenberg](https://github.com/WordPress/gutenberg/labels?q=type). Don't worry if you don't have the required permissions to add labels; the PR reviewer should be able to help with the task."
25-
exit_type: failure
38+
sparse-checkout: tools/pr-meta
39+
show-progress: ${{ runner.debug == '1' && 'true' || 'false' }}
40+
persist-credentials: false
41+
42+
- name: Check for a type label
43+
id: check
44+
env:
45+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
46+
PR_NUMBER: ${{ github.event.number }}
47+
REPO: ${{ github.repository }}
48+
run: |
49+
mkdir -p pr-meta
50+
51+
# Read the labels now rather than from the event payload. Runs
52+
# queue behind each other on the shared lock, so an older one
53+
# can arrive after a newer one already fixed the labels.
54+
type_labels="$(gh api "repos/${REPO}/issues/${PR_NUMBER}/labels" \
55+
--jq '[ .[].name | select( startswith( "[Type]" ) ) ]')"
56+
count="$(printf '%s' "$type_labels" | jq 'length')"
57+
58+
if [ "$count" -eq 1 ]; then
59+
echo "ok=true" >> "$GITHUB_OUTPUT"
60+
exit 0
61+
fi
62+
63+
echo "ok=false" >> "$GITHUB_OUTPUT"
64+
applied="$(printf '%s' "$type_labels" | jq -r 'if length == 0 then "none" else join( ", " ) end')"
65+
66+
cat > pr-meta/body.md <<EOF
67+
This pull request needs exactly one label indicating its type, and has ${count}.
68+
69+
- Required: any label starting with \`[Type]\`.
70+
- Found: ${applied}.
71+
72+
Read more about [Type labels in Gutenberg](https://github.com/WordPress/gutenberg/labels?q=type). If you cannot add labels yourself, a reviewer can do it for you.
73+
EOF
74+
75+
# Runs whatever the check found, since an empty section is what
76+
# clears the warning once the labels are right. Not when the check
77+
# itself failed: no body file then means "labels are fine", and a
78+
# failed API call would erase a warning nothing had disproved.
79+
- name: Report the result
80+
if: ${{ !cancelled() && steps.check.outcome == 'success' }}
81+
uses: ./tools/pr-meta
82+
with:
83+
repo-token: ${{ secrets.GITHUB_TOKEN }}
84+
section: labels
85+
body-path: pr-meta/body.md
86+
87+
- name: Require exactly one type label
88+
if: ${{ steps.check.outputs.ok != 'true' }}
89+
run: exit 1

‎.github/workflows/performance.yml‎

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -263,6 +263,14 @@ jobs:
263263
- name: Add workflow summary
264264
run: cat "${WP_ARTIFACTS_PATH}/summary.md" >> "$GITHUB_STEP_SUMMARY"
265265

266+
- name: Archive performance summary
267+
if: success()
268+
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
269+
with:
270+
name: performance-summary-${{ matrix.shard }}
271+
path: ${{ env.WP_ARTIFACTS_PATH }}/summary.md
272+
if-no-files-found: ignore
273+
266274
- name: Archive performance results
267275
if: success()
268276
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
@@ -331,6 +339,38 @@ jobs:
331339
name: performance-results
332340
path: ${{ env.WP_ARTIFACTS_PATH }}/*.performance-results*.json
333341

342+
- name: Download shard summaries
343+
id: download-summaries
344+
if: ${{ github.event_name == 'pull_request' }}
345+
continue-on-error: true
346+
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
347+
with:
348+
pattern: performance-summary-*
349+
path: ${{ env.WP_ARTIFACTS_PATH }}/summaries
350+
351+
- name: Merge shard summaries
352+
if: ${{ steps.download-summaries.outcome == 'success' }}
353+
run: |
354+
mkdir -p pr-meta
355+
356+
# Every shard renders the same heading and preamble above its
357+
# own suites, so keep them once and take only the tables from
358+
# each. One directory per shard, ordered by name.
359+
printf 'Client side metrics **exclude** the server response time.\n\n' > pr-meta/body.md
360+
find "${WP_ARTIFACTS_PATH}/summaries" -name summary.md -print0 \
361+
| sort -z \
362+
| xargs -0 sed -e '/^## Performance Test Results$/d' \
363+
-e '/^Please note that client side metrics \*\*exclude\*\* the server response time\.$/d' \
364+
>> pr-meta/body.md
365+
366+
- name: Upload the performance report
367+
if: ${{ steps.download-summaries.outcome == 'success' }}
368+
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
369+
with:
370+
name: pr-meta--performance
371+
path: pr-meta
372+
if-no-files-found: ignore
373+
334374
- name: Publish performance results
335375
if: github.event_name == 'push'
336376
env:
@@ -340,3 +380,41 @@ jobs:
340380
COMMITTED_AT="$(git show -s "$GITHUB_SHA" --format="%cI")"
341381
REFERENCE_COMMIT="$(echo "$BRANCHES" | jq -r '.[1].name')"
342382
node tools/release/log-performance-results.js "$CODEHEALTH_PROJECT_TOKEN" trunk "$GITHUB_SHA" "$REFERENCE_COMMIT" "$COMMITTED_AT"
383+
384+
pr-meta:
385+
name: Update PR meta comment
386+
needs: results
387+
# Runs even when the tests failed, so a stale report is cleared.
388+
# Skipped on a fork, whose token cannot write a comment.
389+
if: ${{ !cancelled() && github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository }}
390+
runs-on: 'ubuntu-24.04'
391+
permissions:
392+
contents: read
393+
pull-requests: write
394+
timeout-minutes: 5
395+
# Shared with every other writer of this comment. See bundle-size.yml.
396+
concurrency:
397+
group: pr-meta-${{ github.event.number }}
398+
cancel-in-progress: false
399+
queue: max
400+
401+
steps:
402+
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
403+
with:
404+
sparse-checkout: tools/pr-meta
405+
show-progress: ${{ runner.debug == '1' && 'true' || 'false' }}
406+
persist-credentials: false
407+
408+
- uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
409+
continue-on-error: true
410+
with:
411+
name: pr-meta--performance
412+
path: pr-meta
413+
414+
- uses: ./tools/pr-meta
415+
with:
416+
repo-token: ${{ secrets.GITHUB_TOKEN }}
417+
section: performance
418+
body-path: pr-meta/body.md
419+
commit-sha: ${{ github.event.pull_request.head.sha }}
420+
run-url: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}

0 commit comments

Comments
 (0)