Skip to content

Fix a journey run failing on worker teardown after a launch that gave no window - #607

Merged
zaerl merged 3 commits into
trunkfrom
fix/journey-launch-without-window
Oct 4, 2026
Merged

zaerl merged 3 commits into
trunkfrom
fix/journey-launch-without-window

Conversation

@zaerl

@zaerl zaerl commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

Why

On the macOS runner a journey's launch sometimes gives the test no window: electronApplication.firstWindow: Timeout 30000ms exceeded while waiting for event "window". The test passes on Playwright's retry and is reported as flaky, but the job still fails a minute later on Worker teardown timeout of 60000ms exceeded and "1 error was not a part of any test". It happened on two pull requests in a row on 3 October (runs 37132158563 and 37148706799), in a different test each time, and both went green on a re-run.

What changes

Root cause of the failed run: the app from the launch that gave no window was still running when the worker stopped. Session.close() gave it five seconds and then called proc.kill(). On macOS that is SIGTERM, which a main process that has stopped answering survives. When the worker stops, Playwright asks every process it launched to close and waits for its exit with no limit of its own, so the runner's 60-second teardown limit is what ended it, as a failure.

  • close() now ends the app by force once it has had its five seconds: the process group on macOS and Linux, the process tree on Windows (where the launched process is a shell and the app is its child). It then waits for the exit, with a limit.
  • A launch that gives no window now puts what the main process said of itself in the error: whether Electron became ready, and each window with its address and whether its page was loading or had crashed. An app that could not be reached, and one that gave no answer in five seconds, each say so in their own words.

Not in this change: why the launch gives no window. That is not established. The trace shows the app started in 0.6 s, its main process answered once, no window event arrived in 30 s, and the app then would not quit. The new error text is there so the next occurrence says which state the app was in. There is also no retry of the launch inside the helper: with the teardown fixed, Playwright's own retry in CI is enough, and a retry in the helper would hide the thing still to be found.

How to test this

Platforms: any. No user-visible surface; this is the journeys' engine (tests/e2e/helpers/app.cjs), and nothing under src/ changes.

From the repository root, after npm ci and npm run build:once:

npx playwright test --project=journeys engine.spec
  • closing ends an app that has stopped answering makes the main process busy for good and asserts the app is gone after close(). It is red without the change: with proc.kill() put back in close() the app survives, the assertion fails, and the run ends on "1 error was not a part of any test" about a minute later, which is the failure the macOS runner showed. I checked that on macOS.
  • the main process says what a launch came to reads the state the launch error reports, on a launch that worked.

What must not have happened: no Electron --inspect=0 process left behind after the run (pgrep -fl "Electron --inspect=0" prints nothing on macOS), and no journey slower or failing because its app was killed while quitting normally. All 59 journeys pass locally on macOS.

What cannot be tested by hand: a launch that gives no window cannot be staged without a hook in the app. I checked the error text once by hand for each of its three outcomes (state read, app gone, app not answering), with the helper edited for the run and restored. The Windows branch of the forced kill (taskkill /T /F) is exercised only by CI, through the first test above.

Risks and limitations

  • Review: 2 passes, 8 findings fixed here, 1 not applied, 1 follow-up. Details below.
  • An app that is quitting normally but takes more than five seconds is now ended mid-quit, where SIGTERM used to let it finish; a child it started in a process group of its own would then be left behind. No journey quits that slowly today.
  • Not applied: if the main process exits and one of Chromium's helpers keeps its output pipe open, close() returns without killing anything and Playwright could still wait at teardown. Nothing has shown this happening, so there is no branch for it.
  • The second new test pins the state reader, not the error path it exists for.
  • The packaged smoke was not run: it does not use this helper.

Related

Seen on #605 and #606. Once merged, this goes into redesign/wpds with a merge commit; the launch code differs there, so the merge needs a hand.


Design decisions and alternatives considered
  • Force after the limit, not SIGTERM first. By the limit the app has already been asked to quit through app.quit(). A second polite request is what did not work.
  • Process group and process tree. Playwright starts the app detached on macOS and Linux, so the group is the app and its helpers. On Windows it launches through a shell, so the pid is cmd.exe's and /T is what reaches the app. proc.kill() is avoided on purpose: it marks the process as killed whether or not it went, and Playwright skips its own last forced kill, at worker exit, for a process marked that way.
  • Evidence in the error, not as an attachment. The CI log is where a flaky test is read; the error is in it, an attachment is in an artifact.
  • No shorter wait for the window. 30 s for the window, 5 s for the state and up to 10 s in close() fit the 60 s test timeout, and teardown after a failure has a budget of its own.
Review outcome (required — see AGENTS.md)

Pass 1: 5 [fix here] · 1 [follow-up]. Pass 2 (on the fix-up): 4 [fix here]. 8 fixed, 1 not applied, 1 follow-up.

  • Review: completed — a separate agent context, read-only; head c70587d / base 522b6ea; read the diff against Playwright 1.63's launcher and worker teardown; 5 [fix here], 1 [follow-up].
    • Fixed: the comment gave the wrong reason for the hang (it is Playwright waiting for the exit; the killed mark only matters at its last forced kill); "did not answer" covered an app that had died as well as one that was stuck; the comment did not say a slow normal quit is now cut off; the test polled where its invariant said "by the time close() returns".
    • Not applied: the pipe-held-open path in Risks, which nothing has demonstrated.
    • Follow-up: the second test does not reach the error path.
  • Review: completed — a second separate agent context, read-only; head bc15afb / base 522b6ea; 4 [fix here], all fixed.
    • The direct assertion from pass 1 could be red on Windows with nothing wrong, because close() waits there for the shell and the app can be a moment behind it: the poll is back and the comment says why.
    • The launch error could lose the original error on a rejection that is not an Error, and one of its three sentences had no full stop.
    • TESTING.md said "did not answer" means stuck; it now says most likely.
    • The forced kill's comment said SIGTERM for both platforms and understated what a kill at the limit leaves behind.
  • Since review: bc15afb → a450f63 is the second pass's fixes only; engine spec and all 59 journeys re-run on a450f63, lint clean.
Implementation notes
  • Timeline of the failed launch in run 37148706799, from its trace: launch 637 ms, a main-process evaluate answered in 4 ms, firstWindow timed out at 30 s, "Close context" never returned, the fixture gave up after 5 s, and the worker teardown then waited 60 s.
  • Playwright 1.63: worker teardown calls gracefullyCloseAll(), which calls the app's close(), which awaits the browser context's close promise; that resolves only when the process exits.
  • Local checks on macOS: lint clean, npm test 1733 pass, npm run test:e2e 59 pass.

Nothing on screen changes, so there are no screenshots.

🤖 Generated with Claude Code

zaerl and others added 3 commits October 4, 2026 09:52
…ed no window

On the macOS runner a launch sometimes gives the test no window. The test
passes on the retry, but the run still failed a minute later on "Worker
teardown timeout": the app from the failed launch was still running.

close() ended a stuck app with proc.kill(). That sends SIGTERM, which a main
process that has stopped answering does not act on, and it marks the process
as killed whether or not it went, so Playwright skips its own forced kill when
the worker stops and waits for an exit that never comes.

close() now ends the process group (the process tree on Windows) by force once
the app has had five seconds to quit, and waits for it to go. A launch that
gives no window also puts what the main process said of itself in the error,
so the next one says which state the app was in.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ssert the kill directly

Review fix-ups. The hang at worker teardown is Playwright waiting for the
app's process to exit, not the killed mark, which only matters at its last
forced kill; the comments now say so. The launch error tells an app that could
not be reached from one that did not answer. The engine test asserts the app
has gone when close() returns, without polling.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…kill did

Second review pass. On Windows close() waits for the shell the app was
launched through, so the app itself can be a moment behind: the engine test
polls again. The forced kill's comment says what proc.kill() did on each
platform and what a kill at the limit leaves behind, and the launch error ends
its sentence and survives a rejection that is not an Error.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Oct 4, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration
  • Configuration used: Repository: WordPress/contributor-toolkit/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: fd5d0372-8e49-4e63-bc98-d3e3cdd06036

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@zaerl
zaerl merged commit 9ee21dd into trunk Oct 4, 2026
8 checks passed
@zaerl
zaerl deleted the fix/journey-launch-without-window branch October 4, 2026 08:33
zaerl added a commit that referenced this pull request Oct 4, 2026
Brings in the journey engine's forced kill (#607): an app still running
five seconds after it was asked to quit is ended, and a launch that gives
no window says what its main process knew. The launch here asks for the
profile and runs a journey's beforeWindow before it waits for the window,
so the evidence is taken around that wait, in this branch's order.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
zaerl added a commit that referenced this pull request Oct 4, 2026
…kens (#608)

## Why

Item 9 of #557 is to replace every hardcoded colour with the design
system's tokens, with as little hand-written CSS as possible. Parts 1
and 2 did the site page and its panels. What was left was inside the
three dialogs still built on the older library: "Review & submit
changes" with its diff and its three destinations, the "Update to latest
trunk?" question, and the mail dialog. Between them they held 95 inline
`style` attributes and 87 colour values, including a dark diff pane and
a dark raw-mail pane that no longer matched the light terminal and logs.

## What changes

The three dialogs keep their frames, buttons and inputs from
`@wordpress/components`. Everything drawn by hand inside them is now the
design system's.

- **The diff** is on the strong surface inside a hairline, and each kind
of line (added, removed, hunk, file name, the app's own header) is a
class painted with tokens, as the prototype has them. **The raw mail**
is on the surface the logs and terminal are on.
- **Boxes that warned, failed or informed** are `Notice`s: the old-trunk
warning, the unreadable diff, nothing to send, someone else's work in
the checkout, the branch based on today's trunk, the update question's
failure, and test mode.
- **Lines that say how something went** are `Text` in one of three
shared classes, `success-text`, `warning-text` and `problem-text`. These
also replace the apply card's and the details sidebar's own copies, and
`.error-text` from part 1 now uses the same red (its token gave a
near-black `#470000`).
- **Destinations** are stacks and text inside a card per group. The
update question's two answers, its file list and the mail's panes are
classes in `shell.css`.
- The rule that gave every button in an older dialog a right margin is
gone; the layouts use gaps.

After this, no inline `style` and no colour value is left in any
component under `src/renderer`.

**Deliberately not in this PR:**

- Turning these dialogs into the design system's `Dialog`, or the review
into the prototype's three-column drawer with a file list. That is a
redesign of the screen, not colour work, and is not listed in #557.
- The three discards that still ask through `window.confirm`.
- The lint rule that forbids a hardcoded colour in the renderer. That is
part 4.
- Mail HTML rendering is unchanged and tracked separately.

## How to test this

Platforms: any. The journeys below drive each dialog on macOS and
Windows and assert the colours against the tokens, read from the window
itself. From the repository root, after `npm ci` and `npm run
build:once`:

```
npx playwright test --project=journeys review-changes.spec trunk-update.spec mail.spec open-pull-request.spec
```

Each new assertion is red without the style it claims; I broke the rules
one at a time to check (10 mutations, 10 red):

- `review-changes.spec.js`: an added and a removed line are in the
success and error colours on their washes; a failed save is in the error
colour and a saved file's line in the success colour.
- `trunk-update.spec.js`: the chosen answer is ringed in the brand
colour and the other is not, the ring moves with the choice, the chosen
answer's detail is in the full text colour, and the answer that loses
work stays red, chosen or not.
- `mail.spec.js`: the raw mail is on the weak surface.

**Worth a look by hand, since no journey judges taste** (any platform,
the current head):

1. On a site with an edit, press **Review & submit changes**. The diff
is light, with green added and red removed lines on a tint; the
destinations are two cards.
2. From the site's menu choose **Update to latest trunk** with an edit
in the tree. The chosen answer is ringed in blue; choosing **Discard
them** moves the ring and keeps that answer red.
3. Start the dev server on a site that has sent mail, open a mail, and
press **Raw**. The pane is light.

**What must not have happened:** no wording, role or accessible name
changed (the existing journeys find everything by role and text and pass
unchanged, all 99), and nothing outside these dialogs moved except the
two colour classes named above.

**What cannot be tested by hand here:** the GitHub device code and a
real pull request result need a GitHub sign-in; the journeys stand in
for GitHub and cover both states.

## Risks and limitations

- Review: 2 passes, 6 findings fixed here, 2 accepted as they are.
Details below.
- **Contrast is lower than the dark pane's, though every pair passes
4.5:1.** Added lines are 4.86:1 and hunk lines 4.50:1, where the dark
pane had about 9.7:1. These are the prototype's tokens for a diff.
- **Visible changes beyond colour:** destination cards have the design
system's 24px padding (was 14/16px); titles are the design system's
medium heading; the GitHub device code is 20px (was 24px); test mode is
a neutral notice and not a dashed box; the mail dialog's two tabs sit
flush, which is the library's default; a stand-alone link in a
destination is clickable on its words only, not across the column.
- The error under the site page's changes note turns from near-black to
red, with `.error-text`.
- Accepted, not fixed: no journey checks the 12px size of the four
stand-alone links.
- The strings in these components are still not translatable. That was
so before, and is not changed here.

## Related

Part of #557, after
#605 and
#606.

---

<details>
<summary>Design decisions and alternatives considered</summary>

- **Frames and controls stay the older library's.** A dialog half in
each library's buttons would look worse than either, and the frame is
what a later redesign of the review replaces.
- **`Notice` with nothing spoken.** The design system's notice speaks
its text when it mounts. Each one here either carries `role="alert"`, as
the box it replaces did, or was silent before and stays so; none is
announced twice.
- **`Text` renders a span.** Every one is a flex or grid item, or is
given a block element, so no line runs into the next.
- **Shared classes for text colour** and not one per card: three classes
replace six rules that said the same thing.
- **The pressed answer's ring is drawn inside its border**, so choosing
does not move the other answer by a pixel, as the old 2px border did.

</details>

<details>
<summary>Review outcome (required — see AGENTS.md)</summary>

**Pass 1: 4 [fix here] · 2 [follow-up]. Pass 2 (on the fix-up): 2 [fix
here], both low. 6 fixed, 2 accepted.**

- **Review:** completed — a separate agent context, read-only; head
`50129cd` (rebased since, as `369cf6e`) / base `9570450`; compared each
component with its old version, read `Notice`'s implementation, computed
contrast for every new pair; 4 [fix here], 2 [follow-up].
- Fixed: the chosen answer's detail text was 4.34:1 on its tint (now the
full text colour, 14.6:1); four stand-alone links had grown from 12px to
13px; the mail journey's text-colour check passed without the rule it
was for.
- Fixed though marked follow-up: the test helper `tokenColour` could not
notice a misspelled token (it now refuses one the window does not have);
`.error-text` painted the same error in a different red.
- Accepted: the mail dialog's tabs sitting flush, the library's default.
- **Review:** completed — a second separate agent context, read-only;
head `86555dd` / base `23d46e7`; 2 [fix here], both low.
- Fixed: nothing pinned the losing answer's detail staying red once
chosen, which holds on source order alone; a journey now asserts it.
  - Accepted: no journey checks the links' size.
- **Since review:** `86555dd → 1460d81` adds that one assertion;
`trunk-update.spec` re-run, and red with the two rules swapped.

</details>

<details>
<summary>Implementation notes</summary>

- The branch was rebased once, before it was pushed, onto
`redesign/wpds` after trunk's journey-engine fix
(#607) was merged
into it.
- `ui.tokenColour` and `ui.paintOf` are new shared helpers in
`tests/e2e/helpers/ui.cjs`; `logs.spec.js` and `terminal.spec.js` use
the first in place of their own copies.
- Local checks on macOS: lint clean, `npm test` and `npm run
test:electron` pass (1863 tests), `npm run test:e2e` 99 pass, the
affected journeys pass in a 1024×728 window and with the status read
slowed by a second, `npm run test:e2e:packaged` 15 pass.

</details>

<details>
<summary>Screenshots or recording</summary>

Not attached: I have no way to upload images from the command line. The
three steps under "Worth a look by hand" show every changed surface, and
Buildkite's build for this branch can be used for them.

</details>

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
zaerl added a commit that referenced this pull request Oct 4, 2026
…r's components (#609)

## Why

Parts 1 to 3 of item 9 in #557 left no colour value and no inline
`style` in any component under `src/renderer`: the window is painted
from classes in `src/renderer/shell.css` with the design system's
tokens. Nothing stopped one coming back. A colour typed into a component
is one the theme cannot reach, and `TESTING.md` rules out the only other
check, a test that reads a component's source.

## What changes

Two lint rules of this repository's own, switched on for
`src/renderer/**/*.{js,jsx,cjs}`:

- **`toolkit/no-hardcoded-colors`** reports a colour written out in a
string or in the fixed part of a template: a hex value of 3, 4, 6 or 8
digits, or a function that takes a colour's channels (`rgb()`, `hsl()`,
`oklch()`, `color()` and the like). A token wrapped in `var()` passes,
and so does `color-mix()` of tokens.
- **`toolkit/no-inline-styles`** reports a `style` attribute on a JSX
element.

They live in `eslint-rules/`, registered as a local plugin in
`eslint.config.mjs`. No new dependency: the rules are plain modules,
tested with ESLint's own `RuleTester`.

**Not in this PR:**

- **The stylesheet.** `shell.css` is not checked by these rules, and it
is where all the styling now is. That is the next pull request:
Stylelint with the design system's own rules.
- A colour's name (`red`, `white`), a colour put together from pieces, a
`style` passed in a spread or set from an effect. Each rule's header
says what it does not read. None of these exists in the renderer today.
- `src/main.js`, whose patch window still carries a few inline colours.
It is not the renderer.

## How to test this

Platforms: any. No user-visible surface; this is lint and its tests.
From the repository root, after `npm ci`:

```
npm run lint
node --test tests/unit/eslint-rules.test.cjs
```

Both pass. To see the rules bite, add `style={{ borderColor: "#dcdcde"
}}` to any element in `src/renderer/components/destination.jsx` and run
`npx eslint src/renderer`: it reports the inline style and the colour. I
did that, and the same with an `rgb()` in
`src/renderer/terminal-theme.cjs`.

`tests/unit/eslint-rules.test.cjs` runs each rule over code that must
pass and code that must not. I removed each guard in the two patterns in
turn (the lookbehinds, the lookahead, the case flag, the string-only
check, reading a template as the program gets it, `color()` in the list)
and the suite went red every time.

**What must not have happened:** the rules must not fire on anything in
the tree as it is (`npm run lint` is clean with no disable comment
added), and `eslint-rules/` must not be packaged into the app
(`build.files` in `package.json` does not include it).

## Risks and limitations

- Review: 2 passes, 9 findings fixed here, 1 kept as it is. Details
below.
- **A reference of three or four digits written out in a string reads as
a colour**: `'Fixed in #607'` is reported, because `#607` is a grey and
`#000` must not pass. Nothing in the tree does this; the renderer builds
such strings from the number (`#%d`, `#${ n }`), and the message says
so. A fragment made only of hex letters, `'#add'`, is reported too
unless it follows a word or a slash.
- Kept: these could have been three selectors given to the core rule
`no-restricted-syntax`, with no rule files. They are rules of their own
because the colour rule's message names what it found and the patterns
are tested by themselves.

## Related

Part of #557, after
#605,
#606 and
#608.

---

<details>
<summary>Review outcome (required — see AGENTS.md)</summary>

**Pass 1: 5 [fix here] · 1 [follow-up]. Pass 2 (on the fix-up): 4 [fix
here]. 9 fixed, 1 kept.**

- **Review:** completed — a separate agent context, read-only; head
`e8903a2` / base `5b7e1fb`; ran the test file on Node and on Electron's
Node, eslint over the renderer, and probed both rules with throwaway
code; 5 [fix here], 1 [follow-up].
- Fixed: a test that passed without the guard it named; the header
claiming a written-out reference could not be flagged; a fragment after
a slash being flagged; four wordings that claimed more than the code
does (the `TESTING.md` sentence, both messages, the rule's description).
- Fixed though marked follow-up: tests for uppercase hex, hex after `:`
and `(`, a template that cannot be read, a namespaced attribute.
- Kept: custom rules over `no-restricted-syntax`, with the reason now in
each header.
- **Review:** completed — a second separate agent context, read-only;
head `022aebf` / base `5b7e1fb`; removed each guard in memory and re-ran
the suite; 4 [fix here], all fixed.
- Three guards could be removed with the suite still green; two now have
a test, and the third decided nothing and is gone.
  - Two comments named the wrong case as pinning the string-only check.
- `color()` and `device-cmyk()` were missing from the functions the rule
reads.
  - One of the two reasons given for a custom style rule did not hold.
- **Since review:** `022aebf → b9bc2e2` is those four fixes only; the
test file, `npm test`, `npm run test:electron` and `npm run lint` re-run
on it.

</details>

<details>
<summary>Implementation notes</summary>

- The preset from `@wordpress/eslint-plugin` does not configure
`no-restricted-syntax`, so nothing is overridden either way.
- `RuleTester` is given `node:test`'s `describe` and `it`, so a failing
case fails `npm test`; checked with a deliberately wrong case.
- The journeys and the packaged smoke were not run: nothing under `src/`
or `tests/e2e/` changes.

</details>

Nothing on screen changes, so there are no screenshots.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
@zaerl zaerl added this to the v2.0.0 milestone Oct 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant