Repository navigation
ci(.github): start a new doc-check chat per run and stop it when the run ends - #32185
Merged
bpmct merged 3 commits intoNov 13, 2026
Merged
Conversation
…run ends doc-check reused one chat for every push to a PR, so a re-run sent the whole earlier review again, and a review kept running after its job stopped waiting. On 2026-11-13, 58 review chats and 45 of their sub-agents were running on the shared model while 5 workflow runs were in progress. Each run now starts a new chat in the PR's review workspace and gets the trace of the latest finished review in its prompt. A new run cancels the one in progress, and every run stops its review and workspace when it ends. The wait is 5 hours, and a review that runs past it is stopped and fails the job. This combines the workflow half of #32061 with #31720. Linear: DOCS-1745 Co-authored-by: Ben Potter <me@bpmct.net>
bpmct
approved these changes
Nov 13, 2026
A push no longer cancels a review in progress. GitHub keeps one pending run per PR and replaces it on each push, so the next review covers the latest head. A 5-minute debounce job, which each push cancels, collapses a burst of pushes into one review. A merge still cancels the running review through the delete-workspace job. Generated by Coder Agents on behalf of @bpmct.
bpmct
deleted the
vigilante/docs-1745-doc-check-start-a-new-chat-per-run-wait
branch
November 13, 2026 21:14
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
doc-check reused one chat for every push to a PR, and a review kept running in Coder after its job stopped waiting for it.
This PR starts a new chat for each run, waits up to 5 hours for it, and stops the review and its workspace when the run ends for any reason.
On dogfood at 2026-11-13 17:28 UTC, 58 doc-check review chats and 45 of their sub-agents had status
running, while 5 workflow runs were in progress and 1 was queued.34 of the 58 chats were created before that day, and the oldest on 2026-09-29.
The count covers only chats shared with me, so it is a lower bound.
This combines the workflow half of #32061 with the stop script from #31720, and changes no skill file.
Changes
force-new-chatreplaces the one chat per PR, so a re-run no longer carries every earlier turn on the PR.The prompt gets a
## Previous runsection with thedoc-check-traceof the latest finished review.The job finds
doc-check-pr-<N>, wakes and starts it, and passes its ID to the chat.On the first run the chat creates the workspace under that name.
The review job's concurrency group is keyed on the PR URL with
cancel-in-progress: false. GitHub keeps one pending run per group and replaces it on each push, so the next review covers the latest head.A
debouncejob sleeps 5 minutes onsynchronize, in its own group withcancel-in-progress: true, so a burst of pushes starts one review. Other triggers do not wait.Merging the PR still cancels a running review (the
delete-workspacejob keepscancel-in-progress: true), then deletes the workspace..github/scripts/doc-check-stop-review.shruns before the review and in a finalalways()step.It drops queued messages, interrupts the chat and its sub-agents, and stops the workspace.
wait-timeout-secondsgoes from 2400 to 18000, and the job timeout from 50 to 330 minutes.The job now stays in its concurrency group for the whole review, so the next run for the PR waits for it.
.github/scripts/doc-check-previous-trace.shreads the 5 newest review chats for the PR and prints the first trace with anoutcome:line.It reads more than one chat because a cancelled run's chat has no trace.
A failed lookup costs only that context and never fails the job.
GITHUB_OUTPUTdelimiter for the prompt is random, because the trace is model output.Behavior changes to check
doc-checklabel forced one review, on thelabeledeventTesting
Static checks, all passing:
actionlint,zizmor,shellcheck --external-sources, andshfmt -d.Script tests, run locally:
doc-check-stop-review.test.sh: 48 cases against a fake API.doc-check-previous-trace.test.sh: 22 cases, including CRLF in the trace text and in the PR URL.The survivor was a redundant trim, which I removed.
End to end on dogfood, on this PR, 2026-11-13 UTC:
doc-check-pr-32185and had 2 running sub-agents at 17:48:23.ready_for_review, 17:55:31## Previous run.Not covered:
The fake-API tests cover the script, and the summary text is a few lines of shell.
delete-workspacecancelling a review) has not run live..test.shsuites.The workflow runs on
ubuntu-latestonly, and the scripts were not run on macOS or Windows.Follow-ups, not in this PR
Supersedes #31720 and the workflow half of #32061.
Linear: DOCS-1745
Plan and decision log
This is PR 1 of a 5-part stack from the design doc "Design: reliable, low-cost doc-check on Solstice".
Decisions in this PR:
A re-run's first prompt was 1.01 times the previous run's last prompt, so each push re-sent the whole history of the PR.
A new chat per run removes that, and stopping stale reviews removes load that no job is watching.
Reviews on the local model took a median of 61 minutes when clean and 247 minutes with findings (2026-11-04 to 11-11).
A shorter wait would stop most reviews with findings before they post.
With
cancel-in-progress, a new push still ends the review of the old head at once.This reverses ci(.github/workflows): don't fail doc-check when the review outlives the wait #32097, where a timeout was a notice.
A green job with no review reads as a clean result.
The first step catches a review left by a run whose last step did not finish, for example a runner that died.
It parses model output, and it has to handle chats with no trace, a trace with no outcome, and CRLF.
The prompt says so, and asks for a review of the whole PR.
A delta review needs the skip logic and the ledger from later PRs in the stack.
The skill changes are gated by an eval that does not exist yet.
Follow-up commit
64a09655a1(finish-then-rerun and debounce) generated by Coder Agents on behalf of @bpmct.Generated by Coder Agents on behalf of @nickvigilante.