Repository navigation
feat(coderd): key chat diff statuses by ref - #29081
Conversation
2e6e3f8 to
75833b2
Compare
75833b2 to
1a7076c
Compare
|
@codex review |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1a7076c4bd
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
e370f06 to
caa4caa
Compare
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: caa4caa84c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
caa4caa to
c6eef7d
Compare
|
@codex review |
|
Codex Review: Didn't find any major issues. Bravo. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
c6eef7d to
12de2af
Compare
12de2af to
50c7b30
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 50c7b30cd0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| ADD COLUMN reported_at timestamp with time zone DEFAULT now() NOT NULL; | ||
|
|
||
| UPDATE chat_diff_statuses | ||
| SET reported_at = updated_at; |
There was a problem hiding this comment.
maybe it will be apparent once I scroll some more, but what is going to be the diff between those columns?
There was a problem hiding this comment.
So the chat_diff_statuses column can be updated separately from a PR being reported as existing. Name is maybe a bit shit. updated_at is our generic update thing. Happy to change this if too odd
There was a problem hiding this comment.
I am using the Linear agent to discuss because I did find it odd and this is what tells me:
updated_at used to advance during background PR refreshes, so it could not safely order the primary ref. The change removes that update. After that, both timestamps advance only in UpsertChatDiffStatusReference, and both are initialized from the same value for existing rows.
So reported_at only makes sense if we want to preserve updated_at as a generic “last changed” timestamp and later allow non-report mutations to advance it again. As written, it’s redundant; ordering by updated_at would produce the same result.
There was a problem hiding this comment.
Not sure what model is running underneath, but maybe it's made use of further in the stack and it just lacks the context
There was a problem hiding this comment.
Your Linear agent's summary is right about the history but misses the second half of the design. Here is the full reasoning (LLM-generated, Coder Agents):
reported_at exists so the primary ref (row 0 of the chat's diff statuses) changes only when the agent reports a ref. Ordering by updated_at breaks in two ways:
- Refresh reordering: each ref refreshes on its own schedule, so the last one refreshed became primary. A current branch with no PR writes nothing on refresh, while an old branch with a PR keeps getting bumped. The chat then shows the old PR indefinitely.
- Discovery race: a slow diff GET that finishes after a newer report could move the old row back to primary.
UpdateChatDiffStatusReferenceURLtouches only the URL, andDiscoveryWriteDoesNotReorderPrimarycovers this case.
The result is that only the agent-report upsert sets reported_at. Refreshes, claims, backoffs and URL writes cannot reorder the primary. You are right that today both columns advance together in practice, and that updated_at ordering would give the same result right now. The separation is the invariant: updated_at stays free to advance on any future non-report mutation without silently changing which PR the chat displays.
There was a problem hiding this comment.
the migration to modify the keys looks safe, it's not a big table on df
| Rows | 6,942 live, 515 dead; about 75–140 new rows a day |
|---|---|
| Table data | 15.2 MB, but only about 1.7 MB is live data (242 B per row on average) |
| Indexes | 12.8 MB, of which 9.7 MB is the stale_at index |
There was a problem hiding this comment.
We agree with the analysis and removed reported_at in 53bc0cd. The migration no longer adds the column, and the ordering now reads updated_at.
What carries the invariant instead: this PR also removes the updated_at = NOW() that main's worker refresh set, so updated_at now advances only when the agent reports a ref. Refreshes, claims, backoffs, PR clears and URL writes cannot reorder the primary. The rule to keep for future queries on this table: do not add updated_at to a write that must not change the primary ref.
This reply was generated by Coder Agents.
| // Authorize read on the parent chat. | ||
| _, err := q.GetChatByID(ctx, chatID) | ||
| if err != nil { | ||
| return nil, err |
There was a problem hiding this comment.
previously we were returning an empty object, but now it's nil, I guess it makes some difference for consumers, and ChatStusesByChatIDs still returns an empty array instead of nil
There was a problem hiding this comment.
Returning nil here does not change behavior. Only the error path returns nil, and callers do not read the result when err is not nil. GetChatDiffStatusesByChatIDs also returns nil on its error path. Its []database.ChatDiffStatus{} is the success result for an empty ID list, which main already had. When there are no rows, both queries return nil, because sqlc does not emit empty slices. All callers check len(), and diff_statuses is omitempty, so nil and empty give the same JSON. return nil, err is also the usual form in dbauthz.go.
This reply was generated by Coder Agents.
| case err == nil: | ||
| diffStatus = &status | ||
| case !xerrors.Is(err, sql.ErrNoRows): | ||
| diffStatuses, err := api.Database.GetChatDiffStatusesByChatID(ctx, chat.ID) |
There was a problem hiding this comment.
doesn't this break current fe?
There was a problem hiding this comment.
ok, nvm 1562 handles it
There was a problem hiding this comment.
probably should remove 1562-1564 if frontend handles arrays going forward
There was a problem hiding this comment.
We need to keep these lines. In this PR, diff_status is the only diff field in the API. If we remove them, the chat page does not show the PR, and TestGetChatDiffStatus fails. #29082 adds diff_statuses and marks diff_status as deprecated, but still fills it. The chat list, child chats, and watch events also fill it, and at the top of the stack the frontend still reads it from watch events. Removing the field is a breaking change to /api/v2, and multiple-PR support does not need it.
This reply was generated by Coder Agents.
53bc0cd to
745eca7
Compare
2668b37 to
8ccd48b
Compare
9f3ded7 to
082e0ac
Compare
ibetitsmike
left a comment
There was a problem hiding this comment.
Approved after schema simplification but imho the regex in the migration needs a test.
| -- Strip it with the same pattern as originCredentials in coderd/x/gitsync. | ||
| -- Each chat still has one row here, so this cannot create duplicate keys. | ||
| UPDATE chat_diff_statuses | ||
| SET git_remote_origin = regexp_replace(git_remote_origin, '^(https?://)[^/]*@', '\1') |
There was a problem hiding this comment.
Imho this type of migration with regex needs a migration test to make sure we're handling edge cases correctly. The repo has prior art so should be doable for a bot.
There was a problem hiding this comment.
Added in ffb7a98. testMigration000606ChatDiffStatusOriginCredentials is a table test and a step in TestMigrationChain. It adds one chat for each case at 605, applies 606, and checks each origin.
- The migration strips: a token, a user and password, a password with an
@,http://, a port, no path, and an@in the path after the credentials. - The migration keeps: an origin without credentials, an
@only in the path, scp-style SSH,ssh://git@..., and an empty origin.
When I changed the regex to [^/@]*@, the test failed on the case with @ in the password.
This reply was generated by Coder Agents.
chat_diff_statuses held one row per chat, so a chat could only track one pull request at a time and a branch switch carried the previous PR's URL onto the new branch's data. Re-key the table to (chat_id, git_remote_origin, git_branch) so every ref the agent reports keeps its own row. Existing rows satisfy the new key without a rewrite: both columns are NOT NULL with '' defaults and the old key capped rows at one per chat. The worker claims stale rows by the full key. Retained refs keep polling on their own schedule, so no pause state or resume protocol is needed. Telemetry reports one diff row per chat via DISTINCT ON. The diff GET's discovery write now keys its upsert to the stored ref. Under the old single-row key an empty ref key was safe; under the composite key it would create a stray row the worker ignores. Old coderd instances must finish before this migration runs: their ON CONFLICT (chat_id) writes have no arbiter under the new key.
The primary ref is the most recently reported one, but its ordering column was updated_at, the generic mutation time. A discovery GET that resolves a changed PR URL late could bump the old ref's updated_at past a newer report and steal the primary back. The table now tracks reported_at, set only by reports. Primary ordering, the chat-list tiebreak, and telemetry dedup read it. Refreshes, claims, backoffs, and URL writes cannot reorder the primary anymore. The discovery write uses a dedicated query that never touches ordering columns.
A remote URL can contain a token, such as https://<token>@github.com/o/r.git. coder gitaskpass sends that URL as it is, and the server stored it in chat_diff_statuses.git_remote_origin. MarkStale now removes the user and password from HTTP(S) remotes before it stores them. Migration 000600 removes them from existing rows before it changes the primary key. SSH remotes do not change.
… in migration 000606
082e0ac to
ffb7a98
Compare
Part 1 of 6.
chat_diff_statuseshad one row per chat, so a chat could only track one PR, and switching branches overwrote the old branch's PR data. The primary key is now(chat_id, git_remote_origin, git_branch), so each ref the agent reports gets its own row. The API still returns a singlediff_status: the first row of the chat's statuses, ordered byupdated_at, newest report first. The primary ref changes only when the agent reports a ref: refreshes, claims, backoffs and URL writes do not advanceupdated_at, so they cannot reorder the primary.The gitsync worker now claims, backs off and clears rows by the full key, not the whole chat. When the diff GET discovers a PR URL, it updates the existing ref row and no longer inserts one. Telemetry picks one row per chat.
NOTE: Migration
000606swaps the primary key under an ACCESS EXCLUSIVE lock. The table is small. Existing rows already fit the new key, so no data is rewritten. The down migration keeps only the primary row per chat. Stop old coderd instances before migrating: theirON CONFLICT (chat_id)upserts fail once the old key is gone.Generated by Coder Agents.