Skip to content

test(codersdk): cover chat client request errors - #30960

Merged
ibetitsmike merged 5 commits into
mainfrom
mike/codersdk-chat-request-error-tests
Oct 30, 2026
Merged

ibetitsmike merged 5 commits into
mainfrom
mike/codersdk-chat-request-error-tests

Conversation

@ibetitsmike

@ibetitsmike ibetitsmike commented Oct 18, 2026 •

Copy link
Copy Markdown
Collaborator

Adds TestChatClientRequests, a test of the two chat client request failures that no other test reaches. It is the test-first layer for the refactor above it, and it only adds test lines (+42).

Why it is needed: the next layers replace the request handling in 70 chat client methods with two shared helpers. The coderd tests already run those methods against a real server, so they cover success paths, error statuses, and request options. A real server cannot produce two failures:

  • a transport error, which must come back as an error that errors.Is matches. If a method swallowed it, callers would get a zero value and no error. If it wrapped it with %v, callers that check errors.Is(err, context.Canceled) would stop matching.
  • a 200 response with a body that is not valid JSON, which must come back as an error, not a zero value and no error. Test_ReadBodyAsJSON covers the decoder itself, not that a method returns its error.

The test calls ChatModel (JSON response) and DeleteChatModel (no content), which #30961 moves onto the two helpers, through a fake transport.

Proof: the test passes on main and on this branch. With go test -overlay, it fails when ChatModel returns no error for a transport failure, wraps it with %v, or drops the decode error, and it passes when the transport error is wrapped with %w. Cases I dropped because existing tests already fail on the same mutants: removing the status check from ChatModel fails TestChatModelACL (expects 404), and dropping the request options from ChatModelACLAvailable fails TestChatModelACLAvailable. I also dropped the check that another 2xx status is an error, because coderd never returns one on these routes.

Validation: go test ./codersdk/... -count=1 passes, golangci-lint run ./codersdk/ reports 0 issues, and the pre-commit hook passed.

Stack

This stack removes two unused chat exports from codersdk and moves 70 chat client methods in codersdk/chats.go onto two shared request helpers. Client behavior does not change, except that #30961 closes an unexpected-status response body once instead of twice.

  1. refactor: remove unused chat exports from codersdk #30959 Remove unused chat exports from codersdk
  2. test(codersdk): cover chat client request errors #30960 Cover chat client request errors
  3. refactor(codersdk): add chat client request helpers #30961 Add chat client request helpers, for the project and model methods
  4. refactor(codersdk): move chat settings client methods onto request helpers #30962 Move the chat settings methods onto the request helpers
  5. refactor(codersdk): move chat client methods onto request helpers #30963 Move the chat methods onto the request helpers

Xum, an AI agent, opened this PR for @ibetitsmike and wrote this description.

@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-30T11:10:00.231232Z 16abee4 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep them coming!

Reviewed commit: 3e6e0b090a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@ibetitsmike ibetitsmike changed the title test(codersdk): cover chat client request errors test(codersdk): cover chat client request errors and options Oct 18, 2026
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

Xum, an AI agent, requested this review for @ibetitsmike.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Already looking forward to the next diff.

Reviewed commit: 9bf77fef2f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@ibetitsmike
ibetitsmike marked this pull request as ready for review October 19, 2026 12:14
@linear-code

linear-code Bot commented Oct 21, 2026

Copy link
Copy Markdown

CODAGT-1445

@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

Xum, an AI agent, requested this review for @ibetitsmike.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Bravo.

Reviewed commit: 95506e7b4b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Base automatically changed from mike/codersdk-chat-dead-exports to main October 28, 2026 22:28
ibetitsmike added a commit that referenced this pull request Oct 28, 2026
Removes two unused exports from `codersdk/chats.go`: the `ChatGitChange`
type and `IsChatGitWatchFallbackMessage`. Regenerating the TypeScript
types drops the `ChatGitChange` interface from
`site/src/api/typesGenerated.ts`.

**Why removal is safe:** neither symbol has a caller anywhere in this
repository, including `site/`, `cli/`, `enterprise/`, `scaletest/` and
tests. A GitHub code search across the coder organization finds only
forks of this file and a copy of `typesGenerated.ts`.
`IsChatGitWatchFallbackMessage` lost its only caller when #25190 removed
the agents TUI. The comment on the git watch messages loses the
sentences about that helper and the "keep in sync" note:
`coderd/exp_chats.go` uses these constants directly.

**Kept on purpose:**

- `NewDynamicTool`, `DynamicToolCall`, `DynamicToolResponse` and the
`invopop/jsonschema` import: bots outside this repository build on them
(#24036).
- `ChatMessageSource`: also unused, but #30861 adds a function right
above it, and deleting it here would make that PR conflict. It can go
after #30861 merges.
- `AllChatMessagePartTypes`, `ChatDebugTerminalStatuses` and
`WithChatDiffStatusRef`: `scripts/apitypings`, a guard test and a coderd
test use them.

Validation: the pre-commit hook passed (`make gen`, formatting,
golangci-lint over every package and test file, and the slim build).
`make site/src/api/typesGenerated.ts` produces exactly the TypeScript
change in this PR.

## Stack

This stack removes two unused chat exports from `codersdk` and moves 72
chat client methods in `codersdk/chats.go` onto two shared request
helpers. Client behavior does not change, except that #30961 closes an
unexpected-status response body once instead of twice.

1. #30959 Remove unused chat exports from codersdk
2. #30960 Cover chat client request errors and options
3. #30961 Add chat client request helpers, for the project and model
methods
4. #30962 Move the chat settings methods onto the request helpers
5. #30963 Move the chat methods onto the request helpers

> Xum, an AI agent, opened this PR for @ibetitsmike and wrote this
description.
@ibetitsmike
ibetitsmike force-pushed the mike/codersdk-chat-request-error-tests branch from 95506e7 to 51089c6 Compare October 28, 2026 22:28

@ethanndickson ethanndickson left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need this test? I'm not sure what it guards that we don't already cover.

The existing coderd tests already run the chat client against coderd, so the success paths, error statuses and query options already go through the code this stack changes. Bad JSON is also covered by the ReadBodyAsJSON tests. That seems to leave whether a transport or decode error gets passed back, and whether a different 2xx counts as an error, which is a small surface for a return err and a status check.

Some of the assertions also seem tied to how the code is written rather than what callers see. For example, the check that the transport error is never wrapped would fail if we added context to it, which callers using errors.Is or errors.As wouldn't notice. Not every surviving mutant points at a missing test.

Is there a regression this catches that the existing tests would miss?

Generated by an agent on behalf of the user

@ibetitsmike
ibetitsmike force-pushed the mike/codersdk-chat-request-error-tests branch from 51089c6 to e727382 Compare October 29, 2026 22:57
@ibetitsmike ibetitsmike changed the title test(codersdk): cover chat client request errors and options test(codersdk): cover chat client request errors Oct 30, 2026
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@ethanndickson You're right that most of it duplicated existing coverage. I removed those cases in 16abee4:

  • Error statuses: removing the status check from ChatModel already fails TestChatModelACL (it expects 404).
  • Request options: dropping the options from ChatModelACLAvailable already fails TestChatModelACLAvailable.
  • Another 2xx status: coderd never returns one on these routes, so no caller could see that difference.
  • *url.Error type check and zero-value checks: these test how the code is written, not what callers see. The transport case now uses errors.Is only, so adding context with %w still passes.

Two cases stay because a real server can't produce them, so no coderd test reaches them:

  • Transport error: if a helper swallowed it, callers would get a zero value and a nil error. If a helper wrapped it with %v, errors.Is(err, context.Canceled) would stop matching.
  • Invalid JSON on a 200: if a helper dropped the ReadBodyAsJSON error, callers would get a zero value and a nil error. Test_ReadBodyAsJSON tests the decoder, not that the helper returns its error.

Both fail under go test -overlay mutants of these changes. #30961 keeps its Close count checks in a single ClosesResponseBodyOnce case, since the double Close it fixes is also something a real server can't show.

Xum, an AI agent, posted this for @ibetitsmike.

@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

Xum, an AI agent, posted this comment for @ibetitsmike.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Another round soon, please!

Reviewed commit: 16abee4689

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

No test reaches the transport error branch of any chat client method,
and none sends an invalid JSON body. TestChatClientRequestErrors covers
both, and an unexpected status, through ChatModel and DeleteChatModel.
The next layers move those methods onto shared request helpers.
TestChatClientRequestErrors also answers with 202 Accepted, so a status
check that accepts any 2xx code fails the test.
TestChatClientRequestErrors did not check that request options reach
the request, so a request helper that dropped them would still pass.
Add a case that sends query options through ChatModelACLAvailable, and
rename the test to TestChatClientRequests because it now covers more
than errors.
The TransportError case only checked errors.Is, which still passes when a
method wraps the transport error. http.Client.Do returns the fake
transport's error as a *url.Error, so also require that type.
…aches

Review asked what TestChatClientRequests guards that existing tests miss.
coderd tests already run chat client methods against a real server, so
error statuses (TestChatModelACL expects 404 from ChatModel) and request
options (TestChatModelACLAvailable filters by search and limit) already
fail when a method drops its status check or options. A different 2xx
status is not something coderd returns, so that case guards no caller.

Keep the two cases no other test can produce: a transport error and an
undecodable success body must come back as errors. Assert the transport
error with errors.Is only, so adding context with %w still passes.
@ibetitsmike
ibetitsmike force-pushed the mike/codersdk-chat-request-error-tests branch from 16abee4 to f641994 Compare October 30, 2026 15:51
@ibetitsmike
ibetitsmike merged commit 60c4206 into main Oct 30, 2026
27 checks passed
@ibetitsmike
ibetitsmike deleted the mike/codersdk-chat-request-error-tests branch October 30, 2026 19:08
@github-actions github-actions Bot locked and limited conversation to collaborators Oct 30, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants