Skip to content

feat: add types, context, and model normalization#23914

Merged
ThomasK33 merged 4 commits into
mainfrom
devtools/02-types-context-model-normalization
Apr 13, 2026
Merged

feat: add types, context, and model normalization#23914
ThomasK33 merged 4 commits into
mainfrom
devtools/02-types-context-model-normalization

Conversation

@ThomasK33

@ThomasK33 ThomasK33 commented Apr 1, 2026

Copy link
Copy Markdown
Member

Summary

Add the chatdebug package foundations: shared types for debug runs and steps, context helpers for propagating debug state through the request chain, and the model normalization layer that converts provider-specific LLM request/response formats into a canonical bounded representation.

This is PR 2/9 in the chat debug logging stack.

Changes

  • Types (coderd/x/chatd/chatdebug/types.go): canonical Go types for normalized content parts, tool definitions, usage metrics, and step request/response structures.
  • Context helpers (coderd/x/chatd/chatdebug/context.go): context.Context-based propagation of active debug run/step metadata through the call chain.
  • Model normalization (coderd/x/chatd/chatdebug/model.go): wraps fantasy.LanguageModel to intercept Stream/Generate calls and normalize provider-specific fantasy.Call inputs and fantasy.StreamPart outputs into the canonical representation, including tool calls, tool results, system prompts, and usage data.
  • Tests (coderd/x/chatd/chatdebug/model_test.go): unit tests for the normalization wrapper covering stream capture, generate capture, and edge cases.

Stack overview

  1. Database schema & SDK types
  2. → Types, context, and model normalization (this PR)
  3. Recorder, transport, and redaction
  4. Service and summary aggregation
  5. Chat lifecycle wiring
  6. HTTP handlers and API docs
  7. Frontend API layer and panel utilities
  8. Debug panel components and settings
  9. Storybook stories

Generated with mux • Model: anthropic:claude-opus-4-6 • Thinking: xhigh

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 524ffa6830

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/x/chatd/chatdebug/model.go Outdated
Comment thread coderd/x/chatd/chatdebug/model.go
@ThomasK33 ThomasK33 force-pushed the devtools/02-types-context-model-normalization branch from 524ffa6 to e897b1c Compare April 1, 2026 14:59
@ThomasK33 ThomasK33 force-pushed the devtools/01-database-schema-sdk-types branch 2 times, most recently from 115c169 to a0089c6 Compare April 1, 2026 16:59
@ThomasK33 ThomasK33 force-pushed the devtools/02-types-context-model-normalization branch 2 times, most recently from f4b6ab8 to 93e4b64 Compare April 1, 2026 17:02
@ThomasK33 ThomasK33 force-pushed the devtools/01-database-schema-sdk-types branch from a0089c6 to 7aba550 Compare April 1, 2026 17:02
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 93e4b64dfa

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/x/chatd/chatdebug/stubs.go Outdated
Comment thread coderd/x/chatd/chatdebug/model.go Outdated
@ThomasK33 ThomasK33 force-pushed the devtools/01-database-schema-sdk-types branch from 7aba550 to b90b087 Compare April 1, 2026 17:39
@ThomasK33 ThomasK33 force-pushed the devtools/02-types-context-model-normalization branch 2 times, most recently from 7777092 to d4c98a0 Compare April 1, 2026 18:01
@ThomasK33 ThomasK33 force-pushed the devtools/01-database-schema-sdk-types branch from b90b087 to 2dcf606 Compare April 1, 2026 18:01
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d4c98a0908

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/x/chatd/chatdebug/model.go Outdated
Comment thread coderd/x/chatd/chatdebug/model.go
@ThomasK33 ThomasK33 force-pushed the devtools/01-database-schema-sdk-types branch from 2dcf606 to 9870581 Compare April 1, 2026 18:31
@ThomasK33 ThomasK33 force-pushed the devtools/02-types-context-model-normalization branch from d4c98a0 to 39891e7 Compare April 1, 2026 18:31
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 39891e7471

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/x/chatd/chatdebug/model.go Outdated
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5a35d113bb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/x/chatd/chatdebug/model.go Outdated
Comment thread coderd/x/chatd/chatdebug/model.go Outdated

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bf17fb780c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/x/chatd/chatdebug/context.go
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 250d29f722

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/x/chatd/chatdebug/model.go Outdated
Comment thread coderd/x/chatd/chatdebug/model.go Outdated
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f3837aa5aa

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/x/chatd/chatdebug/model.go Outdated
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ccd6eec944

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/x/chatd/chatdebug/context.go

@mafredri mafredri left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Major progress. All prior findings resolved: sync.Once replaced with mutex (last-writer-wins for retries), reference-counted step counter cleanup, ReasoningStart/ToolInput stream parts captured, unconsumed-stream safety net via context.AfterFunc, GenerateObject and normalizeToolResultOutput test coverage, ErrNilModelResult sentinel error, SDK parity tests. The #24137 tracking ticket items are done.

Severity breakdown: 0 P1-P2, 3 P3 (2 new inline, 1 convention scan in body).

All 4 reviewers converged on the same dead code at model.go:489 (and its sibling at 621). The second if condition is unreachable after the first already mutates streamStatus. One new P3 from Chopper on error-typed stream parts with nil Error producing StatusError with no diagnostic context.

Convention scan found 8 em-dash characters (U+2014) across model.go (lines 365, 463), model_coverage_test.go (lines 18, 305, 310, 325), and model_internal_test.go (lines 365, 750). These should be double-hyphens (--) per project convention.

Lower-priority notes not posted inline:

  • model_internal_test.go:412 capturedHandle declared but never assigned; test name promises interrupted-status verification it doesn't deliver.
  • model.go:997 redundant nil guard on *ToolResultContent inconsistent with other pointer arms (both already guarded by isNilInterfaceValue).
  • stubs.go:61 trackRunRef/releaseRunRef has a theoretical TOCTOU between decrement and delete, exploitable only if two ContextWithRun calls with the same UUID overlap with a GC cycle.

Takumi: "The concurrency model is clean. I traced every interleaving. No new concurrency bugs found."

🤖 This review was automatically generated with Coder Agents.

Comment thread coderd/x/chatd/chatdebug/model.go Outdated
Comment thread coderd/x/chatd/chatdebug/model.go
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@mafredri mafredri left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

F16 (dead code) and F17 (nil-error diagnostic) both addressed in ce601d4. The unreachable second if blocks are deleted, and synthetic error payloads are added for nil-Error stream parts.

F18 (em-dash characters in comments) and F19 (capturedHandle never assigned in test) are silent -- no author response or code change since R16. Both were posted in the review body, not as inline comments, which may have contributed to the silence.

Review is blocked until F18 and F19 are addressed, acknowledged, or responded to.

🤖 This review was automatically generated with Coder Agents.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ce601d47dc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/x/chatd/chatdebug/model.go
Comment thread coderd/x/chatd/chatdebug/model.go Outdated

@mafredri mafredri left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All 19 prior findings resolved. F18 (em-dashes) and F19 (test capturedHandle) addressed in 9157a5c. Takumi confirms concurrency is clean for the second consecutive panel round.

Severity breakdown: 0 P0-P2, 2 P3, 5 Nit.

Two new P3s from the panel: Meruem found a concrete TOCTOU interleaving in the reference-counted step counter cleanup, and a tautological test assertion in StreamCompletedAfterFinish. Zoro flagged the rune-truncation duplication between boundText and TruncateLabel.

Netero's first pass caught 3 nits (tc:=tc Go 1.22 artifact, double-hyphen em-dash substitutes, maxInt vs builtin max) and Mafuuu caught 1 (test assertions using len() where production uses rune count).

Zoro: "Same-package duplication that shares an identical edge-case set is a maintenance sibling waiting to diverge."

🤖 This review was automatically generated with Coder Agents.

Comment thread coderd/x/chatd/chatdebug/stubs.go
Comment thread coderd/x/chatd/chatdebug/model_internal_test.go Outdated
Comment thread coderd/x/chatd/chatdebug/model.go
Comment thread coderd/x/chatd/chatdebug/stubs_internal_test.go Outdated
Comment thread coderd/x/chatd/chatdebug/model_coverage_test.go Outdated
Comment thread coderd/x/chatd/chatdebug/stubs_internal_test.go Outdated
Comment thread coderd/x/chatd/chatdebug/model_normalization_test.go Outdated

@mafredri mafredri left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All 8 R18 findings addressed in 5ec36e0. The TOCTOU in releaseRunRef is fixed with CompareAndSwap+CompareAndDelete. The tautological test assertion is rewritten with direct handle verification. The rune-truncation duplication is extracted into truncateRunes. All nits (tc:=tc, maxInt, len() in tests, double-hyphen substitutes) cleaned up.

Severity breakdown: 0 P0-P3, 1 Nit.

4 panel reviewers, 0 findings. Takumi: concurrency clean for the 4th consecutive panel round. Razor (wildcard): independently verified all claims match behavior across all source and test files.

One remaining Netero nit (F28): 3 double-hyphen em-dash substitutes in prose comments at model.go:365, 466 and model_internal_test.go:709. The R18 fix caught model_coverage_test.go but missed these.

Takumi: "The CAS-based ref counting in releaseRunRef closes the TOCTOU from F24 to a negligible residual window. All shared state is properly guarded."

Razor: "After full diff review, test execution, and tracing the reference-counting, stream finalization, and normalization paths: no new defects. The code's claims match its behavior at every point I tested."

🤖 This review was automatically generated with Coder Agents.

Comment thread coderd/x/chatd/chatdebug/model.go Outdated
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f3a8793276

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/x/chatd/chatdebug/stubs.go Outdated

@mafredri mafredri left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Status: APPROVED

F28 (last remaining double-hyphen nit) addressed in f3a8793. All 28 findings resolved across 20 rounds.

Netero first pass: clean across all 8 sections. Panel: 0 findings from all 4 reviewers (Mafuuu, Pariston, Meruem, Knov). Pariston independently tested three adversarial angles (lossy normalization, stream finalization holes, dual-package type drift) and found all three guarded.

The code is clean. The normalization wrapper correctly intercepts at the LLM call boundary, the concurrency model is sound (confirmed by Takumi across 4 consecutive panel rounds and Meruem across 3), the field coverage test catches drift from the fantasy library, and every prior finding has been addressed in code or tracked with a ticket.

Pariston: "I spent the review trying to build a case that this change solves the wrong problem, or solves the right problem at the wrong level. I could not."

🤖 This review was automatically generated with Coder Agents.

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f03adedb86

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/x/chatd/chattest/fakemodel.go
Comment thread coderd/x/chatd/chatdebug/model.go Outdated

@mafredri mafredri left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Status: APPROVED

The mutex simplification in f03aded replaces the CAS-based ref counting with a straightforward refCountMu serializing both trackRunRef and releaseRunRef. All 4 panel reviewers + Netero confirm 0 findings. Mafuuu verified the serialization against 3 adversarial scenarios including rapid RunID reuse and double-registration. Pariston independently confirmed the mutex eliminates the TOCTOU window and the nextStepNumber invariant holds (live RunContext implies ref count >= 1, so no deletion races). Kite praised the simplification.

All 28 findings resolved. Two consecutive clean panel rounds (R20, R21) with 0 findings from 9 distinct reviewers across both rounds.

Pariston: "The problem is correctly understood, the solution is proportional to the problem, and the latest mutex simplification is at the right causal level. After 20 rounds and 28 resolved findings, the premises are solid."

🤖 This review was automatically generated with Coder Agents.

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

1 similar comment
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Another round soon, please!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

…zation

Change-Id: If8181146f2f06d0d01b5fdb1046eaff930b7ba5d
Signed-off-by: Thomas Kosiewski <tk@coder.com>
- Remove tc := tc loop variable capture (Go 1.22+)
- Replace maxInt helper with builtin max
- Replace -- em-dash substitutes with proper punctuation
- Use utf8.RuneCountInString() in test assertions instead of len()
- Fix tautological test in TestDebugModel_StreamCompletedAfterFinish
- Extract shared truncateRunes helper from boundText/TruncateLabel
- Fix TOCTOU race in releaseRunRef using CompareAndSwap/CompareAndDelete
- Handle ExecutableProviderTool in normalizeTools for provider tool ID
- Fix tool-input attribution loss by tracking per-call-ID stream parts

Change-Id: Ib59ae0902d254cf6cdcc4124a75c5b32911f8bc3
Signed-off-by: Thomas Kosiewski <tk@coder.com>
Replace three remaining ASCII -- occurrences in prose comments
with semicolons per project convention.

Change-Id: I5a9a8153a682581bd765a3e5574fe813075aba97
Signed-off-by: Thomas Kosiewski <tk@coder.com>
The previous CAS(0,0) approach was a no-op that did not actually
claim the zero state, leaving a window where trackRunRef could
increment between the CAS and the delete. Serialize both functions
under a mutex to eliminate the TOCTOU entirely.

Change-Id: I1c889f1d431ca1c39674404a148aee0fb73f77ce
Signed-off-by: Thomas Kosiewski <tk@coder.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants