Repository navigation
feat: buffer CLI session logs and flush on connection failure - #29752
Merged
Merged
Conversation
code-asher
reviewed
Sep 28, 2026
spikecurtis
removed their request for review
September 28, 2026 07:13
Implements CLI-4 (connection-log RFC req 13) for the session commands: ssh, start, stop, and update. Adds a shared helper (cli/logbuffer.go) that builds a per-invocation file logger which, unless --verbose (-v) is set, runs at Info and keeps a rolling in-memory history of debug entries via a slog flight recorder. The history is written to the log file automatically when the command logs its terminal error on exit, so the detail leading up to a failure is available without logging debug during normal operation. - New shared --log-buffer-size / CODER_LOG_BUFFER_SIZE (default 1000, 0 disables, clamped to 10000). - Logs default to the XDG state dir (~/.local/state/coder/logs) via github.com/adrg/xdg; the most recent 50 files are retained. - start/stop/update gain a --log-dir (CODER_LOG_DIR) and set the client logger so HTTP request detail is captured; ssh keeps its existing --log-dir / CODER_SSH_LOG_DIR and now logs by default. - restart is excluded to match the client_session_id opt-in set.
aqandrew
force-pushed
the
cli-4-session-log-buffer
branch
from
September 28, 2026 22:17
111aae0 to
d703984
Compare
…n error Adds a global --log-buffer-size / CODER_LOG_BUFFER_SIZE option (default 10) and a root bufferedLoggerMiddleware that keeps below-level entries in a flight recorder and emits them to stderr only when a command returns an error. Workspace commands (start, stop, update) opt in and no longer write session log files; ssh keeps its file logger to avoid corrupting its stdio stream.
code-asher
approved these changes
Sep 29, 2026
code-asher
left a comment
Member
There was a problem hiding this comment.
Cool! I think we should be consistent with the "flight recorder" name but otherwise it looks good to go to me.
Collect glob, stat, and remove failures with errors.Join and return them instead of silently ignoring them, and log them at Warn in the ssh session log file. Pruning stays best effort and never fails the command.
Rename the buffered-logger naming to flight recorder for consistency with the underlying slog API: --flight-recorder-size / CODER_FLIGHT_RECORDER_SIZE, flightRecorderMiddleware, the flightRecorder helper, the annotationFlightRecorder annotation, and cli/flightrecorder.go. No behavior change.
aqandrew
added a commit
that referenced
this pull request
Sep 29, 2026
## What Runs the workspace agent logger at `Info` with a slog flight recorder (`v3.2.0`) instead of unconditionally at `Debug`. Entries below the current log level are kept in a rolling in-memory ring and flushed to the sinks when the agent loses its connection to coderd (and before any `Error`+ log), so the detail leading up to a failure is available without writing debug logs during normal operation. Implements CLIENT-672 (connection-log RFC req 13) for the workspace agent. Depends on the `cdr.dev/slog/v3` `v3.2.0` bump (already on `main`). ## How - `cli/agent.go`: - The main agent logger is `AppendSinks(...).Leveled(level).FlightRecorder(flightRecorderSize)`, where `level` is `Info` by default and `Debug` under the global `-v` / `CODER_VERBOSE`. The recorder is always attached; at `Debug` it is inert, which keeps the setup correct if the configured level changes in the future. The reaper logger is unchanged. - New `--agent-flight-recorder-size` / `CODER_AGENT_FLIGHT_RECORDER_SIZE` (default `1000`, `0` disables the flight recorder), so the ring size is configurable per RFC req 13. The agent flag is distinct from CLI-4's global `--flight-recorder-size` (#29752) so the two don't collide when `coder agent` inherits the root global. - `workspaceAgent` is a `RootCmd` method so it reuses the root command's global verbose flag instead of defining an agent-specific one. - `agent/agent.go` (`runLoop`): on a lost coderd connection (the `io.EOF` disconnect and the generic run error) the agent calls `logger.Flush(ctx)` to emit the recorded history and then logs at `Warn`. These are `Warn` rather than `Error` because reconnectable disconnects are expected and `slogtest` fails tests on any `Error` log; the manual flush is what surfaces the recorded detail. Graceful shutdown paths (`ctx` canceled, agent closed) are not flushed. - Tests: `agent.TestAgent_FlushFlightRecorderOnDisconnect` covers the flush-on-disconnect path; `cli.TestWorkspaceAgent/DisabledServers` runs with verbose since its debug messages are otherwise recorded. Golden `--help` output regenerated. ## Default log verbosity change This changes the default agent-log verbosity. Today `coder-agent.log` receives `Debug` unconditionally; with this change, normal operation logs `Info`, and recorded below-level entries are emitted only when the agent loses its coderd connection (via a manual flush) or before any `Error`+ log, or always under `-v` / `CODER_VERBOSE`. This is the intended behavior of the feature: a quiet steady state with full detail retained for diagnosing failures. Operators who want the previous always-debug output can run with `-v` / `CODER_VERBOSE`. ## Alignment with #29752 (CLI) This mirrors the CLI-4 (CLI log buffer) design so the two implementations agree: - **Level model:** Info by default, debug available on demand via the global `-v` / `CODER_VERBOSE`. - **Flight recorder size:** default `1000`, `0` disables. CLI-4 exposes a global `--flight-recorder-size`; the agent uses a distinct `--agent-flight-recorder-size` / `CODER_AGENT_FLIGHT_RECORDER_SIZE` (the `CODER_AGENT_*` env convention) so the inherited global and the agent flag coexist without collision. - **Flush trigger:** the recorded history is emitted before any `Error`+ log via slog's auto-flush. The agent additionally flushes manually on a lost coderd connection, which it logs at `Warn` (reconnectable, and `Error` would fail `slogtest`-based tests); the CLI's failures are fatal, so it relies on the auto-flush-on-error. <details> <summary>Rollout context</summary> Cross-repo sequence: coder/slog flight recorder (released as v3.2.0) -> go.mod bump in coder/coder (#29600, merged) -> this PR (CLIENT-672, agent) and CLI-4 (CLI), which are independent consumers and can merge in any order. </details> --- <sub>Opened by Coder Agents on behalf of @aqandrew.</sub>
aqandrew
enabled auto-merge (squash)
September 29, 2026 21:34
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds in-memory log buffering for CLI diagnostics so the debug detail leading up to a failure is available without logging debug during normal operation. Below-level entries are kept in a slog flight recorder (
v3.2.0) and emitted only when a command returns an error. This implements CLI-4 (connection-log RFC req 13). Depends on thecdr.dev/slog/v3v3.2.0bump (already onmain).How
cli/root.go):--log-buffer-size/CODER_LOG_BUFFER_SIZE(default10,0disables, clamped to10000), shared by all commands instead of duplicated per command.bufferedLoggerMiddlewareruns the invocation logger atInfo(orDebugwith-v) with a flight recorder, and on a returned error callsFlushto replay the buffered history to stderr.Flushdoes not add a duplicate error line; the returned error is still rendered by the normal formatter.annotationBufferedLoggerannotation so it does not clobber the ~17 commands that build their own logger oninv.Logger(which would otherwise double-log). Future commands opt in with one line.start/stop/update: opt into the middleware and justclient.SetLogger(inv.Logger)so codersdk request/response debug is captured and flushed to stderr on failure. They no longer write session log files and dropped their per-command--log-dir/--log-buffer-size.ssh: keeps its own file logger (writing to stderr would corrupt its stdio stream), now using the global buffer size and the sharedbufferedLoggerhelper. It retains--log-dir/CODER_SSH_LOG_DIR, stdio/PID filename handling, and file pruning (50 most recent).restartis excluded, matching theclient_session_idopt-in set (DEVEX-664).Default behavior change
start/stop/updatenow emit buffered diagnostics (recent below-level entries, e.g. sdk request/response) to stderr when they fail; on success they stay quiet.coder sshcontinues to write a per-connection diagnostic file by default under the XDG state dir (~/.local/state/coder/logs), pruned to the 50 most recent. Opt out of buffering with--log-buffer-size 0.Review changes (Asher)
bufferedLoggerhelper used by ssh and the workspace commands.10to avoid flooding the terminal on failure.Alignment with #29671 (agent)
Same core model:
Infodefault with a flight recorder,-vfor full debug, and the shared--log-buffer-sizesemantics (0disables). Differences: the agent writes to a file and defaults the buffer to1000, while the CLI prints to stderr and defaults to10; both now rely on slog's auto-flush-on-error (the CLI additionally flushes via the middleware on a returned error).Note
The agent PR (#29671) adds an agent-scoped
--log-buffer-size/CODER_AGENT_LOG_BUFFER_SIZEon the hiddencoder agentcommand. Once both land, the global--log-buffer-sizehere and the agent's own flag will need reconciling oncoder agent. Flagging for whichever merges second.Testing
cli.TestClampLogBufferSize,TestPruneSessionLogs,TestDefaultSessionLogDir, andTestBufferedLogger_*cover clamping, ssh log retention, the default state dir, buffer-until-error, the middleware's flush path, and the verbose escape hatch.start/stop/update/sshcommand tests pass. Golden--helpoutput anddocs/reference/cliregenerated. (TestSSH/Envis a pre-existing interactive-PTY test that does not run in all sandboxes; it fails identically onmainhere.)Rollout context
Cross-repo sequence: coder/slog flight recorder (released as v3.2.0) -> go.mod bump in coder/coder (#29600, merged) -> CLIENT-672 (agent, #29671) and this PR (CLI-4), which are independent consumers and can merge in any order.
Opened by Coder Agents on behalf of @aqandrew.