Repository navigation
Conversation
Docs previewCheck off each page once it's been reviewed. If a page changes in a later push, its checkbox clears automatically so it gets a fresh look. Pages not yet wired into the docs navigation aren't listed here. |
462179d to
596abb3
Compare
596abb3 to
b14818c
Compare
21fb010 to
dd08483
Compare
dd08483 to
4a05a54
Compare
e38f9e4 to
2d8f222
Compare
Documentation CheckUpdates Needed
Automated review via Coder Agents |
2d8f222 to
e48b0ac
Compare
code-asher
left a comment
There was a problem hiding this comment.
I think we have some conflicts between a connection "type" and the app name over that connection type that we should try to iron out.
| -- The enum held only families, so VS Code forks logged as vscode. Rewrites | ||
| -- the table under an exclusive lock. | ||
| ALTER TABLE connection_logs | ||
| ALTER COLUMN type TYPE text; |
There was a problem hiding this comment.
Could make sense to also rename the column to app_name? Type to me implies a grouping like we had before, but now these are individual app names.
Although for web apps it is still a grouping. 🤔 I suppose we could put the slug here instead? But I see we already have slug_or_port.
And we still have types like tunnel. hrmm. Maybe we need both type and app_name? For apps like VS Code we could set the type to ssh? Then we could use slug_or_port to store the app name vscode? (possibly renaming it to app_name_or_port)
There was a problem hiding this comment.
Actually true, this was a bit inconsistent and confusing. type was mixing apps with connection types. Went with your app_name_or_port idea, plus a rename of type:
typeis nowkind, a closed enum of what observed the connection: the agent forsshandreconnecting_pty, coderd forworkspace_app,port_forwarding, andtunnel. VS Code, Cursor, and Zed are allssh, as you suggested.slug_or_portis nowapp_name_or_port, read bykind: the app an agent reported, a workspace app slug, or a port.
Both are renames only, so the table is no longer rewritten and the enum stays. Families are never stored: the API type is the grouping again, derived from the app registry on read, so Cursor returns type: vscode with app_name: cursor, and adding an app or family needs no migration. Older rows keep vscode/jetbrains in kind with no app name, and read as that app. There's also a new app: filter that matches an agent app name or a workspace app slug.
There was a problem hiding this comment.
Awesome!!
I think we should replace vscode and jetbrains now in the migration with ssh (and populate their app_name_or_port with vscode or jetbrains as appropriate) to match the data going forward.
And then, if we do that, the "kind" name might need changing. The comment says "what observed the connection" which makes me think "subsystem" although maybe it is more like "transport" (or dare I say "connection type"). Although transport means something specific so maybe subsystem would be better.
Like the data goes through SSH, or a workspace app, or a port forward, or the agent's reconnecting-pty HTTP API, many of which flow through (and are observed, if not reported by) both coderd and the agent.
There was a problem hiding this comment.
And then, where we report the connection, instead of mapping the app name to the transport (like mapping jetbrains/vscode to SSH), we should just rely on what we know the actual subsystem is (so for agentssh.NewServer obviously we know it is always ssh, etc).
There was a problem hiding this comment.
RFC says to get rid of the enum and we should! The values that exist don't cover all the possibilities now or in future, and lead to us skipping connection logs (if the kind is unknown) or writing something not true (like assuming anything we don't recognize is SSH).
There was a problem hiding this comment.
Okay I've replaced the connection type with with source which is a string with 4 known values as we agreed above, by grouping ssh and reconnecting_pty it actually made the queries and logic everywhere simpler!
I'll now run a more comprehensive benchmark and see if this table is unbounded and how many rows might there be.
Edit: Added the synthetic benchmarks in the description.
There was a problem hiding this comment.
Yeah I think the problem is that we had the connection "type" tangled up with the name of the app making that connection. App names are dynamic; new ones can appear at any time, requiring coderd source change to start tracking them, which was the problem.
Connection types are static, the only way new types appear is if someone modifies the source. Since the source is being modified, the enum can be modified at the same time. And since the types are well-known and static, an enum seems a natural fit to me.
I think at the time of the RFC we were thinking the type column was essentially just a static "app name", or at least that is how I was thinking about it, and so that we could therefore drop it (and its enum) entirely.
We could still drop type and its enum entirely, if we are not particularly concerned with how the connection was made or what subsystem it went through, just that it exists and what app it was for, and fall back to the "type" for the app name when the app name is unknown (so an SSH connection would show "ssh" for the app name instead of "ssh" for the type and a blank app name).
But that would also collapse some things into each other, like web VS Code versus desktop VS Code would look the same instead of one type being "workspace_app" and another "ssh", both with the "vscode" app name.
There was a problem hiding this comment.
Also perhaps interesting to note that even the reconnecting pty can be used by different applications; currently it is used by the web terminal but can also be used on the command line via the cli.
There was a problem hiding this comment.
I do think that we should still do string over an enum because we do not know if this list is truly exhaustive. We can do the check in Go but the DB type can be string. Now for agent vs. ssh and reconnecting_pty, if we want to differentiate between say normal CLI (ssh) and reconnecting_pty CLI then we should have the following sources (we can call it protocol now I suppose): ssh, reconnecting_pty, workspace_app, port_forwarding, tunnel, unknown with the following semantics:
┌──────────────────┬──────────────────┬────────────────────┐
│ Old type │ → protocol │ → app_name_or_port │
├──────────────────┼──────────────────┼────────────────────┤
│ ssh │ ssh │ NULL │
├──────────────────┼──────────────────┼────────────────────┤
│ vscode │ ssh │ vscode │
├──────────────────┼──────────────────┼────────────────────┤
│ jetbrains │ ssh │ jetbrains │
├──────────────────┼──────────────────┼────────────────────┤
│ reconnecting_pty │ reconnecting_pty │ NULL │
├──────────────────┼──────────────────┼────────────────────┤
│ web types │ unchanged │ unchanged │
└──────────────────┴──────────────────┴────────────────────┘
In this case, we support proper protocol and we can even expose it in the API (for example, if we wanna tell reconnecting_pty from the web terminal vs. the CLI).
Actually, in this case, the agent.proto type: enum would still be deprecated but we can add a protocol: string so let's resolve this here before we modify #29675...
@spikecurtis @code-asher Thoughts?
There was a problem hiding this comment.
In considering whether to have the DB type be enum or string, I think the right question to think about is how important is constraining the set of values? If you do an enum, it's constrained at the database and we literally cannot represent an errant value.
If we do a string, then we lose strong constraints, but gain a lot of flexibility. It's easier to add new things.
For this case, where we are doing a sort of best-effort log of connections, I don't think we gain that much by making the constraints air-tight, and the flexibility to add new things without a DB schema migration are appealing.
55ab9fd to
71742a3
Compare
6cd8d30 to
5738c29
Compare
e906056 to
bed4132
Compare
The registry now gives reconnecting_pty an icon, so the deployment stats test expects it. SessionAppPresentation builds on AppDisplayName, which #29674 adds with the same definition.
99508ee to
f964b80
Compare
863403a to
c0ca459
Compare
spikecurtis
left a comment
There was a problem hiding this comment.
This whole thing is still half baked.
It changes the database to "source" and removes type, but we're still talking about Connection Types on the codersdk and API.
We're flying by the seat of our pants here. We deviated from the RFC, for legit reasons, but we can't be making a change as big and far reaching as this one PR by PR. It needs a new RFC that actually plans this out.
connection_logs.type mixed app names with connection types. Rename it to kind, which records what observed the connection, and rename slug_or_port to app_name_or_port: the app an agent reported, a workspace app slug, or a port. The API type is derived from the app registry on read, so a Cursor session reads as type vscode with app_name cursor. type:vscode matches every app in the family, type:unknown matches apps outside the registry, and the new app: filter matches an agent app name or a workspace app slug. The site's type labels are generated from the registry, and the family is labeled Visual Studio Code.
Replace the kind enum with source, a text column with four known values: agent, workspace_app, port_forwarding, and tunnel. Agent rows keep the app in app_name_or_port, which tells SSH from the web terminal, so a new app or agent feature needs no migration. The migration converts old rows too: vscode, jetbrains, ssh, and reconnecting_pty become source agent with that app name. Both columns convert in one ALTER, so the table is rewritten once, about 1.7s per million rows. db2sdk maps a row's source and app to the API type, and sdk2db maps a type filter back, so the two directions sit together and a test pins them. The agent's connection type maps to an app name in sdk2db, and an unspecified type logs as app unknown instead of being dropped. The VS Code family is labeled VS Code Family, apart from the VS Code app.
Table the rows each migration direction converts, and use fixed SQL, which also fixes the gosec G202 lint on the concatenated column list.
c0ca459 to
f6c9581
Compare
Separates how a connection was established from the originating app identity, following the Completing Scalable IDE Session Tracking RFC. This is the storage, HTTP API, filters and frontend half of phase 3; #29675 adds explicit agent reporting on top.
connection_logs.connection_methodis a validated enum;app_name_or_portremains nullable text. Historical VS Code and JetBrains rows become SSH with their identity retained. Missing client identity stays NULL and HTTPapp_nameis omitted. Workspace-app slugs and forwarded ports remain destinations inweb_info.slug_or_port, never client identities. The deprecated HTTPtypestays available as a plain string, with SSH family compatibility mappings; TypeScript/OpenAPI and the UI use the new method field.Legacy
type:and lifecyclestatus:filters remain supported. Newmethod:and normalized client-onlyapp:filters intersect consistently in list and count queries. PTY is labeled Reconnecting PTY, because browser terminals and the CLI share that method.Upgrade impact: the migration rewrites the table under an ACCESS EXCLUSIVE lock. On synthetic indexed PostgreSQL 13.21 data, with 2 CPUs and 6 GB on a shared host, observed upgrade lock durations were 5.83 s / 49.7 s / 99.5 s for 1M / 10M / 25M rows. Rollback durations were 4.15 s / 51.3 s / 400.1 s; the last was an unrepeated disk-bound outlier, not a typical estimate. Upgrade WAL was approximately 354 MB / 3.5 GB / 8.8 GB. These replace the earlier source/text benchmarks and do not measure waiting to acquire the lock under production traffic. All seeded rows were checked after upgrade and rollback. Downgrading collapses client identities to historical families; empty destination strings and legacy non-destination values normalize to NULL.
Go SDK consumers must adapt
ConnectionLog.TypefromConnectionTypetostring. Service-log fieldsTypeandSlugOrPortbecomeConnectionMethodandAppNameOrPort; the unrelated lowercase direct/DERP routing fieldconnection_methodis unchanged.Agreed implementation contract and verification evidence
Verification: make gen, make lint and make pre-commit passed; the full Go suite completed 34,783 tests with 50 skips; the site production build passed; all 285 frontend unit files passed (4,477 tests, 2 skips) on a rerun with two workers. The initial concurrent pre-push frontend run had unrelated timeouts; the remaining focused suites passed on unchanged main and this branch when rerun after contention subsided. Connection-log stories passed, including deliberately conflicting deprecated type values. Migration mapping, rollback, PostgreSQL 13/17 single-transaction checks and large-table measurements completed. Raw benchmark artifacts were provided in the implementation chat.
Phase 6 duration grouping and agent raw port-forward reporting are deliberately excluded. JetBrains per-product propagation is a separate researched follow-up, not claimed complete here.
Refs #27412. Generated with Coder Agents on behalf of @EhabY.