Skip to content

feat(site): rich preview for the spawn_computer_use_agent tool - #23684

Merged
hugodutka merged 1 commit into
mainfrom
hugodutka/computer-use-tool-card
Mar 26, 2026
Merged

hugodutka merged 1 commit into
mainfrom
hugodutka/computer-use-tool-card

Conversation

@hugodutka

@hugodutka hugodutka commented Mar 26, 2026 •

Copy link
Copy Markdown
Collaborator

Adds preview cards for the spawn_computer_use_agent tool. Spawning that agent now renders a rich saying "Spawning computer use sub-agent". The "waiting for" tool now displays an inline desktop preview which can be clicked to reveal the desktop in the sidebar.

Screen.Recording.2026-03-26.at.18.47.05.mov

@github-actions github-actions Bot added the community Pull Requests and issues created by the community. label Mar 26, 2026
@hugodutka
hugodutka force-pushed the hugodutka/computer-use-tool-card branch from 8ebf2f0 to 1ff62f4 Compare March 26, 2026 18:16
@hugodutka
hugodutka marked this pull request as ready for review March 26, 2026 18:20
@mafredri
mafredri self-requested a review March 26, 2026 18:26
@kylecarbs

Copy link
Copy Markdown
Member

@hugodutka for a preview I don't like how big it is. I'd rather it was smaller and said like "Using the Computer" in shimmer text - that would be dope. Like a little 128px height card. With a clear "Click to view" that would open the side-panel or something.

I'm also happy to fix this up afterwards.

@hugodutka

Copy link
Copy Markdown
Collaborator Author

@kylecarbs I'd rather change this afterwards on Monday. I think Danielle is planning to do a major frontend refactor tomorrow, so it'd be more convenient for me to lock the logic in and change the look later.

@mafredri mafredri left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clean work overall. The DesktopPanelContext design is solid, the connectionOverride testing seam avoids fragile module mocks, and the aria-expanded fix is a nice accessibility improvement done without being asked. Storybook coverage across all VNC connection states is thorough.

1 P2, 5 P3s, 1 observation, and 6 nits across 11 inline comments.

Meruem: "Each hook instance creates its own RFB instance, WebSocket, reconnect timers, and ResizeObserver. Multiple wait_agent or message_agent calls referencing the same desktop subagent each get their own inline preview, each opening its own WebSocket. The worst case is N inline previews plus the sidebar panel: N+1 concurrent VNC connections to one desktop."

Hisoka: "The stories look correct at a glance and break on interaction. That's my favorite kind."

🤖 This review was automatically generated with Coder Agents.

@hugodutka
hugodutka force-pushed the hugodutka/computer-use-tool-card branch from 1ff62f4 to 88e69a1 Compare March 26, 2026 21:27

@mafredri mafredri left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing the noVNC private internals issue. Reading canvas dimensions from the DOM is cleaner.

The SidebarTabView stories (F2 from round 1) still throw on tab click since onActiveTabChange is undefined at runtime. That's the one remaining item worth fixing before merge. The rest is acknowledged/deferred or nits.

🤖 This review was automatically generated with Coder Agents.

@hugodutka
hugodutka force-pushed the hugodutka/computer-use-tool-card branch from 88e69a1 to 7d18cfe Compare March 26, 2026 21:44
@hugodutka
hugodutka enabled auto-merge (squash) March 26, 2026 21:47

@mafredri mafredri left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All findings addressed. The showDesktopPreviews prop is a clean solution to F3, letting historical messages show MonitorIcon without opening dead VNC connections. Stories fixed, dead code removed, nits resolved.

No new issues in the delta.

🤖 This review was automatically generated with Coder Agents.

@hugodutka
hugodutka merged commit f87d6e6 into main Mar 26, 2026
27 checks passed
@hugodutka
hugodutka deleted the hugodutka/computer-use-tool-card branch March 26, 2026 21:52
@github-actions github-actions Bot locked and limited conversation to collaborators Mar 26, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

community Pull Requests and issues created by the community.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants