Add Kitesurf support to Browser Tools by cjol · Pull Request #2098 · cloudflare/agents

cjol · 2026-08-12T15:29:00Z

This PR adds connection-scoped Kitesurf support to Agents Browser Tools and uses it in the ai-chat example. Fixes #2095.

Why

Browser Tools previously assumed Browser Run always acquires a reconnectable Chromium session. Kitesurf instead acquires the browser directly through /v1/devtools/browser?browser=kitesurf, where the WebSocket itself owns the browser lifetime.
Treating Kitesurf's analytics session identifier as reconnectable would make pause, reuse, cleanup, and Live View behavior incorrect.
Kitesurf Quick Actions exist over the public REST API, but the Worker binding's quickAction() RPC cannot currently carry an engine selector. Passing browser in its options is rejected as part of the action body.
This PR therefore adds the verified CDP path while rejecting unsupported durable lifecycle features. A possible connection-pinned reuse mode remains a separate design direction because it could not survive socket loss or Durable Object restart.

Public API Surface

Symbol	Kind	Notes
`ConnectBrowserOptions.browser`	Additive option	Selects the `"kitesurf"` engine for direct CDP acquisition.
`BrowserConnectorSessionOptions.browser`	Additive option	Selects Kitesurf for `createBrowserTools()` and `createBrowserRuntime()`.

Selecting Kitesurf defaults Quick Actions off because the binding cannot select that engine. Callers can explicitly request Quick Actions to create a mixed toolset where CDP uses Kitesurf and Quick Actions use Chromium.

Architectural Changes

Chromium
acquire session -> persist session id -> connect/reconnect WebSocket -> delete session

Kitesurf
open /v1/devtools/browser?browser=kitesurf WebSocket -> use browser -> close connection

Kitesurf is exposed as one-shot and connection-scoped. The connector does not expose protocol discovery, Live View, shared session management, recording, keep-alive, or pause/resume for it.
A durable execution marker prevents a paused or lost Kitesurf connection from silently continuing in a fresh browser.
The design notes distinguish this current policy from a future best-effort mode that could pin an open WebSocket across codemode calls without promising durable reconnection.

Code Changes

browser-run.ts acquires Kitesurf directly over WebSocket and rejects incompatible Browser Run options before making a request.
BrowserConnector selects the Kitesurf acquisition path, narrows its model-facing tools and instructions, and fails explicitly when a connection can no longer resume.
browser_execute preserves canonical screenshot results for AI SDK UIs while toModelOutput sends the model only a compact attachment summary.
The Kitesurf-specific tool description documents the raw CDP call shape, attachment handles, bounded readiness polling, and screenshot return contract without asking models to search for CDP commands through codemode.search().
examples/ai-chat demonstrates natural-language Kitesurf navigation and inline screenshot rendering with a remote Browser Run binding.
Browser documentation records the supported surface, current Quick Action limitation, and connection-pinned reuse boundary.

changeset-bot · 2026-08-12T15:29:04Z

🦋 Changeset detected

Latest commit: be30ecd

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages

Name	Type
agents	Minor
@cloudflare/agent-think	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

pkg-pr-new · 2026-08-12T15:39:34Z

Open in StackBlitz

agents

npm i https://pkg.pr.new/agents@2098

@cloudflare/ai-chat

npm i https://pkg.pr.new/@cloudflare/ai-chat@2098

@cloudflare/codemode

npm i https://pkg.pr.new/@cloudflare/codemode@2098

hono-agents

npm i https://pkg.pr.new/hono-agents@2098

@cloudflare/shell

npm i https://pkg.pr.new/@cloudflare/shell@2098

@cloudflare/think

npm i https://pkg.pr.new/@cloudflare/think@2098

@cloudflare/voice

npm i https://pkg.pr.new/@cloudflare/voice@2098

@cloudflare/worker-bundler

npm i https://pkg.pr.new/@cloudflare/worker-bundler@2098

commit: be30ecd

devin-ai-integration · 2026-08-12T16:25:44Z

Pushed cc89773 for the two Devin Review findings (on top of c29c3c7):

keepAliveMs: 0 + Kitesurf: rather than rejecting it (your allows explicitly disabled Kitesurf session options test says explicitly-disabled options are fine), connectBrowser now never forwards keep_alive on the Kitesurf path, so keep_alive=0 can't reach the service. Non-zero values still throw. Test extended to cover keepAliveMs: 0.
examples/ai-chat: formatToolOutput no longer keys redaction off the strict browser_screenshot predicate — any tool result with a large data string is redacted in the transcript, while <img> rendering stays strict.

… path

devin-ai-integration · 2026-08-14T08:54:56Z

Pushed 5596609 for the two new Devin Review findings:

examples/ai-chat/src/server.ts: hoisted the tool set into a tools const and pass it to convertToModelMessages(this.messages, { tools }), so browser_execute's toModelOutput also shrinks persisted screenshots when history is replayed.
packages/agents/src/browser/tanstack-ai.ts: the wrapper now applies browser_execute's toModelOutput to the result before returning it, so the TanStack path gets the same compact summary instead of the raw base64.

…inding path

…tesurf

devin-ai-integration

Devin Review found 2 new potential issues.

View 5 additional findings in Devin Review.

devin-ai-integration · 2026-08-14T10:25:35Z

+    // This wrapper hands the result straight to the model, so apply the same
+    // model-facing summary the AI SDK path gets from `toModelOutput` — a
+    // screenshot is kept out of the model context.
+    const modelOutput = await executeTool.toModelOutput?.({
+      toolCallId: crypto.randomUUID(),
+      input: { code },
+      output: result
+    });
+    if (modelOutput?.type === "text" || modelOutput?.type === "error-text") {
+      return modelOutput.value;
+    }
+    if (modelOutput?.type === "json") {
+      return modelOutput.value;
+    }
    return result;


🟡 Screenshots taken through the TanStack browser tool are thrown away while the model is told they were attached

The screenshot bytes are discarded and replaced with a summary sentence (toModelOutput at packages/agents/src/browser/tanstack-ai.ts:72-79) before the tool result is handed back to the caller, so a TanStack host can no longer show the picture and the model is told the image is "attached to the chat" when nothing was attached.

Impact: Users of the TanStack integration lose every screenshot their agent takes, and the model reports an attachment that does not exist.

Why the AI SDK path keeps the image but the TanStack path does not

In the AI SDK path (packages/agents/src/browser/ai.ts:403-409) toModelOutput is only the model-facing projection: the full tool output — including the canonical { type: "browser_screenshot", mediaType, data } value preserved by transformBrowserResult (packages/agents/src/browser/ai.ts:262-269) — still reaches the UI through the tool result part, which is exactly what examples/ai-chat/src/client.tsx:105-121 renders.

The TanStack wrapper has a single return channel, so returning modelOutput.value collapses both channels: for a screenshot result it returns only "Screenshot captured successfully (…) and attached to the chat." (packages/agents/src/browser/ai.ts:271-281), dropping status, executionId, calls, and the image data that the wrapper previously returned. Also note type === "error-text" is returned as an ordinary success value, losing the error signal.

Prompt for agents

In packages/agents/src/browser/tanstack-ai.ts the wrapper now replaces the tool result with the AI-SDK model-facing projection produced by browser_execute's toModelOutput (defined in packages/agents/src/browser/ai.ts). For canonical screenshot results that projection is only the sentence "Screenshot captured successfully (…) and attached to the chat.", so the TanStack caller loses the image bytes and the rest of the ProxyToolOutput (status, executionId, calls), and the sentence is factually wrong because there is no chat attachment channel in this wrapper. Consider either returning a shape that keeps the screenshot payload available to the host (e.g. return the raw result and let the host decide), or making the model-facing summary wording channel-neutral and ensuring the image is still reachable. Also handle the error-text case distinctly instead of returning it as a successful result value.

Was this helpful? React with 👍 or 👎 to provide feedback.

Partly addressed in be30ecd; the rest is a genuine tradeoff, so flagging rather than flipping again.

TanStack ServerTool has one return channel — whatever the host gets is what goes into the model's context — so the image bytes cannot be both available to the host and kept out of the model. This thread asks for the opposite of #2098 (comment) predecessor finding (screenshots sent to the model unbounded), which is what the projection was added for.

What changed:

the envelope is preserved for the screenshot case: { ...result, result: <summary> } keeps status, executionId and calls, only the base64 is replaced;

error-text now throws instead of being returned as a success value;

the summary wording is channel-neutral ("the image is kept for the UI and omitted here") since there is no chat attachment in this wrapper.

If TanStack hosts should be able to render screenshots, the right fix is a separate out-of-band channel (e.g. writing the image to R2/a workspace and returning a reference) rather than returning raw base64 — happy to do that as a follow-up if @cjol wants it.

…nshot previews

feat(agents): add Kitesurf browser support

6f7703d