Skip to content

improvement(desktop-browser): stable element refs, wait observations, and fewer false refusals - #9953

Merged
waleedlatif1 merged 10 commits into
stagingfrom
feat/browser-use-hillclimb-2
Nov 12, 2026
Merged

waleedlatif1 merged 10 commits into
stagingfrom
feat/browser-use-hillclimb-2

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

  • Element refs stay stable: an unchanged element keeps its ref across snapshots, and a ref from an earlier snapshot still resolves while its element lives (previously any later snapshot, including query/scoped observations, invalidated every ref the model held)
  • Text lines register the identity name the resolver recomputes — aria-labelled text lines and lines over 120 characters were never actionable by ref
  • A still-mounted control whose own menu/disclosure opened keeps its ref instead of reporting "identity drifted" (hover menus failed right after a successful hover)
  • browser_wait_for ends with a fresh snapshot, so the model doesn't spend a round re-observing
  • browser_type types into a top-page field covered by a non-modal overlay (keyboard focus, as Playwright fill does); modal backgrounds and framed fields still refuse
  • One bounded ref store per page (5000) and per driver scope (LRU), shared frame/target helpers

Type of Change

  • Improvement

Testing

  • Red/green unit tests for each change (page-functions.test.ts, driver.test.ts, post-action-observation.test.ts); 534 browser-agent tests pass
  • Lint, type-check, block registry, generators clean; check:audits only failed knip on the docs app's generated .content import, and bun run test only failed two desktop tests that also fail on staging when run as root (permission-refusal tests)
  • Production-loop benchmark (worker main agent, Opus 5.5 high, Sim's Electron driver; capbench 56 + WebGames 104 + REAL 121 paired episodes, fresh seeds, arms run concurrently): success 214 → 215, rounds/episode −12%, failed tool calls −36%, cost −9%, wall −7%

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing (new tests pass the test-audit authoring gate)
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel

vercel Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
docs Skipped Skipped Nov 12, 2026 9:29pm UTC

Request Review

@greptile-apps

greptile-apps Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; no new blocking issue was found.

Summary

The PR keeps browser element refs stable, adds fresh observations after waits, and allows typing through non-modal covers.

  • Browser element refs stay with the same live element across snapshots.
  • Top-page fields accept typing through non-modal overlays.
  • Browser waits now show a fresh page snapshot when they finish.

Reviews (3) · Last reviewed commit: "fix(desktop-browser): bound remembered r..." · Reviewed by Greptile

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 6 files

Reply with feedback, questions, or to request a fix.

Turn on auto-fix | Re-trigger cubic

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Nov 12, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 7 files

Reply with feedback, questions, or to request a fix.

Turn on auto-fix | Re-trigger cubic

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Nov 12, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 7 files

Reply with feedback, questions, or to request a fix.

Turn on auto-fix | Re-trigger cubic

This branch was previously deployed

1 inactive deployment
Preview — a49506eb Deployed Nov 12, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant