Skip to content

fix(mothership): back off the transport poll instead of retrying every second - #9945

Merged
waleedlatif1 merged 2 commits into
stagingfrom
fix/transport-poll-backoff
Nov 12, 2026
Merged

waleedlatif1 merged 2 commits into
stagingfrom
fix/transport-poll-backoff

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

  • The outbound transport receiver retried every failed poll after a fixed 1 s, forever. A self-hosted deployment whose COPILOT_API_KEY the worker rejects polls once a second, all day, and writes a warning on every attempt
  • A refused key (401/403) now backs off from 30 s to a 10-minute ceiling with jitter, and still reconnects by itself once the key is accepted
  • Other failures (lost connection, worker deploy, 5xx, a bad batch) back off from 1 s to 4 s, so with jitter a healthy deployment is back within the worker's 5 s run-control deadline
  • One warning per outage, and another only when it changes kind (outage → refused key), plus one info line on reconnect. The refused-key warning names COPILOT_API_KEY
  • The wait is interruptible, so SIGTERM never sits behind a long backoff

Type of Change

  • Bug fix

Testing

  • receiver.test.ts: four new backoff tests (refused key, transient ceiling, reset after a good poll, reset when an outage changes kind). All failed on the old fixed 1 s retry; removing the reset on a kind change turns its test red
  • Transport suites: 22 passing
  • biome and a type-check of the changed file; CI runs the full audits and type-check

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing (new tests pass the test-audit authoring gate)
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel

vercel Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
docs Skipped Skipped Nov 12, 2026 8:06pm UTC

Request Review

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 2 files

Reply with feedback, questions, or to request a fix.

Turn on auto-fix | Re-trigger cubic

@greptile-apps

greptile-apps Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; the earlier reset issue is fixed and no new actionable issues remain.

Summary

The PR adds longer waits for refused keys, capped waits for other failures, fewer repeated warnings, and interruptible waits.

  • Transport polls slow down when a worker keeps refusing requests.

Reviews (2) · Last reviewed commit: "fix(mothership): end a transport outage ..." · Reviewed by Greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Nov 12, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 2 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Turn on auto-fix | Re-trigger cubic

This branch was previously deployed

1 inactive deployment
Preview — 1b7cac0d Deployed Nov 12, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant