Skip to content

[DNM] kafka: bump sarama dependency to v1.60.1 and do not backoff on the first time error handling - #12848

Open
3AceShowHand wants to merge 2 commits into
pingcap:masterfrom
3AceShowHand:bump-sarama-remove-retry
Open

[DNM] kafka: bump sarama dependency to v1.60.1 and do not backoff on the first time error handling#12848
3AceShowHand wants to merge 2 commits into
pingcap:masterfrom
3AceShowHand:bump-sarama-remove-retry

Conversation

@3AceShowHand

@3AceShowHand 3AceShowHand commented Sep 4, 2026 •
edited by coderabbitai Bot
Loading

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: close #xxx

What is changed and how it works?

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Questions

Will it cause performance regression or break compatibility?
Do you need to update user documentation, design documentation or monitoring documentation?

Release note 

Please refer to [Release Notes Language Style Guide](https://pingcap.github.io/tidb-dev-guide/contribute-to-tidb/release-notes-style-guide.html) to write a quality release note.

If you don't think this PR needs a release note then fill it with `None`.

Summary by CodeRabbit

  • Bug Fixes

    • Retry sequences now begin immediately, while subsequent retries continue using randomized backoff intervals.
    • Retry backoff correctly resets after an extended period without errors.
    • Capture ownership now reliably tracks peer additions and removals, including when processor activity is unavailable.

  • Chores

    • Updated Kafka client and supporting module dependencies.
    • Removed obsolete dependency references and adjusted Kafka retry configuration documentation.

@ti-chi-bot ti-chi-bot Bot added do-not-merge/needs-linked-issue release-note Denotes a PR that will be considered when it comes time to generate release notes. labels Sep 4, 2026
@ti-chi-bot

ti-chi-bot Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign lance6716 for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot Bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Sep 4, 2026
@coderabbitai

coderabbitai Bot commented Sep 4, 2026 •
edited
Loading

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: e2ccfed0-c7e9-42fe-9053-0dd0ace1ada5

📥 Commits

Reviewing files that changed from the base of the PR and between 559a3b6 and 3d51014.

📒 Files selected for processing (2)
  • cdc/capture/capture.go
  • cdc/capture/capture_test.go

Included review availability: Your plan provides up to 2 included reviews per hour; 1 remains after this review.

📝 Walkthrough

Walkthrough

The pull request refreshes Go dependencies, removes the PingCAP Sarama replacement, changes first-retry backoff behavior, and passes callback-enabled global state to the capture owner’s etcd worker. Tests cover retry timing and peer tracking.

Changes

Dependency updates

Layer / File(s) Summary
Sarama and module dependency refresh
go.mod, pkg/sink/kafka/sarama.go		 Sarama and supporting modules are upgraded. Unused indirect dependencies and the PingCAP Sarama replacement are removed. The Kafka retry comment now references Net.MaxOpenRequests=1.

Retry backoff behavior

Layer / File(s) Summary
First-retry backoff handling
pkg/retry/error_retry.go, pkg/retry/error_retry_test.go		 GetRetryBackoff returns zero for the first retry in a new error series. Tests validate the initial zero backoff and the subsequent randomized range.

Capture owner peer tracking

Layer / File(s) Summary
Owner peer tracking and validation
cdc/capture/capture.go, cdc/capture/capture_test.go		 campaignOwner passes the callback-enabled global state to runEtcdWorker. The integration test verifies peer registration and removal without processor ticks.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: ⚪ Minimal · up to 3d510

The change updates dependency, retry, and owner peer-tracking behavior; the supplied test coverage validates peer registration and removal, with no concrete merge-blocking risk identified.

Sequence Diagram(s)

sequenceDiagram
  participant campaignOwner
  participant runEtcdWorker
  participant Etcd
  participant MessageRouter
  campaignOwner->>runEtcdWorker: pass callback-enabled globalState
  runEtcdWorker->>Etcd: watch capture peer changes
  Etcd-->>runEtcdWorker: peer added or deleted
  runEtcdWorker->>MessageRouter: register or remove peer
Loading

Poem

A rabbit starts the first retry bright
New peers appear in etcd’s light
Old peers hop from the router’s view
Sarama follows upstream too
Tests guard each change in view

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description retains the template but does not explain the problem, implementation, test results, compatibility impact, documentation impact, or release note. It also leaves the placeholder issue r… Replace the placeholders with a valid issue reference and complete every applicable section. Describe the Sarama upgrade, first-error retry behavior, and global-state fix. Identify the tests that were added or run, answer both Questions ite…
Docstring Coverage ⚠️ Warning Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 4 functions across 5 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly identifies the two primary changes: updating Sarama to v1.60.1 and removing backoff for the first Kafka error. The additional global-state fix is not required in the title.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Full details: Description check

Explanation

The description retains the template but does not explain the problem, implementation, test results, compatibility impact, documentation impact, or release note. It also leaves the placeholder issue reference as close #xxx``.

Resolution

Replace the placeholders with a valid issue reference and complete every applicable section. Describe the Sarama upgrade, first-error retry behavior, and global-state fix. Identify the tests that were added or run, answer both Questions items, and provide a release note or None.

  • Fix all pre-merge checks with AI
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Warning

Some tools did not complete. Review the errors below.

🔧 golangci-lint (2.13.2)

Error: can't load config: unsupported version of the configuration: "" See https://golangci-lint.run/docs/product/migration-guide for migration instructions
The command is terminated due to an error: can't load config: unsupported version of the configuration: "" See https://golangci-lint.run/docs/product/migration-guide for migration instructions

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@3AceShowHand

3AceShowHand commented Sep 4, 2026

Copy link
Copy Markdown
Contributor Author

/retest

@ti-chi-bot ti-chi-bot Bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Sep 7, 2026
@ti-chi-bot

ti-chi-bot Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

[FORMAT CHECKER NOTIFICATION]

Notice: To remove the do-not-merge/needs-linked-issue label, please provide the linked issue number on one line in the PR body, for example: Issue Number: close #123 or Issue Number: ref #456.

📖 For more info, you can check the "Contribute Code" section in the development guide.

@ti-chi-bot

ti-chi-bot Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

@3AceShowHand: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-dm-integration-test 3d51014 link true /test pull-dm-integration-test
pull-cdc-integration-kafka-test 3d51014 link true /test pull-cdc-integration-kafka-test

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

do-not-merge/needs-linked-issue release-note Denotes a PR that will be considered when it comes time to generate release notes. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@3AceShowHand