Skip to content

feat(pubsublite): Support Google Managed Kafka backend#573

Open
brownsuga918 wants to merge 3 commits into
googleapis:mainfrom
brownsuga918:gmk-support
Open

feat(pubsublite): Support Google Managed Kafka backend#573
brownsuga918 wants to merge 3 commits into
googleapis:mainfrom
brownsuga918:gmk-support

Conversation

@brownsuga918

Copy link
Copy Markdown

This PR implements Google Managed Kafka (GMK) backend support for the Python Pub/Sub Lite client.

It matches the Go implementation by adding:

  1. Gating to PublisherClient and SubscriberClient to detect MessagingBackend.MANAGED_KAFKA and route to Kafka-based implementations.
  2. KafkaPublisherClient (sync & async) using confluent_kafka.Producer.
  3. KafkaAsyncSingleSubscriber (used in multiplexed subscriber clients) using confluent_kafka.Consumer with thread-safe offset commits.
  4. Automatic consumer group ID derivation from Pub/Sub Lite subscription path.
  5. Header translation including event time parsing and multi-valued header flattening.
  6. OAuth callback support using Google credentials.

Tested thoroughly with unit tests mocking confluent_kafka.

@brownsuga918 brownsuga918 requested review from a team as code owners June 13, 2026 19:36
@product-auto-label product-auto-label Bot added the size: xl Pull request size is extra large. label Jun 13, 2026
@product-auto-label product-auto-label Bot added the api: pubsublite Issues related to the googleapis/python-pubsublite API. label Jun 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: pubsublite Issues related to the googleapis/python-pubsublite API. size: xl Pull request size is extra large.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants