Skip to content

PubSub Subscriber will not stop when messages have not been acked or nacked #3752

@Xeli

Description

@Xeli

Issue
When using the Subscriber of the PubSub client, if any message is not ack'ed or nack'ed the subscriber does not shut down anymore.

Why would I want this?
In my case, I want to read PubSub messages using Apache Flink. Flink has a technique to keep itself consistent called checkpointing. Only on every checkpoint will should I ack messages. When a part of Flink crashes/exits/shuts down we want to ignore all messages since the last checkpoint and have PubSub resend them.

This means messages will only be acknowledged on a checkpoint (maybe every 500ms) and when shutting down the last couple of messages won't be acknowledged.

How to reproduce
I've edited the pubsub example in this repo to show what I mean:

public static void main(String... args) throws Exception {
    ProjectSubscriptionName subscription = ProjectSubscriptionName.of("bolcom-stg-jey-streaming-886", "streaming-test");
    MessageReceiver receiver =
            new MessageReceiver() {
                @Override
                public void receiveMessage(PubsubMessage message, AckReplyConsumer consumer) {
                    System.out.println("Received message: " + message.getData().toStringUtf8());
                    //DO NOT ack() or nack() and the subscriber will never stop
                    //consumer.ack();
                }
            };
    Subscriber subscriber = null;
    try {
        subscriber = Subscriber.newBuilder(subscription, receiver)
                               .build();
        subscriber.addListener(
                new Subscriber.Listener() {
                    @Override
                    public void failed(Subscriber.State from, Throwable failure) {
                        // Handle failure. This is called when the Subscriber encountered a fatal error and is shutting down.
                       System.err.println(failure);
                    }
               },
                MoreExecutors.directExecutor());
        subscriber.startAsync().awaitRunning();

        // In this example, we will pull messages for one minute (60,000ms) then stop.
        // In a real application, this sleep-then-stop is not necessary.
        // Simply call stopAsync().awaitTerminated() when the server is shutting down, etc.
        Thread.sleep(5000);
    } finally {
        if (subscriber != null) {
            System.out.println("Stopping");
            subscriber.stopAsync().awaitTerminated();
        }
    }
}

but basically, all that has changed is I do not ack() or nack() anything.

Question:
The example above won't ever return from the awaitTerminated() call and thus will not exit. Is this behavior by design? Or should I terminate in a different manner?

Metadata

Metadata

Assignees

Labels

api: pubsubIssues related to the Pub/Sub API.triage meI really want to be triaged.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions