-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Description
Issue
When using the Subscriber of the PubSub client, if any message is not ack'ed or nack'ed the subscriber does not shut down anymore.
Why would I want this?
In my case, I want to read PubSub messages using Apache Flink. Flink has a technique to keep itself consistent called checkpointing. Only on every checkpoint will should I ack messages. When a part of Flink crashes/exits/shuts down we want to ignore all messages since the last checkpoint and have PubSub resend them.
This means messages will only be acknowledged on a checkpoint (maybe every 500ms) and when shutting down the last couple of messages won't be acknowledged.
How to reproduce
I've edited the pubsub example in this repo to show what I mean:
public static void main(String... args) throws Exception {
ProjectSubscriptionName subscription = ProjectSubscriptionName.of("bolcom-stg-jey-streaming-886", "streaming-test");
MessageReceiver receiver =
new MessageReceiver() {
@Override
public void receiveMessage(PubsubMessage message, AckReplyConsumer consumer) {
System.out.println("Received message: " + message.getData().toStringUtf8());
//DO NOT ack() or nack() and the subscriber will never stop
//consumer.ack();
}
};
Subscriber subscriber = null;
try {
subscriber = Subscriber.newBuilder(subscription, receiver)
.build();
subscriber.addListener(
new Subscriber.Listener() {
@Override
public void failed(Subscriber.State from, Throwable failure) {
// Handle failure. This is called when the Subscriber encountered a fatal error and is shutting down.
System.err.println(failure);
}
},
MoreExecutors.directExecutor());
subscriber.startAsync().awaitRunning();
// In this example, we will pull messages for one minute (60,000ms) then stop.
// In a real application, this sleep-then-stop is not necessary.
// Simply call stopAsync().awaitTerminated() when the server is shutting down, etc.
Thread.sleep(5000);
} finally {
if (subscriber != null) {
System.out.println("Stopping");
subscriber.stopAsync().awaitTerminated();
}
}
}
but basically, all that has changed is I do not ack() or nack() anything.
Question:
The example above won't ever return from the awaitTerminated() call and thus will not exit. Is this behavior by design? Or should I terminate in a different manner?