feat(pubsub): add publish telemetry headers for publish attempt observability - #14338

There is a race condition in scheduleQueueProcessing when scheduling a new hedged request. If hedgingQueue.peek() returns null (because the queue was empty at that moment), but another thread adds an item to the queue and calls scheduleQueueProcessing right before isQueueProcessingScheduled.set(false) is executed, the compareAndSet in the second thread will fail (since it is still true). The first thread will then set the flag to false and return. As a result, the newly added item will remain in the queue without any scheduled task to process it. Using a loop to re-check the queue after setting the flag to false resolves this issue.

  private void scheduleQueueProcessing() {
    while (true) {
      if (!isQueueProcessingScheduled.compareAndSet(false, true)) {
        return;
      }
      HedgedRequest nextItem = hedgingQueue.peek();
      if (nextItem == null) {
        isQueueProcessingScheduled.set(false);
        if (!hedgingQueue.isEmpty()) {
          continue;
        }
        return;
      }

      long delay = Math.max(0, nextItem.getSendAfterMs() - clock.millisTime());

      queueProcessingFuture =
          executor.schedule(
              new Runnable() {
                @Override
                public void run() {
                  processQueue();
                }
              },
              delay,
              TimeUnit.MILLISECONDS);
      return;
    }
  }

The queueProcessingFuture field is written to by both the publisher thread (via startHedgedCall) and the executor thread (via processQueue) without any synchronization or volatile visibility. This can lead to data races and visibility issues, particularly during shutdown where queueProcessingFuture is read and cancelled. Marking this field as volatile ensures proper thread visibility.

I don't think this is the right publish time. We should use the publish time from the publish() call per the proto comment: "Time at which the publish() call was initiated in the client library". This will likely require a few more changes to plumb that value to here, so I'd be okay with just having the hedged attempt set for now.

I've removed this from the changes.