Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: apache/pulsar-client-cpp
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: main
Choose a base ref
...
head repository: apache/pulsar-client-cpp
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: branch-3.4
Choose a head ref
Checking mergeability… Don’t worry, you can still create the pull request.
  • 17 commits
  • 54 files changed
  • 2 contributors

Commits on Nov 1, 2023

  1. Release 3.4.0

    BewareMyPower committed Nov 1, 2023
    Configuration menu
    Copy the full SHA
    272e1a1 View commit details
    Browse the repository at this point in the history

Commits on Nov 7, 2023

  1. Fix ProducerBusy or ConsumerBusy error when configuring multiple brok…

    …ers per connection (#337)
    
    ### Motivation
    
    This is a catch up for apache/pulsar#21144
    
    When a producer or consumer reconnects, a random number will be generated as the key suffix in `ConnectionPool` to create or get the `ClientConnection` object from the pool.
    
    https://github.com/apache/pulsar-client-cpp/blob/81cc562f7b366fad97e1b80c07ef9334a808390d/lib/ConnectionPool.cc#L75
    
    If a new connection is created with the same producer or consumer name to the broker, the broker will respond with a  `ProducerBusy` or `ConsumerBusy` error so that the reconnection will never succeed.
    
    ### Modifications
    
    - Add an overload of `ConnectionPool::getConnectionAsync` that accepts an integer parameter as the key suffix. If it's not specified, generate the random number as the suffix. In this method, choose the executor by `key suffix % size`.
    - Generate the random number and save it when creating the `HandlerBase` object. When connecting the owner broker of its topic, pass that index so that the reconnection will always reuse the same `ClientConnection` object.
    
    ### Verifying this change
    
    `ProducerTest.testReconnectMultiConnectionsPerBroker` is added to protected the change.
    
    (cherry picked from commit 6f115e7)
    BewareMyPower committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    f337eff View commit details
    Browse the repository at this point in the history

Commits on Nov 17, 2023

  1. Use absolute path to find credential files in unit tests (#340)

    Fixes #339
    
    ### Motivation
    
    Currently the relative path is used in unit tests to specify the token
    file (`.test-token.txt`) and the credential directory (`./test-conf`).
    If we don't run the `pulsar-tests` binary in a subdirectory, it won't
    be able to read these files.
    
    ### Modifications
    
    Add the macro `-DTOKEN_PATH="..." -DTEST_CONF_DIR="..."` to specify
    absolute paths according to the `PROJECT_SOURCE_DIR`, which is defined
    by CMake as the absolute path of the project directory.
    
    ### Verifications
    
    ```bash
    cmake -B build
    cmake --build build -j8
    # Run the test in project directory
    ./build/tests/pulsar-tests --gtest_filter='AuthPlugin*:*testRSAEncryption:*testEncryptionFailure:*DecryptionFailedMessages'
    # Run the test in a subdirectory
    cd build
    ./tests/pulsar-tests --gtest_filter='AuthPlugin*:*testRSAEncryption:*testEncryptionFailure:*DecryptionFailedMessages'
    ```
    
    (cherry picked from commit 47fb809)
    BewareMyPower committed Nov 17, 2023
    Configuration menu
    Copy the full SHA
    39d39a9 View commit details
    Browse the repository at this point in the history
  2. Fix close() returns ResultAlreadyClosed after unsubscribe or close (#338

    )
    
    Fixes #88
    
    ### Motivation
    
    When `close` is called if the consumer has already called `unsubscribe`
    or `close`, it should not fail. See
    
    https://github.com/apache/pulsar/blob/428c18c8d0c3d135189920740192982e11ffb2bf/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ConsumerImpl.java#L1034
    
    ### Modifications
    
    Use the same close logic with Java client.
    
    Add `testCloseAgainBeforeCloseDone` and `testCloseAfterUnsubscribe` to
    verify the new behaviors of `Consumer::close`.
    
    (cherry picked from commit a8402da)
    BewareMyPower committed Nov 17, 2023
    Configuration menu
    Copy the full SHA
    1674a08 View commit details
    Browse the repository at this point in the history
  3. Fix lazy partitioned producer might send duplicated messages (#342)

    Fixes #341
    
    ### Motivation
    
    When a lazy partitioned producer sends two messages, the flow is:
    1. `start` is called to grab the connection via `grab()`.
    2. Generate 0 as the sequence id of the 1st message.
    3. Add the 1st message into the queuea.
    4. The connection is established, `msgSequenceGenerator_` is reset from
       1 to 0.
    5. When sending the 2nd message, 0 is also generated as the sequence id.
    
    Then two messages have the same sequence id.
    
    ### Modifications
    
    For lazy partitioned producers, if the internal producer is not started,
    sending the message in the callback of its future.
    
    Add `ChunkDedupTest#testLazyPartitionedProducer` to verify it since only
    the `tests/chunkdedup/docker-compose.yml` enables the deduplication.
    
    (cherry picked from commit bb16f24)
    BewareMyPower committed Nov 17, 2023
    Configuration menu
    Copy the full SHA
    8408e65 View commit details
    Browse the repository at this point in the history
  4. Fix crash when removing connection from the pool (#347)

    Fixes #346
    
    ### Motivation
    
    #336 changes the key of
    the `ClientConnection` in `ConnectionPool`, while in
    `ClientConnection::close`, it still passes the old key (logical address)
    to `ConnectionPool::remove`, which results in the connection could never
    be removed and destroyed until being deleted as a stale connection.
    
    What's worse, if the key does not exist, the iterator returned by
    `std::map::find` will still be dereferenced, which might cause crash in
    some platforms. See
    https://github.com/apache/pulsar-client-cpp/blob/8d32fd254e294d1fabba73aed70115a434b341ef/lib/ConnectionPool.cc#L122-L123
    
    ### Modifications
    
    - Avoid dereferencing the iterator if it's invalid in
      `ConnectionPool::remove`.
    - Store the key suffix in `ClientConnection` and pass the correct key to
      `ConnectionPool::remove` in `ClientConnection::close`
    - Add `ClientTest.testConnectionClose` to verify
      `ClientConnection::close` can remove itself from the pool and the
      connection will be destroyed eventually.
    
    (cherry picked from commit 6d47e94)
    BewareMyPower committed Nov 17, 2023
    Configuration menu
    Copy the full SHA
    aee12e2 View commit details
    Browse the repository at this point in the history
  5. Log topic lookup result (#351)

    (cherry picked from commit c771b12)
    erobot authored and BewareMyPower committed Nov 17, 2023
    Configuration menu
    Copy the full SHA
    806698e View commit details
    Browse the repository at this point in the history
  6. Fix bad_weak_ptr when close a ClientConnection during construction (#350

    )
    
    Fixes #348
    
    Fixes #349
    
    ### Motivation
    
    When `close` is called in `ClientConnection`'s constructor,
    `shared_from_this()` will be called, which results in a
    `std::bad_weak_ptr` error. This error does not happen before
    #317 because
    `shared_from_this()` could only be called when the `producers` or
    `consumers` field is not empty.
    
    ### Modifications
    
    Throw `ResultAuthenticationError` when there is anything wrong with
    authentication in `ClientConnection`'s constructor. Then catch the
    result in `ConnectionPool::getConnectionAsync` and returned the failed
    future.
    
    In addition, check `authentication_` even for non-TLS URLs. Otherwise,
    the null authentication will be used to construct `CommandConnect`.
    
    Add `testInvalidPlugin` and `testTlsConfigError` to verify the changes.
    
    (cherry picked from commit 7bb94f4)
    BewareMyPower committed Nov 17, 2023
    Configuration menu
    Copy the full SHA
    fa8decd View commit details
    Browse the repository at this point in the history
  7. Release 3.4.1

    BewareMyPower committed Nov 17, 2023
    Configuration menu
    Copy the full SHA
    e6ec207 View commit details
    Browse the repository at this point in the history

Commits on Nov 21, 2023

  1. Fix the flush callback might be called repeatedly (#353)

    Fixes #352
    
    ### Motivation
    
    #303 adds the flush
    callback to the last `OpSendMsg` instead of adding to the batch message
    container. However, `batchMessageAndSend` will create an `OpSendMsg` and
    add it to the `pendingMessagesQueue_`.
    
    https://github.com/apache/pulsar-client-cpp/blob/7bb94f45b917ed30b5302ac93ffa1f1942fc6313/lib/ProducerImpl.cc#L384-L389
    
    In the code above, `pendingMessagesQueue_` could never be empty and the
    callback will be added again by `opSendMsg->addTrackerCallback`. The 1st
    time it's added in `createOpSendMsg` or `createOpSendMsgs` called by
    `batchMessageAndSend`.
    
    ### Motivation
    
    Add the callback to the last `OpSendMsg only when the batch message
    container is empty.
    
    In `testFlushBatch`, replace the `flush` call with the `flushAsync` call
    and verify the callback is only called once after it's completed.
    
    (cherry picked from commit 37ea769)
    BewareMyPower committed Nov 21, 2023
    Configuration menu
    Copy the full SHA
    4d087b5 View commit details
    Browse the repository at this point in the history
  2. Fix Protobuf symbols not found in libpulsarwithdeps.a when building o…

    …n macOS (#354)
    
    ### Motivation
    
    #290 brings a regression
    that on macOS, Protobuf is always found with CMake Config mode, which
    does not set the `Protobuf_LIBRARIES` variable so that the
    libpulsarwithdeps.a misses the symbols of Protobuf.
    
    ### Modifications
    
    When `LINK_STATIC` is ON, use CMake Module mode to find the Protobuf.
    
    Add `build-static-library.sh` to build libraries with static
    dependencies and verify these libraries in PR workflow. Upload the
    pre-built binaries in the build workflow.
    
    (cherry picked from commit f75b39b)
    BewareMyPower committed Nov 21, 2023
    Configuration menu
    Copy the full SHA
    1c725b4 View commit details
    Browse the repository at this point in the history

Commits on Dec 6, 2023

  1. Gather the macOS binaries when releasing (#355)

    (cherry picked from commit 0bbc155)
    BewareMyPower committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    478d3c3 View commit details
    Browse the repository at this point in the history
  2. Fix HTTP lookup segfault when the redirect host cannot be resolved (#356

    )
    
    ### Motivation
    
    When the host of the redirect URL cannot be resolved, segmentation fault
    will happen:
    https://github.com/apache/pulsar-client-cpp/blob/0bbc15502905d19c630d237b5e102bfb996bb098/lib/CurlWrapper.h#L173-L175
    
    In this case, `curl` will be `nullptr`. Assigning a nullptr to a
    `std::string` is an undefined behavior that might cause segfault.
    
    ### Modifications
    
    Check if `url` is nullptr in `CurlWrapper::get` and before assigning it
    to the `redirectUrl` field. Improve the
    `HTTPLookupService::sendHTTPRequest` method by configuring the
    `maxLookupRedirects` instead of a loop and print more detailed error
    messages.
    
    (cherry picked from commit d209482)
    BewareMyPower committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    6192987 View commit details
    Browse the repository at this point in the history
  3. Install Version.h when installing by CMakeLists (#361)

    ### Motivation
    
    The macOS pre-built binaries don't include the Version.h generated from
    CMake. It's because `cmake --target install` does not install the
    `Version.h`, which is generated by `configure_file`.
    
    ### Modifications
    
    Install the `Version.h`.
    
    (cherry picked from commit 751c807)
    BewareMyPower committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    8595f97 View commit details
    Browse the repository at this point in the history
  4. Fix accessing destroyed objects in the callback of async_wait (#362)

    Fixes #358
    Fixes #359
    
    ### Motivation
    
    `async_wait` is not used correctly in some places. A callback that
    captures the `this` pointer or reference to `this` is passed to
    `async_wait`, if this object is destroyed when the callback is called,
    an invalid memory access will happen.
    
    ### Modifications
    
    Use the following pattern in all `async_wait` calls.
    
    ```c++
    std::weak_ptr<T> weakSelf{shared_from_this()};
    timer_->async_wait([weakSelf](/* ... */) {
        if (auto self = weakSelf.lock()) {
            self->foo();
        }
    });
    ```
    
    (cherry picked from commit 24ab12c)
    BewareMyPower committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    6cb391a View commit details
    Browse the repository at this point in the history
  5. Fix tlsTrustCertsFilePath config is not applied for OAuth2 (#364)

    ### Motivation
    
    #313 has reverted the
    fix of #190, which
    applies the `tlsTrustCertsFilePath` config for OAuth2 authentication.
    
    The macOS pre-built libraries are affected most because the bundled CA
    path is empty.
    
    ### Modification
    
    Apply the `tlsTrustCertsFilePath` for OAuth2.
    
    (cherry picked from commit 27cba3e)
    BewareMyPower committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    afe54da View commit details
    Browse the repository at this point in the history
  6. Release 3.4.2

    BewareMyPower committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    1cb1bf8 View commit details
    Browse the repository at this point in the history
Loading