-
Notifications
You must be signed in to change notification settings - Fork 76
Comparing changes
Open a pull request
base repository: apache/pulsar-client-cpp
base: d040039
head repository: apache/pulsar-client-cpp
compare: 1cb1bf8
- 17 commits
- 54 files changed
- 2 contributors
Commits on Nov 1, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 272e1a1 - Browse repository at this point
Copy the full SHA 272e1a1View commit details
Commits on Nov 7, 2023
-
Fix ProducerBusy or ConsumerBusy error when configuring multiple brok…
…ers per connection (#337) ### Motivation This is a catch up for apache/pulsar#21144 When a producer or consumer reconnects, a random number will be generated as the key suffix in `ConnectionPool` to create or get the `ClientConnection` object from the pool. https://github.com/apache/pulsar-client-cpp/blob/81cc562f7b366fad97e1b80c07ef9334a808390d/lib/ConnectionPool.cc#L75 If a new connection is created with the same producer or consumer name to the broker, the broker will respond with a `ProducerBusy` or `ConsumerBusy` error so that the reconnection will never succeed. ### Modifications - Add an overload of `ConnectionPool::getConnectionAsync` that accepts an integer parameter as the key suffix. If it's not specified, generate the random number as the suffix. In this method, choose the executor by `key suffix % size`. - Generate the random number and save it when creating the `HandlerBase` object. When connecting the owner broker of its topic, pass that index so that the reconnection will always reuse the same `ClientConnection` object. ### Verifying this change `ProducerTest.testReconnectMultiConnectionsPerBroker` is added to protected the change. (cherry picked from commit 6f115e7)
Configuration menu - View commit details
-
Copy full SHA for f337eff - Browse repository at this point
Copy the full SHA f337effView commit details
Commits on Nov 17, 2023
-
Use absolute path to find credential files in unit tests (#340)
Fixes #339 ### Motivation Currently the relative path is used in unit tests to specify the token file (`.test-token.txt`) and the credential directory (`./test-conf`). If we don't run the `pulsar-tests` binary in a subdirectory, it won't be able to read these files. ### Modifications Add the macro `-DTOKEN_PATH="..." -DTEST_CONF_DIR="..."` to specify absolute paths according to the `PROJECT_SOURCE_DIR`, which is defined by CMake as the absolute path of the project directory. ### Verifications ```bash cmake -B build cmake --build build -j8 # Run the test in project directory ./build/tests/pulsar-tests --gtest_filter='AuthPlugin*:*testRSAEncryption:*testEncryptionFailure:*DecryptionFailedMessages' # Run the test in a subdirectory cd build ./tests/pulsar-tests --gtest_filter='AuthPlugin*:*testRSAEncryption:*testEncryptionFailure:*DecryptionFailedMessages' ``` (cherry picked from commit 47fb809)
Configuration menu - View commit details
-
Copy full SHA for 39d39a9 - Browse repository at this point
Copy the full SHA 39d39a9View commit details -
Fix close() returns ResultAlreadyClosed after unsubscribe or close (#338
) Fixes #88 ### Motivation When `close` is called if the consumer has already called `unsubscribe` or `close`, it should not fail. See https://github.com/apache/pulsar/blob/428c18c8d0c3d135189920740192982e11ffb2bf/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ConsumerImpl.java#L1034 ### Modifications Use the same close logic with Java client. Add `testCloseAgainBeforeCloseDone` and `testCloseAfterUnsubscribe` to verify the new behaviors of `Consumer::close`. (cherry picked from commit a8402da)
Configuration menu - View commit details
-
Copy full SHA for 1674a08 - Browse repository at this point
Copy the full SHA 1674a08View commit details -
Fix lazy partitioned producer might send duplicated messages (#342)
Fixes #341 ### Motivation When a lazy partitioned producer sends two messages, the flow is: 1. `start` is called to grab the connection via `grab()`. 2. Generate 0 as the sequence id of the 1st message. 3. Add the 1st message into the queuea. 4. The connection is established, `msgSequenceGenerator_` is reset from 1 to 0. 5. When sending the 2nd message, 0 is also generated as the sequence id. Then two messages have the same sequence id. ### Modifications For lazy partitioned producers, if the internal producer is not started, sending the message in the callback of its future. Add `ChunkDedupTest#testLazyPartitionedProducer` to verify it since only the `tests/chunkdedup/docker-compose.yml` enables the deduplication. (cherry picked from commit bb16f24)
Configuration menu - View commit details
-
Copy full SHA for 8408e65 - Browse repository at this point
Copy the full SHA 8408e65View commit details -
Fix crash when removing connection from the pool (#347)
Fixes #346 ### Motivation #336 changes the key of the `ClientConnection` in `ConnectionPool`, while in `ClientConnection::close`, it still passes the old key (logical address) to `ConnectionPool::remove`, which results in the connection could never be removed and destroyed until being deleted as a stale connection. What's worse, if the key does not exist, the iterator returned by `std::map::find` will still be dereferenced, which might cause crash in some platforms. See https://github.com/apache/pulsar-client-cpp/blob/8d32fd254e294d1fabba73aed70115a434b341ef/lib/ConnectionPool.cc#L122-L123 ### Modifications - Avoid dereferencing the iterator if it's invalid in `ConnectionPool::remove`. - Store the key suffix in `ClientConnection` and pass the correct key to `ConnectionPool::remove` in `ClientConnection::close` - Add `ClientTest.testConnectionClose` to verify `ClientConnection::close` can remove itself from the pool and the connection will be destroyed eventually. (cherry picked from commit 6d47e94)
Configuration menu - View commit details
-
Copy full SHA for aee12e2 - Browse repository at this point
Copy the full SHA aee12e2View commit details -
Log topic lookup result (#351)
(cherry picked from commit c771b12)
Configuration menu - View commit details
-
Copy full SHA for 806698e - Browse repository at this point
Copy the full SHA 806698eView commit details -
Fix bad_weak_ptr when close a ClientConnection during construction (#350
) Fixes #348 Fixes #349 ### Motivation When `close` is called in `ClientConnection`'s constructor, `shared_from_this()` will be called, which results in a `std::bad_weak_ptr` error. This error does not happen before #317 because `shared_from_this()` could only be called when the `producers` or `consumers` field is not empty. ### Modifications Throw `ResultAuthenticationError` when there is anything wrong with authentication in `ClientConnection`'s constructor. Then catch the result in `ConnectionPool::getConnectionAsync` and returned the failed future. In addition, check `authentication_` even for non-TLS URLs. Otherwise, the null authentication will be used to construct `CommandConnect`. Add `testInvalidPlugin` and `testTlsConfigError` to verify the changes. (cherry picked from commit 7bb94f4)
Configuration menu - View commit details
-
Copy full SHA for fa8decd - Browse repository at this point
Copy the full SHA fa8decdView commit details -
Configuration menu - View commit details
-
Copy full SHA for e6ec207 - Browse repository at this point
Copy the full SHA e6ec207View commit details
Commits on Nov 21, 2023
-
Fix the flush callback might be called repeatedly (#353)
Fixes #352 ### Motivation #303 adds the flush callback to the last `OpSendMsg` instead of adding to the batch message container. However, `batchMessageAndSend` will create an `OpSendMsg` and add it to the `pendingMessagesQueue_`. https://github.com/apache/pulsar-client-cpp/blob/7bb94f45b917ed30b5302ac93ffa1f1942fc6313/lib/ProducerImpl.cc#L384-L389 In the code above, `pendingMessagesQueue_` could never be empty and the callback will be added again by `opSendMsg->addTrackerCallback`. The 1st time it's added in `createOpSendMsg` or `createOpSendMsgs` called by `batchMessageAndSend`. ### Motivation Add the callback to the last `OpSendMsg only when the batch message container is empty. In `testFlushBatch`, replace the `flush` call with the `flushAsync` call and verify the callback is only called once after it's completed. (cherry picked from commit 37ea769)
Configuration menu - View commit details
-
Copy full SHA for 4d087b5 - Browse repository at this point
Copy the full SHA 4d087b5View commit details -
Fix Protobuf symbols not found in libpulsarwithdeps.a when building o…
…n macOS (#354) ### Motivation #290 brings a regression that on macOS, Protobuf is always found with CMake Config mode, which does not set the `Protobuf_LIBRARIES` variable so that the libpulsarwithdeps.a misses the symbols of Protobuf. ### Modifications When `LINK_STATIC` is ON, use CMake Module mode to find the Protobuf. Add `build-static-library.sh` to build libraries with static dependencies and verify these libraries in PR workflow. Upload the pre-built binaries in the build workflow. (cherry picked from commit f75b39b)
Configuration menu - View commit details
-
Copy full SHA for 1c725b4 - Browse repository at this point
Copy the full SHA 1c725b4View commit details
Commits on Dec 6, 2023
-
Gather the macOS binaries when releasing (#355)
(cherry picked from commit 0bbc155)
Configuration menu - View commit details
-
Copy full SHA for 478d3c3 - Browse repository at this point
Copy the full SHA 478d3c3View commit details -
Fix HTTP lookup segfault when the redirect host cannot be resolved (#356
) ### Motivation When the host of the redirect URL cannot be resolved, segmentation fault will happen: https://github.com/apache/pulsar-client-cpp/blob/0bbc15502905d19c630d237b5e102bfb996bb098/lib/CurlWrapper.h#L173-L175 In this case, `curl` will be `nullptr`. Assigning a nullptr to a `std::string` is an undefined behavior that might cause segfault. ### Modifications Check if `url` is nullptr in `CurlWrapper::get` and before assigning it to the `redirectUrl` field. Improve the `HTTPLookupService::sendHTTPRequest` method by configuring the `maxLookupRedirects` instead of a loop and print more detailed error messages. (cherry picked from commit d209482)
Configuration menu - View commit details
-
Copy full SHA for 6192987 - Browse repository at this point
Copy the full SHA 6192987View commit details -
Install Version.h when installing by CMakeLists (#361)
### Motivation The macOS pre-built binaries don't include the Version.h generated from CMake. It's because `cmake --target install` does not install the `Version.h`, which is generated by `configure_file`. ### Modifications Install the `Version.h`. (cherry picked from commit 751c807)
Configuration menu - View commit details
-
Copy full SHA for 8595f97 - Browse repository at this point
Copy the full SHA 8595f97View commit details -
Fix accessing destroyed objects in the callback of async_wait (#362)
Fixes #358 Fixes #359 ### Motivation `async_wait` is not used correctly in some places. A callback that captures the `this` pointer or reference to `this` is passed to `async_wait`, if this object is destroyed when the callback is called, an invalid memory access will happen. ### Modifications Use the following pattern in all `async_wait` calls. ```c++ std::weak_ptr<T> weakSelf{shared_from_this()}; timer_->async_wait([weakSelf](/* ... */) { if (auto self = weakSelf.lock()) { self->foo(); } }); ``` (cherry picked from commit 24ab12c)
Configuration menu - View commit details
-
Copy full SHA for 6cb391a - Browse repository at this point
Copy the full SHA 6cb391aView commit details -
Fix tlsTrustCertsFilePath config is not applied for OAuth2 (#364)
### Motivation #313 has reverted the fix of #190, which applies the `tlsTrustCertsFilePath` config for OAuth2 authentication. The macOS pre-built libraries are affected most because the bundled CA path is empty. ### Modification Apply the `tlsTrustCertsFilePath` for OAuth2. (cherry picked from commit 27cba3e)
Configuration menu - View commit details
-
Copy full SHA for afe54da - Browse repository at this point
Copy the full SHA afe54daView commit details -
Configuration menu - View commit details
-
Copy full SHA for 1cb1bf8 - Browse repository at this point
Copy the full SHA 1cb1bf8View commit details
This comparison is taking too long to generate.
Unfortunately it looks like we can’t render this comparison for you right now. It might be too big, or there might be something weird with your repository.
You can try running this command locally to see the comparison on your machine:
git diff d040039...1cb1bf8