Fix buffer overflows on data from external servers by alexey-milovidov · Pull Request #115706 · ClickHouse/ClickHouse

alexey-milovidov · 2026-08-21T02:44:57Z

Places where data that comes from an external server was used without checking it against what was
requested locally:

ReadBufferFromAzureBlobStorage::readBigAt requests n bytes and copies body_stream->Length()
bytes into a buffer that has room for n. The length is the Content-Length of the response, so
an endpoint that returns more data than the requested range overflows the buffer. The S3
implementation caps the copy at n; do the same here.
The handler of a MySQL BIT field copies value.size() bytes into a UInt64 on the stack. The
length comes from the MySQL wire protocol, while a BIT value holds at most 64 bits.
The sequential Azure read path had the same "do not trust the remote length" defect:
initialize derived total_size from BodyStream::Length, so with read_until_position = 100,
offset = 0 and a 64-byte buffer, an endpoint answering the 100-byte ranged request with
Content-Length = 128 delivered 28 bytes from outside the requested range to the caller before
the right-bound check tripped on the following call. The bound is now derived from
read_until_position, which is set locally.
ETag is an optional response header, and Azure::ETag::ToString aborts the process when the
tag is absent - in release builds too, because AZURE_ASSERT_MSG expands to a bare std::abort
under NDEBUG, dropping the message with it. Three call sites converted it unconditionally on a
value taken straight from a response: setMetadataFromResponse on the details of every
Download, AzureObjectStorage::getObjectMetadata on the answer to GetProperties, and both
listing paths (listObjects and the async iterator) on the Etag element of every blob in the
answer to ListBlobs. All of them are reachable by an endpoint that simply omits the header, and
all but the first are outside the read-buffer flow. The guard now lives in one place,
AzureBlobStorage::getETagOrEmpty, and every call site goes through it; no unguarded
Azure::ETag::ToString is left in the tree.

Additionally, readBigAt reported the full requested size even when the endpoint kept returning
short responses and the retry budget was exhausted, so callers could use uninitialized data; it
now returns the number of bytes actually copied, and says so at warning level. It does not throw:
readBigAt is documented to stop at the end of the file and return the number of bytes read, and
the callers that cannot accept a short read (CachedInMemoryReadBufferFromFile, for one) already
turn it into UNEXPECTED_END_OF_FILE themselves.

parseMySQLBitValue also decoded short BIT payloads incorrectly on a big-endian host: the bytes
were written into the object representation of the result and reversed only on a little-endian one,
so a value shorter than 8 bytes stayed left-aligned and "\x01\x02" read back as
0x0102000000000000 instead of 0x0102. It is now assembled by shifting, which needs no byte
order at all.

Beyond the overflows, the Azure read and copy paths trusted the endpoint about where an object
ends and about which generation of it they were talking to:

Where the data ends is decided locally. read_until_position when the caller set one,
otherwise the size the object had when it was listed or headed. That size is not an optional
hint: it is the length of the file for every layer above the buffer -
ReadBufferFromRemoteFSGather lays the next object of a file out right behind it,
AsynchronousBoundedReadBuffer takes it as its own right bound, ReaderExecutor clamps object
reads to it, and both caches address the data inside a file of that length - so the buffer ends
the read there as well, and an endpoint that answers with more cannot push the excess under
offsets that belong to something else. Where neither is available, reaching the Content-Range
total of a response is treated as a question rather than an answer: the download is reopened at
that offset and the end of the file is reported only if the fresh response delivers nothing
either, or if the endpoint refuses the range with 416, which is how a real one answers a range
past the end of a blob. An endpoint that caps an open-ended GET therefore cannot truncate the
file silently.
Which generation is read is pinned. Every request of one logical read - the first Download,
a retry, and a reopen after a premature end of the response - carries If-Match on the
generation selected at read setup, and the ETag of every response is compared with it, so a
blob overwritten in place mid-read raises FILE_CHANGED_DURING_READ instead of handing the
caller bytes stitched together from two objects. The offset of the returned Content-Range is
checked against the requested one, so an endpoint that ignores the range cannot deliver the wrong
bytes under the right offsets.
Which generation is copied and deleted is pinned too. A native copy carries
x-ms-source-if-match, the read-and-write fallback pins its reads, and a DELETE that follows a
copy removes only the generation that was copied. The paths that copy-then-delete - the
after_processing step of ObjectStorageQueue, the MOVE and unlink operations of a
plain_rewritable disk, and backups - name the generation once and use it for every request, and
refuse to act at all when the endpoint will not name one, rather than acting blind. The one
exception is a plain_rewritable move or hard link whose endpoint stops naming generations after
the copy to the destination has been made: the operation is refused before the source is deleted,
and its rollback removes the destination blob by key, because load rebuilds a directory from
the blobs under its key and would otherwise import the uncommitted file on the next start.
The S3 copies have the same shape, as far as S3 allows. copyS3File and copyS3FileRange
carry the ETag of the selected generation as x-amz-copy-source-if-match on the CopyObject
and on every UploadPartCopy, and map a 412 to S3_OBJECT_CHANGED_DURING_READ before any other
route is tried. The S3 backup writer (copyFileFromDisk, copyFile) and reader name the
generation of the source with one HeadObject, check the measured size against the file being
backed up or against the backup metadata (a restore of a backup file replaced by a longer object
is refused rather than restored as its first bytes or copied whole), and use it for the native
copy and for the pinned ReadBufferFromS3 of the fallback; the after_processing
move and delete of ObjectStorageQueue pin their copy and their delete to the generation the
listing reported, pin the read that ingests the file to that same generation whatever
s3_validate_etag_on_read says (as they do on Azure), and a
S3_OBJECT_CHANGED_DURING_READ anywhere in the post-processing aborts the batch like
FILE_CHANGED_DURING_READ does on Azure. s3_validate_etag_on_read = 0 opts the S3 backup
copies out of the pinning only, as it does every other S3 read, and not out of the size check.
The read-and-write fallback of the backup writer reads through the disk (so it honours
read_from_filesystem_cache) for every disk whose object keys are generated and never
rewritten in place, and checks after the buffer is built that the file still names the object
that was measured (a file rewritten in between names a new key, and is refused with
S3_OBJECT_CHANGED_DURING_READ); only a plain / plain_rewritable disk gets the direct read
pinned with If-Match. A plain_rewritable move or hard link names the generation of its
source on Azure and on S3 (pinToTheGenerationThatIsThereNow), copies exactly that one, and
refuses a source generation whose size is not the one the metadata records for the file
(refuseAGenerationOfAnotherSize), so a target can never be recorded with the size of a
generation it did not copy. The S3 delete is pinned too: S3 evaluates If-Match on a
DeleteObject and the ETag element of every object of a DeleteObjects on general purpose
buckets, so the delete that follows a copy (plain_rewritable unlink, move and hard link, the
after_processing move of ObjectStorageQueue), the rollback deletes of a move and a hard link,
and after_processing = 'delete' carry the generation that was copied or ingested and are refused
with FILE_CHANGED_DURING_READ when another writer has replaced it; a batch deletes every object
it can before it reports the replaced one. An S3-compatible endpoint that ignores If-Match on a
DELETE deletes by key, as it did before the header was sent. The rollback of a
plain_rewritable unlink or move whose delete already went through restores the blob it saved
aside without writing over a key that another writer has recreated since, on Azure and on S3
alike (restoreTheSavedBlobWithoutWritingOver): the saved blob is read pinned to its generation
and put back with a create-if-absent write (If-None-Match: *), and a refused restore leaves the
saved blob in the bucket at its temporary key rather than being retried blind.

src/Disks/tests/gtest_azure_read_buffer.cpp drives the reader, the copies, the deletes, the queue
post-processing and the plain_rewritable move operations against a fake Azure HTTP transport that
misbehaves in each of the ways above - a response longer than the requested range, a response
shorter than it, a Content-Range total that under-reports or shrinks, a range refused with 416,
a HEAD without an ETag or with a 404, an ignored range, and an overwrite placed at an exact
point of a request sequence. src/Disks/tests/gtest_azure_object_storage_metadata.cpp covers
getObjectMetadata, listObjects and iterate against an endpoint that omits the header;
src/Processors/tests/gtest_mysql_bit_value.cpp covers the BIT payload at every width from 0 to
8 bytes, with expectations written as numbers so that they hold on a host of either endianness.
src/IO/tests/gtest_writebuffer_s3.cpp drives the pinned S3 copies and deletes, and the
plain_rewritable move and hard-link operations on S3, against an S3 mock that keeps a
generation per object and evaluates x-amz-copy-source-if-match, If-Match on DeleteObject
and the ETag element of DeleteObjects; 05211_s3_backup_native_copy_etag
backs a table on an S3 disk up to S3 with a failpoint that makes the pinned copy carry a generation
the source never had, and 05210_s3_backup_plain_read_etag / 05194_s3_backup_archive_reopen_etag
do the same for the reads of a backup; 05212_s3_backup_restore_replaced_file replaces a data file
inside an S3 backup with a longer object and asserts that both the buffered restore and the native
copy of the restore refuse it; 05213_s3_backup_archive_without_etag makes HeadObject report no
ETag (failpoint s3_head_omit_etag) and asserts that the restore of an archive is refused up front,
because its reopens could not be pinned to one generation.

Changelog category (leave one):

Critical Bug Fix (crash, data loss, RBAC)

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

Fix buffer overflows and a server abort on malformed responses from external servers: an Azure Blob Storage endpoint that returns more data than was requested or omits the ETag header, and a MySQL server that returns an oversized BIT value. Azure reads, copies and deletes, and S3 copies made by backups and by S3Queue post-processing, are now pinned to the object generation they selected, so an object overwritten in place is reported instead of silently mixing two generations. Also fix the decoding of a MySQL BIT value shorter than 8 bytes on big-endian hosts.

Workflow [PR]
Sync PR [sync-upstream/pr/115706]

clickhouse-gh · 2026-08-21T02:46:01Z

Workflow [PR], commit [3584315]

AI Review

Summary

This PR hardens Azure/S3 reads, copies, deletes, and backup paths against malformed responses and in-place overwrites, and it closes most of the earlier review findings. I still see two live correctness issues in the current head and one author-accepted residual data-loss window, so the end-to-end generation-pinning contract is not complete yet.

Findings

❌ Blockers

[src/Backups/BackupIO_S3.cpp:488-547, 588-611, 711-857] The new "s3_validate_etag_on_read = 0 disables pinning but keeps size validation" contract is still false after the validating HeadObject. checkBackupFile() and the writer-side HeadObject only validate object A at one instant; with pinning off they then drop the ETag, so a same-key replacement to larger B between that HeadObject and the later GET / CopyObject is still restored or copied under the old metadata size. Buffered restore reads the first recorded bytes of B, while whole-object native copy copies all of B because CopyObject ignores the caller's size argument. Suggested fix: if backups must keep the "size check still on" contract when pinning is disabled, keep these read/copy paths pinned whenever the declared size is correctness-critical, or narrow the contract/tests/comments to state that the size check only catches replacements already visible to the HeadObject.
[dismissed by author -- https://github.com/Fix buffer overflows on data from external servers #115706#discussion_r3954284925] [src/Disks/DiskObjectStorage/MetadataStorages/PlainRewritable/MetadataStorageFromPlainRewritableObjectStorageOperations.cpp:205-217, 666-695, 804-914] plain_rewritable destination naming still relies on a separate HEAD after copyObject. If another writer replaces remote_path_to in that gap, nameTheGenerationThatWasJustWritten() binds the operation to the foreign generation instead of the one this transaction copied. undo() can still delete another writer's blob, and the committed metadata can describe path_to with the source file's old size even though the live object is already different. Suggested fix: only claim "the move/hard link deletes exactly what it copied" if the copy primitive can return the generation it created, or switch these writes to a flow that can prove the destination is still the blob just written.

⚠️ Majors

[src/Disks/IO/ReadBufferFromAzureBlobStorage.h:120-128, src/Disks/IO/ReadBufferFromAzureBlobStorage.cpp:610-615] The new known_object_size contract is still inconsistent with tryGetFileSize(). The buffer now treats known_object_size as the file length it will serve, but its base WithFileSize cache is never seeded, so wrappers like CachedInMemoryReadBufferFromFile force a fresh GetProperties() and can learn a different length from the one nextImpl() / readBigAt() will actually honor. Suggested fix: initialize ReadBufferFromFileBase::file_size from known_object_size (or return known_object_size from tryGetFileSize() before the remote metadata probe) and add a focused page-cache regression.

Final Verdict

Status: ⚠️ Request changes

Minimum required actions:

Fix or explicitly narrow the S3 backup "size check without pinning" contract.
Make ReadBufferFromAzureBlobStorage::getFileSize() agree with the known_object_size the reader actually serves.
Keep the plain_rewritable copyObject -> HEAD residual window in scope instead of treating it as closed.

clickhouse-gh · 2026-08-21T04:58:20Z

Build profile diff (arm_release)

Comparing bdbae4e82 with master a5efd3a38 (stripped binary size, per-symbol sizes and ThinLTO time; object sizes against the warmup build of 7df5d31fe; compile times per translation unit against the most recent warmup build that recompiled it).

✅ No significant changes.

Binary sizes

programs/clickhouse-stripped: smaller than the master baseline by the known offset between the two builds, so the difference is not shown. A delta that differs from the offset by more than 50% of it is shown, in either direction.

The official master build is compiled with -g and a pull request build is not, and XRay counts debug instructions towards its instrumentation threshold, so master instruments thousands of functions more and its binary is ~0.4% larger no matter what the pull request does.

Object file sizes

50 object files changed (+359.06 KiB total), 0 added.

Object file	Master	PR	Δ
`src/CMakeFiles/dbms.dir/Disks/DiskObjectStorage/MetadataStorages/PlainRewritable/MetadataStorageFro…`	351.87 KiB	417.60 KiB	+65.73 KiB (+18.68%)
`src/CMakeFiles/dbms.dir/Backups/BackupIO_S3.cpp.o`	321.84 KiB	385.81 KiB	+63.97 KiB (+19.88%)
`src/CMakeFiles/dbms.dir/Disks/IO/ReadBufferFromAzureBlobStorage.cpp.o`	237.50 KiB	291.45 KiB	+53.95 KiB (+22.72%)
`src/CMakeFiles/dbms.dir/Backups/BackupIO_AzureBlobStorage.cpp.o`	279.51 KiB	321.05 KiB	+41.54 KiB (+14.86%)
`src/CMakeFiles/clickhouse_common_io.dir/Disks/DiskObjectStorage/ObjectStorages/AzureBlobStorage/Azu…`	410.77 KiB	438.62 KiB	+27.85 KiB (+6.78%)
`src/CMakeFiles/dbms.dir/Disks/DiskObjectStorage/ObjectStorages/AzureBlobStorage/AzureObjectStorage.…`	410.77 KiB	438.62 KiB	+27.85 KiB (+6.78%)
`src/CMakeFiles/dbms.dir/Storages/ObjectStorageQueue/ObjectStorageQueuePostProcessor.cpp.o`	349.74 KiB	367.03 KiB	+17.29 KiB (+4.94%)

716 more object files are built by the master warmup baseline only (it builds every object-file target, a pull request build only clickhouse-bundle) and not compared.

Compile time of recompiled translation units

363 translation units recompiled, 2696 s compile time in total, 363 of them have a recent master baseline.

Median compile-time ratio to the baselines is ×1.09 (machine-speed difference or a change affecting every TU); per-TU deltas below are relative to that ratio.
The matched translation units cost +221.4 s (+9%) in total before that adjustment.

Job report

… tests Address review: `ReadBufferFromAzureBlobStorage::readBigAt` returned the requested size even when the endpoint kept returning short responses and the retry budget was exhausted, so callers could use uninitialized data. Return the number of bytes actually copied instead. Extract `copyFromAzureBodyStream` and `parseMySQLBitValue` and cover both fixes with unit tests: an endpoint returning more data than requested, a short response, and an oversized MySQL `BIT` payload.

…-buffer-overflows

`ReadBufferFromAzureBlobStorage::initialize` derived `total_size` from `BodyStream::Length`, which is the `Content-Length` chosen by the remote endpoint. With `read_until_position = 100`, `offset = 0` and a 64-byte reading buffer, an endpoint that answers the 100-byte ranged request with `Content-Length = 128` made `nextImpl` hand out 64 bytes twice and only trip the right-bound check on the following call, so 28 bytes from outside the requested range had already escaped into the caller. With a reading buffer larger than the range, all 28 escape in a single call. Derive the bound of the current download from `read_until_position`, which is set locally by the caller, instead of trusting the length of the response. The size of the blob from the same response is not used for this: it comes from the same untrusted place, and an endpoint that answers a ranged request with `200 OK` would make it truncate a legitimate read. Fold the tests of `copyFromAzureBodyStream` into a new `src/Disks/tests/gtest_azure_read_buffer.cpp` and add regressions that drive `nextImpl` through `setReadUntilPosition` against a fake Azure HTTP transport whose response is longer than the requested range. Without the change they fail with `Attempt to read beyond right offset (128 > 99)`.

`ETag` is an optional response header, and `Azure::ETag::ToString` calls `AZURE_ASSERT_MSG`, which aborts the process - in release builds too, because that macro is not compiled out with `NDEBUG`. `setMetadataFromResponse` called it unconditionally on the response of every `Download`, both on the sequential path and in `readBigAt`, so an endpoint that answers without an `ETag` took the server down. Found while writing the regression for the ranged-read bound: a mocked response without that header aborted the unit test binary with `SIGABRT`.

SmitaRKulkarni · 2026-08-24T10:00:13Z

+        /// The endpoint kept returning short responses. Report how much was actually copied:
+        /// the caller must not treat the tail of its buffer as initialized.
+        LOG_DEBUG(log, "AzureBlobStorage readBigAt for file {} got only {} bytes out of {} requested after {} attempts",
+            path, initial_n - n, initial_n, max_single_download_retries);
+    }


Maybe we should throw error (like UNEXPECTED_END_OF_FILE) here or increase level of log to warning. wdyt ?

🕵 I took the second half of your suggestion and raised it to LOG_WARNING (it went in with 8329b8c, which touches the same file - the commit message only mentions the ETag work). Exhausting the retry budget against an endpoint that keeps answering short is not a normal condition and should be visible without turning up the log level of the whole server.

I did not make it throw, and I would rather leave that to you to overrule if you disagree. readBigAt is documented in SeekableReadBuffer as "Stops either after n bytes, or at end of file, or on exception. Returns number of bytes read", so a short return is the documented end-of-file signal, and the callers that cannot accept one already convert it into UNEXPECTED_END_OF_FILE themselves - see CachedInMemoryReadBufferFromFile, which compares the result against the requested size and throws exactly that. Throwing inside readBigAt would duplicate that error and take the choice away from callers that legitimately read up to the end of the file. What this pull request fixes is the previous behaviour of returning initial_n unconditionally, which told the caller that bytes it never received were initialized.

SmitaRKulkarni

Rest all LGTM

`parseMySQLBitValue` copied the payload into the object representation of the result and reversed it only on a little-endian host. On a big-endian host a value shorter than 8 bytes therefore stayed left-aligned: `"\x01\x02"` read back as `0x0102000000000000` instead of `0x0102`, a wrong result for every `MYSQL_TYPE_BIT` column on the big-endian targets we build for. Assemble the integer by shifting instead, which needs no byte order at all. The unit test now covers every width from 0 to 8 bytes; its expectations are numbers rather than byte patterns, so they hold on a host of either endianness.

…d` path `Azure::ETag::ToString` aborts the process when the tag is absent, in release builds too, because `AZURE_ASSERT_MSG` expands to `std::abort` under `NDEBUG`. Guarding it in `ReadBufferFromAzureBlobStorage::setMetadataFromResponse` left three call sites that take the same untrusted value straight from a response: `AzureObjectStorage::getObjectMetadata` on the answer to `GetProperties`, and the two listing paths (`listObjects` and the iterator) on the `Etag` element of every blob in the answer to `ListBlobs`. All three are reachable outside the read-buffer flow, so an Azure-compatible endpoint that omits the optional tag could still take the server down. Put the guard in one place, `AzureBlobStorage::getETagOrEmpty`, and route all four call sites through it. There is no unguarded `Azure::ETag::ToString` left in the tree. `src/Disks/tests/gtest_azure_object_storage_metadata.cpp` drives `getObjectMetadata`, `listObjects` and `iterate` against a fake Azure HTTP transport that answers without an `ETag`.

The transport policy of the Azure SDK buffers the body of every response it does not stream, by calling `ReadToEnd` on the body stream unconditionally. A response constructed without one therefore dereferences a null pointer, which is what the answer to the HEAD request of `GetProperties` did: the test crashed before it ever reached the `ETag` conversion it was meant to cover. Serve every response, including the empty body of the HEAD, from a body stream that owns its data.

The AI review on #115706 found two remaining gaps in the Azure hardening: - `getTotalSizeOfCurrentDownload` shrank the bound of a right-bounded read to the remote `Content-Length`, so a short `206` response moved the end of the file before the locally requested `read_until_position`, breaking the `supportsRightBoundedReads` promise. The local bound is now authoritative: on a premature end of the response, `nextImpl` reopens the download at the current offset, and throws `UNEXPECTED_END_OF_FILE` once the retry budget is exhausted, instead of silently reporting a truncated file. - `copyFromAzureBodyStream` capped the copy by the reported `Content-Length`, so an endpoint under-reporting it truncated a `readBigAt` even when the bytes were already in the response body. The size of the destination is the only bound needed: `ReadToCount` stops at the actual end of the body. The unit tests now drive the reader through a range-aware fake transport: `ShortRangeResponse` asserts that a bounded read reassembles the full range through reopens, the new `TruncatedBlob` asserts the exception when the blob really ends before the bound, and the new `LengthSmallerThanData` pins down that a lying short `Content-Length` does not truncate the copy. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…-buffer-overflows

alexey-milovidov · 2026-08-27T03:35:17Z

🕵 @groeneai, investigate the failure: https://s3.amazonaws.com/clickhouse-test-reports/json.html?PR=115706&sha=439fb44121b960987a8ab86ba3fa2d9e46e1d1f1&name_0=PR&name_1=Fast%20test%20%28arm_darwin%29 — 03312_squashing_with_low_card_mem_usage failed in Fast test (arm_darwin), and it is unrelated to this PR: the same test failed on 2026-08-25, 2026-08-22, 2026-08-19, 2026-08-18, 2026-08-05, and 2026-08-01 across unrelated PRs (#116225, #111985, #111694, #112805, #107075, #115297, and others), so it looks like a recurring flaky memory-usage threshold test. Please provide a fix in a separate PR. If the fix is already in progress, link it here.

groeneai · 2026-08-27T04:15:58Z

The fix is already open: #113605 (CI: apply the runner free-space guard on macOS runners too), open since 2026-08-06 with a review requested from @ leshikus and no review yet.

One correction to the diagnosis: this is disk, not memory. The only failing leaf in the report you linked is

Code: 243. DB::Exception: Cannot reserve 317.74 MiB, not enough space. (NOT_ENOUGH_SPACE) (version 26.9.1.1)

on runner ip-172-31-9-239. All 13 failures of this test on Fast test (arm_darwin) in the last 30 days are that same code 243, with the requested reservation constant at 317.51-317.82 MiB and zero MEMORY_LIMIT_EXCEEDED rows, so the test's own footprint is not the variable. The host is: in the same window that check has 31 code-243 rows spread over 11 different tests, 18 pull requests and 6 runners, 0 on master, and 7 of them failed to reserve only 1.00-2.00 MiB (02476_analyzer_identifier_hints, 03640_skip_indexes_with_or, 04043_text_index_in_with_preprocessor and others). A volume that cannot hand out 1 MiB has no large test to blame. 03312_squashing_with_low_card_mem_usage is the most frequent victim only because it asks for the most space, so shrinking or tagging it would move the failures to the next-largest test instead of stopping them.

What #113605 changes: Runner.check_post_run() returns early on macOS before it ever runs df (ci/praktika/infrastructure/runner/runner-init.py:430), so those long-lived hosts have never enforced the free-space threshold Linux runners do. It runs the check on macOS as well, at job start in addition to post-run, and pins df -k so the block count means the same on BSD and GNU df.

…null `memcpy` in the test `supportsReadAt` advertises positioned reads on a buffer on which nothing has been read yet, but `readBigAt` dereferenced `blob_client`, which is only created by the sequential path and by `tryGetFileSize`. It is also called concurrently on the same buffer, so it must not create the shared member either: it now uses a call-local `BlobClient` when the shared one does not exist. `AzureReadUntilPosition.TruncatedBlob` failed under UBSan because the fake transport served an empty response body, and `memcpy` was then called with a null source pointer.

…-buffer-overflows

CLAassistant · 2026-09-12T13:30:32Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ alexey-milovidov
❌ Alexey Milovidov

Alexey Milovidov seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

…-buffer-overflows

…_CHANGED_DURING_READ` in the queue Two gaps in the "read one generation or fail" contract, both on the S3 side: - `BackupReaderS3::readFile` delegated to `readFilePinnedToGeneration` with an empty token, so every non-archive read of an unversioned backup (`BackupImpl::readFileImpl`, the buffered fallback of `copyFileToDisk`) carried no `If-Match`. A key rewritten between two requests of the same buffer (a retry, or the reopen after a `seek`) was restored as a splice of two generations without any error. Now an ordinary read names the generation with one `HeadObject`, as the Azure reader does, and every `GET` is pinned to it. `s3_validate_etag_on_read = 0` opts plain reads out, as for every other S3 read; an archive session that already names a generation stays pinned regardless. Test: `05210_s3_backup_plain_read_etag` (the `s3_read_inject_etag_mismatch` failpoint makes the restore of a plain backup refuse with `S3_OBJECT_CHANGED_DURING_READ`; with the setting off, the old unpinned behaviour is observed, which is also what the test looked like before this change). - `ObjectStorageQueueSource::prepareCommitRequests` released the path without charging the retry budget only for Azure's `FILE_CHANGED_DURING_READ`. `ReadBufferFromS3` reports the very same race as `S3_OBJECT_CHANGED_DURING_READ`, so an S3 object rewritten after the listing burnt the budget and could end up with a terminal `failed` node that hides every later generation at that key. Both codes are the rewritten-generation case now. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…amed `plain_rewritable` rollback blobs Four findings of the AI review of 2026-09-13: - `copyS3File` and `copyS3FileRange` take the `ETag` of the generation of the source the caller selected and carry it as `x-amz-copy-source-if-match` on the `CopyObject` and on every `UploadPartCopy`; a `412` is mapped to `S3_OBJECT_CHANGED_DURING_READ` before any other route (multipart, read-and-write fallback) is tried. `S3ObjectStorage::copyObject` passes the generation the caller carries. - `BackupWriterS3::copyFileFromDisk` and `copyFile` name the generation of the source with one `HeadObject`, verify the measured size against it, and thread it through the native copy and through a pinned `ReadBufferFromS3` in the fallback, the same shape as the Azure writer. `BackupReaderS3::copyToDiskImpl` pins its native copy to the generation an ordinary read is pinned to. `s3_validate_etag_on_read = 0` opts the copies out, as for every other S3 read. - `ObjectStorageQueuePostProcessor::moveS3Objects` measures the source with a `HeadObject` that has to describe the ingested generation and pins the copy to it; `ChangedGeneration` treats `S3_OBJECT_CHANGED_DURING_READ` as batch-aborting like `FILE_CHANGED_DURING_READ`. The S3 delete stays by key, as S3 has no conditional `DeleteObject` on general purpose buckets. - The `plain_rewritable` copy and move operations remove the destination blob by key in `undo` when the endpoint names no generation for it, instead of leaving it under the key of the file, where `load` would import it as the file on the next start. Tests: `CopyS3FileRoutingTest.{WholeCopyPinnedToTheCurrentGenerationGoesThrough, WholeCopyPinnedToAReplacedGenerationIsRefused,RangedCopyPinnedToAReplacedGenerationIsRefused, UnpinnedCopyCarriesNoPrecondition}` against the S3 mock, which now has per-object generations and evaluates the precondition; `AzurePlainRewritableMove.ADestinationWhoseGenerationCannotBeNamedIsRefusedAndRemoved` and the extended `AzurePlainRewritableHardLink.ADestinationWhoseGenerationCannotBeNamedIsRefused`; `05211_s3_backup_native_copy_etag` with the new `s3_copy_inject_etag_mismatch` failpoint. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…enerationIsRefused` `Build (arm_tidy)` failed with `bugprone-unused-local-non-trivial-variable` on the `source` string of the new `CopyS3FileRoutingTest`; the test only needs the object put, not its content. https://s3.amazonaws.com/clickhouse-test-reports/praktika.html?PR=115706&sha=62545674d94be6668f696c05fa5736704c07f44a&name_0=PR&name_1=Build%20(arm_tidy) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…-buffer-overflows Conflict in `MySQLSource.cpp`: master added a length check to the `BIT` value decoding (8807675, with test 04870) in the place this branch had already replaced with `parseMySQLBitValue`. Kept the helper (it also fixes the big-endian decoding) and made it throw `INCORRECT_DATA` with master's message, so that master's test holds. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…post-processing retries fast on `S3_OBJECT_CHANGED_DURING_READ` Two findings of the AI review on `62545674d94b`. `BackupReaderS3::readFilePinnedToGeneration` received `expected_file_size` and ignored it, so an unversioned backup key replaced by a longer object before the read was still restored: the buffered restore copies exactly the number of bytes the backup metadata records and would restore the first bytes of the replacement, and the native S3-to-S3 copy of `copyToDiskImpl` would `CopyObject` the whole replacement. `checkBackupFile` now takes one `HeadObject` per read that needs one, compares the size with the backup metadata, checks a generation named by the caller, and names the generation a plain read is pinned to, so that what is measured is what is read; a mismatch throws `S3_OBJECT_CHANGED_DURING_READ` before a byte is read or copied. The native copy of `copyToDiskImpl` and its read-and-write fallback go through the same check with the size of the file. The size check applies to unpinned reads too (`s3_validate_etag_on_read = 0` opts out of the generation, not of the size). The Azure reader already worked this way. `ObjectStorageQueuePostProcessor::doWithRetries` rethrew `FILE_CHANGED_DURING_READ` without retrying but retried `S3_OBJECT_CHANGED_DURING_READ`, so an S3 move or delete that saw the generation change first could end with another error from a later retry, and in non-`EXCLUSIVE` mode that other error is only logged while the path is marked processed. Both codes are rethrown at once now. Test `05212_s3_backup_restore_replaced_file` backs a table on the local disk and one on an S3 disk up to S3, replaces `n.bin` of each inside the backup with a longer object, and asserts that the buffered restore and the native copy of the restore are refused, pinned or not, while an intact backup restores through both paths. Verified against a build without the check: all four restores went through silently. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…geTree settings The test overwrites `all_1_1_0/n.bin` inside the backup, so the part has to be stored as separate files. The randomized MergeTree setting `min_bytes_for_full_part_storage` made the part packed into one file instead, the replacement went to a key that no restore reads, and all four restores went through silently. Pin `min_bytes_for_full_part_storage = 0` in both tables. CI: https://s3.amazonaws.com/clickhouse-test-reports/praktika.html?PR=115706&sha=584623865b8691996884ac74e2142ec7c32a4356&name_0=PR&name_1=Stateless%20tests%20(amd_debug,%20parallel) PR: #115706 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…an archive whose generation cannot be named Two fail-closed gaps found by the review: - `ObjectStorageQueue` on S3 with `after_processing = 'move'` recorded the listing `ETag` as the ingested generation while the read itself was pinned to it only when `s3_validate_etag_on_read` was on. With the setting off, an object listed as generation `A` and overwritten to `B` before the `GET` was read as `B`, the move pinned to `A` refused it with `S3_OBJECT_CHANGED_DURING_READ`, the Keeper commit was skipped, and `B` was ingested a second time on the next pass. `afterProcessingNeedsIngestedGeneration` now includes the S3 `MOVE`, so the read is pinned through `require_read_pinned_to_generation` independently of the setting, and a file whose listing carries no `ETag` is failed instead of read, as on Azure. The S3 move in `ObjectStorageQueuePostProcessor` refuses an untagged source instead of moving whatever generation exists. An S3 `DELETE` addresses the object by key and is unchanged. - `BackupImpl::openArchive` pins every reopen of the archive to the token of `getFileGeneration`, but the S3 and Azure readers returned an empty token when the endpoint reported no `ETag`, which pinned nothing: a same-size replacement of the archive between two handles was read as two archives. Both readers now throw (`S3_ERROR`, `AZURE_BLOB_STORAGE_ERROR`) up front. Tests: `AzureIngestedGeneration.AzureMoveAndDeleteNeedIt` pins the policy per storage type and action; `AzureBackupReader.ArchiveGenerationCannotBeNamedWithoutETag`; `05213_s3_backup_archive_without_etag` uses the new failpoint `s3_head_omit_etag` to make `HeadObject` report no `ETag` and asserts that the restore of a `.tar.gz` backup is refused with either value of `s3_validate_etag_on_read`. PR: #115706 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…sure the source whatever the settings say The read-and-write fallback of `BackupWriterS3::copyFileFromDisk` read the object directly with `ReadBufferFromS3` for every disk, so a backup with `allow_s3_native_copy = 0` no longer honoured `read_from_filesystem_cache`: nothing was served from the filesystem cache of the disk and nothing was counted in `CachedReadBufferReadFromCacheBytes` / `CachedReadBufferReadFromSourceBytes`. This made `test_backup_restore_s3/test.py::test_backup_with_fs_cache` fail in CI: https://s3.amazonaws.com/clickhouse-test-reports/praktika.html?PR=115706&sha=debef90cdcae498acaf2229578e934ec0b470709&name_0=PR&name_1=Integration%20tests%20(amd_asan_ubsan%2C%20db%20disk%2C%20old%20analyzer%2C%204%2F8) (#115706) The fallback now reads through the disk (`readFile` / `readEncryptedFile` with the backup's `read_settings`) for every disk whose object keys are generated: such a disk never writes an object in place, so the key itself pins the generation and the read honours the cache settings. Only a `plain` / `plain_rewritable` disk, whose objects are named by path and rewritten in place, keeps the direct read pinned with `If-Match`. Also, per the AI review: the `HeadObject` that measures the source against the size the backup records is now taken unconditionally in `copyFileFromDisk` and `copyFile`; `s3_validate_etag_on_read = 0` opts the copy out of the `ETag` pinning only, the same way it does for the reads of `BackupReaderS3`. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…ize is not the recorded one The move and hard-link operations record the target with the `FileRemoteInfo` snapshot of the source, but copy whatever generation is at the source key. A blob written over the file out of band with another size would be copied whole and recorded with the old size, so every later read of the target would stop short of its end. `pinToTheGenerationThatIsThereNow` already names the generation and its size on Azure. The new `refuseAGenerationOfAnotherSize` compares that size against the recorded one and throws `FILE_CHANGED_DURING_READ` before anything is written; the move pins its source before it touches the target, and the hard link copies the pinned generation instead of the bare key. An object storage that does not pin names no generation and is not measured. Three gtests in `gtest_azure_read_buffer.cpp` cover the matching size, the mismatch (nothing copied, nothing deleted) and the unnamed generation. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…-buffer-overflows

…-buffer-overflows Master merged #119054, an independent fix of the same `readBigAt` overflow in `ReadBufferFromAzureBlobStorage`. Resolution: - `ReadBufferFromAzureBlobStorage.cpp`: take master's thread-safe lazy `getBlobClient` (created under `call_once`) everywhere, which supersedes this branch's call-local blob client in `readBigAt`; keep this branch's `checkReturnedRange` / `checkReturnedETag`, `copyFromAzureBodyStream`, the null body stream guard and the short-read warning; keep master's `chassert(bytes_copied <= n)`. - `gtest_azure_read_buffer.cpp` (add/add): keep this branch's file and port master's two tests `AzureReadBigAt.DoesNotTrustResponseLength` and `AzureReadBigAt.ReturnsAccumulatedCountOnTruncatedResponse` onto its `makeFreshBuffer` harness. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…_buffer.cpp` by the merge Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

… backup fallback read of a rewritten file Two blockers of the AI review on the previous head: - `pinToTheGenerationThatIsThereNow` named the generation of the source blob for Azure only, so on `s3_plain_rewritable` a move or hard link still copied the blob by bare key and recorded the target with the stale size of the `FileRemoteInfo` snapshot when the blob had been written over out of band. It now names the generation on S3 as well (one `HeadObject`): `S3ObjectStorage::copyObject` carries it as `x-amz-copy-source-if-match` on the `CopyObject` and on every `UploadPartCopy`, and `refuseAGenerationOfAnotherSize` measures it against the recorded size. An S3 endpoint that reports no `ETag` is refused with `S3_ERROR`, the way an Azure one is with `AZURE_BLOB_STORAGE_ERROR`. The S3 delete stays by key, as documented in the header. - `BackupWriterS3::copyFileFromDisk`: the read-and-write fallback reads through the disk for a disk with generated object keys (it has to, to honour the filesystem cache settings), but the disk read resolves the objects of `src_path` from the metadata when the buffer is built, not when `src_key` was measured and named, so a file rewritten in between would be read under another key. The metadata is now consulted again after the buffer is built, and a file that no longer names `src_key` is refused with `S3_OBJECT_CHANGED_DURING_READ`. A rewrite after the buffer is built cannot reach a buffer that already holds its objects. Tests: `S3PlainRewritablePinningTest.{NamesTheGenerationAndItsSize,AGenerationOfAnotherSizeIsRefused, ACopyOfAReplacedGenerationIsRefused,AnObjectWithoutAnETagCannotBePinned}` in `gtest_writebuffer_s3.cpp`, over an `S3ObjectStorage` on the mock endpoint that keeps a generation per key. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…created key `restoreTheSavedBlobWithoutWritingOver` was fail-closed on Azure only: for S3 it fell back to `copyObject` by key, so a rollback of an `unlink` or a `move` whose delete had already gone through wrote over a blob that another writer recreated at the same key in between - a generation this transaction never saw. S3 has the same pieces Azure does: the restore now reads the saved blob pinned to its generation (`If-Match`) and puts it back with a create-if-absent write (`If-None-Match: *`) on S3 too, and a refused write leaves the saved blob in the bucket at its temporary key rather than retrying blind. The object storages that pin are named by one predicate, `pinsGenerations`, shared with `pinToTheGenerationThatIsThereNow`. Tests: `S3PlainRewritableRollbackTest.{ARecreatedKeyIsNotRestoredOver, AFreeKeyIsRestored,ASavedBlobWithoutAGenerationIsNotRestored}`. The mock S3 client now serves a `GetObject` body over a `Poco::Net::HTTPBasicStreamBuf`, which is what `ReadBufferFromIStream` reads from, honours `If-Match` on `GetObject` and `If-None-Match: *` on `PutObject`, and records the `If-None-Match` of every put. The recreated-key test fails on a control binary with the S3 half reverted. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…-buffer-overflows

…able` and in `ObjectStorageQueue` The `plain_rewritable` `unlink`, `move` and hard-link operations, and the `after_processing` step of `ObjectStorageQueue`, copy one generation of an object and then delete it. On Azure the delete already carried the generation as `If-Match`; on S3 it was still a `DeleteObject` by key, so an object that another writer replaced between the pinned copy and the delete was deleted without the newer generation ever having been copied or ingested. The same held for the rollback deletes of a move and a hard link, which `nameTheGenerationThatWasJustWritten` "named" on S3 with a bare key. S3 evaluates `If-Match` on `DeleteObject` and the `ETag` element of every object of a `DeleteObjects` on general purpose buckets (`412 Precondition Failed` when the generation is not the one named), so the S3 side now has the Azure shape: - `deleteFileFromS3` takes the `ETag` to match and sends it as `If-Match`; a `412` (or the `409` of a concurrent write) is reported as `FILE_CHANGED_DURING_READ`, which `if_exists` does not swallow. `deleteFilesFromS3` takes a parallel list of `ETag`s, sets the `ETag` element of every pinned object, deletes every object it can before it reports the replaced one, and leaves the replaced one out of `successful_keys`; the one-by-one fallback does the same. - `S3ObjectStorage::removeObjectImpl` / `removeObjectsImpl` forward `StoredObject::etag`. - `nameTheGenerationThatWasJustWritten` names the generation on S3 as it does on Azure, so the rollback delete of a move and of a hard link is pinned, and a destination whose generation cannot be named is refused with `S3_ERROR`. - `ObjectStorageQueue` with `after_processing = 'delete'` refuses an untagged S3 object the way it refuses an untagged Azure blob; the move path was already handing the ingested generation to `removeObjectIfExists`. An S3-compatible endpoint that ignores `If-Match` on a `DELETE` deletes by key, as it did before the header was sent. Tests: the S3 mock evaluates `If-Match` on `DeleteObject` and the `ETag` element of `DeleteObjects`, and records both. `S3PlainRewritableDeleteTest` covers the pinned, refused, batch and unpinned deletes; `S3PlainRewritableOperationTest` drives the production move and hard-link operations: a move deletes the generation it copied, a move whose source is replaced before the delete refuses it and rolls back pinned, a hard-link rollback does not delete a generation another writer put at the destination, and a destination whose generation cannot be named is refused. Also fixes the `arm_tidy` build: `google-runtime-int` on a `long long` cast in the mock `GetObject`. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…_processing = 'delete'` too `afterProcessingNeedsIngestedGeneration` returned `true` on S3 only for `MOVE`, so with `s3_validate_etag_on_read = 0` the read that ingests a file whose post-processing is a `DELETE` was not pinned to the generation the listing reported, while the `DeleteObjects` that followed was (since f7a3e37). A generation `B` written over the listed `A` between the listing and the read could then be ingested as `A`, refused by the pinned delete as `A`, and ingested again on the next pass. Now the predicate is the same for S3 and Azure: a `MOVE` and a `DELETE` both need the ingested generation, the read is pinned to it whatever `s3_validate_etag_on_read` says, and an untagged object is refused before it is read for both actions. Addresses the review thread on `ObjectStorageQueueSource.cpp`. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

clickhouse-gh · 2026-09-14T09:44:13Z

+    /// being validated, so it is a hard end of the data of an unbounded read: every layer above
+    /// this buffer already treats it as the length of the file - `ReadBufferFromRemoteFSGather`
+    /// places the following object right behind it, `ReaderExecutor` clamps object reads to it, the
+    /// caches address the data within a file of that length, and `AsynchronousBoundedReadBuffer`


known_object_size still doesn't reach getFileSize(). ReadBufferFromAzureBlobStorage is constructed with ReadBufferFromFileBase() and tryGetFileSize() later does a fresh GetProperties() instead of returning this pre-read size first, so outer wrappers can learn a different EOF than nextImpl() / readBigAt().

The concrete breakage is the page-cache path: CachedInMemoryReadBufferFromFile snapshots in_->getFileSize() in its ctor. If a blob was listed as 100 bytes and then overwritten to 200 before an unpinned read (use_page_cache_for_object_storage = 1, s3_validate_etag_on_read = 0), the cache wrapper takes file_size = 200 while readBigAt() still clamps at known_object_size = 100, so the first cold miss throws UNEXPECTED_END_OF_FILE instead of serving the listed 100-byte file. If the replacement is shorter, the cache truncates to that newer HEAD size for the same reason. Please seed file_size from known_object_size (like ReadBufferFromS3 seeds it from file_size_) and add a focused page-cache regression where LIST says 100 but the later HEAD says 200.

clickhouse-gh · 2026-09-14T09:47:47Z

+            /// `checkBackupFile`): the native copy carries it as `x-amz-copy-source-if-match`, and
+            /// every `GET` of the fallback as `If-Match`. A versioned URI is pinned by its version and
+            /// gets no token, and `s3_validate_etag_on_read = 0` opts the copy out of the pinning, as
+            /// it does every other plain read of the backup, but not out of the size check.


s3_validate_etag_on_read = 0 still opts these backup paths out of more than generation pinning. checkBackupFile() does validate the object seen by this HeadObject, but in the unpinned mode it then drops the generation and the later GET / CopyObject runs by key alone.

That leaves a post-HeadObject hole the new comments say is closed. Concrete trace: metadata says 100 bytes, HeadObject sees 100-byte object A, the key is then replaced by 150-byte object B, and this code continues with src_etag = "". The buffered restore path builds ReadBufferFromS3(file_size = 100, expected_etag = ""), so it reads the first 100 bytes of B and succeeds. The whole-object native copy path calls copyS3File(..., size = 100, src_etag = ""), but CopyObject ignores that size argument and copies all 150 bytes of B. BackupWriterS3::copyFileFromDisk() / copyFile() below have the same shape.

So the current split only catches replacements that happened before the HeadObject; it does not preserve the promised "pinning off, size check still on" invariant after it. If backups need that contract, these reads/copies still have to carry the selected generation whenever the declared size matters, or the contract/tests/comments need to be narrowed accordingly.

Fix buffer overflows on data from external servers

a21418c

clickhouse-gh Bot added pr-critical-bugfix pr-must-backport Pull request should be backported intentionally. Use this label with great care! labels Aug 21, 2026

clickhouse-gh Bot reviewed Aug 21, 2026

View reviewed changes

Comment thread src/Disks/IO/ReadBufferFromAzureBlobStorage.cpp Outdated

clickhouse-gh Bot reviewed Aug 21, 2026

View reviewed changes

Comment thread src/Disks/IO/ReadBufferFromAzureBlobStorage.cpp Outdated

Comment thread src/Processors/Sources/MySQLSource.cpp Outdated

alexey-milovidov added 2 commits August 23, 2026 23:06

Merge remote-tracking branch 'origin/master' into fix-external-server…

4fe0875

…-buffer-overflows

clickhouse-gh Bot reviewed Aug 23, 2026

View reviewed changes

Comment thread src/Disks/IO/ReadBufferFromAzureBlobStorage.cpp Outdated

alexey-milovidov added 3 commits August 24, 2026 07:26

Add a blank line before parseMySQLBitValue

055e41d

SmitaRKulkarni self-assigned this Aug 24, 2026

SmitaRKulkarni reviewed Aug 24, 2026

View reviewed changes

Comment thread src/Processors/Sources/MySQLSource.cpp

SmitaRKulkarni reviewed Aug 24, 2026

View reviewed changes

clickhouse-gh Bot reviewed Aug 24, 2026

View reviewed changes

Comment thread src/Disks/IO/ReadBufferFromAzureBlobStorage.cpp Outdated

alexey-milovidov added 3 commits August 24, 2026 16:08

clickhouse-gh Bot reviewed Aug 24, 2026

View reviewed changes

Comment thread src/Disks/IO/ReadBufferFromAzureBlobStorage.cpp Outdated

Comment thread src/Disks/IO/ReadBufferFromAzureBlobStorage.cpp Outdated

alexey-milovidov mentioned this pull request Aug 25, 2026

Measure the compressed size of aggregate states in estimateSizeOfCompressedState #111985

Open

alexey-milovidov and others added 2 commits August 27, 2026 03:34

Merge remote-tracking branch 'origin/master' into fix-external-server…

d2ecffa

…-buffer-overflows

clickhouse-gh Bot reviewed Aug 27, 2026

View reviewed changes

Comment thread src/Disks/IO/ReadBufferFromAzureBlobStorage.cpp

alexey-milovidov added 2 commits August 29, 2026 01:51

Merge remote-tracking branch 'origin/master' into fix-external-server…

512ed6c

…-buffer-overflows

clickhouse-gh Bot reviewed Sep 12, 2026

View reviewed changes

Comment thread src/Backups/BackupIO_S3.cpp

Comment thread src/Storages/ObjectStorageQueue/ObjectStorageQueueSource.cpp Outdated

alexey-milovidov and others added 2 commits September 13, 2026 04:36

Merge remote-tracking branch 'origin/master' into fix-external-server…

15b3a58

…-buffer-overflows

clickhouse-gh Bot reviewed Sep 13, 2026

View reviewed changes

Comment thread src/Storages/ObjectStorageQueue/ObjectStorageQueueSource.cpp

Comment thread src/Backups/BackupIO_S3.cpp Outdated

Comment thread ...tadataStorages/PlainRewritable/MetadataStorageFromPlainRewritableObjectStorageOperations.cpp

clickhouse-gh Bot reviewed Sep 13, 2026

View reviewed changes

Comment thread src/Backups/BackupIO_S3.cpp Outdated

alexey-milovidov and others added 3 commits September 13, 2026 13:26

clickhouse-gh Bot reviewed Sep 13, 2026

View reviewed changes

Comment thread src/Storages/ObjectStorageQueue/ObjectStorageQueueSource.cpp

Comment thread src/Backups/BackupImpl.cpp

alexey-milovidov and others added 3 commits September 13, 2026 15:49

Merge remote-tracking branch 'origin/master' into HEAD

debef90

clickhouse-gh Bot added comp-object-storage Object storage connectivity (S3/GCS/Azure) including credentials, retries, multipart, etc. and removed comp-azure Azure Blob Storage integration (credentials, multipart, Azure-specific IO). labels Sep 13, 2026

clickhouse-gh Bot reviewed Sep 13, 2026

View reviewed changes

Comment thread ...tadataStorages/PlainRewritable/MetadataStorageFromPlainRewritableObjectStorageOperations.cpp

Comment thread src/Backups/BackupIO_S3.cpp

alexey-milovidov and others added 10 commits September 14, 2026 01:08

Merge remote-tracking branch 'origin/master' into fix-external-server…

bdbae4e

…-buffer-overflows

Remove two conflict marker lines left at the end of `gtest_azure_read…

c9ef19f

…_buffer.cpp` by the merge Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Merge remote-tracking branch 'origin/master' into fix-external-server…

4b482f4

…-buffer-overflows

clickhouse-gh Bot reviewed Sep 14, 2026

View reviewed changes

Comment thread src/Storages/ObjectStorageQueue/ObjectStorageQueueSource.cpp

clickhouse-gh Bot reviewed Sep 14, 2026

View reviewed changes

Uh oh!

Conversation

alexey-milovidov commented Aug 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changelog category (leave one):

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

Uh oh!

clickhouse-gh Bot commented Aug 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

AI Review

Summary

Findings

Final Verdict

Uh oh!

Uh oh!

clickhouse-gh Bot commented Aug 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Build profile diff (arm_release)

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

SmitaRKulkarni Aug 24, 2026

Choose a reason for hiding this comment

Uh oh!

alexey-milovidov Aug 24, 2026

Choose a reason for hiding this comment

Uh oh!

SmitaRKulkarni left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

alexey-milovidov commented Aug 27, 2026

Uh oh!

groeneai commented Aug 27, 2026

Uh oh!

Uh oh!

CLAassistant commented Sep 12, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

clickhouse-gh Bot Sep 14, 2026

Choose a reason for hiding this comment

Uh oh!

clickhouse-gh Bot Sep 14, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

alexey-milovidov commented Aug 21, 2026 •

edited

Loading

clickhouse-gh Bot commented Aug 21, 2026 •

edited

Loading

clickhouse-gh Bot commented Aug 21, 2026 •

edited

Loading