Cherry pick #99854 to 26.4: Fix unauthenticated TablesStatusRequest in interserver mode - #110498
Closed
robot-clickhouse-ci-2 wants to merge 18 commits into
Closed
Cherry pick #99854 to 26.4: Fix unauthenticated TablesStatusRequest in interserver mode#110498robot-clickhouse-ci-2 wants to merge 18 commits into
robot-clickhouse-ci-2 wants to merge 18 commits into
Conversation
The test uses `CLICKHOUSE_DATABASE` for table isolation and operates at the TCP connection level; parallel execution poses no hazard. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Move `!is_interserver_authenticated` check into `processTablesStatusRequest()` inside the `if (is_interserver_mode)` block, as suggested by reviewer - Remove unnecessary `IF NOT EXISTS` / `IF EXISTS` from CREATE/DROP TABLE in test - Rename `vu`/`ws`/`read_vu`/`read_ws` to `varbyte`/`varstring`/`read_varbyte`/`read_varstring` - Replace terse AI-generated comments with protocol-level documentation explaining the Hello packet layout, TablesStatusRequest layout, and expected outcome - Rewrite chained server-Hello reads as one call per line with field annotations - Add comment explaining why two TCP close paths (graceful FIN vs RST) both count as success Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ed_table_existence_request
The previous fix rejected any interserver `TablesStatusRequest` until the connection was `is_interserver_authenticated`, but that flag is only set after a query's secret hash validates. `TablesStatusRequest` is sent during `Distributed` connection establishment (ConnectionEstablisher), before any query, so the blanket check broke every `<secret>` cluster. Instead, authenticate the request itself: - Bump the TCP protocol revision and add DBMS_MIN_REVISION_WITH_INTERSERVER_SECRET_TABLES_STATUS. - Client (`Connection::getTablesStatus`): when connecting in interserver mode to a new-enough peer, send a SHA-256 over `salt + nonce + cluster_secret + "TablesStatusRequest"`, reusing the salt/nonce already exchanged during the Hello (mirrors the per-query hash). - Server (`processTablesStatusRequest`): validate that hash before serving the response. Peers too old to send a hash are rejected only when the new server setting `interserver_tables_status_require_auth` is enabled (default false), so rolling upgrades keep working; operators flip it on once the whole cluster is upgraded to close the disclosure entirely. Replaces the raw-socket stateless test (which assumed the old reject-always behavior and predates chunked framing) with a `.sql` test that drives the authenticated path over the `test_cluster_interserver_secret` cluster; the existing `test_distributed_inter_server_secret` integration test covers the legitimate secret-cluster path. Design: pull_request/99854-interserver-tables-status-auth/design_proposal.md
…ection The stateless `04036_interserver_tables_status_auth.sql` exercised only the legitimate (authenticated) path, which succeeds on master too, so it could not demonstrate the fix and broke `Bugfix validation` (which requires a changed test to fail on the pre-fix binary). Replace it with `test_interserver_tables_status_auth`: a node configured with `interserver_tables_status_require_auth = 1` is sent an old-protocol interserver `TablesStatusRequest` (no secret hash) over a raw socket. The fixed server rejects it and closes the connection without disclosing table status; a pre-fix binary does not know the setting, serves the request, and the test fails — which is what `Bugfix validation` expects. The legitimate secret-cluster path remains covered by `test_distributed_inter_server_secret`. Verified locally against a server with the setting on (rejected, no data) and off (returns a TablesStatusResponse), matching the post-fix and pre-fix behaviors.
…ion test Follow-up to the interserver TablesStatusRequest authentication, addressing review comments on #99854: - Authenticate before decoding the request body. The secret hash is now sent (client) and validated (server) *before* `TablesStatusRequest::read`, so an unauthenticated peer can no longer force the server to decode an unauthenticated request. - Default `interserver_tables_status_require_auth` to `true` (secure by default): old-protocol peers that send no hash are rejected unless an operator opts out for a mixed-version rolling upgrade. Documented the upgrade caveat in the setting description. - Extend the integration test to also cover the new-protocol rejection path: two nodes configure the same cluster with different secrets, so a peer that signs the request with the wrong secret is rejected by hash validation during connection establishment (asserted via node_b's log), in addition to the old-protocol no-hash rejection. Verified locally: authenticated path still succeeds with the hash sent first and the setting defaulting on; old-protocol peer rejected by default.
…auth to true
The old-protocol rejection branch's comment still read as opt-in ("rejected only
when the operator has opted in"), but the setting now defaults to `true`, so
hash-less old-protocol clients are rejected by default and `false` is the
temporary rolling-upgrade opt-out. Comment-only change.
# Conflicts: # src/Core/ServerSettings.cpp
`processUnexpectedTablesStatusRequest` read the request body directly, so on an interserver connection with protocol revision >= 54486 an out-of-place `TablesStatusRequest` would decode the 32-byte hash prefix as the request body and fail with unrelated parse errors instead of `UNEXPECTED_PACKET_FROM_CLIENT`. Consume the same prefix as `processTablesStatusRequest`. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
# Conflicts: # src/Core/ProtocolDefines.h
The cluster-secret hash proving a peer may issue a `TablesStatusRequest` covered only `salt + nonce + cluster_secret + "TablesStatusRequest"`, not the requested table list, so a relayed hash could be reused to read the status of arbitrary tables. Fold an order-independent, length-prefixed digest of `request.tables` into the hash on both client and server, mirroring the per-query secret hash that `processQuery` computes over the query text. The hash precedes the body on the wire, so the server deserializes the body to recompute the digest before validating (as `processQuery` reads the query before validating), using `StringWithMemoryTracking` for the hash duplicate. Table resolution still happens only after validation, and an old-protocol unauthenticated request rejected by `interserver_tables_status_require_auth` is refused before its body is read. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…_existence_request Fix unauthenticated TablesStatusRequest in interserver mode
Contributor
|
Closing: this branch will not receive the interserver |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Original pull-request #99854
Do not merge this PR manually
This pull-request is a first step of an automated backporting.
It contains changes similar to calling
git cherry-picklocally.If you intend to continue backporting the changes, then resolve all conflicts if any.
Otherwise, if you do not want to backport them, then just close this pull-request.
The check results does not matter at this step - you can safely ignore them.
Troubleshooting
If the conflicts were resolved in a wrong way
If this cherry-pick PR is completely screwed by a wrong conflicts resolution, and you want to recreate it:
pr-cherrypicklabel from the PRYou also need to check the Original pull-request for
pr-backports-createdlabel, and delete if it's presented thereThe PR source
The PR is created in the CI job