Skip to content

Cherry pick #99854 to 26.4: Fix unauthenticated TablesStatusRequest in interserver mode - #110498

Closed
robot-clickhouse-ci-2 wants to merge 18 commits into
backport/26.4/99854from
cherrypick/26.4/99854
Closed

Cherry pick #99854 to 26.4: Fix unauthenticated TablesStatusRequest in interserver mode#110498
robot-clickhouse-ci-2 wants to merge 18 commits into
backport/26.4/99854from
cherrypick/26.4/99854

Conversation

@robot-clickhouse-ci-2

Copy link
Copy Markdown
Contributor

Original pull-request #99854

Do not merge this PR manually

This pull-request is a first step of an automated backporting.
It contains changes similar to calling git cherry-pick locally.
If you intend to continue backporting the changes, then resolve all conflicts if any.
Otherwise, if you do not want to backport them, then just close this pull-request.

The check results does not matter at this step - you can safely ignore them.

Troubleshooting

If the conflicts were resolved in a wrong way

If this cherry-pick PR is completely screwed by a wrong conflicts resolution, and you want to recreate it:

  • delete the pr-cherrypick label from the PR
  • delete this branch from the repository

You also need to check the Original pull-request for pr-backports-created label, and delete if it's presented there

The PR source

The PR is created in the CI job

tiandiwonder and others added 18 commits March 18, 2026 09:17
The test uses `CLICKHOUSE_DATABASE` for table isolation and operates at
the TCP connection level; parallel execution poses no hazard.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Move `!is_interserver_authenticated` check into `processTablesStatusRequest()`
  inside the `if (is_interserver_mode)` block, as suggested by reviewer
- Remove unnecessary `IF NOT EXISTS` / `IF EXISTS` from CREATE/DROP TABLE in test
- Rename `vu`/`ws`/`read_vu`/`read_ws` to `varbyte`/`varstring`/`read_varbyte`/`read_varstring`
- Replace terse AI-generated comments with protocol-level documentation explaining
  the Hello packet layout, TablesStatusRequest layout, and expected outcome
- Rewrite chained server-Hello reads as one call per line with field annotations
- Add comment explaining why two TCP close paths (graceful FIN vs RST) both count as success

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The previous fix rejected any interserver `TablesStatusRequest` until the
connection was `is_interserver_authenticated`, but that flag is only set
after a query's secret hash validates. `TablesStatusRequest` is sent during
`Distributed` connection establishment (ConnectionEstablisher), before any
query, so the blanket check broke every `<secret>` cluster.

Instead, authenticate the request itself:

- Bump the TCP protocol revision and add
  DBMS_MIN_REVISION_WITH_INTERSERVER_SECRET_TABLES_STATUS.
- Client (`Connection::getTablesStatus`): when connecting in interserver
  mode to a new-enough peer, send a SHA-256 over
  `salt + nonce + cluster_secret + "TablesStatusRequest"`, reusing the
  salt/nonce already exchanged during the Hello (mirrors the per-query hash).
- Server (`processTablesStatusRequest`): validate that hash before serving
  the response. Peers too old to send a hash are rejected only when the new
  server setting `interserver_tables_status_require_auth` is enabled
  (default false), so rolling upgrades keep working; operators flip it on
  once the whole cluster is upgraded to close the disclosure entirely.

Replaces the raw-socket stateless test (which assumed the old reject-always
behavior and predates chunked framing) with a `.sql` test that drives the
authenticated path over the `test_cluster_interserver_secret` cluster; the
existing `test_distributed_inter_server_secret` integration test covers the
legitimate secret-cluster path.

Design: pull_request/99854-interserver-tables-status-auth/design_proposal.md
…ection

The stateless `04036_interserver_tables_status_auth.sql` exercised only the
legitimate (authenticated) path, which succeeds on master too, so it could not
demonstrate the fix and broke `Bugfix validation` (which requires a changed
test to fail on the pre-fix binary).

Replace it with `test_interserver_tables_status_auth`: a node configured with
`interserver_tables_status_require_auth = 1` is sent an old-protocol interserver
`TablesStatusRequest` (no secret hash) over a raw socket. The fixed server
rejects it and closes the connection without disclosing table status; a pre-fix
binary does not know the setting, serves the request, and the test fails — which
is what `Bugfix validation` expects. The legitimate secret-cluster path remains
covered by `test_distributed_inter_server_secret`.

Verified locally against a server with the setting on (rejected, no data) and off
(returns a TablesStatusResponse), matching the post-fix and pre-fix behaviors.
…ion test

Follow-up to the interserver TablesStatusRequest authentication, addressing
review comments on #99854:

- Authenticate before decoding the request body. The secret hash is now sent
  (client) and validated (server) *before* `TablesStatusRequest::read`, so an
  unauthenticated peer can no longer force the server to decode an unauthenticated
  request.

- Default `interserver_tables_status_require_auth` to `true` (secure by default):
  old-protocol peers that send no hash are rejected unless an operator opts out
  for a mixed-version rolling upgrade. Documented the upgrade caveat in the
  setting description.

- Extend the integration test to also cover the new-protocol rejection path: two
  nodes configure the same cluster with different secrets, so a peer that signs
  the request with the wrong secret is rejected by hash validation during
  connection establishment (asserted via node_b's log), in addition to the
  old-protocol no-hash rejection.

Verified locally: authenticated path still succeeds with the hash sent first and
the setting defaulting on; old-protocol peer rejected by default.
…auth to true

The old-protocol rejection branch's comment still read as opt-in ("rejected only
when the operator has opted in"), but the setting now defaults to `true`, so
hash-less old-protocol clients are rejected by default and `false` is the
temporary rolling-upgrade opt-out. Comment-only change.
# Conflicts:
#	src/Core/ServerSettings.cpp
`processUnexpectedTablesStatusRequest` read the request body directly, so
on an interserver connection with protocol revision >= 54486 an
out-of-place `TablesStatusRequest` would decode the 32-byte hash prefix as
the request body and fail with unrelated parse errors instead of
`UNEXPECTED_PACKET_FROM_CLIENT`. Consume the same prefix as
`processTablesStatusRequest`.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
# Conflicts:
#	src/Core/ProtocolDefines.h
The cluster-secret hash proving a peer may issue a `TablesStatusRequest` covered
only `salt + nonce + cluster_secret + "TablesStatusRequest"`, not the requested
table list, so a relayed hash could be reused to read the status of arbitrary
tables. Fold an order-independent, length-prefixed digest of `request.tables`
into the hash on both client and server, mirroring the per-query secret hash
that `processQuery` computes over the query text.

The hash precedes the body on the wire, so the server deserializes the body to
recompute the digest before validating (as `processQuery` reads the query before
validating), using `StringWithMemoryTracking` for the hash duplicate. Table
resolution still happens only after validation, and an old-protocol
unauthenticated request rejected by `interserver_tables_status_require_auth` is
refused before its body is read.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…_existence_request

Fix unauthenticated TablesStatusRequest in interserver mode
@robot-clickhouse-ci-2 robot-clickhouse-ci-2 added pr-cherrypick Cherry-pick of merge-commit before backporting. Do not use manually - automated use only! do not test disable testing on pull request pr-critical-bugfix labels Jul 15, 2026
@tiandiwonder

Copy link
Copy Markdown
Contributor

Closing: this branch will not receive the interserver TablesStatusRequest auth backport. It sits below protocol revision 54487 and is missing intervening revisions (54485 client_agent and/or 54486 internal_query_flag); reaching 54487 would require backporting the client_agent feature (a system.query_log schema change) into a stable branch, and there is no clean revision-independent gating. Full rationale on the original PR #99854.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do not test disable testing on pull request pr-cherrypick Cherry-pick of merge-commit before backporting. Do not use manually - automated use only! pr-critical-bugfix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants