Skip to content

Prefer the local DataNode when LOAD creates DataRegions - #19291

Open
luoluoyuyu wants to merge 1 commit into
apache:masterfrom
luoluoyuyu:feature/load-local-node-affinity
Open

luoluoyuyu wants to merge 1 commit into
apache:masterfrom
luoluoyuyu:feature/load-local-node-affinity

Conversation

@luoluoyuyu

Copy link
Copy Markdown
Member

When a LOAD has to create the DataRegions of the TsFile it is loading, the Region allocator picks the replicas purely by the configured policy (GCR random weight, GREEDY free disk, PGP subgraph). Data that is partitioned by DataNode and loaded on the DataNode that produced it therefore usually ends up on another node, and every read of it has to cross the network.

LOAD now accepts an opt-in attribute that pins the local DataNode as the first replica of each DataRegion created by that request:

LOAD '/data/example.tsfile' WITH ('prefer-local-node'='true')

The first replica is the DataNode that the write path routes the RegionGroup to, so the loaded data is also written locally. The remaining rf - 1 replicas are still chosen by the configured policy (GCR, GREEDY or PGP), which keeps all of its balancing keys in charge and only steers one slot.

The preference is off by default and best-effort:

  • it only affects the DataRegions created by that request; existing Regions are never moved and SchemaRegions are never hinted;
  • it is honoured only while the local DataNode is Running and still a placement candidate, so it can neither place a replica on a DataNode that went down nor fail an otherwise servable request. A DataNode that fails while the RegionGroups are being created is repaired by the Region creation procedure with a regular replica re-allocation;
  • the replica order is re-sorted when a replica is added or removed later, so the locality is best-effort rather than sticky;
  • the attribute is carried through statement splitting, the tree-to-table conversion, the analysis-time tablet conversion, the fallback and rewrite conversions, the asynchronous load handoff and the active-load retry path. Pipe-generated loads never set it.

The hint travels as TDataPartitionReq.preferredDataNodeId to the ConfigNode, where each allocator seeds the pinned replica and keeps its own policy for the rest of the replica set.


This PR has:

  • been self-reviewed.
    • concurrent read
    • concurrent write
    • concurrent read and write
  • added documentation for new or modified features or behaviors.
  • added Javadocs for most classes and all non-trivial methods.
  • added or updated version, license, or notice information
  • added comments explaining the "why" and the intent of the code wherever would not be obvious
    for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold
    for code coverage.
  • added integration tests.
  • been tested in a test IoTDB cluster.

Key changed/added classes (or packages if there are too many classes) in this PR

When a LOAD has to create the DataRegions of the TsFile it is loading,
the Region allocator picks the replicas purely by the configured policy
(GCR random weight, GREEDY free disk, PGP subgraph). Data that is
partitioned by DataNode and loaded on the DataNode that produced it
therefore usually ends up on another node, and every read of it has to
cross the network.

LOAD now accepts an opt-in attribute that pins the local DataNode as the
first replica of each DataRegion created by that request:

  LOAD '/data/example.tsfile' WITH ('prefer-local-node'='true')

The first replica is the DataNode that the write path routes the
RegionGroup to, so the loaded data is also written locally. The remaining
rf - 1 replicas are still chosen by the configured policy (GCR, GREEDY or
PGP), which keeps all of its balancing keys in charge and only steers one
slot.

The preference is off by default and best-effort:

- it only affects the DataRegions created by that request; existing
  Regions are never moved and SchemaRegions are never hinted;
- it is honoured only while the local DataNode is Running and still a
  placement candidate, so it can neither place a replica on a DataNode
  that went down nor fail an otherwise servable request. A DataNode that
  fails while the RegionGroups are being created is repaired by the Region
  creation procedure with a regular replica re-allocation;
- the replica order is re-sorted when a replica is added or removed later,
  so the locality is best-effort rather than sticky;
- the attribute is carried through statement splitting, the tree-to-table
  conversion, the analysis-time tablet conversion, the fallback and
  rewrite conversions, the asynchronous load handoff and the active-load
  retry path. Pipe-generated loads never set it.

The hint travels as TDataPartitionReq.preferredDataNodeId to the
ConfigNode, where each allocator seeds the pinned replica and keeps its
own policy for the rest of the replica set.
@luoluoyuyu
luoluoyuyu force-pushed the feature/load-local-node-affinity branch from b5c8936 to af2a847 Compare November 17, 2026 01:35

@Caideyipi Caideyipi left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the current diff at af2a847. No actionable issues found.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants