Skip to content

PR #47012: [xla:gpu] Fine grained PJRT definition events for result buffers - #125375

Draft
copybara-service[bot] wants to merge 1 commit into
masterfrom
exported_pr_964881846
Draft

PR #47012: [xla:gpu] Fine grained PJRT definition events for result buffers#125375
copybara-service[bot] wants to merge 1 commit into
masterfrom
exported_pr_964881846

Conversation

@copybara-service

Copy link
Copy Markdown

PR #47012: [xla:gpu] Fine grained PJRT definition events for result buffers

Imported from GitHub PR openxla/xla#47012

StreamExecutor PJRT currently gives every result buffer the same definition event, recorded after the entire executable has been scheduled. This unnecessarily delays consumers of results whose final device work was scheduled earlier.

This change integrates XLA:GPU’s ThunkExecutor definition tracking with PJRT (built in top of openxla/xla#46914):

  • Extends RawExecuteResult with optional per-result definition events. Results without an individual event fall back to the executable-wide event.
  • Creates a definition-event promise for every result before asynchronous dispatch.
  • Maps GPU buffer allocations to their corresponding PJRT result leaves.
  • Installs a ThunkExecutor definition callback during execution. The callback records one event on the stream that scheduled the allocation’s final use and resolves every result sharing that allocation.
  • Resolves any promises not handled by the definition callback with the executable-wide event, preserving existing behavior for untracked allocations and failures.
  • Updates CommonPjRtClient::CreateOutputs to attach each result’s individual definition event.
  • Materializes the allocation-readiness sync point before launch. Allocation readiness remains separate from result-content definition.

A lot of churn is because of previous commits that didn't run clang-format when moved to ABSL_ASSIGN_OR_RETURN.
Copybara import of the project:

--
ecf6e6883e9675499d739363ac0deccad1f805ad by Eugene Zhulenev ezhulenev@openxla.org:

[xla:gpu] Fine grained PJRT definition events for result buffers

--
4b5421ffd0d7dda06589833522850c243ba9e4d3 by Eugene Zhulenev ezhulenev@openxla.org:

Add individually_defined_output_indices to pjrt execute options

--
c29e1629d25740aaf5edeefae257ecd0eb59b686 by Eugene Zhulenev ezhulenev@openxla.org:

When individually_defined_output_indices empty use default event

Merging this change closes #47012

FUTURE_COPYBARA_INTEGRATE_REVIEW=openxla/xla#47012 from ezhulenev:thunk-executor-definition-callback c29e1629d25740aaf5edeefae257ecd0eb59b686

…uffers

Imported from GitHub PR openxla/xla#47012

StreamExecutor PJRT currently gives every result buffer the same definition event, recorded after the entire executable has been scheduled. This unnecessarily delays consumers of results whose final device work was scheduled earlier.

This change integrates XLA:GPU’s ThunkExecutor definition tracking with PJRT (built in top of openxla/xla#46914):

  - Extends `RawExecuteResult` with optional per-result definition events. Results without an individual event fall back to the executable-wide event.
  - Creates a definition-event promise for every result before asynchronous dispatch.
  - Maps GPU buffer allocations to their corresponding PJRT result leaves.
  - Installs a `ThunkExecutor` definition callback during execution. The callback records one event on the stream that scheduled the allocation’s final use and resolves every result sharing that allocation.
  - Resolves any promises not handled by the definition callback with the executable-wide event, preserving existing behavior for untracked allocations and failures.
  - Updates `CommonPjRtClient::CreateOutputs` to attach each result’s individual definition event.
  - Materializes the allocation-readiness sync point before launch. Allocation readiness remains separate from result-content definition.

A lot of churn is because of previous commits that didn't run clang-format when moved to `ABSL_ASSIGN_OR_RETURN`.
Copybara import of the project:

--
ecf6e6883e9675499d739363ac0deccad1f805ad by Eugene Zhulenev <ezhulenev@openxla.org>:

[xla:gpu] Fine grained PJRT definition events for result buffers

--
4b5421ffd0d7dda06589833522850c243ba9e4d3 by Eugene Zhulenev <ezhulenev@openxla.org>:

Add individually_defined_output_indices to pjrt execute options

--
c29e1629d25740aaf5edeefae257ecd0eb59b686 by Eugene Zhulenev <ezhulenev@openxla.org>:

When individually_defined_output_indices empty use default event

Merging this change closes #47012

FUTURE_COPYBARA_INTEGRATE_REVIEW=openxla/xla#47012 from ezhulenev:thunk-executor-definition-callback c29e1629d25740aaf5edeefae257ecd0eb59b686
PiperOrigin-RevId: 964881846
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant