PR #47012: [xla:gpu] Fine grained PJRT definition events for result buffers - #125375
Draft
copybara-service[bot] wants to merge 1 commit into
Draft
PR #47012: [xla:gpu] Fine grained PJRT definition events for result buffers#125375copybara-service[bot] wants to merge 1 commit into
copybara-service[bot] wants to merge 1 commit into
Conversation
…uffers Imported from GitHub PR openxla/xla#47012 StreamExecutor PJRT currently gives every result buffer the same definition event, recorded after the entire executable has been scheduled. This unnecessarily delays consumers of results whose final device work was scheduled earlier. This change integrates XLA:GPU’s ThunkExecutor definition tracking with PJRT (built in top of openxla/xla#46914): - Extends `RawExecuteResult` with optional per-result definition events. Results without an individual event fall back to the executable-wide event. - Creates a definition-event promise for every result before asynchronous dispatch. - Maps GPU buffer allocations to their corresponding PJRT result leaves. - Installs a `ThunkExecutor` definition callback during execution. The callback records one event on the stream that scheduled the allocation’s final use and resolves every result sharing that allocation. - Resolves any promises not handled by the definition callback with the executable-wide event, preserving existing behavior for untracked allocations and failures. - Updates `CommonPjRtClient::CreateOutputs` to attach each result’s individual definition event. - Materializes the allocation-readiness sync point before launch. Allocation readiness remains separate from result-content definition. A lot of churn is because of previous commits that didn't run clang-format when moved to `ABSL_ASSIGN_OR_RETURN`. Copybara import of the project: -- ecf6e6883e9675499d739363ac0deccad1f805ad by Eugene Zhulenev <ezhulenev@openxla.org>: [xla:gpu] Fine grained PJRT definition events for result buffers -- 4b5421ffd0d7dda06589833522850c243ba9e4d3 by Eugene Zhulenev <ezhulenev@openxla.org>: Add individually_defined_output_indices to pjrt execute options -- c29e1629d25740aaf5edeefae257ecd0eb59b686 by Eugene Zhulenev <ezhulenev@openxla.org>: When individually_defined_output_indices empty use default event Merging this change closes #47012 FUTURE_COPYBARA_INTEGRATE_REVIEW=openxla/xla#47012 from ezhulenev:thunk-executor-definition-callback c29e1629d25740aaf5edeefae257ecd0eb59b686 PiperOrigin-RevId: 964881846
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR #47012: [xla:gpu] Fine grained PJRT definition events for result buffers
Imported from GitHub PR openxla/xla#47012
StreamExecutor PJRT currently gives every result buffer the same definition event, recorded after the entire executable has been scheduled. This unnecessarily delays consumers of results whose final device work was scheduled earlier.
This change integrates XLA:GPU’s ThunkExecutor definition tracking with PJRT (built in top of openxla/xla#46914):
RawExecuteResultwith optional per-result definition events. Results without an individual event fall back to the executable-wide event.ThunkExecutordefinition callback during execution. The callback records one event on the stream that scheduled the allocation’s final use and resolves every result sharing that allocation.CommonPjRtClient::CreateOutputsto attach each result’s individual definition event.A lot of churn is because of previous commits that didn't run clang-format when moved to
ABSL_ASSIGN_OR_RETURN.Copybara import of the project:
--
ecf6e6883e9675499d739363ac0deccad1f805ad by Eugene Zhulenev ezhulenev@openxla.org:
[xla:gpu] Fine grained PJRT definition events for result buffers
--
4b5421ffd0d7dda06589833522850c243ba9e4d3 by Eugene Zhulenev ezhulenev@openxla.org:
Add individually_defined_output_indices to pjrt execute options
--
c29e1629d25740aaf5edeefae257ecd0eb59b686 by Eugene Zhulenev ezhulenev@openxla.org:
When individually_defined_output_indices empty use default event
Merging this change closes #47012
FUTURE_COPYBARA_INTEGRATE_REVIEW=openxla/xla#47012 from ezhulenev:thunk-executor-definition-callback c29e1629d25740aaf5edeefae257ecd0eb59b686