[xpu] Support high stream for ProcessGroupXCCL#163049
[xpu] Support high stream for ProcessGroupXCCL#163049Chao1Han wants to merge 3 commits intopytorch:mainfrom
Conversation
pytorch-bot
bot
added
oncall: distributed
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/163049
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 65d78d2 with merge base f2bb22f ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
Support high priority stream for xccl, test case add in #2049 We need merge this pr first and upstream op register pytorch/pytorch#163049 and then test case could be pass --------- Co-authored-by: mengfei25 <mengfei.li@Intel.com>
There was a problem hiding this comment.
Pull Request Overview
This PR adds high priority stream support for ProcessGroupXCCL, bringing it in line with CUDA's stream priority capabilities. The implementation enables XPU streams to execute with higher priority compared to other streams.
- Adds a new constructor overload for ProcessGroupXCCL that accepts store, rank, and size parameters with default low priority stream configuration
- Extends the Options class to include
is_high_priority_streamparameter with proper Python bindings - Provides read/write access to the high priority stream option through Python properties
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
Chao1Han
force-pushed
the
high_stream
branch
from
203f701 to
b4f70bf
Compare
Support high priority stream for xccl, test case add in #2049 We need merge this pr first and upstream op register pytorch/pytorch#163049 and then test case could be pass --------- Co-authored-by: mengfei25 <mengfei.li@Intel.com>
|
@Chao1Han You need to update torch-xpu-ops as well. |
Sure, let me update pin commit also here. |
guangyey
added
release notes: xpu
|
To add the ciflow label This helps ensure we don't trigger CI on this PR until it is actually authorized to do so. Please ping one of the reviewers if you do not have access to approve and run workflows. |
|
I think you'd better update the pin in a separate PR. |
Sure, I will wait for the pin commit update before merging this PR. |
|
@pytorchbot rebase -b main |
guangyey
added
ciflow/xpu
|
To add the ciflow label This helps ensure we don't trigger CI on this PR until it is actually authorized to do so. Please ping one of the reviewers if you do not have access to approve and run workflows. |
|
To add the ciflow label This helps ensure we don't trigger CI on this PR until it is actually authorized to do so. Please ping one of the reviewers if you do not have access to approve and run workflows. |
guangyey
added
ciflow/trunk
|
@pytorchmergebot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Add high priority stream support for ProcessGroupXCCL. Just like CUDA, XPU streams also support execution with higher priority compared to other streams. Implementation in intel/torch-xpu-ops#1715, add register here. Pull Request resolved: pytorch#163049 Approved by: https://github.com/guangyey, https://github.com/gujinghui, https://github.com/EikanWang, https://github.com/albanD
Feature #1715 and register pytorch/pytorch#163049 merged. add some high priority stream test case
After #163049, this PR fixes the type annotations to match the actual implementation for ProcessGroupXCCL::Options. Pull Request resolved: #166418 Approved by: https://github.com/guangyey, https://github.com/ezyang
After #163049, this PR fixes the type annotations to match the actual implementation for ProcessGroupXCCL::Options. Pull Request resolved: #166418 Approved by: https://github.com/guangyey, https://github.com/ezyang
After pytorch#163049, this PR fixes the type annotations to match the actual implementation for ProcessGroupXCCL::Options. Pull Request resolved: pytorch#166418 Approved by: https://github.com/guangyey, https://github.com/ezyang
7 participants
Add high priority stream support for ProcessGroupXCCL. Just like CUDA, XPU streams also support execution with higher priority compared to other streams. Implementation in intel/torch-xpu-ops#1715, add register here.
cc @H-Huang @awgu @wanchaol @fegin @fduwjj @wz337 @wconstab @d4l3k @pragupta @msaroufim @dcci