Support XPU in --nproc-per-node option to torchrun by moksiuc · Pull Request #159474 · pytorch/pytorch

moksiuc · 2025-07-30T13:15:13Z

Support both --nproc-per-node=xpu and autodetection of XPU device in case of --nproc-per-node=auto

cc @H-Huang @awgu @wanchaol @fegin @fduwjj @wz337 @wconstab @d4l3k @pragupta @ezyang @msaroufim @dcci

Support both --nproc-per-node=xpu and autodetection of XPU device in case of --nproc-per-node=auto

pytorch-bot · 2025-07-30T13:15:17Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/159474

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit eef7e98 with merge base be8095b ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

moksiuc · 2025-07-30T13:20:13Z

@pytorchbot label "topic: not user facing"

torch/distributed/run.py

pytorch-bot · 2025-07-31T11:38:32Z

To add the ciflow label ciflow/trunk please first approve the workflows that are awaiting approval (scroll to the bottom of this page).

This helps ensure we don't trigger CI on this PR until it is actually authorized to do so. Please ping one of the reviewers if you do not have access to approve and run workflows.

moksiuc · 2025-07-31T12:40:55Z

I had to make additional modification due to lint error:

Lint for torch/distributed/run.py:

Error (MYPY) [union-attr]
Item "None" of "device | None" has no attribute "type"

    709  |        elif nproc_per_node == "auto":
    710  |            if torch.accelerator.is_available():
    711  |                num_proc = torch.accelerator.device_count()
>>> 712  |                device_type = torch.accelerator.current_accelerator().type
    713  |            else:
    714  |                num_proc = os.cpu_count()
    715  |                device_type = "cpu"

torch/distributed/run.py

moksiuc · 2025-09-10T08:12:16Z

@gujinghui

moksiuc · 2025-09-11T07:04:15Z

@guangyey , what should be done further for this PR to be merged ?

guangyey · 2025-09-11T07:08:36Z

Hey @moksiuc, since @d4l3k is quite busy, I’ll remind him to take another look at this soon.

d4l3k

LGTM

guangyey · 2025-09-12T05:46:09Z

@pytorchbot merge

pytorchmergebot · 2025-09-12T05:48:05Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

Support both --nproc-per-node=xpu and autodetection of XPU device in case of --nproc-per-node=auto Pull Request resolved: pytorch#159474 Approved by: https://github.com/tsocha, https://github.com/guangyey, https://github.com/d4l3k Co-authored-by: Yu, Guangye <106960996+guangyey@users.noreply.github.com>

Support XPU in --nproc-per-node option to torchrun

aa0a90b

Support both --nproc-per-node=xpu and autodetection of XPU device in case of --nproc-per-node=auto

pytorch-bot bot added the oncall: distributed Add this issue/PR to distributed oncall triage queue label Jul 30, 2025

pytorchbot added the open source label Jul 30, 2025

pytorch-bot bot added the topic: not user facing topic category label Jul 30, 2025

tsocha approved these changes Jul 30, 2025

View reviewed changes

albanD requested a review from d4l3k July 30, 2025 13:33

albanD added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Jul 30, 2025

guangyey reviewed Jul 31, 2025

View reviewed changes

torch/distributed/run.py Outdated Show resolved Hide resolved

guangyey approved these changes Jul 31, 2025

View reviewed changes

guangyey added this to PyTorch Intel Jul 31, 2025

guangyey added the ciflow/xpu Run XPU CI tasks label Jul 31, 2025

moksiuc added 2 commits July 31, 2025 14:35

Merge branch 'refs/heads/main' into moksiucik_torchrun_xpu

c33e136

Apply review comment

249f7c2

pytorch-bot bot removed the ciflow/xpu Run XPU CI tasks label Jul 31, 2025

guangyey added ciflow/xpu Run XPU CI tasks ciflow/trunk Trigger trunk jobs on your pull request labels Jul 31, 2025

pytorch-bot bot removed the ciflow/trunk Trigger trunk jobs on your pull request label Jul 31, 2025

guangyey added the ciflow/trunk Trigger trunk jobs on your pull request label Jul 31, 2025

moksiuc added 2 commits July 31, 2025 15:39

Merge branch 'refs/heads/main' into moksiucik_torchrun_xpu

2aa3ebc

Fix lint error

e38a821

pytorch-bot bot removed ciflow/trunk Trigger trunk jobs on your pull request ciflow/xpu Run XPU CI tasks labels Jul 31, 2025

guangyey previously approved these changes Aug 1, 2025

View reviewed changes

guangyey added the ciflow/xpu Run XPU CI tasks label Aug 1, 2025

guangyey reviewed Aug 1, 2025

View reviewed changes

torch/distributed/run.py Outdated Show resolved Hide resolved

Update torch/distributed/run.py

3bea668

Merge branch 'refs/heads/main' into moksiucik_torchrun_xpu

36a5a47

pytorch-bot bot removed the ciflow/trunk Trigger trunk jobs on your pull request label Aug 11, 2025

d4l3k requested changes Aug 11, 2025

View reviewed changes

torch/distributed/run.py Show resolved Hide resolved

torch/distributed/run.py Show resolved Hide resolved

moksiuc requested a review from d4l3k August 12, 2025 08:47

guangyey added the ciflow/xpu Run XPU CI tasks label Aug 13, 2025

Merge branch 'refs/heads/main' into moksiucik_torchrun_xpu

29d6cd5

pytorch-bot bot removed the ciflow/xpu Run XPU CI tasks label Sep 1, 2025

guangyey added the ciflow/xpu Run XPU CI tasks label Sep 2, 2025

Merge branch 'refs/heads/main' into moksiucik_torchrun_xpu

8f2b885

pytorch-bot bot removed the ciflow/xpu Run XPU CI tasks label Sep 2, 2025

moksiuc requested a review from EikanWang September 10, 2025 08:08

Merge branch 'refs/heads/main' into moksiucik_torchrun_xpu

eef7e98

d4l3k approved these changes Sep 11, 2025

View reviewed changes

pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Sep 12, 2025

pytorchmergebot added the merging label Sep 12, 2025

pytorchmergebot added the Merged label Sep 12, 2025

pytorchmergebot closed this in 66c0f14 Sep 12, 2025

github-project-automation bot moved this to Done in PyTorch Intel Sep 12, 2025

pytorchmergebot removed the merging label Sep 12, 2025

moksiuc deleted the moksiucik_torchrun_xpu branch December 10, 2025 08:45

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support XPU in --nproc-per-node option to torchrun#159474

Support XPU in --nproc-per-node option to torchrun#159474
moksiuc wants to merge 22 commits intopytorch:mainfrom
moksiuc:moksiucik_torchrun_xpu

moksiuc commented Jul 30, 2025 •

edited by pytorch-bot bot

Loading

Uh oh!

pytorch-bot bot commented Jul 30, 2025 •

edited

Loading

Uh oh!

moksiuc commented Jul 30, 2025

Uh oh!

Uh oh!

pytorch-bot bot commented Jul 31, 2025

Uh oh!

moksiuc commented Jul 31, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

moksiuc commented Sep 10, 2025

Uh oh!

moksiuc commented Sep 11, 2025

Uh oh!

guangyey commented Sep 11, 2025

Uh oh!

d4l3k left a comment

Uh oh!

guangyey commented Sep 12, 2025

Uh oh!

pytorchmergebot commented Sep 12, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

Conversation

moksiuc commented Jul 30, 2025 • edited by pytorch-bot bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Jul 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/159474

✅ No Failures

Uh oh!

moksiuc commented Jul 30, 2025

Uh oh!

Uh oh!

pytorch-bot bot commented Jul 31, 2025

Uh oh!

moksiuc commented Jul 31, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

moksiuc commented Sep 10, 2025

Uh oh!

moksiuc commented Sep 11, 2025

Uh oh!

guangyey commented Sep 11, 2025

Uh oh!

d4l3k left a comment

Choose a reason for hiding this comment

Uh oh!

guangyey commented Sep 12, 2025

Uh oh!

pytorchmergebot commented Sep 12, 2025

Merge started

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

moksiuc commented Jul 30, 2025 •

edited by pytorch-bot bot

Loading

pytorch-bot bot commented Jul 30, 2025 •

edited

Loading