Skip to content

dist/debug: support py-spy (native+subprocess) stacks#169147

Closed
d4l3k wants to merge 5 commits intogh/d4l3k/5/basefrom
gh/d4l3k/5/head
Closed

dist/debug: support py-spy (native+subprocess) stacks#169147
d4l3k wants to merge 5 commits intogh/d4l3k/5/basefrom
gh/d4l3k/5/head

Conversation

@d4l3k
Copy link
Member

@d4l3k d4l3k commented Nov 26, 2025 •
edited
Loading

Stack from ghstack (oldest at bottom):

This adds support for getting subprocess+native+Python stack traces if py-spy is installed in the Python environment. This handler is implemented in C++ and doesn't depend on Python GIL

Test plan:

run simple debug server script

20251202_12h42m17s_grim

@d4l3k
Update
e2630c9 
[ghstack-poisoned]
@pytorch-bot pytorch-bot bot added the release notes: distributed (c10d) release notes category label Nov 26, 2025
@d4l3k d4l3k mentioned this pull request Nov 26, 2025
Closed
@pytorch-bot
Copy link

pytorch-bot bot commented Nov 26, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/169147

Note: Links to docs will display an error until the docs builds have been completed.

❌ 7 New Failures

As of commit a197e2b with merge base 082e96b (image):

NEW FAILURES - The following jobs have failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

d4l3k added a commit that referenced this pull request Nov 26, 2025
@d4l3k
dist/debug: support py-spy native stacks
53693e6 
ghstack-source-id: 930040e
Pull-Request: #169147
This was referenced Nov 26, 2025
Closed
Closed
@d4l3k d4l3k requested a review from fduwjj November 26, 2025 23:46
@d4l3k
Update
29bd4c2 
[ghstack-poisoned]
d4l3k added a commit that referenced this pull request Dec 1, 2025
@d4l3k
dist/debug: support py-spy native stacks
86e5e5d 
ghstack-source-id: fb97124
Pull-Request: #169147
@d4l3k
Update
f78a32c 
[ghstack-poisoned]
d4l3k added a commit that referenced this pull request Dec 2, 2025
@d4l3k
dist/debug: support py-spy native stacks
9946461 
ghstack-source-id: 737bf58
Pull-Request: #169147
@d4l3k
Update
41a7f3b 
[ghstack-poisoned]
d4l3k added a commit that referenced this pull request Dec 2, 2025
@d4l3k
dist/debug: support py-spy native stacks
b5748cb 
ghstack-source-id: 1d9a924
Pull-Request: #169147
d4l3k added a commit that referenced this pull request Dec 2, 2025
@d4l3k
dist/debug: support py-spy native stacks
7f3ef6a 
ghstack-source-id: 1d9a924
Pull-Request: #169147
@d4l3k
Copy link
Member Author

d4l3k commented Dec 2, 2025

@pytorchbot merge -i

@pytorch-bot pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Dec 2, 2025
@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Dec 2, 2025

Merge started

Your change will be merged while ignoring the following 0 checks:

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Dec 2, 2025

Merge failed

Reason: 1 jobs have failed, first few of them are: trunk / win-vs2022-cuda12.8-py3 / build

Details for Dev Infra team Raised by workflow job

@d4l3k
Update
a197e2b 
[ghstack-poisoned]
d4l3k added a commit that referenced this pull request Dec 2, 2025
@d4l3k
dist/debug: support py-spy native stacks
b50640a 
ghstack-source-id: 547b462
Pull-Request: #169147
@d4l3k d4l3k changed the title dist/debug: support py-spy native stacks dist/debug: support py-spy (native+subprocess) stacks Dec 2, 2025
@d4l3k
Copy link
Member Author

d4l3k commented Dec 2, 2025

@pytorchbot merge

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Dec 2, 2025

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Dec 2, 2025

Merge failed

Reason: 2 jobs have failed, first few of them are: trunk / linux-jammy-rocm-py3.10 / test (default, 3, 6, linux.rocm.gpu.gfx942.1), trunk / linux-jammy-rocm-py3.10 / test (default, 5, 6, linux.rocm.gpu.gfx942.1)

Details for Dev Infra team Raised by workflow job

@d4l3k
Copy link
Member Author

d4l3k commented Dec 2, 2025

Failures appear unrelated

@d4l3k
Copy link
Member Author

d4l3k commented Dec 2, 2025

@pytorchbot merge -i

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Dec 2, 2025

Merge started

Your change will be merged while ignoring the following 6 checks: trunk / linux-jammy-rocm-py3.10 / test (default, 2, 6, linux.rocm.gpu.gfx942.1), trunk / linux-jammy-rocm-py3.10 / test (default, 4, 6, linux.rocm.gpu.gfx942.1), trunk / linux-jammy-rocm-py3.10 / test (default, 3, 6, linux.rocm.gpu.gfx942.1), trunk / linux-jammy-rocm-py3.10 / test (default, 6, 6, linux.rocm.gpu.gfx942.1), trunk / linux-jammy-rocm-py3.10 / test (distributed, 1, 3, linux.rocm.gpu.gfx942.4), trunk / linux-jammy-rocm-py3.10 / test (default, 5, 6, linux.rocm.gpu.gfx942.1)

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Dec 2, 2025

Merge failed

Reason: 1 jobs have failed, first few of them are: trunk / linux-jammy-rocm-py3.10 / test (default, 1, 6, linux.rocm.gpu.gfx942.1)

Details for Dev Infra team Raised by workflow job

@d4l3k
Copy link
Member Author

d4l3k commented Dec 2, 2025

@pytorchbot merge -i

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Dec 2, 2025

Merge started

Your change will be merged while ignoring the following 7 checks: trunk / linux-jammy-rocm-py3.10 / test (default, 2, 6, linux.rocm.gpu.gfx942.1), trunk / linux-jammy-rocm-py3.10 / test (default, 4, 6, linux.rocm.gpu.gfx942.1), trunk / linux-jammy-rocm-py3.10 / test (default, 3, 6, linux.rocm.gpu.gfx942.1), trunk / linux-jammy-rocm-py3.10 / test (default, 1, 6, linux.rocm.gpu.gfx942.1), trunk / linux-jammy-rocm-py3.10 / test (default, 6, 6, linux.rocm.gpu.gfx942.1), trunk / linux-jammy-rocm-py3.10 / test (distributed, 1, 3, linux.rocm.gpu.gfx942.4), trunk / linux-jammy-rocm-py3.10 / test (default, 5, 6, linux.rocm.gpu.gfx942.1)

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@d4l3k d4l3k deleted the gh/d4l3k/5/head branch December 3, 2025 00:12
JacobSzwejbka pushed a commit that referenced this pull request Dec 8, 2025
@d4l3k @JacobSzwejbka
dist/debug: support py-spy (native+subprocess) stacks (#169147)
f9c41c8 
This adds support for getting subprocess+native+Python stack traces if `py-spy` is installed in the Python environment. This handler is implemented in C++ and doesn't depend on Python GIL

Test plan:

run simple debug server script

<img width="1603" height="1092" alt="20251202_12h42m17s_grim" src="https://github.com/user-attachments/assets/8c2c167e-666a-4b8c-807f-110ca177ed84" />

Pull Request resolved: #169147
Approved by: https://github.com/fduwjj
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@fduwjj fduwjj fduwjj approved these changes

Assignees

No one assigned

Labels

ciflow/trunk Trigger trunk jobs on your pull request Merged release notes: distributed (c10d) release notes category release notes: distributed (tools)

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@d4l3k @pytorchmergebot @fduwjj