[Inductor] Fix combo kernels for cpu backend by karthickai · Pull Request #167781 · pytorch/pytorch

karthickai · 2025-11-14T00:41:18Z

Stack from ghstack (oldest at bottom):

This PR fixes two issues
Fixes: #167780 combo_kernel fails with CppScheduling backend
Fixes: #168067 combo_kernel fails with mixed cpu/cuda nodes

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @kadeng @muchulee8 @amjames @chauhang @aakhundov @coconutruben @jataylo @mlazos @chenyang78

[ghstack-poisoned]

pytorch-bot · 2025-11-14T00:41:21Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/167781

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 4 Unrelated Failures

As of commit f46c098 with merge base e770c95 ():

NEW FAILURE - The following job has failed:

trunk / linux-jammy-cuda12.8-py3.10-gcc11 / test (default, 4, 5, lf.linux.g6.4xlarge.experimental.nvidia.gpu) (gh)
test/inductor/test_cooperative_reductions.py::CooperativeReductionTests::test_reduce_split

UNSTABLE - The following jobs are marked as unstable, possibly due to flakiness on trunk:

pull / linux-jammy-py3.14-clang12 / test (default, 2, 5, linux.4xlarge) (gh) (#169587)
The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.
pull / linux-jammy-py3.14-clang12 / test (default, 3, 5, linux.4xlarge) (gh) (#169587)
The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.
pull / linux-jammy-py3.14-clang12 / test (default, 4, 5, linux.4xlarge) (gh) (#169587)
The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.
pull / linux-jammy-py3.14-clang12 / test (default, 5, 5, linux.4xlarge) (gh) (#169587)
The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

ghstack-source-id: e8bd682 Pull Request resolved: #167781

Fixes: #167780 cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov coconutruben mlazos [ghstack-poisoned]

ghstack-source-id: 27e6b5d Pull Request resolved: #167781

Fixes: #167780 cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov coconutruben mlazos [ghstack-poisoned]

This PR fixes two issues Fixes: #167780 combo_kernel fails with CppScheduling backend Fixes: #168067 combo_kernel fails with mixed cpu/cuda nodes cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov coconutruben mlazos [ghstack-poisoned]

mlazos · 2025-11-19T08:19:34Z

torch/_inductor/scheduler.py

        subkernel_nodes = nodes
        device = subkernel_nodes[0].get_device()

+        if not all(node.get_device() == device for node in subkernel_nodes):


Hmm why does this case happen? we should only try and make combokernels in the combokernel pass with groups of nodes on the same device

group_nodes_for_combo_kernels doesn't filter by device. While I was debugging for this issue (#168067) and found mixed devices

> /home/karthickps/issues/torch/_inductor/scheduler.py(6127)speedup_by_combo_kernel() -> breakpoint() (Pdb) [node.get_device() for node in subkernel_nodes] [device(type='cuda', index=0), device(type='cpu')]

the code was only checking device = subkernel_nodes[0].get_device() (first node), so mixed-device groups slip through. This check blocks them and solves the mixed device issue.

or else we can add this check in _default_group_nodes_for_combo_kernels

I discussed with Michael, I'm moving this filter to _default_group_nodes_for_combo_kernels and adding assert in speedup_by_combo_kernel

This PR fixes two issues Fixes: #167780 combo_kernel fails with CppScheduling backend Fixes: #168067 combo_kernel fails with mixed cpu/cuda nodes cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov coconutruben mlazos [ghstack-poisoned]

pytorchmergebot · 2025-12-02T01:07:30Z

Starting merge as part of PR stack under #168109

This PR fixes two issues Fixes: #167780 combo_kernel fails with CppScheduling backend Fixes: #168067 combo_kernel fails with mixed cpu/cuda nodes cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov coconutruben mlazos [ghstack-poisoned]

ghstack-source-id: affe011 Pull Request resolved: #167781

This PR fixes two issues Fixes: #167780 combo_kernel fails with CppScheduling backend Fixes: #168067 combo_kernel fails with mixed cpu/cuda nodes cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov coconutruben mlazos [ghstack-poisoned]

karthickai · 2025-12-04T23:33:01Z

@pytorchbot merge -i

pytorchmergebot · 2025-12-04T23:35:01Z

Merge started

Your change will be merged while ignoring the following 5 checks: pull / linux-jammy-py3.14-clang12 / test (default, 3, 5, linux.4xlarge), pull / linux-jammy-py3.14-clang12 / test (default, 2, 5, linux.4xlarge), pull / linux-jammy-py3.14-clang12 / test (default, 5, 5, linux.4xlarge), pull / linux-jammy-py3.14-clang12 / test (default, 4, 5, linux.4xlarge), trunk / linux-jammy-cuda12.8-py3.10-gcc11 / test (default, 4, 5, lf.linux.g6.4xlarge.experimental.nvidia.gpu)

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

…rgs (#168127) Fixes: #168124 This PR fixes triton compilation failures in combo kernels when combining multiple kernels with random ops (or any ops that creates args with value equal to 1). The fix adds the missing logic to populate the `constants` for args marked as compile-time constants, matching the behavior of regular Triton kernels. Pull Request resolved: #168127 Approved by: https://github.com/mlazos ghstack dependencies: #167781

This PR fixes two issues Fixes: pytorch#167780 combo_kernel fails with CppScheduling backend Fixes: pytorch#168067 combo_kernel fails with mixed cpu/cuda nodes Pull Request resolved: pytorch#167781 Approved by: https://github.com/mlazos

…rgs (pytorch#168127) Fixes: pytorch#168124 This PR fixes triton compilation failures in combo kernels when combining multiple kernels with random ops (or any ops that creates args with value equal to 1). The fix adds the missing logic to populate the `constants` for args marked as compile-time constants, matching the behavior of regular Triton kernels. Pull Request resolved: pytorch#168127 Approved by: https://github.com/mlazos ghstack dependencies: pytorch#167781

This PR fixes two issues Fixes: #167780 combo_kernel fails with CppScheduling backend Fixes: #168067 combo_kernel fails with mixed cpu/cuda nodes Pull Request resolved: #167781 Approved by: https://github.com/mlazos

…rgs (#168127) Fixes: #168124 This PR fixes triton compilation failures in combo kernels when combining multiple kernels with random ops (or any ops that creates args with value equal to 1). The fix adds the missing logic to populate the `constants` for args marked as compile-time constants, matching the behavior of regular Triton kernels. Pull Request resolved: #168127 Approved by: https://github.com/mlazos ghstack dependencies: #167781

ghstack-source-id: ba6c7d2 Pull Request resolved: pytorch/pytorch#167781

ghstack-source-id: 7156ce0 Pull Request resolved: pytorch/pytorch#167781

[WIP][Inductor] Fix combo kernels for cpu backend

7f62da0

[ghstack-poisoned]

pytorch-bot bot added ciflow/inductor module: inductor labels Nov 14, 2025

karthickai added a commit that referenced this pull request Nov 14, 2025

[WIP][Inductor] Fix combo kernels for cpu backend

ded2894

ghstack-source-id: e8bd682 Pull Request resolved: #167781

karthickai marked this pull request as draft November 14, 2025 00:42

karthickai added release notes: inductor ciflow/trunk Trigger trunk jobs on your pull request labels Nov 14, 2025

karthickai changed the title ~~[WIP][Inductor] Fix combo kernels for cpu backend~~ [Inductor] Fix combo kernels for cpu backend Nov 17, 2025

karthickai requested a review from mlazos November 17, 2025 20:11

Update on "[Inductor] Fix combo kernels for cpu backend"

1589528

Fixes: #167780 cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov coconutruben mlazos [ghstack-poisoned]

karthickai added a commit that referenced this pull request Nov 18, 2025

[WIP][Inductor] Fix combo kernels for cpu backend

b8ac208

ghstack-source-id: 27e6b5d Pull Request resolved: #167781

karthickai mentioned this pull request Nov 18, 2025

[WIP][Inductor] Enable combo_kernels by default #168050

Draft

karthickai added 2 commits November 17, 2025 21:57

Update on "[Inductor] Fix combo kernels for cpu backend"

cc59358

Fixes: #167780 cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov coconutruben mlazos [ghstack-poisoned]

karthickai marked this pull request as ready for review November 18, 2025 20:53

karthickai mentioned this pull request Nov 18, 2025

[Inductor] handle GroupedSchedulerNode in combo kernel fusion #168109

Closed

karthickai mentioned this pull request Nov 19, 2025

[Inductor] Fix combo kernels by populating constants for equal_to_1 args #168127

Closed

mlazos reviewed Nov 19, 2025

View reviewed changes

This was referenced Nov 19, 2025

[WIP][Inductor] harden test for combo_kernels enablement #168195

Closed

Inductor codegen is missing support for CPPScheduling #168303

Closed

karthickai added the ciflow/rocm Trigger "default" config CI on ROCm label Nov 21, 2025

karthickai added 4 commits November 21, 2025 10:24

karthickai mentioned this pull request Nov 24, 2025

[Inductor] Fix combo kernels missing store masks #168939

Closed

karthickai mentioned this pull request Dec 2, 2025

[WIP] Filter out reduction nodes for combo kernels #169332

Closed

karthickai added 5 commits December 1, 2025 19:05

karthickai added a commit that referenced this pull request Dec 2, 2025

[WIP][Inductor] Fix combo kernels for cpu backend

5dfcb82

ghstack-source-id: affe011 Pull Request resolved: #167781

mlazos approved these changes Dec 3, 2025

View reviewed changes

karthickai mentioned this pull request Dec 3, 2025

[Inductor] Fix combo kernel heuristics for persistent_reduction #169509

Closed

karthickai added 2 commits December 3, 2025 23:31

pytorchmergebot added the merging label Dec 4, 2025

pytorchmergebot added the Merged label Dec 4, 2025

pytorchmergebot closed this in 6211bfd Dec 4, 2025

pytorchmergebot removed the merging label Dec 4, 2025

tiendatngcs pushed a commit to tiendatngcs/pytorch-Dec25 that referenced this pull request Dec 10, 2025

[WIP][Inductor] Fix combo kernels for cpu backend

46d51b2

ghstack-source-id: ba6c7d2 Pull Request resolved: pytorch/pytorch#167781

tiendatngcs pushed a commit to tiendatngcs/pytorch-Dec25 that referenced this pull request Dec 10, 2025

[WIP][Inductor] Fix combo kernels for cpu backend

a1e69a1

ghstack-source-id: 7156ce0 Pull Request resolved: pytorch/pytorch#167781

github-actions bot deleted the gh/karthickai/12/head branch January 4, 2026 02:22

licy666 mentioned this pull request Feb 3, 2026

[RFC][Inductor] enablement of combo-kernels - experimental horizontal optimization in torchinductor #170268

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Inductor] Fix combo kernels for cpu backend#167781

[Inductor] Fix combo kernels for cpu backend#167781
karthickai wants to merge 24 commits intogh/karthickai/12/basefrom
gh/karthickai/12/head

karthickai commented Nov 14, 2025 •

edited by pytorch-bot bot

Loading

Uh oh!

pytorch-bot bot commented Nov 14, 2025 •

edited

Loading

Uh oh!

mlazos Nov 19, 2025 •

edited

Loading

Uh oh!

karthickai Nov 19, 2025

Uh oh!

karthickai Nov 19, 2025

Uh oh!

karthickai Nov 21, 2025

Uh oh!

pytorchmergebot commented Dec 2, 2025

Uh oh!

karthickai commented Dec 4, 2025

Uh oh!

pytorchmergebot commented Dec 4, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

karthickai commented Nov 14, 2025 • edited by pytorch-bot bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Nov 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/167781

❌ 1 New Failure, 4 Unrelated Failures

Uh oh!

mlazos Nov 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

karthickai Nov 19, 2025

Choose a reason for hiding this comment

Uh oh!

karthickai Nov 19, 2025

Choose a reason for hiding this comment

Uh oh!

karthickai Nov 21, 2025

Choose a reason for hiding this comment

Uh oh!

pytorchmergebot commented Dec 2, 2025

Uh oh!

karthickai commented Dec 4, 2025

Uh oh!

pytorchmergebot commented Dec 4, 2025

Merge started

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

karthickai commented Nov 14, 2025 •

edited by pytorch-bot bot

Loading

pytorch-bot bot commented Nov 14, 2025 •

edited

Loading

mlazos Nov 19, 2025 •

edited

Loading