[inductor] Enable combo kernels with unbacked inputs by ColinPeppler · Pull Request #162442 · pytorch/pytorch

ColinPeppler · 2025-09-09T00:37:26Z

Internal user tried enabling combo kernels, but ran into "Cannot convert symbols to int". This PR is to enable combo kernels on inputs with data-dependent shapes.

Example exception

  File "/data/users/colinpeppler/pytorch/torch/_inductor/codegen/triton.py", line 4997, in benchmark_combo_kernel
    kernel_code_list = self.generate_combo_kernel_code(
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/users/colinpeppler/pytorch/torch/_inductor/codegen/simd.py", line 1849, in generate_combo_kernel_code
    src_code = kernel.codegen_kernel()
               ^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/users/colinpeppler/pytorch/torch/_inductor/codegen/triton_combo_kernel.py", line 802, in codegen_kernel
    code.splice(self.codegen_kernel_benchmark(num_gb=0))
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/users/colinpeppler/pytorch/torch/_inductor/codegen/triton_combo_kernel.py", line 852, in codegen_kernel_benchmark
    var_names.extend(self.kernel_benchmark_extra_args())
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/users/colinpeppler/pytorch/torch/_inductor/codegen/triton_combo_kernel.py", line 733, in kernel_benchmark_extra_args
    extra_args.append(str(V.graph.sizevars.size_hint(tree.numel)))
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/users/colinpeppler/pytorch/torch/_inductor/sizevars.py", line 584, in size_hint
    return int(out)
           ^^^^^^^^
  File "/home/colinpeppler/.conda/envs/pytorch/lib/python3.12/site-packages/sympy/core/expr.py", line 307, in __int__
    raise TypeError("Cannot convert symbols to int")
torch._inductor.exc.InductorError: TypeError: Cannot convert symbols to int

Stack from ghstack (oldest at bottom):

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @chenyang78 @kadeng @muchulee8 @amjames @chauhang @aakhundov @coconutruben

Differential Revision: D82042230

[ghstack-poisoned]

pytorch-bot · 2025-09-09T00:37:29Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/162442

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 1 Unrelated Failure

As of commit 380c8f4 with merge base 5babb4d ():

NEW FAILURE - The following job has failed:

Lint / lintrunner-noclang / linux-job (gh)
>>> Lint for torch/_inductor/codegen/triton_combo_kernel.py:

FLAKY - The following job failed but was likely due to flakiness present on trunk:

pull / linux-jammy-py3.13-clang12 / test (dynamo_wrapped, 3, 3, linux.2xlarge) (gh) (similar failure)
'test/profiler/test_profiler.py::TestExperimentalUtils::test_profiler_optimizer_single_tensor_pattern'

This comment was automatically generated by Dr. CI and updates every 15 minutes.

ghstack-source-id: 1afff9a Pull Request resolved: #162442

ColinPeppler · 2025-09-09T00:44:08Z

torch/_inductor/codegen/triton_combo_kernel.py

+                V.graph.sizevars.shape_env.has_hint(n.group[-1][-1])
+                and V.graph.sizevars.size_hint(n.group[-1][-1]) > 2048  # type: ignore[arg-type]


For this we can check if there's a hint, and if no hint, then we don't claim it's a long reduction. Since we don't actually know.

ColinPeppler · 2025-09-09T00:51:59Z

torch/_inductor/codegen/triton_combo_kernel.py

            for n in not_reduction
            if not kernel_map[n].inside_reduction
            and len(kernel_map[n].numels) == 2
+            and V.graph.sizevars.shape_env.has_hint(kernel_map[n].numels["x"])


ditto - don't claim it's a large pointwise

ColinPeppler · 2025-09-09T00:56:07Z

torch/_inductor/codegen/wrapper.py

+        is_benchmark_kernel = kernel_name == ""
+        if not is_benchmark_kernel:


We don't want to writeline if we're just using this function for kernel benchmarking.

pytorch/torch/_inductor/codegen/triton.py

Lines 3862 to 3866 in ed77e23

def kernel_benchmark_extra_args(self) -> list[str]:

args = []

if self.need_numel_args():

numel_args: list[sympy.Expr] = []

self.add_numel_to_call_args("", numel_args, [])

Internal user tried enabling combo kernels, but ran into "Cannot convert symbols to int". This PR is to enable combo kernels on inputs with data-dependent shapes. ### Example exception ``` File "/data/users/colinpeppler/pytorch/torch/_inductor/codegen/triton.py", line 4997, in benchmark_combo_kernel kernel_code_list = self.generate_combo_kernel_code( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/users/colinpeppler/pytorch/torch/_inductor/codegen/simd.py", line 1849, in generate_combo_kernel_code src_code = kernel.codegen_kernel() ^^^^^^^^^^^^^^^^^^^^^^^ File "/data/users/colinpeppler/pytorch/torch/_inductor/codegen/triton_combo_kernel.py", line 802, in codegen_kernel code.splice(self.codegen_kernel_benchmark(num_gb=0)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/users/colinpeppler/pytorch/torch/_inductor/codegen/triton_combo_kernel.py", line 852, in codegen_kernel_benchmark var_names.extend(self.kernel_benchmark_extra_args()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/users/colinpeppler/pytorch/torch/_inductor/codegen/triton_combo_kernel.py", line 733, in kernel_benchmark_extra_args extra_args.append(str(V.graph.sizevars.size_hint(tree.numel))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/users/colinpeppler/pytorch/torch/_inductor/sizevars.py", line 584, in size_hint return int(out) ^^^^^^^^ File "/home/colinpeppler/.conda/envs/pytorch/lib/python3.12/site-packages/sympy/core/expr.py", line 307, in __int__ raise TypeError("Cannot convert symbols to int") torch._inductor.exc.InductorError: TypeError: Cannot convert symbols to int ``` cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov coconutruben [ghstack-poisoned]

ghstack-source-id: 77c5d59 Pull Request resolved: #162442

ColinPeppler · 2025-09-09T18:07:45Z

@ColinPeppler has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

laithsakka · 2025-09-09T21:22:05Z

@bobrenjc93 would be nice for example here with unbacked to use override hint if exists instead of unbacked_symint_fallback

pytorchmergebot · 2025-09-10T17:38:53Z

Starting merge as part of PR stack under #162609

pytorchmergebot · 2025-09-10T20:49:09Z

Starting merge as part of PR stack under #162609

``` lintrunner init && lintrunner -m origin/main ``` Pull Request resolved: #162609 Approved by: https://github.com/coconutruben ghstack dependencies: #162442

Internal user tried enabling combo kernels, but ran into "Cannot convert symbols to int". This PR is to enable combo kernels on inputs with data-dependent shapes. ### Example exception ``` File "/data/users/colinpeppler/pytorch/torch/_inductor/codegen/triton.py", line 4997, in benchmark_combo_kernel kernel_code_list = self.generate_combo_kernel_code( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/users/colinpeppler/pytorch/torch/_inductor/codegen/simd.py", line 1849, in generate_combo_kernel_code src_code = kernel.codegen_kernel() ^^^^^^^^^^^^^^^^^^^^^^^ File "/data/users/colinpeppler/pytorch/torch/_inductor/codegen/triton_combo_kernel.py", line 802, in codegen_kernel code.splice(self.codegen_kernel_benchmark(num_gb=0)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/users/colinpeppler/pytorch/torch/_inductor/codegen/triton_combo_kernel.py", line 852, in codegen_kernel_benchmark var_names.extend(self.kernel_benchmark_extra_args()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/users/colinpeppler/pytorch/torch/_inductor/codegen/triton_combo_kernel.py", line 733, in kernel_benchmark_extra_args extra_args.append(str(V.graph.sizevars.size_hint(tree.numel))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/users/colinpeppler/pytorch/torch/_inductor/sizevars.py", line 584, in size_hint return int(out) ^^^^^^^^ File "/home/colinpeppler/.conda/envs/pytorch/lib/python3.12/site-packages/sympy/core/expr.py", line 307, in __int__ raise TypeError("Cannot convert symbols to int") torch._inductor.exc.InductorError: TypeError: Cannot convert symbols to int ``` Differential Revision: [D82042230](https://our.internmc.facebook.com/intern/diff/D82042230) Pull Request resolved: pytorch#162442 Approved by: https://github.com/jansel

``` lintrunner init && lintrunner -m origin/main ``` Pull Request resolved: pytorch#162609 Approved by: https://github.com/coconutruben ghstack dependencies: pytorch#162442

Internal user tried enabling combo kernels, but ran into "Cannot convert symbols to int". This PR is to enable combo kernels on inputs with data-dependent shapes. ### Example exception ``` File "/data/users/colinpeppler/pytorch/torch/_inductor/codegen/triton.py", line 4997, in benchmark_combo_kernel kernel_code_list = self.generate_combo_kernel_code( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/users/colinpeppler/pytorch/torch/_inductor/codegen/simd.py", line 1849, in generate_combo_kernel_code src_code = kernel.codegen_kernel() ^^^^^^^^^^^^^^^^^^^^^^^ File "/data/users/colinpeppler/pytorch/torch/_inductor/codegen/triton_combo_kernel.py", line 802, in codegen_kernel code.splice(self.codegen_kernel_benchmark(num_gb=0)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/users/colinpeppler/pytorch/torch/_inductor/codegen/triton_combo_kernel.py", line 852, in codegen_kernel_benchmark var_names.extend(self.kernel_benchmark_extra_args()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/users/colinpeppler/pytorch/torch/_inductor/codegen/triton_combo_kernel.py", line 733, in kernel_benchmark_extra_args extra_args.append(str(V.graph.sizevars.size_hint(tree.numel))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/users/colinpeppler/pytorch/torch/_inductor/sizevars.py", line 584, in size_hint return int(out) ^^^^^^^^ File "/home/colinpeppler/.conda/envs/pytorch/lib/python3.12/site-packages/sympy/core/expr.py", line 307, in __int__ raise TypeError("Cannot convert symbols to int") torch._inductor.exc.InductorError: TypeError: Cannot convert symbols to int ``` Differential Revision: [D82042230](https://our.internmc.facebook.com/intern/diff/D82042230) Pull Request resolved: pytorch#162442 Approved by: https://github.com/jansel

``` lintrunner init && lintrunner -m origin/main ``` Pull Request resolved: pytorch#162609 Approved by: https://github.com/coconutruben ghstack dependencies: pytorch#162442

Internal user tried enabling combo kernels, but ran into "Cannot convert symbols to int". This PR is to enable combo kernels on inputs with data-dependent shapes. ### Example exception ``` File "/data/users/colinpeppler/pytorch/torch/_inductor/codegen/triton.py", line 4997, in benchmark_combo_kernel kernel_code_list = self.generate_combo_kernel_code( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/users/colinpeppler/pytorch/torch/_inductor/codegen/simd.py", line 1849, in generate_combo_kernel_code src_code = kernel.codegen_kernel() ^^^^^^^^^^^^^^^^^^^^^^^ File "/data/users/colinpeppler/pytorch/torch/_inductor/codegen/triton_combo_kernel.py", line 802, in codegen_kernel code.splice(self.codegen_kernel_benchmark(num_gb=0)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/users/colinpeppler/pytorch/torch/_inductor/codegen/triton_combo_kernel.py", line 852, in codegen_kernel_benchmark var_names.extend(self.kernel_benchmark_extra_args()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/users/colinpeppler/pytorch/torch/_inductor/codegen/triton_combo_kernel.py", line 733, in kernel_benchmark_extra_args extra_args.append(str(V.graph.sizevars.size_hint(tree.numel))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/users/colinpeppler/pytorch/torch/_inductor/sizevars.py", line 584, in size_hint return int(out) ^^^^^^^^ File "/home/colinpeppler/.conda/envs/pytorch/lib/python3.12/site-packages/sympy/core/expr.py", line 307, in __int__ raise TypeError("Cannot convert symbols to int") torch._inductor.exc.InductorError: TypeError: Cannot convert symbols to int ``` Differential Revision: [D82042230](https://our.internmc.facebook.com/intern/diff/D82042230) Pull Request resolved: pytorch#162442 Approved by: https://github.com/jansel

``` lintrunner init && lintrunner -m origin/main ``` Pull Request resolved: pytorch#162609 Approved by: https://github.com/coconutruben ghstack dependencies: pytorch#162442

Internal user tried enabling combo kernels, but ran into "Cannot convert symbols to int". This PR is to enable combo kernels on inputs with data-dependent shapes. ### Example exception ``` File "/data/users/colinpeppler/pytorch/torch/_inductor/codegen/triton.py", line 4997, in benchmark_combo_kernel kernel_code_list = self.generate_combo_kernel_code( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/users/colinpeppler/pytorch/torch/_inductor/codegen/simd.py", line 1849, in generate_combo_kernel_code src_code = kernel.codegen_kernel() ^^^^^^^^^^^^^^^^^^^^^^^ File "/data/users/colinpeppler/pytorch/torch/_inductor/codegen/triton_combo_kernel.py", line 802, in codegen_kernel code.splice(self.codegen_kernel_benchmark(num_gb=0)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/users/colinpeppler/pytorch/torch/_inductor/codegen/triton_combo_kernel.py", line 852, in codegen_kernel_benchmark var_names.extend(self.kernel_benchmark_extra_args()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/users/colinpeppler/pytorch/torch/_inductor/codegen/triton_combo_kernel.py", line 733, in kernel_benchmark_extra_args extra_args.append(str(V.graph.sizevars.size_hint(tree.numel))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/users/colinpeppler/pytorch/torch/_inductor/sizevars.py", line 584, in size_hint return int(out) ^^^^^^^^ File "/home/colinpeppler/.conda/envs/pytorch/lib/python3.12/site-packages/sympy/core/expr.py", line 307, in __int__ raise TypeError("Cannot convert symbols to int") torch._inductor.exc.InductorError: TypeError: Cannot convert symbols to int ``` Differential Revision: [D82042230](https://our.internmc.facebook.com/intern/diff/D82042230) Pull Request resolved: pytorch#162442 Approved by: https://github.com/jansel

``` lintrunner init && lintrunner -m origin/main ``` Pull Request resolved: pytorch#162609 Approved by: https://github.com/coconutruben ghstack dependencies: pytorch#162442

Enable combo kernels with unbacked inputs

0d70b5a

[ghstack-poisoned]

ColinPeppler mentioned this pull request Sep 5, 2025

support unbacked softmax / logsoftmax #162216

Closed

pytorch-bot bot added ciflow/inductor module: inductor labels Sep 9, 2025

ColinPeppler added a commit that referenced this pull request Sep 9, 2025

Enable combo kernels with unbacked inputs

9bf6019

ghstack-source-id: 1afff9a Pull Request resolved: #162442

ColinPeppler commented Sep 9, 2025

View reviewed changes

ColinPeppler added a commit that referenced this pull request Sep 9, 2025

Enable combo kernels with unbacked inputs

5c254fb

ghstack-source-id: 77c5d59 Pull Request resolved: #162442

ColinPeppler added topic: not user facing topic category ciflow/trunk Trigger trunk jobs on your pull request labels Sep 9, 2025

ColinPeppler changed the title ~~Enable combo kernels with unbacked inputs~~ [inductor] Enable combo kernels with unbacked inputs Sep 9, 2025

ColinPeppler requested review from bobrenjc93, eellison, ezyang, jansel, laithsakka and pianpwk September 9, 2025 16:53

jansel approved these changes Sep 9, 2025

View reviewed changes

ColinPeppler mentioned this pull request Sep 10, 2025

[ez] add docstring/typing for codegen_kernel_benchmark #162609

Closed

pytorchmergebot closed this in 94755e8 Sep 10, 2025

pytorchmergebot added the Merged label Sep 10, 2025

github-actions bot deleted the gh/ColinPeppler/81/head branch October 11, 2025 02:07

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[inductor] Enable combo kernels with unbacked inputs#162442

[inductor] Enable combo kernels with unbacked inputs#162442
ColinPeppler wants to merge 2 commits intogh/ColinPeppler/81/basefrom
gh/ColinPeppler/81/head

ColinPeppler commented Sep 9, 2025 •

edited

Loading

Uh oh!

pytorch-bot bot commented Sep 9, 2025 •

edited

Loading

Uh oh!

ColinPeppler Sep 9, 2025 •

edited

Loading

Uh oh!

ColinPeppler Sep 9, 2025

Uh oh!

ColinPeppler Sep 9, 2025

Uh oh!

ColinPeppler commented Sep 9, 2025

Uh oh!

laithsakka commented Sep 9, 2025

Uh oh!

pytorchmergebot commented Sep 10, 2025

Uh oh!

pytorchmergebot commented Sep 10, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

		V.graph.sizevars.shape_env.has_hint(n.group[-1][-1])
		and V.graph.sizevars.size_hint(n.group[-1][-1]) > 2048 # type: ignore[arg-type]

		is_benchmark_kernel = kernel_name == ""
		if not is_benchmark_kernel:

	def kernel_benchmark_extra_args(self) -> list[str]:
	args = []
	if self.need_numel_args():
	numel_args: list[sympy.Expr] = []
	self.add_numel_to_call_args("", numel_args, [])

Conversation

ColinPeppler commented Sep 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Example exception

Uh oh!

pytorch-bot bot commented Sep 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/162442

❌ 1 New Failure, 1 Unrelated Failure

Uh oh!

ColinPeppler Sep 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ColinPeppler Sep 9, 2025

Choose a reason for hiding this comment

Uh oh!

ColinPeppler Sep 9, 2025

Choose a reason for hiding this comment

Uh oh!

ColinPeppler commented Sep 9, 2025

Uh oh!

laithsakka commented Sep 9, 2025

Uh oh!

pytorchmergebot commented Sep 10, 2025

Uh oh!

pytorchmergebot commented Sep 10, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ColinPeppler commented Sep 9, 2025 •

edited

Loading

pytorch-bot bot commented Sep 9, 2025 •

edited

Loading

ColinPeppler Sep 9, 2025 •

edited

Loading