Enable generating generic c_shim that doesn't bypass dispatcher #158974

mikaylagawarecki · 2025-07-23T20:57:27Z

Adds c_shim_aten.{h/cpp} and use this for fill_

Stack from ghstack (oldest at bottom):

-> Enable generating generic c_shim that doesn't bypass dispatcher #158974

This is the generated c_shim_aten.cpp for reference

// WARNING: THIS FILE IS AUTOGENERATED BY torchgen. DO NOT MODIFY BY HAND.
// See https://github.com/pytorch/pytorch/blob/7e86a7c0155295539996e0cf422883571126073e/torchgen/gen.py#L2424-L2436 for details

// This file corresponds to the aten_shimified_ops list in torchgen/aoti/fallback_ops.py


#include <torch/csrc/inductor/aoti_torch/generated/c_shim_aten.h>
#include <torch/csrc/inductor/aoti_torch/utils.h>

#ifndef AT_PER_OPERATOR_HEADERS
#include <ATen/Functions.h>
#include <ATen/CompositeExplicitAutogradFunctions.h>
#include <ATen/CompositeExplicitAutogradNonFunctionalFunctions.h>
#include <ATen/CompositeImplicitAutogradFunctions.h>
#else
#include <ATen/ops/fill.h>

#endif // AT_PER_OPERATOR_HEADERS

using namespace torch::aot_inductor;


AOTITorchError aoti_torch_aten_fill__Scalar(AtenTensorHandle self, double value) {
    AOTI_TORCH_CONVERT_EXCEPTION_TO_ERROR_CODE({
        at::fill_(
            *tensor_handle_to_tensor_pointer(self), value
        );

[ghstack-poisoned]

pytorch-bot · 2025-07-23T20:57:31Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/158974

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

⏳ 1 Pending, 2 Unrelated Failures

As of commit 671b951 with merge base febf3c4 ():

FLAKY - The following job failed but was likely due to flakiness present on trunk:

BC Lint / bc_linter (gh) (suppressed by suppress-bc-linter)
Process completed with exit code 1.

UNSTABLE - The following job is marked as unstable, possibly due to flakiness on trunk:

pull / linux-jammy-py3_9-clang9-xla / test (xla, 1, 1, linux.12xlarge, unstable) (gh) (#158876)
sccache: error: couldn't connect to server

This comment was automatically generated by Dr. CI and updates every 15 minutes.

ghstack-source-id: 3cedfe3 Pull Request resolved: #158974

github-actions · 2025-07-23T21:01:32Z

Attention! PyTorch one of the C-stable API file was changed

You MUST NOT change existing function declarations in this, as this header defines a stable C ABI. If you need to change the signature for a function, introduce a new v2 version of the function and modify code generation to target the new version of the function.

Caused by:

torch/csrc/inductor/aoti_torch/generated/c_shim_generic.h

…tcher" [ghstack-poisoned]

ghstack-source-id: eb8ee19 Pull Request resolved: #158974

…tcher" [ghstack-poisoned]

ghstack-source-id: e8d5697 Pull Request resolved: #158974

…tcher" And use this for `fill_` This is the generated `c_shim_generic.cpp` for reference ```cpp // WARNING: THIS FILE IS AUTOGENERATED BY torchgen. DO NOT MODIFY BY HAND. // See https://github.com/pytorch/pytorch/blob/7e86a7c0155295539996e0cf422883571126073e/torchgen/gen.py#L2424-L2436 for details #include <torch/csrc/inductor/aoti_torch/generated/c_shim_generic.h> #include <torch/csrc/inductor/aoti_torch/utils.h> #ifndef AT_PER_OPERATOR_HEADERS #include <ATen/Functions.h> #include <ATen/CompositeExplicitAutogradFunctions.h> #include <ATen/CompositeExplicitAutogradNonFunctionalFunctions.h> #include <ATen/CompositeImplicitAutogradFunctions.h> #else #include <ATen/ops/fill.h> #endif // AT_PER_OPERATOR_HEADERS using namespace torch::aot_inductor; AOTITorchError aoti_torch_generic_fill__Scalar(AtenTensorHandle self, double value) { AOTI_TORCH_CONVERT_EXCEPTION_TO_ERROR_CODE({ at::fill_( *tensor_handle_to_tensor_pointer(self), value ); }); } ``` [ghstack-poisoned]

ghstack-source-id: d8c6959 Pull Request resolved: #158974

mikaylagawarecki · 2025-07-24T17:15:07Z

torchgen/gen_aoti_c_shim.py

 def gen_static_dispatch_backend_call(
    f: NativeFunction,
-    backend_index: BackendIndex,
+    backend_index: Optional[BackendIndex] = None,


hm I don't think this breaks BC (?)

i think this is fine yea

caffe2/CMakeLists.txt

test/cpp_extensions/libtorch_agnostic_extension/libtorch_agnostic/csrc/kernel.cpp

test/cpp_extensions/libtorch_agnostic_extension/libtorch_agnostic/ops.py

janeyx99 · 2025-07-24T20:40:40Z

torch/csrc/inductor/aoti_torch/generated/c_shim_generic.h

@@ -0,0 +1,18 @@
+


why are there two extra blank lines haha

This isn't from this PR, all the rest of the c_shims are like this, so I'm leaving it as is

torch/csrc/stable/ops.h

torch/csrc/inductor/aoti_torch/generated/c_shim_generic.h

torchgen/aoti/fallback_ops.py

janeyx99 · 2025-07-24T20:45:09Z

torchgen/gen_aoti_c_shim.py

 def gen_static_dispatch_backend_call(
    f: NativeFunction,
-    backend_index: BackendIndex,
+    backend_index: Optional[BackendIndex] = None,


i think this is fine yea

janeyx99 · 2025-07-24T20:46:34Z

torchgen/gen_aoti_c_shim.py

+    if backend_index is None:
+        return f"at::{cpp_sig.name()}"
+    else:
+        return f"at::{backend_index.dispatch_key.lower()}::{cpp_sig.name()}"


this code change looks good to me (and works). cc @albanD --I'd appreciate if you took a glance at this codegen portion and ascertain that we're making the right assumptions!

albanD

No concern beyond the fake dispatch key on my end!

torchgen/model.py

…tcher" And use this for `fill_` This is the generated `c_shim_generic.cpp` for reference ```cpp // WARNING: THIS FILE IS AUTOGENERATED BY torchgen. DO NOT MODIFY BY HAND. // See https://github.com/pytorch/pytorch/blob/7e86a7c0155295539996e0cf422883571126073e/torchgen/gen.py#L2424-L2436 for details #include <torch/csrc/inductor/aoti_torch/generated/c_shim_generic.h> #include <torch/csrc/inductor/aoti_torch/utils.h> #ifndef AT_PER_OPERATOR_HEADERS #include <ATen/Functions.h> #include <ATen/CompositeExplicitAutogradFunctions.h> #include <ATen/CompositeExplicitAutogradNonFunctionalFunctions.h> #include <ATen/CompositeImplicitAutogradFunctions.h> #else #include <ATen/ops/fill.h> #endif // AT_PER_OPERATOR_HEADERS using namespace torch::aot_inductor; AOTITorchError aoti_torch_generic_fill__Scalar(AtenTensorHandle self, double value) { AOTI_TORCH_CONVERT_EXCEPTION_TO_ERROR_CODE({ at::fill_( *tensor_handle_to_tensor_pointer(self), value ); }); } ``` [ghstack-poisoned]

ghstack-source-id: 0d7e2e0 Pull Request resolved: #158974

…tcher" And use this for `fill_` This is the generated `c_shim_generic.cpp` for reference ```cpp // WARNING: THIS FILE IS AUTOGENERATED BY torchgen. DO NOT MODIFY BY HAND. // See https://github.com/pytorch/pytorch/blob/7e86a7c0155295539996e0cf422883571126073e/torchgen/gen.py#L2424-L2436 for details #include <torch/csrc/inductor/aoti_torch/generated/c_shim_generic.h> #include <torch/csrc/inductor/aoti_torch/utils.h> #ifndef AT_PER_OPERATOR_HEADERS #include <ATen/Functions.h> #include <ATen/CompositeExplicitAutogradFunctions.h> #include <ATen/CompositeExplicitAutogradNonFunctionalFunctions.h> #include <ATen/CompositeImplicitAutogradFunctions.h> #else #include <ATen/ops/fill.h> #endif // AT_PER_OPERATOR_HEADERS using namespace torch::aot_inductor; AOTITorchError aoti_torch_generic_fill__Scalar(AtenTensorHandle self, double value) { AOTI_TORCH_CONVERT_EXCEPTION_TO_ERROR_CODE({ at::fill_( *tensor_handle_to_tensor_pointer(self), value ); }); } ``` [ghstack-poisoned]

ghstack-source-id: dd2b052 Pull Request resolved: #158974

…tcher" And use this for `fill_` This is the generated `c_shim_generic.cpp` for reference ```cpp // WARNING: THIS FILE IS AUTOGENERATED BY torchgen. DO NOT MODIFY BY HAND. // See https://github.com/pytorch/pytorch/blob/7e86a7c0155295539996e0cf422883571126073e/torchgen/gen.py#L2424-L2436 for details #include <torch/csrc/inductor/aoti_torch/generated/c_shim_generic.h> #include <torch/csrc/inductor/aoti_torch/utils.h> #ifndef AT_PER_OPERATOR_HEADERS #include <ATen/Functions.h> #include <ATen/CompositeExplicitAutogradFunctions.h> #include <ATen/CompositeExplicitAutogradNonFunctionalFunctions.h> #include <ATen/CompositeImplicitAutogradFunctions.h> #else #include <ATen/ops/fill.h> #endif // AT_PER_OPERATOR_HEADERS using namespace torch::aot_inductor; AOTITorchError aoti_torch_generic_fill__Scalar(AtenTensorHandle self, double value) { AOTI_TORCH_CONVERT_EXCEPTION_TO_ERROR_CODE({ at::fill_( *tensor_handle_to_tensor_pointer(self), value ); }); } ``` [ghstack-poisoned]

ghstack-source-id: a1a871f Pull Request resolved: #158974

…tcher" And use this for `fill_` This is the generated `c_shim_aten.cpp` for reference ```cpp // WARNING: THIS FILE IS AUTOGENERATED BY torchgen. DO NOT MODIFY BY HAND. // See https://github.com/pytorch/pytorch/blob/7e86a7c0155295539996e0cf422883571126073e/torchgen/gen.py#L2424-L2436 for details // This file corresponds to the aten_shimified_ops list in torchgen/aoti/fallback_ops.py #include <torch/csrc/inductor/aoti_torch/generated/c_shim_aten.h> #include <torch/csrc/inductor/aoti_torch/utils.h> #ifndef AT_PER_OPERATOR_HEADERS #include <ATen/Functions.h> #include <ATen/CompositeExplicitAutogradFunctions.h> #include <ATen/CompositeExplicitAutogradNonFunctionalFunctions.h> #include <ATen/CompositeImplicitAutogradFunctions.h> #else #include <ATen/ops/fill.h> #endif // AT_PER_OPERATOR_HEADERS using namespace torch::aot_inductor; AOTITorchError aoti_torch_aten_fill__Scalar(AtenTensorHandle self, double value) { AOTI_TORCH_CONVERT_EXCEPTION_TO_ERROR_CODE({ at::fill_( *tensor_handle_to_tensor_pointer(self), value ); ``` [ghstack-poisoned]

ghstack-source-id: d86f14a Pull Request resolved: #158974

…tcher" And use this for `fill_` This is the generated `c_shim_aten.cpp` for reference ```cpp // WARNING: THIS FILE IS AUTOGENERATED BY torchgen. DO NOT MODIFY BY HAND. // See https://github.com/pytorch/pytorch/blob/7e86a7c0155295539996e0cf422883571126073e/torchgen/gen.py#L2424-L2436 for details // This file corresponds to the aten_shimified_ops list in torchgen/aoti/fallback_ops.py #include <torch/csrc/inductor/aoti_torch/generated/c_shim_aten.h> #include <torch/csrc/inductor/aoti_torch/utils.h> #ifndef AT_PER_OPERATOR_HEADERS #include <ATen/Functions.h> #include <ATen/CompositeExplicitAutogradFunctions.h> #include <ATen/CompositeExplicitAutogradNonFunctionalFunctions.h> #include <ATen/CompositeImplicitAutogradFunctions.h> #else #include <ATen/ops/fill.h> #endif // AT_PER_OPERATOR_HEADERS using namespace torch::aot_inductor; AOTITorchError aoti_torch_aten_fill__Scalar(AtenTensorHandle self, double value) { AOTI_TORCH_CONVERT_EXCEPTION_TO_ERROR_CODE({ at::fill_( *tensor_handle_to_tensor_pointer(self), value ); ``` [ghstack-poisoned]

ghstack-source-id: f39bccc Pull Request resolved: #158974

janeyx99

cool thanks!

mikaylagawarecki · 2025-07-25T18:56:41Z

@pytorchbot merge

pytorchmergebot · 2025-07-25T18:58:47Z

Merge failed

Reason: HTTP Error 500: Internal Server Error

Details for Dev Infra team

Raised by workflow job

mikaylagawarecki · 2025-07-25T19:30:43Z

@pytorchbot merge

pytorchmergebot · 2025-07-25T19:36:09Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

) Adds `c_shim_aten.{h/cpp}` and use this for `fill_` This is the generated `c_shim_aten.cpp` for reference ```cpp // WARNING: THIS FILE IS AUTOGENERATED BY torchgen. DO NOT MODIFY BY HAND. // See https://github.com/pytorch/pytorch/blob/7e86a7c0155295539996e0cf422883571126073e/torchgen/gen.py#L2424-L2436 for details // This file corresponds to the aten_shimified_ops list in torchgen/aoti/fallback_ops.py #include <torch/csrc/inductor/aoti_torch/generated/c_shim_aten.h> #include <torch/csrc/inductor/aoti_torch/utils.h> #ifndef AT_PER_OPERATOR_HEADERS #include <ATen/Functions.h> #include <ATen/CompositeExplicitAutogradFunctions.h> #include <ATen/CompositeExplicitAutogradNonFunctionalFunctions.h> #include <ATen/CompositeImplicitAutogradFunctions.h> #else #include <ATen/ops/fill.h> #endif // AT_PER_OPERATOR_HEADERS using namespace torch::aot_inductor; AOTITorchError aoti_torch_aten_fill__Scalar(AtenTensorHandle self, double value) { AOTI_TORCH_CONVERT_EXCEPTION_TO_ERROR_CODE({ at::fill_( *tensor_handle_to_tensor_pointer(self), value ); ``` Pull Request resolved: #158974 Approved by: https://github.com/albanD, https://github.com/janeyx99

Enable generating generic c_shim that doesn't bypass dispatcher

7306923

[ghstack-poisoned]

mikaylagawarecki added a commit that referenced this pull request Jul 23, 2025

Enable generating generic c_shim that doesn't bypass dispatcher

a058f94

ghstack-source-id: 3cedfe3 Pull Request resolved: #158974

pytorch-bot bot added ciflow/inductor release notes: inductor (aoti) labels Jul 23, 2025

mikaylagawarecki marked this pull request as draft July 23, 2025 21:04

Update on "Enable generating generic c_shim that doesn't bypass dispa…

4841aca

…tcher" [ghstack-poisoned]

mikaylagawarecki added a commit that referenced this pull request Jul 24, 2025

Enable generating generic c_shim that doesn't bypass dispatcher

7eead98

ghstack-source-id: eb8ee19 Pull Request resolved: #158974

Update on "Enable generating generic c_shim that doesn't bypass dispa…

ebc2fee

…tcher" [ghstack-poisoned]

mikaylagawarecki added a commit that referenced this pull request Jul 24, 2025

Enable generating generic c_shim that doesn't bypass dispatcher

90cea3d

ghstack-source-id: e8d5697 Pull Request resolved: #158974

mikaylagawarecki requested a review from janeyx99 July 24, 2025 15:43

mikaylagawarecki marked this pull request as ready for review July 24, 2025 15:43

mikaylagawarecki added a commit that referenced this pull request Jul 24, 2025

Enable generating generic c_shim that doesn't bypass dispatcher

5315cfc

ghstack-source-id: d8c6959 Pull Request resolved: #158974

mikaylagawarecki commented Jul 24, 2025

View reviewed changes

janeyx99 reviewed Jul 24, 2025

View reviewed changes

albanD approved these changes Jul 24, 2025

View reviewed changes

torchgen/model.py Outdated Show resolved Hide resolved

mikaylagawarecki added a commit that referenced this pull request Jul 24, 2025

Enable generating generic c_shim that doesn't bypass dispatcher

277d420

ghstack-source-id: 0d7e2e0 Pull Request resolved: #158974

mikaylagawarecki added a commit that referenced this pull request Jul 25, 2025

Enable generating generic c_shim that doesn't bypass dispatcher

8a62591

ghstack-source-id: dd2b052 Pull Request resolved: #158974

mikaylagawarecki added a commit that referenced this pull request Jul 25, 2025

Enable generating generic c_shim that doesn't bypass dispatcher

c7d6e38

ghstack-source-id: a1a871f Pull Request resolved: #158974

mikaylagawarecki added a commit that referenced this pull request Jul 25, 2025

Enable generating generic c_shim that doesn't bypass dispatcher

35a7c8c

ghstack-source-id: d86f14a Pull Request resolved: #158974

mikaylagawarecki requested a review from janeyx99 July 25, 2025 15:33

mikaylagawarecki added a commit that referenced this pull request Jul 25, 2025

Enable generating generic c_shim that doesn't bypass dispatcher

7588975

ghstack-source-id: f39bccc Pull Request resolved: #158974

janeyx99 added the suppress-bc-linter Suppresses the failures of API backward-compatibility linter (Lint/bc_linter) label Jul 25, 2025

janeyx99 approved these changes Jul 25, 2025

View reviewed changes

pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Jul 25, 2025

pytorchmergebot added the merging label Jul 25, 2025

pytorchmergebot removed the merging label Jul 25, 2025

pytorchmergebot added the merging label Jul 25, 2025

pytorchmergebot closed this in e65ab9a Jul 25, 2025

pytorchmergebot added Merged and removed merging labels Jul 25, 2025

mikaylagawarecki mentioned this pull request Jul 22, 2025

[Tracker] Support flash attention fa3 ABI stable w/ libtorch #154908

Open

51 tasks

github-actions bot deleted the gh/mikaylagawarecki/328/head branch August 25, 2025 02:18

Enable generating generic c_shim that doesn't bypass dispatcher #158974

Enable generating generic c_shim that doesn't bypass dispatcher #158974

Uh oh!

Conversation

mikaylagawarecki commented Jul 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Jul 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/158974

⏳ 1 Pending, 2 Unrelated Failures

Uh oh!

github-actions bot commented Jul 23, 2025

Attention! PyTorch one of the C-stable API file was changed

Uh oh!

mikaylagawarecki Jul 24, 2025

Choose a reason for hiding this comment

Uh oh!

janeyx99 Jul 24, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

janeyx99 Jul 24, 2025

Choose a reason for hiding this comment

Uh oh!

mikaylagawarecki Jul 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

janeyx99 Jul 24, 2025

Choose a reason for hiding this comment

Uh oh!

janeyx99 Jul 24, 2025 • edited by mikaylagawarecki Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

albanD left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

janeyx99 left a comment

Choose a reason for hiding this comment

Uh oh!

mikaylagawarecki commented Jul 25, 2025

Uh oh!

pytorchmergebot commented Jul 25, 2025

Merge failed

Uh oh!

mikaylagawarecki commented Jul 25, 2025

Uh oh!

pytorchmergebot commented Jul 25, 2025

Merge started

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

mikaylagawarecki commented Jul 23, 2025 •

edited

Loading

pytorch-bot bot commented Jul 23, 2025 •

edited

Loading

mikaylagawarecki Jul 25, 2025 •

edited

Loading

janeyx99 Jul 24, 2025 •

edited by mikaylagawarecki

Loading