[Inductor][Quant]Support qconv_pointwise.tensor and qconv2d_pointwise.binary_tensor by jiayisunx · Pull Request #166608 · pytorch/pytorch

jiayisunx · 2025-10-30T03:18:31Z

Stack from ghstack (oldest at bottom):

-> [Inductor][Quant]Support qconv_pointwise.tensor and qconv2d_pointwise.binary_tensor #166608

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @ipiszy @chenyang78 @kadeng @muchulee8 @amjames @chauhang @aakhundov @coconutruben @jataylo @mlazos

pytorch-bot · 2025-10-30T03:18:35Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/166608

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (1 Unrelated Failure)

As of commit 9c168c4 with merge base aa504d4 ():

UNSTABLE - The following job is marked as unstable, possibly due to flakiness on trunk:

trunk / linux-jammy-py3-clang12-executorch / test (executorch, 1, 1, linux.2xlarge, unstable) (gh) (#166072)
backends/xnnpack/test/recipes/test_xnnpack_recipes.py::TestXnnpackRecipes::test_int8_static_quant_recipe

This comment was automatically generated by Dr. CI and updates every 15 minutes.

ghstack-source-id: b3adc3a Pull-Request: #166608

[ghstack-poisoned]

ghstack-source-id: 5b8f2a8 Pull-Request: #166608

[ghstack-poisoned]

…e.binary_tensor ghstack-source-id: 74c9809 Pull-Request: #166608

[ghstack-poisoned]

…e.binary_tensor ghstack-source-id: f8c216a Pull-Request: #166608

[ghstack-poisoned]

…e.binary_tensor ghstack-source-id: 2788dfb Pull-Request: #166608

[ghstack-poisoned]

…e.binary_tensor ghstack-source-id: 00a23ea Pull-Request: #166608

…e.binary_tensor ghstack-source-id: d7ddd94 Pull-Request: #166608

…e.binary_tensor ghstack-source-id: b8a2aa7 Pull-Request: #166608

…e.binary_tensor ghstack-source-id: aa58157 Pull-Request: #166608

[ghstack-poisoned]

…e.binary_tensor ghstack-source-id: 9c081ce Pull-Request: #166608

[ghstack-poisoned]

test/inductor/test_mkldnn_pattern_matcher.py

Xia-Weiwen · 2025-11-18T08:11:49Z

torch/_inductor/fx_passes/quantization.py

                "none", None, "none", [], ""
            ): generate_pattern_with_output_quant(
-                get_qconv_pt2e_pattern(1),
+                get_qconv_pt2e_pattern(users=1),


Why don't we need to consider the tensor variant here?

These fusions have migrated to torchao, so I will make more changes to this part directly in torchao.

Then is this change needed? Does it work if we remove this change and depend on torchao's pass?

This change is needed because _register_quantization_lowerings has not been migrated to torchao, and all changes to this file are to modify _register_quantization_lowerings to support fp8.

torch/_inductor/mkldnn_ir.py

…e.binary_tensor ghstack-source-id: 282519c Pull-Request: #166608

[ghstack-poisoned]

…e.binary_tensor ghstack-source-id: db29fcc Pull-Request: #166608

[ghstack-poisoned]

Xia-Weiwen · 2025-11-19T02:43:14Z

torch/_inductor/fx_passes/quantization.py

                "none", None, "none", [], ""
            ): generate_pattern_with_output_quant(
-                get_qconv_pt2e_pattern(1),
+                get_qconv_pt2e_pattern(users=1),


If we always use the tensor variant now and get_qconv_pt2e_pattern returns the default variant here, will the fusion pass will work?

For int8, the qconv pattern here is always the qconv_pointwise.default

Xia-Weiwen · 2025-11-19T06:01:24Z

torch/_inductor/mkldnn_ir.py

        qw: "TensorBox",  # qw
        w_scale: "TensorBox",
-        w_zero_point: "TensorBox",
+        w_zero_point,


Is this change needed?

Yes, in order to pass python type checking.

Xia-Weiwen · 2025-11-19T06:01:33Z

torch/_inductor/mkldnn_lowerings.py

            packed_weight: TensorBox,
            w_scale: TensorBox,
-            w_zp: TensorBox,
+            w_zp,


Xia-Weiwen · 2025-11-19T06:01:44Z

torch/_inductor/mkldnn_lowerings.py

            packed_weight: TensorBox,
            w_scale: TensorBox,
-            w_zp: TensorBox,
+            w_zp,


jiayisunx · 2025-11-25T01:35:47Z

@pytorchbot merge

pytorchmergebot · 2025-11-25T01:37:35Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

….binary_tensor (pytorch#166608) Pull Request resolved: pytorch#166608 Approved by: https://github.com/Xia-Weiwen, https://github.com/mingfeima, https://github.com/jansel

….binary_tensor (#166608) Pull Request resolved: #166608 Approved by: https://github.com/Xia-Weiwen, https://github.com/mingfeima, https://github.com/jansel

….binary_tensor (pytorch#166608) Pull Request resolved: pytorch#166608 Approved by: https://github.com/Xia-Weiwen, https://github.com/mingfeima, https://github.com/jansel

pytorch-bot bot added ciflow/inductor module: inductor labels Oct 30, 2025

jiayisunx added a commit that referenced this pull request Oct 30, 2025

[Quant][PT2E][X86]Support qconv_pointwise.tensor

c450571

ghstack-source-id: b3adc3a Pull-Request: #166608

jiayisunx marked this pull request as draft October 30, 2025 03:22

jiayisunx added a commit that referenced this pull request Oct 30, 2025

[Quant][PT2E][X86]Support qconv_pointwise.tensor

ccfe222

ghstack-source-id: b3adc3a Pull-Request: #166608

Update

ad592ed

[ghstack-poisoned]

pytorchbot added the open source label Oct 30, 2025

jiayisunx added ciflow/trunk Trigger trunk jobs on your pull request release notes: inductor labels Nov 3, 2025

jiayisunx added a commit that referenced this pull request Nov 4, 2025

[Quant][PT2E][X86]Support qconv_pointwise.tensor

b5f376e

ghstack-source-id: 5b8f2a8 Pull-Request: #166608

Update

3dcf4c9

[ghstack-poisoned]

jiayisunx added a commit that referenced this pull request Nov 5, 2025

[Quant][PT2E][X86]Support qconv_pointwise.tensor and qconv2d_pointwis…

e2536a6

…e.binary_tensor ghstack-source-id: 74c9809 Pull-Request: #166608

Update

e130a78

[ghstack-poisoned]

jiayisunx changed the title ~~[Quant][PT2E][X86]Support qconv_pointwise.tensor~~ [Inductor][Quant]Support qconv_pointwise.tensor and qconv2d_pointwise.binary_tensor Nov 10, 2025

jiayisunx added a commit that referenced this pull request Nov 12, 2025

[Quant][PT2E][X86]Support qconv_pointwise.tensor and qconv2d_pointwis…

fb60950

…e.binary_tensor ghstack-source-id: f8c216a Pull-Request: #166608

Update

5e21348

[ghstack-poisoned]

jiayisunx added a commit that referenced this pull request Nov 12, 2025

[Quant][PT2E][X86]Support qconv_pointwise.tensor and qconv2d_pointwis…

6c7e6ba

…e.binary_tensor ghstack-source-id: 2788dfb Pull-Request: #166608

Update

649a16f

[ghstack-poisoned]

jiayisunx added a commit that referenced this pull request Nov 14, 2025

[Quant][PT2E][X86]Support qconv_pointwise.tensor and qconv2d_pointwis…

70add1f

…e.binary_tensor ghstack-source-id: 00a23ea Pull-Request: #166608

jiayisunx added a commit that referenced this pull request Nov 14, 2025

[Quant][PT2E][X86]Support qconv_pointwise.tensor and qconv2d_pointwis…

9049a2c

…e.binary_tensor ghstack-source-id: d7ddd94 Pull-Request: #166608

jiayisunx added a commit that referenced this pull request Nov 14, 2025

[Quant][PT2E][X86]Support qconv_pointwise.tensor and qconv2d_pointwis…

547556f

…e.binary_tensor ghstack-source-id: b8a2aa7 Pull-Request: #166608

jiayisunx added a commit that referenced this pull request Nov 14, 2025

[Quant][PT2E][X86]Support qconv_pointwise.tensor and qconv2d_pointwis…

dfbc7c4

…e.binary_tensor ghstack-source-id: b8a2aa7 Pull-Request: #166608

jiayisunx added a commit that referenced this pull request Nov 14, 2025

[Quant][PT2E][X86]Support qconv_pointwise.tensor and qconv2d_pointwis…

fc6ca8e

…e.binary_tensor ghstack-source-id: aa58157 Pull-Request: #166608

jiayisunx added 4 commits November 14, 2025 15:27

Update

6c80205

[ghstack-poisoned]

Update

73643e5

[ghstack-poisoned]

Update

8252d02

[ghstack-poisoned]

Update

5bf89d7

[ghstack-poisoned]

jiayisunx added a commit that referenced this pull request Nov 17, 2025

[Quant][PT2E][X86]Support qconv_pointwise.tensor and qconv2d_pointwis…

3d451bf

…e.binary_tensor ghstack-source-id: 9c081ce Pull-Request: #166608

jiayisunx requested a review from Xia-Weiwen November 17, 2025 05:11

Update

a788461

[ghstack-poisoned]

Xia-Weiwen reviewed Nov 18, 2025

View reviewed changes

jiayisunx added a commit that referenced this pull request Nov 18, 2025

[Quant][PT2E][X86]Support qconv_pointwise.tensor and qconv2d_pointwis…

863ae8b

…e.binary_tensor ghstack-source-id: 282519c Pull-Request: #166608

Update

6aa4954

[ghstack-poisoned]

jiayisunx requested a review from Xia-Weiwen November 19, 2025 01:28

jiayisunx added a commit that referenced this pull request Nov 19, 2025

[Quant][PT2E][X86]Support qconv_pointwise.tensor and qconv2d_pointwis…

38803ab

…e.binary_tensor ghstack-source-id: db29fcc Pull-Request: #166608

Update

9c168c4

[ghstack-poisoned]

Xia-Weiwen reviewed Nov 19, 2025

View reviewed changes

jiayisunx requested a review from Xia-Weiwen November 19, 2025 05:21

Xia-Weiwen approved these changes Nov 19, 2025

View reviewed changes

jiayisunx requested a review from mingfeima November 24, 2025 02:34

mingfeima approved these changes Nov 24, 2025

View reviewed changes

mingfeima marked this pull request as ready for review November 24, 2025 02:40

jiayisunx requested a review from jansel November 24, 2025 05:53

jansel approved these changes Nov 24, 2025

View reviewed changes

pytorchmergebot added the merging label Nov 25, 2025

pytorchmergebot added the Merged label Nov 25, 2025

pytorchmergebot closed this in b3bc797 Nov 25, 2025

pytorchmergebot removed the merging label Nov 25, 2025

github-actions bot deleted the gh/jiayisunx/87/head branch December 25, 2025 02:19

Conversation

jiayisunx commented Oct 30, 2025 • edited by pytorch-bot bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/166608

✅ You can merge normally! (1 Unrelated Failure)

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jiayisunx Nov 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jiayisunx commented Nov 25, 2025

Uh oh!

pytorchmergebot commented Nov 25, 2025

Merge started

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

jiayisunx commented Oct 30, 2025 •

edited by pytorch-bot bot

Loading

pytorch-bot bot commented Oct 30, 2025 •

edited

Loading

jiayisunx Nov 19, 2025 •

edited

Loading