[Quant][FX] Lower QConvAdd2d for onednn backend #91153

leslie-fang-intel · 2022-12-20T03:55:03Z

Stack from ghstack (oldest at bottom):

Summary
Add quantization mappings for QConvAdd2d for int8 inference for onednn backend. The fusion and lowering is supported only in FX mode.

Test plan

python -m pytest test_quantization.py -k test_fuse_conv_bn_add_relu_onednn
python -m pytest test_quantization.py -k test_fuse_conv_bn_add_relu_by_default
python -m pytest test_quantization.py -k test_fuse_conv_bn_add_relu_lowering

cc @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10

[ghstack-poisoned]

pytorch-bot · 2022-12-20T03:55:06Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/91153

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 2cf5520:
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 [ghstack-poisoned]

**Summary** Add quantization mappings for QConvAdd2d for int8 inference for onednn backend. The fusion and lowering is supported only in FX mode. **Test plan** ``` python -m pytest test_quantization.py -k test_fuse_conv_bn_add_relu_onednn python -m pytest test_quantization.py -k test_fuse_conv_bn_add_relu_by_default python -m pytest test_quantization.py -k test_fuse_conv_bn_add_relu_lowering ``` cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 [ghstack-poisoned]

leslie-fang-intel · 2023-01-11T06:12:54Z

@jerryzh168 I have changed according to your comments. Could you help to take a look of this PR again?

ghstack-source-id: fb7521b Pull Request resolved: pytorch#91153

**Summary** Add quantization mappings for QConvAdd2d for int8 inference for onednn backend. The fusion and lowering is supported only in FX mode. **Test plan** ``` python -m pytest test_quantization.py -k test_fuse_conv_bn_add_relu_onednn python -m pytest test_quantization.py -k test_fuse_conv_bn_add_relu_by_default python -m pytest test_quantization.py -k test_fuse_conv_bn_add_relu_lowering ``` cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 [ghstack-poisoned]

leslie-fang-intel · 2023-01-26T02:23:41Z

Hi @jerryzh168 Is there any other comments to this PR? Could you help to take a look again?

**Summary** Add quantization mappings for QConvAdd2d for int8 inference for onednn backend. The fusion and lowering is supported only in FX mode. **Test plan** ``` python -m pytest test_quantization.py -k test_fuse_conv_bn_add_relu_onednn python -m pytest test_quantization.py -k test_fuse_conv_bn_add_relu_by_default python -m pytest test_quantization.py -k test_fuse_conv_bn_add_relu_lowering ``` cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 [ghstack-poisoned]

leslie-fang-intel · 2023-01-31T01:21:06Z

Hi @jerryzh168, Could you also take a review of this PR again?

jerryzh168 · 2023-01-31T01:44:16Z

test/quantization/core/test_quantized_module.py

-            # workaround in this PR to return from here, since the below lowering part enabled in next PR
-            # We will enable below check in next PR
-            return
+        class _FusedModule_two_input_args(torch.nn.intrinsic._FusedModule):


do we need this?

Yes, I think so. That's because original torch.nn.intrinsic._FusedModule only support one input.

jerryzh168 · 2023-01-31T01:45:07Z

test/quantization/fx/test_quantize_fx.py

+
+        options = itertools.product(
+            [True, False],  # with_bn
+            [False],  # with_relu


is relu supported? the name of the test mentioned relu, but we did not test it here

Oh, yes. The support of relu is added in the following up 2 PRs.

jerryzh168 · 2023-01-31T01:45:41Z

test/quantization/fx/test_quantize_fx.py

+    def test_fuse_conv_bn_add_relu_by_default(self):
+        options = itertools.product(
+            [True, False],  # with_bn
+            [False],  # with_relu


same question for relu

The support of relu has been added in the following up PRs.

jerryzh168 · 2023-01-31T01:47:47Z

torch/ao/quantization/fx/_lower_to_native_backend.py

 }

+# The difference between STATIC_LOWER_FUSED_MODULE_TWO_INPUTS_MAP and STATIC_LOWER_FUSED_MODULE_MAP:
+# The refer node inside STATIC_LOWER_FUSED_MODULE_TWO_INPUTS_MAP has 2 dq input nodes.


nit: dq input nodes --> inputs

Thanks for the comments and changed.

jerryzh168

lgtm, thanks!

**Summary** Add quantization mappings for QConvAdd2d for int8 inference for onednn backend. The fusion and lowering is supported only in FX mode. **Test plan** ``` python -m pytest test_quantization.py -k test_fuse_conv_bn_add_relu_onednn python -m pytest test_quantization.py -k test_fuse_conv_bn_add_relu_by_default python -m pytest test_quantization.py -k test_fuse_conv_bn_add_relu_lowering ``` cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 [ghstack-poisoned]

leslie-fang-intel · 2023-02-01T01:11:52Z

@pytorchbot merge

pytorchmergebot · 2023-02-01T01:14:03Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

[Quant][FX] Lower QConvAdd2d for onednn backend

5eb0224

[ghstack-poisoned]

pytorch-bot bot added the release notes: AO frontend label Dec 20, 2022

github-actions bot added the release notes: quantization release notes category label Dec 20, 2022

leslie-fang-intel requested review from Xia-Weiwen and jgong5 December 20, 2022 03:57

leslie-fang-intel marked this pull request as draft December 20, 2022 03:57

leslie-fang-intel changed the title ~~[Quant][FX] Lower QConvAdd2d for onednn backend~~ [WIP][Quant][FX] Lower QConvAdd2d for onednn backend Dec 20, 2022

leslie-fang-intel added ciflow/trunk Trigger trunk jobs on your pull request intel This tag is for PR from Intel labels Dec 20, 2022

pytorchbot added the open source label Dec 20, 2022

Update on "[WIP][Quant][FX] Lower QConvAdd2d for onednn backend"

b0caac8

cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 [ghstack-poisoned]

leslie-fang-intel added 2 commits December 20, 2022 15:11

Update on "[WIP][Quant][FX] Lower QConvAdd2d for onednn backend"

05e9705

cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 [ghstack-poisoned]

Update on "[WIP][Quant][FX] Lower QConvAdd2d for onednn backend"

6308ff5

cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 [ghstack-poisoned]

leslie-fang-intel added 5 commits December 20, 2022 16:44

Update on "[WIP][Quant][FX] Lower QConvAdd2d for onednn backend"

bddb6e2

cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 [ghstack-poisoned]

Update on "[WIP][Quant][FX] Lower QConvAdd2d for onednn backend"

0682977

cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 [ghstack-poisoned]

Update on "[WIP][Quant][FX] Lower QConvAdd2d for onednn backend"

9cb9b11

cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 [ghstack-poisoned]

Update on "[WIP][Quant][FX] Lower QConvAdd2d for onednn backend"

169dedb

cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 [ghstack-poisoned]

Update on "[WIP][Quant][FX] Lower QConvAdd2d for onednn backend"

2f21bac

cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 [ghstack-poisoned]

leslie-fang-intel requested review from jerryzh168 and removed request for z-a-f January 11, 2023 01:45

leslie-fang-intel added a commit to leslie-fang-intel/pytorch that referenced this pull request Jan 13, 2023

[Quant][FX] Lower QConvAdd2d for onednn backend

dbf5033

ghstack-source-id: fb7521b Pull Request resolved: pytorch#91153

leslie-fang-intel added a commit to leslie-fang-intel/pytorch that referenced this pull request Jan 26, 2023

[Quant][FX] Lower QConvAdd2d for onednn backend

461dbc2

ghstack-source-id: fb7521b Pull Request resolved: pytorch#91153

leslie-fang-intel added 8 commits January 26, 2023 10:35

jerryzh168 reviewed Jan 31, 2023

View reviewed changes

jerryzh168 approved these changes Jan 31, 2023

View reviewed changes

leslie-fang-intel added 2 commits January 31, 2023 11:03

pytorchmergebot added the Merged label Feb 1, 2023

pytorchmergebot closed this in ef4118e Feb 1, 2023

facebook-github-bot deleted the gh/leslie-fang-intel/15/head branch June 8, 2023 17:52

[Quant][FX] Lower QConvAdd2d for onednn backend #91153

[Quant][FX] Lower QConvAdd2d for onednn backend #91153

Uh oh!

Conversation

leslie-fang-intel commented Dec 20, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Dec 20, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/91153

✅ No Failures

Uh oh!

leslie-fang-intel commented Jan 11, 2023

Uh oh!

leslie-fang-intel commented Jan 26, 2023

Uh oh!

leslie-fang-intel commented Jan 31, 2023

Uh oh!

jerryzh168 Jan 31, 2023

Choose a reason for hiding this comment

Uh oh!

leslie-fang-intel Jan 31, 2023

Choose a reason for hiding this comment

Uh oh!

jerryzh168 Jan 31, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

leslie-fang-intel Jan 31, 2023

Choose a reason for hiding this comment

Uh oh!

jerryzh168 Jan 31, 2023

Choose a reason for hiding this comment

Uh oh!

leslie-fang-intel Jan 31, 2023

Choose a reason for hiding this comment

Uh oh!

jerryzh168 Jan 31, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

leslie-fang-intel Jan 31, 2023

Choose a reason for hiding this comment

Uh oh!

jerryzh168 left a comment

Choose a reason for hiding this comment

Uh oh!

leslie-fang-intel commented Feb 1, 2023

Uh oh!

pytorchmergebot commented Feb 1, 2023

Merge started

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

leslie-fang-intel commented Dec 20, 2022 •

edited

Loading

pytorch-bot bot commented Dec 20, 2022 •

edited

Loading

jerryzh168 Jan 31, 2023 •

edited

Loading

jerryzh168 Jan 31, 2023 •

edited

Loading