Skip to content

[Quant] Add fused conv2d_add op for onednn backend #90262

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
leslie-fang-intel wants to merge 23 commits into from
Closed

[Quant] Add fused conv2d_add op for onednn backend #90262

leslie-fang-intel wants to merge 23 commits into from

Conversation

@leslie-fang-intel
Copy link
Collaborator

@leslie-fang-intel leslie-fang-intel commented Dec 6, 2022 •
edited
Loading

Stack from ghstack (oldest at bottom):

Summary
Post op fusion can reduce data movement overhead and improve inference performance. This PR adds fused conv2d_add op for onednn backend, which will be used for int8 inference with onednn backend. Cannot call this op with other quantization backends otherwise an error is thrown.

Test Plan

python -m pytest test_quantization.py::TestQuantizedConv

cc @VitalyFedyunin @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10 @gujinghui @PenghuiCheng @jianyuh @min-jean-cho @yanbing-j @Guobing-Chen @Xia-Weiwen

@pytorch-bot pytorch-bot bot added the release notes: quantization release notes category label Dec 6, 2022
@leslie-fang-intel leslie-fang-intel mentioned this pull request Dec 6, 2022
Closed
@pytorch-bot
Copy link

pytorch-bot bot commented Dec 6, 2022 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/90262

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 8baef3e:
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@github-actions github-actions bot added module: cpu CPU specific problem (e.g., perf, algorithm) module: mkldnn Related to Intel IDEEP or oneDNN (a.k.a. mkldnn) integration labels Dec 6, 2022
@leslie-fang-intel leslie-fang-intel marked this pull request as draft December 6, 2022 06:26
@leslie-fang-intel leslie-fang-intel changed the title [Quant] Add fused conv_add op for onednn backend [WIP] [Quant] Add fused conv_add op for onednn backend Dec 6, 2022
@leslie-fang-intel
Update on "[WIP] [Quant] Add fused conv_add op for onednn backend"
8c2cd94 
cc VitalyFedyunin jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 gujinghui PenghuiCheng jianyuh min-jean-cho yanbing-j Guobing-Chen Xia-Weiwen

[ghstack-poisoned]
leslie-fang-intel added a commit that referenced this pull request Dec 6, 2022
@leslie-fang-intel
[Quant] Add fused conv_add op for onednn backend
aecd0dd 
ghstack-source-id: a7f5fb3
Pull Request resolved: #90262
@leslie-fang-intel
Update on "[WIP] [Quant] Add fused conv_add op for onednn backend"
3b1bfae 
cc VitalyFedyunin jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 gujinghui PenghuiCheng jianyuh min-jean-cho yanbing-j Guobing-Chen Xia-Weiwen

[ghstack-poisoned]
leslie-fang-intel added a commit that referenced this pull request Dec 7, 2022
@leslie-fang-intel
[Quant] Add fused conv_add op for onednn backend
43e9c8a 
ghstack-source-id: 324e97d
Pull Request resolved: #90262
@leslie-fang-intel leslie-fang-intel changed the title [WIP] [Quant] Add fused conv_add op for onednn backend [Quant] Add fused conv_add op for onednn backend Dec 7, 2022
@leslie-fang-intel leslie-fang-intel added intel This tag is for PR from Intel ciflow/trunk Trigger trunk jobs on your pull request labels Dec 7, 2022
@leslie-fang-intel leslie-fang-intel changed the title [Quant] Add fused conv_add op for onednn backend [Quant] Add fused conv2d_add op for onednn backend Dec 7, 2022
@leslie-fang-intel leslie-fang-intel mentioned this pull request Dec 7, 2022
Closed
This was referenced Dec 20, 2022
Closed
Closed
Closed
@leslie-fang-intel
Update on "[Quant] Add fused conv2d_add op for onednn backend"
e381622 
**Summary**
Post op fusion can reduce data movement overhead and improve inference performance. This PR adds fused `conv2d_add` op for onednn backend, which will be used for int8 inference with onednn backend. Cannot call this op with other quantization backends otherwise an error is thrown.

**Test Plan**
```
python -m pytest test_quantization.py::TestQuantizedConv
```

cc VitalyFedyunin jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 gujinghui PenghuiCheng jianyuh min-jean-cho yanbing-j Guobing-Chen Xia-Weiwen

[ghstack-poisoned]
@leslie-fang-intel
Update on "[Quant] Add fused conv2d_add op for onednn backend"
98fa382 
**Summary**
Post op fusion can reduce data movement overhead and improve inference performance. This PR adds fused `conv2d_add` op for onednn backend, which will be used for int8 inference with onednn backend. Cannot call this op with other quantization backends otherwise an error is thrown.

**Test Plan**
```
python -m pytest test_quantization.py::TestQuantizedConv
```

cc VitalyFedyunin jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 gujinghui PenghuiCheng jianyuh min-jean-cho yanbing-j Guobing-Chen Xia-Weiwen

[ghstack-poisoned]
leslie-fang-intel added a commit to leslie-fang-intel/pytorch that referenced this pull request Dec 20, 2022
@leslie-fang-intel
[Quant] Add fused conv_add op for onednn backend
3a420e1 
ghstack-source-id: 2b22fc9
Pull Request resolved: pytorch#90262
@leslie-fang-intel
Update on "[Quant] Add fused conv2d_add op for onednn backend"
8fab3df 
**Summary**
Post op fusion can reduce data movement overhead and improve inference performance. This PR adds fused `conv2d_add` op for onednn backend, which will be used for int8 inference with onednn backend. Cannot call this op with other quantization backends otherwise an error is thrown.

**Test Plan**
```
python -m pytest test_quantization.py::TestQuantizedConv
```

cc VitalyFedyunin jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 gujinghui PenghuiCheng jianyuh min-jean-cho yanbing-j Guobing-Chen Xia-Weiwen

[ghstack-poisoned]
@leslie-fang-intel
Update on "[Quant] Add fused conv2d_add op for onednn backend"
9203469 
**Summary**
Post op fusion can reduce data movement overhead and improve inference performance. This PR adds fused `conv2d_add` op for onednn backend, which will be used for int8 inference with onednn backend. Cannot call this op with other quantization backends otherwise an error is thrown.

**Test Plan**
```
python -m pytest test_quantization.py::TestQuantizedConv
```

cc VitalyFedyunin jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 gujinghui PenghuiCheng jianyuh min-jean-cho yanbing-j Guobing-Chen Xia-Weiwen

[ghstack-poisoned]
@leslie-fang-intel
Update on "[Quant] Add fused conv2d_add op for onednn backend"
523001c 
**Summary**
Post op fusion can reduce data movement overhead and improve inference performance. This PR adds fused `conv2d_add` op for onednn backend, which will be used for int8 inference with onednn backend. Cannot call this op with other quantization backends otherwise an error is thrown.

**Test Plan**
```
python -m pytest test_quantization.py::TestQuantizedConv
```

cc VitalyFedyunin jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 gujinghui PenghuiCheng jianyuh min-jean-cho yanbing-j Guobing-Chen Xia-Weiwen

[ghstack-poisoned]
@leslie-fang-intel
Update on "[Quant] Add fused conv2d_add op for onednn backend"
6a72e4e 
**Summary**
Post op fusion can reduce data movement overhead and improve inference performance. This PR adds fused `conv2d_add` op for onednn backend, which will be used for int8 inference with onednn backend. Cannot call this op with other quantization backends otherwise an error is thrown.

**Test Plan**
```
python -m pytest test_quantization.py::TestQuantizedConv
```

cc VitalyFedyunin jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 gujinghui PenghuiCheng jianyuh min-jean-cho yanbing-j Guobing-Chen Xia-Weiwen

[ghstack-poisoned]
jerryzh168
jerryzh168 reviewed Jan 9, 2023
View reviewed changes
test/quantization/core/test_quantized_op.py Outdated
Y_zero_point=st.integers(0, 4),
use_bias=st.booleans(),
use_relu=st.booleans(),
post_op=st.sampled_from(["none", "relu"]),
Copy link
Contributor

@jerryzh168 jerryzh168 Jan 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can be a separate PR, but might make sense to split the conv and conv_relu test as well

Copy link
Collaborator Author

@leslie-fang-intel leslie-fang-intel Jan 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the suggestions, I have split the conv and conv_relu test and so as the other similar test cases.

jerryzh168
jerryzh168 reviewed Jan 9, 2023
View reviewed changes
test/quantization/core/test_quantized_op.py Outdated
Y_zero_point=st.sampled_from([0]),
use_bias=st.booleans(),
use_relu=st.booleans(),
post_op=st.sampled_from(["none", "relu"]),
Copy link
Contributor

@jerryzh168 jerryzh168 Jan 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here

Copy link
Collaborator Author

@leslie-fang-intel leslie-fang-intel Jan 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the suggestions, split them into separate tests.

jerryzh168
jerryzh168 reviewed Jan 9, 2023
View reviewed changes
test/quantization/core/test_quantized_op.py Outdated
Comment on lines 4839 to 4840
if post_op == "add":
qconv = torch.ops.quantized.conv2d_add
Copy link
Contributor

@jerryzh168 jerryzh168 Jan 9, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if this is only "add" we can remove the post_op argument and also this check

Copy link
Collaborator Author

@leslie-fang-intel leslie-fang-intel Jan 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the suggestions, I have removed the post_op argument and the check. In next PR, I will put conv2d_add_relu into a separate test.

@leslie-fang-intel
Update on "[Quant] Add fused conv2d_add op for onednn backend"
74ca35a 
**Summary**
Post op fusion can reduce data movement overhead and improve inference performance. This PR adds fused `conv2d_add` op for onednn backend, which will be used for int8 inference with onednn backend. Cannot call this op with other quantization backends otherwise an error is thrown.

**Test Plan**
```
python -m pytest test_quantization.py::TestQuantizedConv
```

cc VitalyFedyunin jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 gujinghui PenghuiCheng jianyuh min-jean-cho yanbing-j Guobing-Chen Xia-Weiwen

[ghstack-poisoned]
leslie-fang-intel added a commit to leslie-fang-intel/pytorch that referenced this pull request Jan 13, 2023
@leslie-fang-intel
[Quant] Add fused conv_add op for onednn backend
15320c0 
ghstack-source-id: bb2010e
Pull Request resolved: pytorch#90262
leslie-fang-intel added a commit to leslie-fang-intel/pytorch that referenced this pull request Jan 26, 2023
@leslie-fang-intel
[Quant] Add fused conv_add op for onednn backend
5b211b5 
ghstack-source-id: bb2010e
Pull Request resolved: pytorch#90262
@leslie-fang-intel
Update on "[Quant] Add fused conv2d_add op for onednn backend"
55678e3 
**Summary**
Post op fusion can reduce data movement overhead and improve inference performance. This PR adds fused `conv2d_add` op for onednn backend, which will be used for int8 inference with onednn backend. Cannot call this op with other quantization backends otherwise an error is thrown.

**Test Plan**
```
python -m pytest test_quantization.py::TestQuantizedConv
```

cc VitalyFedyunin jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 gujinghui PenghuiCheng jianyuh min-jean-cho yanbing-j Guobing-Chen Xia-Weiwen

[ghstack-poisoned]
@leslie-fang-intel
Update on "[Quant] Add fused conv2d_add op for onednn backend"
8ff2747 
**Summary**
Post op fusion can reduce data movement overhead and improve inference performance. This PR adds fused `conv2d_add` op for onednn backend, which will be used for int8 inference with onednn backend. Cannot call this op with other quantization backends otherwise an error is thrown.

**Test Plan**
```
python -m pytest test_quantization.py::TestQuantizedConv
```

cc VitalyFedyunin jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 gujinghui PenghuiCheng jianyuh min-jean-cho yanbing-j Guobing-Chen Xia-Weiwen

[ghstack-poisoned]
@leslie-fang-intel
Update on "[Quant] Add fused conv2d_add op for onednn backend"
44786a6 
**Summary**
Post op fusion can reduce data movement overhead and improve inference performance. This PR adds fused `conv2d_add` op for onednn backend, which will be used for int8 inference with onednn backend. Cannot call this op with other quantization backends otherwise an error is thrown.

**Test Plan**
```
python -m pytest test_quantization.py::TestQuantizedConv
```

cc VitalyFedyunin jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 gujinghui PenghuiCheng jianyuh min-jean-cho yanbing-j Guobing-Chen Xia-Weiwen

[ghstack-poisoned]
@leslie-fang-intel
Update on "[Quant] Add fused conv2d_add op for onednn backend"
5590d28 
**Summary**
Post op fusion can reduce data movement overhead and improve inference performance. This PR adds fused `conv2d_add` op for onednn backend, which will be used for int8 inference with onednn backend. Cannot call this op with other quantization backends otherwise an error is thrown.

**Test Plan**
```
python -m pytest test_quantization.py::TestQuantizedConv
```

cc VitalyFedyunin jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 gujinghui PenghuiCheng jianyuh min-jean-cho yanbing-j Guobing-Chen Xia-Weiwen

[ghstack-poisoned]
@leslie-fang-intel
Update on "[Quant] Add fused conv2d_add op for onednn backend"
8baef3e 
**Summary**
Post op fusion can reduce data movement overhead and improve inference performance. This PR adds fused `conv2d_add` op for onednn backend, which will be used for int8 inference with onednn backend. Cannot call this op with other quantization backends otherwise an error is thrown.

**Test Plan**
```
python -m pytest test_quantization.py::TestQuantizedConv
```

cc VitalyFedyunin jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 gujinghui PenghuiCheng jianyuh min-jean-cho yanbing-j Guobing-Chen Xia-Weiwen

[ghstack-poisoned]
@leslie-fang-intel
Copy link
Collaborator Author

leslie-fang-intel commented Jan 28, 2023

@pytorchbot merge

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Jan 28, 2023

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@facebook-github-bot facebook-github-bot deleted the gh/leslie-fang-intel/4/head branch June 8, 2023 17:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jerryzh168 jerryzh168 jerryzh168 approved these changes

@jgong5 jgong5 jgong5 approved these changes

@z-a-f z-a-f Awaiting requested review from z-a-f

@salilsdesai salilsdesai Awaiting requested review from salilsdesai

@kimishpatel kimishpatel Awaiting requested review from kimishpatel

@digantdesai digantdesai Awaiting requested review from digantdesai

@jianyuh jianyuh Awaiting requested review from jianyuh

@Xia-Weiwen Xia-Weiwen Awaiting requested review from Xia-Weiwen

@XiaobingSuper XiaobingSuper Awaiting requested review from XiaobingSuper

Assignees

No one assigned

Labels

ciflow/trunk Trigger trunk jobs on your pull request intel This tag is for PR from Intel Merged module: cpu CPU specific problem (e.g., perf, algorithm) module: mkldnn Related to Intel IDEEP or oneDNN (a.k.a. mkldnn) integration open source release notes: quantization release notes category

Projects

PyTorch Intel
Status: Done

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@leslie-fang-intel @jerryzh168 @pytorchmergebot @jgong5 @pytorchbot