[quant][graphmode][fix] dequantize propagation for {add/mul}_scalar #40596
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Summary:
Previously the fusion patterns for {add/mul}_scalar is inconsistent since the op pattern
produces a non-quantized tensor and the op replacement graph produces a quantized tensor
Test Plan:
Reviewers:
Subscribers:
Tasks:
Tags:
[ghstack-poisoned]
facebook-github-bot
added
the
oncall: jit
|
|
||
| def forward(self, x): | ||
| x = self.conv(x) | ||
| # add_scalar |
Contributor
There was a problem hiding this comment.
I thought this was already supported. What was missing?
Contributor
Author
There was a problem hiding this comment.
the fusion pattern is inconsistent
jerryzh168
commented
Jun 25, 2020
Comment on lines
508
to
517
| std::string add_scalar = R"( | ||
| graph(%a_quant, %b_scalar, %alpha): | ||
| %a_dequant = aten::dequantize(%a_quant) | ||
| %r = aten::add(%a_dequant, %b_scalar, %alpha) | ||
| return (%r) )"; | ||
|
|
||
| std::string quantized_add_scalar = R"( | ||
| graph(%a_quant, %b_scalar, %alpha): | ||
| %r = quantized::add_scalar(%a_quant, %b_scalar) | ||
| return (%r) )"; |
Contributor
Author
There was a problem hiding this comment.
original pattern produces a float tensor and quantized pattern produces a quantized tensor
Contributor
There was a problem hiding this comment.
So is it correct to say that the new pattern is: deq->add-> = quantize_add->deq?
Contributor
Author
There was a problem hiding this comment.
new pattern is:
dq - aten::add -quant --> quantized::add_scalar
💊 CI failures summary and remediationsAs of commit 52b3f05 (more details on the Dr. CI page):
ci.pytorch.org: 1 failedThis comment was automatically generated by Dr. CI (expand for details).Follow this link to opt-out of these comments for your Pull Requests.Please report bugs/suggestions on the GitHub issue tracker or post in the (internal) Dr. CI Users group. This comment has been revised 12 times. |
raghuramank100
approved these changes
Jun 26, 2020
…l}_scalar"
Summary:
Previously the fusion patterns for {add/mul}_scalar is inconsistent since the op pattern
produces a non-quantized tensor and the op replacement graph produces a quantized tensor
Test Plan:
Reviewers:
Subscribers:
Tasks:
Tags:
[ghstack-poisoned]
…l}_scalar"
Summary:
Previously the fusion patterns for {add/mul}_scalar is inconsistent since the op pattern
produces a non-quantized tensor and the op replacement graph produces a quantized tensor
Test Plan:
Reviewers:
Subscribers:
Tasks:
Tags:
Differential Revision: [D22251072](https://our.internmc.facebook.com/intern/diff/D22251072)
[ghstack-poisoned]
Contributor
|
This pull request has been merged in e3a9768. |
1 similar comment
Contributor
|
This pull request has been merged in e3a9768. |
jerryzh168
added a commit
to jerryzh168/pytorch
that referenced
this pull request
Jun 26, 2020
…ytorch#40596) Summary: Pull Request resolved: pytorch#40596 Previously the fusion patterns for {add/mul}_scalar is inconsistent since the op pattern produces a non-quantized tensor and the op replacement graph produces a quantized tensor Test Plan: Imported from OSS Differential Revision: D22251072 fbshipit-source-id: e16eb92cf6611578cca1ed8ebde961f8d0610137
jerryzh168
added a commit
that referenced
this pull request
Jun 29, 2020
…40596) Summary: Pull Request resolved: #40596 Previously the fusion patterns for {add/mul}_scalar is inconsistent since the op pattern produces a non-quantized tensor and the op replacement graph produces a quantized tensor Test Plan: Imported from OSS Differential Revision: D22251072 fbshipit-source-id: e16eb92cf6611578cca1ed8ebde961f8d0610137
jerryzh168
added a commit
that referenced
this pull request
Jul 2, 2020
…40596) Summary: Pull Request resolved: #40596 Previously the fusion patterns for {add/mul}_scalar is inconsistent since the op pattern produces a non-quantized tensor and the op replacement graph produces a quantized tensor Test Plan: Imported from OSS Differential Revision: D22251072 fbshipit-source-id: e16eb92cf6611578cca1ed8ebde961f8d0610137
malfet
pushed a commit
that referenced
this pull request
Jul 7, 2020
… aten::repeat (#40933) * [quant][graphmode][fix] dequantize propagation for {add/mul}_scalar (#40596) Summary: Pull Request resolved: #40596 Previously the fusion patterns for {add/mul}_scalar is inconsistent since the op pattern produces a non-quantized tensor and the op replacement graph produces a quantized tensor Test Plan: Imported from OSS Differential Revision: D22251072 fbshipit-source-id: e16eb92cf6611578cca1ed8ebde961f8d0610137 * [quant][graphmode] Support quantization for `aten::apend` (#40743) Summary: Pull Request resolved: #40743 `aten::append` modifies input inplace and the output is ignored, these ops are not supported right now, so we'll need to first make `aten::append` non-inplace by change ``` ignored = aten::append(list, x) ``` to ``` x_list = aten::ListConstruct(x) result = aten::add(list, x_list) ``` and then quantize the aten::add instead. Test Plan: TestQuantizeJitOps.test_general_shape_ops Imported from OSS Differential Revision: D22302151 fbshipit-source-id: 931000388e7501e9dd17bec2fad8a96b71a5efc5
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Stack from ghstack:
Summary:
Previously the fusion patterns for {add/mul}_scalar is inconsistent since the op pattern
produces a non-quantized tensor and the op replacement graph produces a quantized tensor
Test Plan:
Reviewers:
Subscribers:
Tasks:
Tags:
Differential Revision: D22251072