[quant][fx] Allow incrementally remove the items in quantization_patterns.py #74210
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…erns.py Summary: This PR added a codepath for getting patterns (quantize handlers) for the backend_config_dict for native backend when backend_config_dict is None. This would allow us to incrementally define the backend_config_dict for pytorch native backend and gradually remove the entries in quantization_patterns.py Test Plan: python test/test_quantization.py TestQuantizeFx python test/test_quantization.py TestQuantizeFxOps Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]
CI Flow Status⚛️ CI FlowRuleset - Version:
|
jerryzh168
added a commit
that referenced
this pull request
Mar 15, 2022
…erns.py Summary: This PR added a codepath for getting patterns (quantize handlers) for the backend_config_dict for native backend when backend_config_dict is None. This would allow us to incrementally define the backend_config_dict for pytorch native backend and gradually remove the entries in quantization_patterns.py Test Plan: python test/test_quantization.py TestQuantizeFx python test/test_quantization.py TestQuantizeFxOps Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 92f0412 Pull Request resolved: #74210
Contributor
🔗 Helpful links
💊 CI failures summary and remediationsAs of commit 73b4332 (more details on the Dr. CI page): 💚 💚 Looks good so far! There are no failures yet. 💚 💚 This comment was automatically generated by Dr. CI (expand for details).Please report bugs/suggestions to the (internal) Dr. CI Users group. |
jerryzh168
requested review from
andrewor14 and
vkuzo
and removed request for
vkuzo
…zation_patterns.py" Summary: This PR added a codepath for getting patterns (quantize handlers) for the backend_config_dict for native backend when backend_config_dict is None. This would allow us to incrementally define the backend_config_dict for pytorch native backend and gradually remove the entries in quantization_patterns.py Test Plan: python test/test_quantization.py TestQuantizeFx python test/test_quantization.py TestQuantizeFxOps Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]
jerryzh168
added a commit
that referenced
this pull request
Mar 15, 2022
…erns.py Summary: This PR added a codepath for getting patterns (quantize handlers) for the backend_config_dict for native backend when backend_config_dict is None. This would allow us to incrementally define the backend_config_dict for pytorch native backend and gradually remove the entries in quantization_patterns.py Test Plan: python test/test_quantization.py TestQuantizeFx python test/test_quantization.py TestQuantizeFxOps Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 124a9b9 Pull Request resolved: #74210
vkuzo
reviewed
Mar 15, 2022
| weighted_op_int8_dtype_config, | ||
| ], | ||
| # the root module for the pattern, used to query the reference quantized module | ||
| # e.g. for a (torch.nn.ReLU, torch.nn.Linear) pattern, the root will be torch.nn.Linear |
Contributor
There was a problem hiding this comment.
are we considering defining things as (Linear, ReLU) instead of (ReLU, Linear) in the user facing configuration code?
Contributor
Author
There was a problem hiding this comment.
yes, that's the plan, we'll do that a bit later but before the beta release
vkuzo
approved these changes
Mar 15, 2022
Contributor
Author
|
@jerryzh168 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator. |
facebook-github-bot
pushed a commit
that referenced
this pull request
Mar 16, 2022
…erns.py (#74210) Summary: Pull Request resolved: #74210 This PR added a codepath for getting patterns (quantize handlers) for the backend_config_dict for native backend when backend_config_dict is None. This would allow us to incrementally define the backend_config_dict for pytorch native backend and gradually remove the entries in quantization_patterns.py Test Plan: python test/test_quantization.py TestQuantizeFx python test/test_quantization.py TestQuantizeFxOps Imported from OSS Reviewed By: dzdang Differential Revision: D34899783 fbshipit-source-id: 7f31292948d7fc4566e51e175b41511f52d0a880
Contributor
|
Hey @jerryzh168. |
jerryzh168
added a commit
that referenced
this pull request
Mar 23, 2022
…e native backend config" Summary: Following #74210, this PR adds the support for some ops using the DefaultNodeQuantizeHandler in the backend_config_dict defintion for pytorch native backend TODO: There is still a few ops we didn't handle with backend_config_dict path: gelu and softmax, need to discuss if we still need them, if so we can change the test to use backend_config_dict and remove the DefaultNodeQuantizeHandler after that Test Plan: python test/test_quantization.py TestQuantizeFxOps Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D35071437](https://our.internmc.facebook.com/intern/diff/D35071437) [ghstack-poisoned]
jerryzh168
added a commit
that referenced
this pull request
Mar 23, 2022
… config" Summary: Following #74210, this PR adds the support for some ops using the DefaultNodeQuantizeHandler in the backend_config_dict defintion for pytorch native backend TODO: There is still a few ops we didn't handle with backend_config_dict path: gelu and softmax, need to discuss if we still need them, if so we can change the test to use backend_config_dict and remove the DefaultNodeQuantizeHandler after that Test Plan: python test/test_quantization.py TestQuantizeFxOps Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D35071437](https://our.internmc.facebook.com/intern/diff/D35071437) [ghstack-poisoned]
jerryzh168
added a commit
that referenced
this pull request
Mar 23, 2022
…e native backend config" Summary: Following #74210, this PR adds the support for some ops using the DefaultNodeQuantizeHandler in the backend_config_dict defintion for pytorch native backend TODO: There is still a few ops we didn't handle with backend_config_dict path: gelu and softmax, need to discuss if we still need them, if so we can change the test to use backend_config_dict and remove the DefaultNodeQuantizeHandler after that Test Plan: python test/test_quantization.py TestQuantizeFxOps Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D35071437](https://our.internmc.facebook.com/intern/diff/D35071437) [ghstack-poisoned]
jerryzh168
added a commit
that referenced
this pull request
Mar 23, 2022
… config" Summary: Following #74210, this PR adds the support for some ops using the DefaultNodeQuantizeHandler in the backend_config_dict defintion for pytorch native backend TODO: There is still a few ops we didn't handle with backend_config_dict path: gelu and softmax, need to discuss if we still need them, if so we can change the test to use backend_config_dict and remove the DefaultNodeQuantizeHandler after that Test Plan: python test/test_quantization.py TestQuantizeFxOps Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D35071437](https://our.internmc.facebook.com/intern/diff/D35071437) [ghstack-poisoned]
jerryzh168
added a commit
that referenced
this pull request
Mar 23, 2022
…e native backend config" Summary: Following #74210, this PR adds the support for some ops using the DefaultNodeQuantizeHandler in the backend_config_dict defintion for pytorch native backend TODO: There is still a few ops we didn't handle with backend_config_dict path: gelu and softmax, need to discuss if we still need them, if so we can change the test to use backend_config_dict and remove the DefaultNodeQuantizeHandler after that Test Plan: python test/test_quantization.py TestQuantizeFxOps Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D35071437](https://our.internmc.facebook.com/intern/diff/D35071437) [ghstack-poisoned]
jerryzh168
added a commit
that referenced
this pull request
Mar 23, 2022
… config" Summary: Following #74210, this PR adds the support for some ops using the DefaultNodeQuantizeHandler in the backend_config_dict defintion for pytorch native backend TODO: There is still a few ops we didn't handle with backend_config_dict path: gelu and softmax, need to discuss if we still need them, if so we can change the test to use backend_config_dict and remove the DefaultNodeQuantizeHandler after that Test Plan: python test/test_quantization.py TestQuantizeFxOps Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D35071437](https://our.internmc.facebook.com/intern/diff/D35071437) [ghstack-poisoned]
jerryzh168
added a commit
that referenced
this pull request
Mar 23, 2022
…e native backend config" Summary: Following #74210, this PR adds the support for some ops using the DefaultNodeQuantizeHandler in the backend_config_dict defintion for pytorch native backend TODO: There is still a few ops we didn't handle with backend_config_dict path: gelu and softmax, need to discuss if we still need them, if so we can change the test to use backend_config_dict and remove the DefaultNodeQuantizeHandler after that Test Plan: python test/test_quantization.py TestQuantizeFxOps Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D35071437](https://our.internmc.facebook.com/intern/diff/D35071437) [ghstack-poisoned]
jerryzh168
added a commit
that referenced
this pull request
Mar 23, 2022
… config" Summary: Following #74210, this PR adds the support for some ops using the DefaultNodeQuantizeHandler in the backend_config_dict defintion for pytorch native backend TODO: There is still a few ops we didn't handle with backend_config_dict path: gelu and softmax, need to discuss if we still need them, if so we can change the test to use backend_config_dict and remove the DefaultNodeQuantizeHandler after that Test Plan: python test/test_quantization.py TestQuantizeFxOps Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D35071437](https://our.internmc.facebook.com/intern/diff/D35071437) [ghstack-poisoned]
jerryzh168
added a commit
that referenced
this pull request
Mar 23, 2022
…e native backend config" Summary: Following #74210, this PR adds the support for some ops using the DefaultNodeQuantizeHandler in the backend_config_dict defintion for pytorch native backend TODO: There is still a few ops we didn't handle with backend_config_dict path: gelu and softmax, need to discuss if we still need them, if so we can change the test to use backend_config_dict and remove the DefaultNodeQuantizeHandler after that Test Plan: python test/test_quantization.py TestQuantizeFxOps Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D35071437](https://our.internmc.facebook.com/intern/diff/D35071437) [ghstack-poisoned]
jerryzh168
added a commit
that referenced
this pull request
Mar 23, 2022
… config" Summary: Following #74210, this PR adds the support for some ops using the DefaultNodeQuantizeHandler in the backend_config_dict defintion for pytorch native backend TODO: There is still a few ops we didn't handle with backend_config_dict path: gelu and softmax, need to discuss if we still need them, if so we can change the test to use backend_config_dict and remove the DefaultNodeQuantizeHandler after that Test Plan: python test/test_quantization.py TestQuantizeFxOps Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D35071437](https://our.internmc.facebook.com/intern/diff/D35071437) [ghstack-poisoned]
jerryzh168
added a commit
that referenced
this pull request
Mar 23, 2022
…e native backend config" Summary: Following #74210, this PR adds the support for some ops using the DefaultNodeQuantizeHandler in the backend_config_dict defintion for pytorch native backend TODO: There is still a few ops we didn't handle with backend_config_dict path: gelu and softmax, need to discuss if we still need them, if so we can change the test to use backend_config_dict and remove the DefaultNodeQuantizeHandler after that Test Plan: python test/test_quantization.py TestQuantizeFxOps Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D35071437](https://our.internmc.facebook.com/intern/diff/D35071437) [ghstack-poisoned]
jerryzh168
added a commit
that referenced
this pull request
Mar 23, 2022
… config" Summary: Following #74210, this PR adds the support for some ops using the DefaultNodeQuantizeHandler in the backend_config_dict defintion for pytorch native backend TODO: There is still a few ops we didn't handle with backend_config_dict path: gelu and softmax, need to discuss if we still need them, if so we can change the test to use backend_config_dict and remove the DefaultNodeQuantizeHandler after that Test Plan: python test/test_quantization.py TestQuantizeFxOps Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D35071437](https://our.internmc.facebook.com/intern/diff/D35071437) [ghstack-poisoned]
jerryzh168
added a commit
that referenced
this pull request
Mar 24, 2022
…e native backend config" Summary: Following #74210, this PR adds the support for some ops using the DefaultNodeQuantizeHandler in the backend_config_dict defintion for pytorch native backend TODO: There is still a few ops we didn't handle with backend_config_dict path: gelu and softmax, need to discuss if we still need them, if so we can change the test to use backend_config_dict and remove the DefaultNodeQuantizeHandler after that Test Plan: python test/test_quantization.py TestQuantizeFxOps python test/test_quantization.py TestFXNumericSuiteCoreAPIs python test/test_quantization.py TestFXGraphMatcher Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D35071437](https://our.internmc.facebook.com/intern/diff/D35071437) [ghstack-poisoned]
jerryzh168
added a commit
that referenced
this pull request
Mar 24, 2022
… config" Summary: Following #74210, this PR adds the support for some ops using the DefaultNodeQuantizeHandler in the backend_config_dict defintion for pytorch native backend TODO: There is still a few ops we didn't handle with backend_config_dict path: gelu and softmax, need to discuss if we still need them, if so we can change the test to use backend_config_dict and remove the DefaultNodeQuantizeHandler after that Test Plan: python test/test_quantization.py TestQuantizeFxOps python test/test_quantization.py TestFXNumericSuiteCoreAPIs python test/test_quantization.py TestFXGraphMatcher Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D35071437](https://our.internmc.facebook.com/intern/diff/D35071437) [ghstack-poisoned]
jerryzh168
added a commit
that referenced
this pull request
Mar 24, 2022
…e native backend config" Summary: Following #74210, this PR adds the support for some ops using the DefaultNodeQuantizeHandler in the backend_config_dict defintion for pytorch native backend TODO: There is still a few ops we didn't handle with backend_config_dict path: gelu and softmax, need to discuss if we still need them, if so we can change the test to use backend_config_dict and remove the DefaultNodeQuantizeHandler after that Test Plan: python test/test_quantization.py TestQuantizeFxOps python test/test_quantization.py TestFXNumericSuiteCoreAPIs python test/test_quantization.py TestFXGraphMatcher Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D35071437](https://our.internmc.facebook.com/intern/diff/D35071437) [ghstack-poisoned]
jerryzh168
added a commit
that referenced
this pull request
Mar 24, 2022
… config" Summary: Following #74210, this PR adds the support for some ops using the DefaultNodeQuantizeHandler in the backend_config_dict defintion for pytorch native backend TODO: There is still a few ops we didn't handle with backend_config_dict path: gelu and softmax, need to discuss if we still need them, if so we can change the test to use backend_config_dict and remove the DefaultNodeQuantizeHandler after that Test Plan: python test/test_quantization.py TestQuantizeFxOps python test/test_quantization.py TestFXNumericSuiteCoreAPIs python test/test_quantization.py TestFXGraphMatcher Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D35071437](https://our.internmc.facebook.com/intern/diff/D35071437) [ghstack-poisoned]
jerryzh168
added a commit
that referenced
this pull request
Mar 24, 2022
…e native backend config" Summary: Following #74210, this PR adds the support for some ops using the DefaultNodeQuantizeHandler in the backend_config_dict defintion for pytorch native backend TODO: There is still a few ops we didn't handle with backend_config_dict path: gelu and softmax, need to discuss if we still need them, if so we can change the test to use backend_config_dict and remove the DefaultNodeQuantizeHandler after that Test Plan: python test/test_quantization.py TestQuantizeFxOps python test/test_quantization.py TestFXNumericSuiteCoreAPIs python test/test_quantization.py TestFXGraphMatcher Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D35071437](https://our.internmc.facebook.com/intern/diff/D35071437) [ghstack-poisoned]
jerryzh168
added a commit
that referenced
this pull request
Mar 24, 2022
… config" Summary: Following #74210, this PR adds the support for some ops using the DefaultNodeQuantizeHandler in the backend_config_dict defintion for pytorch native backend TODO: There is still a few ops we didn't handle with backend_config_dict path: gelu and softmax, need to discuss if we still need them, if so we can change the test to use backend_config_dict and remove the DefaultNodeQuantizeHandler after that Test Plan: python test/test_quantization.py TestQuantizeFxOps python test/test_quantization.py TestFXNumericSuiteCoreAPIs python test/test_quantization.py TestFXGraphMatcher Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D35071437](https://our.internmc.facebook.com/intern/diff/D35071437) [ghstack-poisoned]
jerryzh168
added a commit
that referenced
this pull request
Mar 24, 2022
…e native backend config" Summary: Following #74210, this PR adds the support for some ops using the DefaultNodeQuantizeHandler in the backend_config_dict defintion for pytorch native backend TODO: There is still a few ops we didn't handle with backend_config_dict path: gelu and softmax, need to discuss if we still need them, if so we can change the test to use backend_config_dict and remove the DefaultNodeQuantizeHandler after that Test Plan: python test/test_quantization.py TestQuantizeFxOps python test/test_quantization.py TestFXNumericSuiteCoreAPIs python test/test_quantization.py TestFXGraphMatcher Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D35071437](https://our.internmc.facebook.com/intern/diff/D35071437) [ghstack-poisoned]
jerryzh168
added a commit
that referenced
this pull request
Mar 24, 2022
… config" Summary: Following #74210, this PR adds the support for some ops using the DefaultNodeQuantizeHandler in the backend_config_dict defintion for pytorch native backend TODO: There is still a few ops we didn't handle with backend_config_dict path: gelu and softmax, need to discuss if we still need them, if so we can change the test to use backend_config_dict and remove the DefaultNodeQuantizeHandler after that Test Plan: python test/test_quantization.py TestQuantizeFxOps python test/test_quantization.py TestFXNumericSuiteCoreAPIs python test/test_quantization.py TestFXGraphMatcher Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D35071437](https://our.internmc.facebook.com/intern/diff/D35071437) [ghstack-poisoned]
facebook-github-bot
pushed a commit
that referenced
this pull request
Mar 25, 2022
…4600) Summary: Pull Request resolved: #74600 Following #74210, this PR adds the support for some ops using the DefaultNodeQuantizeHandler in the backend_config_dict defintion for pytorch native backend TODO: There is still a few ops we didn't handle with backend_config_dict path: gelu and softmax, need to discuss if we still need them, if so we can change the test to use backend_config_dict and remove the DefaultNodeQuantizeHandler after that Test Plan: python test/test_quantization.py TestQuantizeFxOps Imported from OSS Reviewed By: andrewor14 Differential Revision: D35071437 fbshipit-source-id: 70351d2810ca1ac7dc09d4a9c239f6757ccb51ca
pytorchmergebot
pushed a commit
that referenced
this pull request
Mar 25, 2022
…4600) Summary: Pull Request resolved: #74600 Following #74210, this PR adds the support for some ops using the DefaultNodeQuantizeHandler in the backend_config_dict defintion for pytorch native backend TODO: There is still a few ops we didn't handle with backend_config_dict path: gelu and softmax, need to discuss if we still need them, if so we can change the test to use backend_config_dict and remove the DefaultNodeQuantizeHandler after that Test Plan: python test/test_quantization.py TestQuantizeFxOps Imported from OSS Reviewed By: andrewor14 Differential Revision: D35071437 fbshipit-source-id: 70351d2810ca1ac7dc09d4a9c239f6757ccb51ca (cherry picked from commit 5e68f75)
shahofblah
pushed a commit
that referenced
this pull request
Mar 25, 2022
…4600) Summary: Pull Request resolved: #74600 Following #74210, this PR adds the support for some ops using the DefaultNodeQuantizeHandler in the backend_config_dict defintion for pytorch native backend TODO: There is still a few ops we didn't handle with backend_config_dict path: gelu and softmax, need to discuss if we still need them, if so we can change the test to use backend_config_dict and remove the DefaultNodeQuantizeHandler after that Test Plan: python test/test_quantization.py TestQuantizeFxOps Imported from OSS Reviewed By: andrewor14 Differential Revision: D35071437 fbshipit-source-id: 70351d2810ca1ac7dc09d4a9c239f6757ccb51ca (cherry picked from commit 5e68f75)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Stack from ghstack (oldest at bottom):
Summary:
This PR added a codepath for getting patterns (quantize handlers) for the backend_config_dict for native backend when
backend_config_dict is None. This would allow us to incrementally define the backend_config_dict for
pytorch native backend and gradually remove the entries in quantization_patterns.py
Test Plan:
python test/test_quantization.py TestQuantizeFx
python test/test_quantization.py TestQuantizeFxOps
Reviewers:
Subscribers:
Tasks:
Tags:
Differential Revision: D34899783