[quant][eagermode][refactor] Add set/get method for quantization and fusion mappings #43990

jerryzh168 · 2020-09-01T21:31:02Z

Stack from ghstack:

[quant][graphmode][fx] Support quantization for standalone module #44074 [quant][graphmode][fx] Support quantization for custom module
[quant][eagermode][refactor] Add set/get method for quantization and fusion mappings #43990 [quant][eagermode][refactor] Add set/get method for quantization and fusion mappings

Summary:
Allow user to register custom quantization and fusion patterns

Custom quantization API:

register_static_quant_module_mapping
get_static_quant_module_mappings
get_static_quant_module_class
register_dynamic_quant_module_mapping
get_dynamic_quant_module_mappings
register_qat_module_mapping
get_qat_module_mappings

for fx graph mode quantization

register_quantized_operator
get_quantized_operator

Custom fusion API:

register_fuser_method
get_fuser_method

Example:

register_static_quant_module_mapping(torch.nn.Conv2d, torch.nn.quantized.Conv2d)
register_dynamic_quant_module_mapping(torch.nn.Conv2d, torch.nn.quantized.dynamic.Conv2d)
register_qat_module_mapping(torch.nn.Conv2d, torch.nn.qat.Conv2d)
register_fuser_method((torch.nn.Conv2d, torch.nn.BatchNorm2d), fuse_conv_bn)

>> get_static_quant_module_class(torch.nn.Conv2d)
torch.nn.quantized.Conv2d

>> get_fuser_method((torch.nn.Conv2d, torch.nn.BatchNorm2d))
fuser_conv_bn

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: D23485344

…fusion mappings Summary: Allow user to register custom quantization and fusion patterns Test Plan: Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

…fusion mappings Summary: Allow user to register custom quantization and fusion patterns Test Plan: Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 913f70a Pull Request resolved: #43990

dr-ci · 2020-09-01T21:41:08Z

💊 CI failures summary and remediations

As of commit 3dc9ab6 (more details on the Dr. CI page):

1/1 failures possibly* introduced in this PR
- 1/1 non-CircleCI failure(s)

ci.pytorch.org: 1 failed

Failed: pr/pytorch-linux-bionic-rocm3.7-py3.6

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker or post in the (internal) Dr. CI Users group.

See how this bot performed.

This comment has been revised 127 times.

…zation and fusion mappings" Summary: Allow user to register custom quantization and fusion patterns Test Plan: Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

…fusion mappings Summary: Allow user to register custom quantization and fusion patterns Test Plan: Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 77e207d Pull Request resolved: #43990

…zation and fusion mappings" Summary: Allow user to register custom quantization and fusion patterns Test Plan: Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

vkuzo · 2020-09-02T22:12:25Z

nice! any chance we could add an example usage of this API to register a custom pattern to the PR summary? would make it easier to review

z-a-f

LGTM

z-a-f · 2020-09-02T22:39:32Z

torch/quantization/_numeric_suite.py

    *data,
    Logger=OutputLogger,
-    white_list=DEFAULT_NUMERIC_SUITE_COMPARE_MODEL_OUTPUT_WHITE_LIST,
+    white_list=None


Should we change the white_list while we are at it?

i'm afraid this is bc breaking, would be better to change all at once

…zation and fusion mappings" Summary: Allow user to register custom quantization and fusion patterns Custom quantization API: - register_static_quant_module_class - get_static_quant_module_mapping - get_static_quant_module_class - register_dynamic_quant_module_class - get_dynamic_quant_module_mapping - register_qat_module_class - get_qat_module_mapping Custom fusion API: - register_fuser_method - get_fuser_method Example: ``` register_static_quant_module_class(torch.nn.Conv2d, torch.nn.quantized.Conv2d) register_dynamic_quant_module_class(torch.nn.Conv2d, torch.nn.quantized.dynamic.Conv2d) register_qat_module_class(torch.nn.Conv2d, torch.nn.qat.Conv2d) register_fuser_method((torch.nn.Conv2d, torch.nn.BatchNorm2d), fuse_conv_bn) >> get_static_quant_module_class(torch.nn.Conv2d) torch.nn.quantized.Conv2d >> get_fuser_method((torch.nn.Conv2d, torch.nn.BatchNorm2d)) fuser_conv_bn ``` Test Plan: Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D23485344](https://our.internmc.facebook.com/intern/diff/D23485344) [ghstack-poisoned]

jerryzh168 · 2020-09-02T23:29:14Z

nice! any chance we could add an example usage of this API to register a custom pattern to the PR summary? would make it easier to review

done

…zation and fusion mappings" Summary: Allow user to register custom quantization and fusion patterns Custom quantization API: - register_static_quant_module_class - get_static_quant_module_mapping - get_static_quant_module_class - register_dynamic_quant_module_class - get_dynamic_quant_module_mapping - register_qat_module_class - get_qat_module_mapping for fx graph mode quantization - register_quantized_operator - get_quantized_operator Custom fusion API: - register_fuser_method - get_fuser_method Example: ``` register_static_quant_module_class(torch.nn.Conv2d, torch.nn.quantized.Conv2d) register_dynamic_quant_module_class(torch.nn.Conv2d, torch.nn.quantized.dynamic.Conv2d) register_qat_module_class(torch.nn.Conv2d, torch.nn.qat.Conv2d) register_fuser_method((torch.nn.Conv2d, torch.nn.BatchNorm2d), fuse_conv_bn) >> get_static_quant_module_class(torch.nn.Conv2d) torch.nn.quantized.Conv2d >> get_fuser_method((torch.nn.Conv2d, torch.nn.BatchNorm2d)) fuser_conv_bn ``` Test Plan: Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D23485344](https://our.internmc.facebook.com/intern/diff/D23485344) [ghstack-poisoned]

vkuzo · 2020-09-10T16:53:36Z

torch/quantization/quantization_mappings.py

+    F.layer_norm: torch._ops.ops.quantized.layer_norm,
+}
+
+def register_static_quant_module_class(module_class, static_quant_module_class):


optional, how about something like this to improve readability

def register_static_quant_module_swap(fp_source_module_class, static_quant_target_module_class): """Registers a module swap of {fp_source_module_class} to {static_quant_target_module_class} during static quant convert step. static_quant_target_module_class must have from_float defined as a class method."""

sure, will change to register_static_quant_module_mapping

…zation and fusion mappings" Summary: Allow user to register custom quantization and fusion patterns Custom quantization API: - register_static_quant_module_class - get_static_quant_module_mapping - get_static_quant_module_class - register_dynamic_quant_module_class - get_dynamic_quant_module_mapping - register_qat_module_class - get_qat_module_mapping for fx graph mode quantization - register_quantized_operator - get_quantized_operator Custom fusion API: - register_fuser_method - get_fuser_method Example: ``` register_static_quant_module_class(torch.nn.Conv2d, torch.nn.quantized.Conv2d) register_dynamic_quant_module_class(torch.nn.Conv2d, torch.nn.quantized.dynamic.Conv2d) register_qat_module_class(torch.nn.Conv2d, torch.nn.qat.Conv2d) register_fuser_method((torch.nn.Conv2d, torch.nn.BatchNorm2d), fuse_conv_bn) >> get_static_quant_module_class(torch.nn.Conv2d) torch.nn.quantized.Conv2d >> get_fuser_method((torch.nn.Conv2d, torch.nn.BatchNorm2d)) fuser_conv_bn ``` Test Plan: Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D23485344](https://our.internmc.facebook.com/intern/diff/D23485344) [ghstack-poisoned]

…zation and fusion mappings" Summary: Allow user to register custom quantization and fusion patterns Custom quantization API: - register_static_quant_module_mapping - get_static_quant_module_mappings - get_static_quant_module_class - register_dynamic_quant_module_mapping - get_dynamic_quant_module_mappings - register_qat_module_mapping - get_qat_module_mappings for fx graph mode quantization - register_quantized_operator - get_quantized_operator Custom fusion API: - register_fuser_method - get_fuser_method Example: ``` register_static_quant_module_mapping(torch.nn.Conv2d, torch.nn.quantized.Conv2d) register_dynamic_quant_module_mapping(torch.nn.Conv2d, torch.nn.quantized.dynamic.Conv2d) register_qat_module_mapping(torch.nn.Conv2d, torch.nn.qat.Conv2d) register_fuser_method((torch.nn.Conv2d, torch.nn.BatchNorm2d), fuse_conv_bn) >> get_static_quant_module_class(torch.nn.Conv2d) torch.nn.quantized.Conv2d >> get_fuser_method((torch.nn.Conv2d, torch.nn.BatchNorm2d)) fuser_conv_bn ``` Test Plan: Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D23485344](https://our.internmc.facebook.com/intern/diff/D23485344) [ghstack-poisoned]

codecov · 2020-09-11T02:25:00Z

Codecov Report

Merging #43990 into gh/jerryzh168/426/base will decrease coverage by 0.01%.
The diff coverage is 89.34%.

@@                    Coverage Diff                     @@
##           gh/jerryzh168/426/base   #43990      +/-   ##
==========================================================
- Coverage                   68.05%   68.04%   -0.02%     
==========================================================
  Files                         382      384       +2     
  Lines                       49468    49527      +59     
==========================================================
+ Hits                        33667    33699      +32     
- Misses                      15801    15828      +27

Impacted Files	Coverage Δ
torch/quantization/_numeric_suite.py	`95.65% <80.00%> (-0.43%)`	⬇️
torch/quantization/quantization_mappings.py	`85.10% <85.10%> (ø)`
torch/quantization/fuser_method_mappings.py	`88.88% <88.88%> (ø)`
torch/quantization/quantize.py	`90.32% <90.90%> (ø)`
torch/quantization/__init__.py	`87.50% <100.00%> (+1.78%)`	⬆️
torch/quantization/fuse_modules.py	`98.14% <100.00%> (+1.85%)`	⬆️
torch/quantization/fx/fusion_patterns.py	`88.09% <100.00%> (+0.88%)`	⬆️
torch/quantization/fx/quantization_patterns.py	`91.89% <100.00%> (ø)`
torch/quantization/fx/quantize.py	`96.31% <100.00%> (ø)`
torch/testing/_internal/common_quantization.py	`89.57% <100.00%> (ø)`
... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a2a81e1...3dc9ab6. Read the comment docs.

facebook-github-bot · 2020-09-11T06:12:13Z

This pull request has been merged in 0c58a01.

…fusion mappings (#43990) Summary: Pull Request resolved: #43990 Allow user to register custom quantization and fusion patterns Test Plan: Imported from OSS Reviewed By: z-a-f Differential Revision: D23485344 fbshipit-source-id: 4f0174ee6d8000d83de0f73cb370e9a1941d54aa

[quant][eagermode][refactor] Add set/get method for quantization and …

348a63c

…fusion mappings Summary: Allow user to register custom quantization and fusion patterns Test Plan: Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

Update on "[quant][eagermode][refactor] Add set/get method for quanti…

1d218e9

…zation and fusion mappings" Summary: Allow user to register custom quantization and fusion patterns Test Plan: Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

Update on "[quant][eagermode][refactor] Add set/get method for quanti…

27f8654

…zation and fusion mappings" Summary: Allow user to register custom quantization and fusion patterns Test Plan: Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

Update on "[quant][eagermode][refactor] Add set/get method for quanti…

03c5a09

…zation and fusion mappings" Summary: Allow user to register custom quantization and fusion patterns Test Plan: Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

jerryzh168 requested review from raghuramank100 and vkuzo September 2, 2020 21:50

z-a-f approved these changes Sep 2, 2020

View reviewed changes

jerryzh168 mentioned this pull request Sep 2, 2020

[quant][graphmode][fx] Support quantization for standalone module #44074

Closed

jerryzh168 added 9 commits September 2, 2020 16:35

vkuzo reviewed Sep 10, 2020

View reviewed changes

jerryzh168 added 2 commits September 10, 2020 09:57

jerryzh168 added 2 commits September 10, 2020 14:12

facebook-github-bot closed this in 0c58a01 Sep 11, 2020

facebook-github-bot added the merged label Sep 11, 2020

facebook-github-bot deleted the gh/jerryzh168/426/head branch September 14, 2020 14:17

walterddr mentioned this pull request Oct 16, 2020

Enable torch.quantization.default_mappings typechecks during CI #42970

Closed

mruberry added the Merged label Oct 28, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[quant][eagermode][refactor] Add set/get method for quantization and fusion mappings #43990

[quant][eagermode][refactor] Add set/get method for quantization and fusion mappings #43990

Uh oh!

jerryzh168 commented Sep 1, 2020 •

edited

Loading

Uh oh!

dr-ci bot commented Sep 1, 2020 •

edited

Loading

Uh oh!

vkuzo commented Sep 2, 2020

Uh oh!

z-a-f left a comment

Uh oh!

z-a-f Sep 2, 2020

Uh oh!

jerryzh168 Sep 2, 2020

Uh oh!

jerryzh168 commented Sep 2, 2020

Uh oh!

vkuzo Sep 10, 2020

Uh oh!

jerryzh168 Sep 10, 2020

Uh oh!

codecov bot commented Sep 11, 2020 •

edited

Loading

Uh oh!

facebook-github-bot commented Sep 11, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

[quant][eagermode][refactor] Add set/get method for quantization and fusion mappings #43990

[quant][eagermode][refactor] Add set/get method for quantization and fusion mappings #43990

Uh oh!

Conversation

jerryzh168 commented Sep 1, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dr-ci bot commented Sep 1, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💊 CI failures summary and remediations

ci.pytorch.org: 1 failed

Uh oh!

vkuzo commented Sep 2, 2020

Uh oh!

z-a-f left a comment

Choose a reason for hiding this comment

Uh oh!

z-a-f Sep 2, 2020

Choose a reason for hiding this comment

Uh oh!

jerryzh168 Sep 2, 2020

Choose a reason for hiding this comment

Uh oh!

jerryzh168 commented Sep 2, 2020

Uh oh!

vkuzo Sep 10, 2020

Choose a reason for hiding this comment

Uh oh!

jerryzh168 Sep 10, 2020

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Sep 11, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

facebook-github-bot commented Sep 11, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

jerryzh168 commented Sep 1, 2020 •

edited

Loading

dr-ci bot commented Sep 1, 2020 •

edited

Loading

codecov bot commented Sep 11, 2020 •

edited

Loading