[quant][eagermode] Custom module support #44835

jerryzh168 · 2020-09-16T22:14:25Z

Stack from ghstack:

[quant][eagermode] Custom module support #44835 [quant][eagermode] Custom module support

Summary:
This is for feature parity with fx graph mode quantization

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: D23745086

Summary: This is for feature parity with fx graph mode quantization Test Plan: Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

Summary: This is for feature parity with fx graph mode quantization Test Plan: Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D23745086](https://our.internmc.facebook.com/intern/diff/D23745086) [ghstack-poisoned]

dr-ci · 2020-09-16T22:25:32Z

💊 CI failures summary and remediations

As of commit 6e66c85 (more details on the Dr. CI page):

1/1 failures possibly* introduced in this PR
- 1/1 non-CircleCI failure(s)

ci.pytorch.org: 1 failed

Failed: pr/pytorch-linux-bionic-rocm3.7-py3.6

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker or post in the (internal) Dr. CI Users group.

See how this bot performed.

This comment has been revised 27 times.

Summary: This is for feature parity with fx graph mode quantization Test Plan: Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D23745086](https://our.internmc.facebook.com/intern/diff/D23745086) [ghstack-poisoned]

Summary: This is for feature parity with fx graph mode quantization Test Plan: Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 7b74091 Pull Request resolved: #44835

raghuramank100 · 2020-09-18T17:49:53Z

torch/quantization/quantize.py

-        if type(mod) not in SWAPPABLE_MODULES:
+        # both swappable modules and observed custom modules are
+        # swapped as one unit
+        if type(mod) not in SWAPPABLE_MODULES and \


When we support custom module fusion, we will need to change code here. Currently, it looks like Swappable modules defines the list of fusions that are supported. Thoughts on how this can be addressed? There main use case would be : nn.module A + nn.module B => Custom Fused Module C

we can make this configuration with registrations as well, can be done in a separate PR before we need this feature

raghuramank100 · 2020-09-18T17:50:42Z

test/quantization/test_quantize.py

        self.checkDynamicQuantizedModule(quantized_model.emb, torch.nn.quantized.EmbeddingBag, torch.quint8)

+    @skipIfNoFBGEMM
+    def test_custom_module_class(self):


Similar request as in the previous PR, lets add tests for each of the workflows (PTQ, Dynamic and QAT) and also for modules with multiple input/outputs (LSTMs).

for dynamic/qat, will do that after the mixed mode support change.
lstm test will be done after zafar write the lstm pr. but I'll add multiple input/output support in a separate PR

Summary: This is for feature parity with fx graph mode quantization Test Plan: Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D23745086](https://our.internmc.facebook.com/intern/diff/D23745086) [ghstack-poisoned]

Summary: This is for feature parity with fx graph mode quantization Test Plan: Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: dc04c72 Pull Request resolved: #44835

z-a-f

LGTM

z-a-f · 2020-09-22T21:59:13Z

torch/quantization/quantize.py

-        if type(mod) not in SWAPPABLE_MODULES:
+        # both swappable modules and observed custom modules are
+        # swapped as one unit
+        if type(mod) not in SWAPPABLE_MODULES and \


nit: will the SWAPPABLE_MODULES be a globa variable, or are we planning on replacing it with a getter function?

yeah we can do that when there is a need

raghuramank100

If we can add more tests like requested in the other PR for both eager and IR custom module support, that would be great.

codecov · 2020-09-23T01:23:11Z

Codecov Report

❗ No coverage uploaded for pull request base (gh/jerryzh168/437/base@e64f20f). Click here to learn what that means.
The diff coverage is n/a.

@@                    Coverage Diff                    @@
##             gh/jerryzh168/437/base   #44835   +/-   ##
=========================================================
  Coverage                          ?   67.92%           
=========================================================
  Files                             ?      385           
  Lines                             ?    50169           
  Branches                          ?        0           
=========================================================
  Hits                              ?    34079           
  Misses                            ?    16090           
  Partials                          ?        0

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e64f20f...6e66c85. Read the comment docs.

Summary: This is for feature parity with fx graph mode quantization Test Plan: Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D23745086](https://our.internmc.facebook.com/intern/diff/D23745086) [ghstack-poisoned]

Summary: This is for feature parity with fx graph mode quantization Test Plan: Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 0b5e297 Pull Request resolved: #44835

Summary: This is for feature parity with fx graph mode quantization Test Plan: Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D23745086](https://our.internmc.facebook.com/intern/diff/D23745086) [ghstack-poisoned]

Summary: This is for feature parity with fx graph mode quantization Test Plan: Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 7a078ac Pull Request resolved: #44835

facebook-github-bot · 2020-09-24T02:15:09Z

This pull request has been merged in f93ead6.

[quant][eagermode] Custom module support

038ade0

Summary: This is for feature parity with fx graph mode quantization Test Plan: Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

jerryzh168 mentioned this pull request Sep 16, 2020

[quant][graphmode][fx] Custom module support #44766

Closed

facebook-github-bot added the fx label Sep 16, 2020

Update on "[quant][eagermode] Custom module support"

7c36a84

Summary: This is for feature parity with fx graph mode quantization Test Plan: Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

Update on "[quant][eagermode] Custom module support"

1114fca

Summary: This is for feature parity with fx graph mode quantization Test Plan: Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D23745086](https://our.internmc.facebook.com/intern/diff/D23745086) [ghstack-poisoned]

Update on "[quant][eagermode] Custom module support"

a1b97e6

Summary: This is for feature parity with fx graph mode quantization Test Plan: Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D23745086](https://our.internmc.facebook.com/intern/diff/D23745086) [ghstack-poisoned]

jerryzh168 requested review from raghuramank100 and vkuzo September 17, 2020 20:37

raghuramank100 reviewed Sep 18, 2020

View reviewed changes

Update on "[quant][eagermode] Custom module support"

b528690

Summary: This is for feature parity with fx graph mode quantization Test Plan: Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D23745086](https://our.internmc.facebook.com/intern/diff/D23745086) [ghstack-poisoned]

Update on "[quant][eagermode] Custom module support"

e05c2f0

Summary: This is for feature parity with fx graph mode quantization Test Plan: Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D23745086](https://our.internmc.facebook.com/intern/diff/D23745086) [ghstack-poisoned]

Update on "[quant][eagermode] Custom module support"

9469667

Summary: This is for feature parity with fx graph mode quantization Test Plan: Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D23745086](https://our.internmc.facebook.com/intern/diff/D23745086) [ghstack-poisoned]

z-a-f approved these changes Sep 22, 2020

View reviewed changes

raghuramank100 approved these changes Sep 22, 2020

View reviewed changes

Update on "[quant][eagermode] Custom module support"

2a75b5f

Summary: This is for feature parity with fx graph mode quantization Test Plan: Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D23745086](https://our.internmc.facebook.com/intern/diff/D23745086) [ghstack-poisoned]

Update on "[quant][eagermode] Custom module support"

6e66c85

Summary: This is for feature parity with fx graph mode quantization Test Plan: Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D23745086](https://our.internmc.facebook.com/intern/diff/D23745086) [ghstack-poisoned]

facebook-github-bot closed this in f93ead6 Sep 23, 2020

facebook-github-bot added the merged label Sep 24, 2020

facebook-github-bot deleted the gh/jerryzh168/437/head branch September 27, 2020 14:15

mruberry added the Merged label Oct 28, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[quant][eagermode] Custom module support #44835

[quant][eagermode] Custom module support #44835

Uh oh!

jerryzh168 commented Sep 16, 2020 •

edited

Loading

Uh oh!

dr-ci bot commented Sep 16, 2020 •

edited

Loading

Uh oh!

raghuramank100 Sep 18, 2020

Uh oh!

jerryzh168 Sep 22, 2020

Uh oh!

raghuramank100 Sep 18, 2020

Uh oh!

jerryzh168 Sep 22, 2020

Uh oh!

z-a-f left a comment

Uh oh!

z-a-f Sep 22, 2020

Uh oh!

jerryzh168 Sep 22, 2020

Uh oh!

raghuramank100 left a comment

Uh oh!

codecov bot commented Sep 23, 2020 •

edited

Loading

Uh oh!

facebook-github-bot commented Sep 24, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

[quant][eagermode] Custom module support #44835

[quant][eagermode] Custom module support #44835

Uh oh!

Conversation

jerryzh168 commented Sep 16, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dr-ci bot commented Sep 16, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💊 CI failures summary and remediations

ci.pytorch.org: 1 failed

Uh oh!

raghuramank100 Sep 18, 2020

Choose a reason for hiding this comment

Uh oh!

jerryzh168 Sep 22, 2020

Choose a reason for hiding this comment

Uh oh!

raghuramank100 Sep 18, 2020

Choose a reason for hiding this comment

Uh oh!

jerryzh168 Sep 22, 2020

Choose a reason for hiding this comment

Uh oh!

z-a-f left a comment

Choose a reason for hiding this comment

Uh oh!

z-a-f Sep 22, 2020

Choose a reason for hiding this comment

Uh oh!

jerryzh168 Sep 22, 2020

Choose a reason for hiding this comment

Uh oh!

raghuramank100 left a comment

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Sep 23, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

facebook-github-bot commented Sep 24, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

jerryzh168 commented Sep 16, 2020 •

edited

Loading

dr-ci bot commented Sep 16, 2020 •

edited

Loading

codecov bot commented Sep 23, 2020 •

edited

Loading