[quant] Add quant APIs to save/load observer state_dict #44846

supriyar · 2020-09-16T23:08:31Z

Stack from ghstack:

[quant] Add quant APIs to save/load observer state_dict #44846 [quant] Add quant APIs to save/load observer state_dict

Summary:
The save function traverses the model state dict to pick out the observer stats
load function traverse the module hierarchy to load the state dict into module attributes depending on observer type

Test Plan:
python test/test_quantization.py TestQuantizeFx.test_save_observer_state_dict

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: D23746821

Summary: The save function traverses the model state dict to pick out the observer stats load function traverse the module hierarchy to load the state dict into module attributes depending on observer type Test Plan: python test/test_quantization.py TestQuantizeFx.test_save_observer_state_dict Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

Summary: The save function traverses the model state dict to pick out the observer stats load function traverse the module hierarchy to load the state dict into module attributes depending on observer type Test Plan: python test/test_quantization.py TestQuantizeFx.test_save_observer_state_dict Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: e1275f7 Pull Request resolved: #44846

Summary: The save function traverses the model state dict to pick out the observer stats load function traverse the module hierarchy to load the state dict into module attributes depending on observer type Test Plan: python test/test_quantization.py TestQuantizeFx.test_save_observer_state_dict Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D23746821](https://our.internmc.facebook.com/intern/diff/D23746821) [ghstack-poisoned]

Summary: The save function traverses the model state dict to pick out the observer stats load function traverse the module hierarchy to load the state dict into module attributes depending on observer type Test Plan: python test/test_quantization.py TestQuantizeFx.test_save_observer_state_dict Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 296c73e Pull Request resolved: #44846

dr-ci · 2020-09-17T21:36:55Z

💊 CI failures summary and remediations

As of commit efc434f (more details on the Dr. CI page):

💚 💚 Looks good so far! There are no failures yet. 💚 💚

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker or post in the (internal) Dr. CI Users group.

See how this bot performed.

This comment has been revised 49 times.

codecov · 2020-09-18T01:07:59Z

Codecov Report

❗ No coverage uploaded for pull request base (gh/supriyar/181/base@b9633ff). Click here to learn what that means.
The diff coverage is n/a.

@@                   Coverage Diff                   @@
##             gh/supriyar/181/base   #44846   +/-   ##
=======================================================
  Coverage                        ?   68.07%           
=======================================================
  Files                           ?      396           
  Lines                           ?    51293           
  Branches                        ?        0           
=======================================================
  Hits                            ?    34920           
  Misses                          ?    16373           
  Partials                        ?        0

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b9633ff...efc434f. Read the comment docs.

torch/quantization/observer.py

Summary: The save function traverses the model state dict to pick out the observer stats load function traverse the module hierarchy to load the state dict into module attributes depending on observer type Test Plan: python test/test_quantization.py TestQuantizeFx.test_save_observer_state_dict Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D23746821](https://our.internmc.facebook.com/intern/diff/D23746821) [ghstack-poisoned]

Summary: The save function traverses the model state dict to pick out the observer stats load function traverse the module hierarchy to load the state dict into module attributes depending on observer type Test Plan: python test/test_quantization.py TestQuantizeFx.test_save_observer_state_dict Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 923da1a Pull Request resolved: #44846

Summary: The save function traverses the model state dict to pick out the observer stats load function traverse the module hierarchy to load the state dict into module attributes depending on observer type Test Plan: python test/test_quantization.py TestQuantizeFx.test_save_observer_state_dict Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D23746821](https://our.internmc.facebook.com/intern/diff/D23746821) [ghstack-poisoned]

Summary: The save function traverses the model state dict to pick out the observer stats load function traverse the module hierarchy to load the state dict into module attributes depending on observer type Test Plan: python test/test_quantization.py TestQuantizeFx.test_save_observer_state_dict Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 8e69ddd Pull Request resolved: #44846

torch/quantization/observer.py

raghuramank100 · 2020-09-18T17:08:28Z

test/quantization/test_quantize_fx.py


        # save state_dict of model
-        import io
+        obs_dict = torch.quantization.get_observer_state_dict(model)


Currently we seem to be doing this only for activation observers. As we move to QAT, we will need similar functionality, but we will need to consider weight fake-quants too. Is there a plan to add weight observers to the state dict?

Summary: The save function traverses the model state dict to pick out the observer stats load function traverse the module hierarchy to load the state dict into module attributes depending on observer type Test Plan: python test/test_quantization.py TestQuantizeFx.test_save_observer_state_dict Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D23746821](https://our.internmc.facebook.com/intern/diff/D23746821) [ghstack-poisoned]

…line with a title. Use 1 line only, 67 chars or less> Summary: The save function traverses the model state dict to pick out the observer stats load function traverse the module hierarchy to load the state dict into module attributes depending on observer type Test Plan: python test/test_quantization.py TestQuantizeFx.test_save_observer_state_dict python test/test_quantization.py TestQuantizeJitPasses.test_save_observer_state_dict Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 63d3db8 Pull Request resolved: #44846

Summary: The save function traverses the model state dict to pick out the observer stats load function traverse the module hierarchy to load the state dict into module attributes depending on observer type Test Plan: python test/test_quantization.py TestQuantizeFx.test_save_observer_state_dict Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D23746821](https://our.internmc.facebook.com/intern/diff/D23746821) [ghstack-poisoned]

…line with a title. Use 1 line only, 67 chars or less> Summary: The save function traverses the model state dict to pick out the observer stats load function traverse the module hierarchy to load the state dict into module attributes depending on observer type Test Plan: python test/test_quantization.py TestQuantizeFx.test_save_observer_state_dict python test/test_quantization.py TestQuantizeJitPasses.test_save_observer_state_dict Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: df483ce Pull Request resolved: #44846

torch/quantization/observer.py

vkuzo · 2020-09-22T23:16:10Z

torch/quantization/observer.py

+                                                                        missing_keys, unexpected_keys, error_msgs)
+
+    @torch.jit.export
+    def _load_from_state_dict_script(self, state_dict: Union[Dict[str, torch.Tensor], Dict[str, torch.Tensor]],


this seems to call _load_from_state_dict, just curious on what would break if this function wasn't there? I might be missing something

I noticed that without this, the call to load_from_state_dict was calling the one defined in module.py. Once we script the model we lose access to the overridden load_from_state_dict so we need to call a different fn here to get around it.

vkuzo · 2020-09-22T23:17:13Z

torch/quantization/observer.py

+            _is_observer_script_module(module, "torch.quantization.observer.MovingAveragePerChannelMinMaxObserver")
+    return False
+
+def get_observer_state_dict(mod):


thoughts about making most of these FB-only, since the use case is pretty specific? Or are there OSS needs as well?

That may be true right now, but Raghu mentioned these will come in use when we develop the numeric suite and auto quantizer for OSS. One can argue that the OSS use cases might not need this functionality for script models, but it might be better to keep it all in the same place.

Summary: The save function traverses the model state dict to pick out the observer stats load function traverse the module hierarchy to load the state dict into module attributes depending on observer type Test Plan: python test/test_quantization.py TestQuantizeFx.test_save_observer_state_dict Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D23746821](https://our.internmc.facebook.com/intern/diff/D23746821) [ghstack-poisoned]

…line with a title. Use 1 line only, 67 chars or less> Summary: The save function traverses the model state dict to pick out the observer stats load function traverse the module hierarchy to load the state dict into module attributes depending on observer type Test Plan: python test/test_quantization.py TestQuantizeFx.test_save_observer_state_dict python test/test_quantization.py TestQuantizeJitPasses.test_save_observer_state_dict Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: c27c255 Pull Request resolved: #44846

Summary: The save function traverses the model state dict to pick out the observer stats load function traverse the module hierarchy to load the state dict into module attributes depending on observer type Test Plan: python test/test_quantization.py TestQuantizeFx.test_save_observer_state_dict Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: f0d591a Pull Request resolved: pytorch/pytorch#44846

Summary: The save function traverses the model state dict to pick out the observer stats load function traverse the module hierarchy to load the state dict into module attributes depending on observer type Test Plan: python test/test_quantization.py TestQuantizeFx.test_save_observer_state_dict Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D23746821](https://our.internmc.facebook.com/intern/diff/D23746821) [ghstack-poisoned]

…line with a title. Use 1 line only, 67 chars or less> Summary: The save function traverses the model state dict to pick out the observer stats load function traverse the module hierarchy to load the state dict into module attributes depending on observer type Test Plan: python test/test_quantization.py TestQuantizeFx.test_save_observer_state_dict python test/test_quantization.py TestQuantizeJitPasses.test_save_observer_state_dict Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 69c958d Pull Request resolved: #44846

Summary: The save function traverses the model state dict to pick out the observer stats load function traverse the module hierarchy to load the state dict into module attributes depending on observer type Test Plan: python test/test_quantization.py TestQuantizeFx.test_save_observer_state_dict Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D23746821](https://our.internmc.facebook.com/intern/diff/D23746821) [ghstack-poisoned]

…line with a title. Use 1 line only, 67 chars or less> Summary: The save function traverses the model state dict to pick out the observer stats load function traverse the module hierarchy to load the state dict into module attributes depending on observer type Test Plan: python test/test_quantization.py TestQuantizeFx.test_save_observer_state_dict python test/test_quantization.py TestQuantizeJitPasses.test_save_observer_state_dict Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 0b6130c Pull Request resolved: #44846

Summary: The save function traverses the model state dict to pick out the observer stats load function traverse the module hierarchy to load the state dict into module attributes depending on observer type Test Plan: python test/test_quantization.py TestQuantizeFx.test_save_observer_state_dict Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D23746821](https://our.internmc.facebook.com/intern/diff/D23746821) [ghstack-poisoned]

…line with a title. Use 1 line only, 67 chars or less> Summary: The save function traverses the model state dict to pick out the observer stats load function traverse the module hierarchy to load the state dict into module attributes depending on observer type Test Plan: python test/test_quantization.py TestQuantizeFx.test_save_observer_state_dict python test/test_quantization.py TestQuantizeJitPasses.test_save_observer_state_dict Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 246eac0 Pull Request resolved: #44846

facebook-github-bot · 2020-09-29T10:22:13Z

This pull request has been merged in 489af4d.

This was referenced Sep 16, 2020

[quant][fx] Add node name as prefix to observer module name #44765

Closed

[quant][qat] Ensure fake_quant and observer can be disabled on scriptmodule #44773

Closed

supriyar requested review from jerryzh168, qizzzh, radkris-git and vkuzo and removed request for radkris-git September 16, 2020 23:12

vkuzo reviewed Sep 18, 2020

View reviewed changes

torch/quantization/observer.py Show resolved Hide resolved

vkuzo reviewed Sep 18, 2020

View reviewed changes

torch/quantization/observer.py Outdated Show resolved Hide resolved

vkuzo reviewed Sep 18, 2020

View reviewed changes

torch/quantization/observer.py Show resolved Hide resolved

supriyar requested a review from raghuramank100 September 18, 2020 15:46

raghuramank100 reviewed Sep 18, 2020

View reviewed changes

torch/quantization/observer.py Outdated Show resolved Hide resolved

raghuramank100 reviewed Sep 18, 2020

View reviewed changes

torch/quantization/observer.py Outdated Show resolved Hide resolved

raghuramank100 reviewed Sep 18, 2020

View reviewed changes

supriyar mentioned this pull request Sep 19, 2020

[quant] Make observer fields as paramteres instead of buffers #45001

Closed

supriyar mentioned this pull request Sep 19, 2020

[quant] Make observer fields as paramteres instead of buffers #45003

Closed

supriyar added 2 commits September 20, 2020 11:54

supriyar mentioned this pull request Sep 21, 2020

[quant] Make observer fields as paramteres instead of buffers #45054

Closed

vkuzo reviewed Sep 22, 2020

View reviewed changes

supriyar requested review from raghuramank100 and vkuzo September 24, 2020 21:38

raghuramank100 approved these changes Sep 29, 2020

View reviewed changes

facebook-github-bot closed this in 489af4d Sep 29, 2020

facebook-github-bot added the merged label Sep 29, 2020

facebook-github-bot deleted the gh/supriyar/181/head branch October 2, 2020 14:17

mruberry added the Merged label Oct 28, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[quant] Add quant APIs to save/load observer state_dict #44846

[quant] Add quant APIs to save/load observer state_dict #44846

Uh oh!

supriyar commented Sep 16, 2020 •

edited

Loading

Uh oh!

dr-ci bot commented Sep 17, 2020 •

edited

Loading

Uh oh!

codecov bot commented Sep 18, 2020 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

raghuramank100 Sep 18, 2020

Uh oh!

Uh oh!

Uh oh!

vkuzo Sep 22, 2020

Uh oh!

supriyar Sep 22, 2020

Uh oh!

vkuzo Sep 22, 2020

Uh oh!

supriyar Sep 22, 2020 •

edited

Loading

Uh oh!

facebook-github-bot commented Sep 29, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

[quant] Add quant APIs to save/load observer state_dict #44846

[quant] Add quant APIs to save/load observer state_dict #44846

Uh oh!

Conversation

supriyar commented Sep 16, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dr-ci bot commented Sep 17, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💊 CI failures summary and remediations

Uh oh!

codecov bot commented Sep 18, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

raghuramank100 Sep 18, 2020

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

vkuzo Sep 22, 2020

Choose a reason for hiding this comment

Uh oh!

supriyar Sep 22, 2020

Choose a reason for hiding this comment

Uh oh!

vkuzo Sep 22, 2020

Choose a reason for hiding this comment

Uh oh!

supriyar Sep 22, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented Sep 29, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

supriyar commented Sep 16, 2020 •

edited

Loading

dr-ci bot commented Sep 17, 2020 •

edited

Loading

codecov bot commented Sep 18, 2020 •

edited

Loading

supriyar Sep 22, 2020 •

edited

Loading