[quant][pyper] make embedding_bag quantization static #44008

supriyar · 2020-09-02T01:23:23Z

Stack from ghstack:

[quant][pyper] Support quantization of ops in fork-wait subgraph #44048 [quant][pyper] Support quantization of ops in fork-wait subgraph
[quant][pyper] make embedding_bag quantization static #44008 [quant][pyper] make embedding_bag quantization static
[quant][pyper] Support aten::embedding_bag quantization in graph mode #43989 [quant][pyper] Support aten::embedding_bag quantization in graph mode

Summary:
embedding_bag requires only quantization of weights (no dynamic quantization of inputs)
So the type of quantization is essentially static (without calibration)
This will enable pyper to do fc and embedding_bag quantization using the same API call

Test Plan:
python test/test_quantization.py test_embedding_bag

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: D23467019

Summary: embedding_bag requires only quantization of weights (no dynamic quantization of inputs) So the type of quantization is essentially static (without calibration) This will enable pyper to do fc and embedding_bag quantization using the same API call Test Plan: python test/test_quantization.py test_embedding_bag Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

Summary: embedding_bag requires only quantization of weights (no dynamic quantization of inputs) So the type of quantization is essentially static (without calibration) This will enable pyper to do fc and embedding_bag quantization using the same API call Test Plan: python test/test_quantization.py test_embedding_bag Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 8d24934 Pull Request resolved: #44008

dr-ci · 2020-09-02T01:35:09Z

💊 CI failures summary and remediations

As of commit 24789a1 (more details on the Dr. CI page):

1/2 failures possibly* introduced in this PR
- 1/1 non-CircleCI failure(s)
1/2 broken upstream at merge base 6474057 on Sep 04 from 3:37am to 11:28am PDT (13 commits; f8f35fd - 0c2bc4f)

🚧 1 fixed upstream failure:

These were probably caused by upstream breakages that were already fixed.

Please rebase on the viable/strict branch (expand for instructions)

If your commit is newer than viable/strict, you can try basing on an older, stable commit:

git fetch https://github.com/pytorch/pytorch viable/strict
git rebase --onto FETCH_HEAD $(git merge-base origin/master HEAD)

If your commit is older than viable/strict:

git fetch https://github.com/pytorch/pytorch viable/strict
git rebase FETCH_HEAD

Check out the recency history of this "viable master" tracking branch.

pytorch_xla_linux_bionic_py3_6_clang9_test on Sep 04 from 3:37am to 11:28am PDT (13 commits; f8f35fd - 0c2bc4f)
- 🔁 rerun

ci.pytorch.org: 1 failed

Failed: pr/pytorch-linux-bionic-rocm3.7-py3.6

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker or post in the (internal) Dr. CI Users group.

See how this bot performed.

This comment has been revised 16 times.

Summary: embedding_bag requires only quantization of weights (no dynamic quantization of inputs) So the type of quantization is essentially static (without calibration) This will enable pyper to do fc and embedding_bag quantization using the same API call Test Plan: python test/test_quantization.py test_embedding_bag Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D23467019](https://our.internmc.facebook.com/intern/diff/D23467019) [ghstack-poisoned]

codecov · 2020-09-02T22:22:28Z

Codecov Report

Merging #44008 into gh/supriyar/171/base will decrease coverage by 0.08%.
The diff coverage is n/a.

@@                   Coverage Diff                    @@
##           gh/supriyar/171/base   #44008      +/-   ##
========================================================
- Coverage                 69.35%   69.26%   -0.09%     
========================================================
  Files                       381      381              
  Lines                     47313    47239      -74     
========================================================
- Hits                      32812    32722      -90     
- Misses                    14501    14517      +16

Impacted Files	Coverage Δ
torch/testing/_internal/common_nn.py	`79.20% <0.00%> (-3.88%)`	⬇️
torch/testing/_internal/common_utils.py	`76.61% <0.00%> (-0.62%)`	⬇️
...ch/testing/_internal/common_methods_invocations.py	`91.12% <0.00%> (-0.59%)`	⬇️
torch/jit/quantized.py	`56.44% <0.00%> (-0.17%)`	⬇️
torch/quantization/fx/quantize.py	`96.51% <0.00%> (-0.15%)`	⬇️
torch/jit/annotations.py	`92.27% <0.00%> (-0.14%)`	⬇️
torch/jit/_recursive.py	`93.96% <0.00%> (-0.06%)`	⬇️
torch/jit/frontend.py	`90.82% <0.00%> (-0.04%)`	⬇️
torch/jit/_script.py	`91.00% <0.00%> (-0.03%)`	⬇️
torch/nn/modules/activation.py	`96.62% <0.00%> (-0.02%)`	⬇️
... and 27 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 119b273...24789a1. Read the comment docs.

raghuramank100 · 2020-09-03T18:34:56Z

test/quantization/test_quantize_jit.py

+            int8_qconfig = QConfig(activation=PlaceholderObserver.with_args(dtype=torch.float,
+                                                                            custom_op_name="embedding_bag_byte"),
+                                   weight=PlaceholderObserver.with_args(custom_op_name="embedding_bag_byte"))
+            m = prepare_jit(m, {'embedding1' : int4_qconfig, 'embedding2' : int8_qconfig})


What about eager mode? Currently we expose embedding bag as a module in torch/nn/quantized/dynamic. Should we change that too?. Strictly speaking this case straddles the boundary of static vs dynamic: Output activations are in fp32 (like dynamic). Inputs are addresses and require no quantization. Weights alone are quantized.

I'm not sure about eager mode. Currently for static quant we expect the users to provide calibration fn when they call quantize. Weight only quantization shouldn't require that step. So in that sense it fits better into dynamic.

Summary: embedding_bag requires only quantization of weights (no dynamic quantization of inputs) So the type of quantization is essentially static (without calibration) This will enable pyper to do fc and embedding_bag quantization using the same API call Test Plan: python test/test_quantization.py test_embedding_bag Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D23467019](https://our.internmc.facebook.com/intern/diff/D23467019) [ghstack-poisoned]

facebook-github-bot · 2020-09-05T20:13:31Z

This pull request has been merged in 164b96c.

supriyar requested a review from apaszke as a code owner September 2, 2020 01:23

supriyar mentioned this pull request Sep 2, 2020

[quant][pyper] Support aten::embedding_bag quantization in graph mode #43989

Closed

facebook-github-bot added the oncall: jit Add this issue/PR to JIT oncall triage queue label Sep 2, 2020

supriyar mentioned this pull request Sep 2, 2020

[quant][pyper] Support quantization of ops in fork-wait subgraph #44048

Closed

supriyar requested review from raghuramank100 and vkuzo September 2, 2020 20:50

vkuzo approved these changes Sep 3, 2020

View reviewed changes

raghuramank100 reviewed Sep 3, 2020

View reviewed changes

raghuramank100 approved these changes Sep 3, 2020

View reviewed changes

facebook-github-bot closed this in 164b96c Sep 5, 2020

facebook-github-bot added the merged label Sep 5, 2020

facebook-github-bot deleted the gh/supriyar/171/head branch September 9, 2020 14:18

mruberry added the Merged label Oct 28, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[quant][pyper] make embedding_bag quantization static #44008

[quant][pyper] make embedding_bag quantization static #44008

Uh oh!

supriyar commented Sep 2, 2020 •

edited

Loading

Uh oh!

dr-ci bot commented Sep 2, 2020 •

edited

Loading

Uh oh!

codecov bot commented Sep 2, 2020 •

edited

Loading

Uh oh!

raghuramank100 Sep 3, 2020

Uh oh!

supriyar Sep 3, 2020

Uh oh!

facebook-github-bot commented Sep 5, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

[quant][pyper] make embedding_bag quantization static #44008

[quant][pyper] make embedding_bag quantization static #44008

Uh oh!

Conversation

supriyar commented Sep 2, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dr-ci bot commented Sep 2, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💊 CI failures summary and remediations

🚧 1 fixed upstream failure:

ci.pytorch.org: 1 failed

Uh oh!

codecov bot commented Sep 2, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

raghuramank100 Sep 3, 2020

Choose a reason for hiding this comment

Uh oh!

supriyar Sep 3, 2020

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented Sep 5, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

supriyar commented Sep 2, 2020 •

edited

Loading

dr-ci bot commented Sep 2, 2020 •

edited

Loading

codecov bot commented Sep 2, 2020 •

edited

Loading