[quant] Create nn.quantized.dynamic.EmbeddingBag #43088

supriyar · 2020-08-15T00:13:55Z

Stack from ghstack:

[quant] Add benchmarks for quantized embeddingbag module #43296 [quant] Add benchmarks for quantized embeddingbag module
[quant] Enable from_float for quantized Embedding_Bag #43176 [quant] Enable from_float for quantized Embedding_Bag
[quant] Make offsets an optional argument #43090 [quant] Make offsets an optional argument
[quant] Create nn.quantized.dynamic.EmbeddingBag #43088 [quant] Create nn.quantized.dynamic.EmbeddingBag

Summary:
Create quantized module that the user can use to perform embedding bag quantization
The module uses the EmbeddingPackedParams to store the weights which can be serialized /deserialized
using TorchBind custom classes (C++ get/setstate code)
Following PR will add support for from_float to convert from float to quantized module

Test Plan:
python test/test_quantization.py TestDynamicQuantizedModule.test_embedding_bag_api

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: D23167519

Summary: Create quantized module that the user can use to perform embedding bag quantization The module uses the EmbeddingPackedParams to store the weights which can be serialized /deserialized using TorchBind custom classes (C++ get/setstate code) Following PR will add support for `from_float` to convert from float to quantized module Test Plan: python test/test_quantization.py TestDynamicQuantizedModule.test_embedding_bag_api Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

Summary: Create quantized module that the user can use to perform embedding bag quantization The module uses the EmbeddingPackedParams to store the weights which can be serialized /deserialized using TorchBind custom classes (C++ get/setstate code) Following PR will add support for `from_float` to convert from float to quantized module Test Plan: python test/test_quantization.py TestDynamicQuantizedModule.test_embedding_bag_api Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 531e030 Pull Request resolved: #43088

dr-ci · 2020-08-15T00:23:23Z

💊 CI failures summary and remediations

As of commit 5f44c98 (more details on the Dr. CI page):

✅ None of the CI failures appear to be your fault 💚

1/1 broken upstream at merge base c8bc298 since Aug 20

🚧 1 ongoing upstream failure:

These were probably caused by upstream breakages that are not fixed yet:

pytorch_linux_backward_compatibility_check_test since Aug 20
- 🔁 rerun

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker or post in the (internal) Dr. CI Users group.

See how this bot performed.

This comment has been revised 17 times.

Summary: Create quantized module that the user can use to perform embedding bag quantization The module uses the EmbeddingPackedParams to store the weights which can be serialized /deserialized using TorchBind custom classes (C++ get/setstate code) Following PR will add support for `from_float` to convert from float to quantized module Test Plan: python test/test_quantization.py TestDynamicQuantizedModule.test_embedding_bag_api Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

Summary: Create quantized module that the user can use to perform embedding bag quantization The module uses the EmbeddingPackedParams to store the weights which can be serialized /deserialized using TorchBind custom classes (C++ get/setstate code) Following PR will add support for `from_float` to convert from float to quantized module Test Plan: python test/test_quantization.py TestDynamicQuantizedModule.test_embedding_bag_api Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D23167519](https://our.internmc.facebook.com/intern/diff/D23167519) [ghstack-poisoned]

vkuzo

lg!

vkuzo · 2020-08-18T18:24:56Z

test/quantization/test_quantized_module.py

+        module_out = qemb(indices, offsets)
+
+        # Call the qembedding_bag operator directly
+        ref = torch.ops.quantized.embedding_bag_byte(w_packed, indices, offsets, mode=0,


optional: do we need to check other modes as well? Not sure if that's not needed / done elsewhere / etc

We currently don't have support for 4-bit qtensor. Once we add that then we can repeat this check for that op as well.

vkuzo · 2020-08-18T18:25:20Z

torch/_utils.py

        _, scale, zero_point = quantizer_params
        tensor = torch._empty_affine_quantized(size, scale=scale, zero_point=zero_point, dtype=storage.dtype)
-    elif qscheme == torch.per_channel_affine:
+    elif qscheme == torch.per_channel_affine or qscheme == torch.per_channel_affine_float_qparams:


nit: qscheme in (a, b) ?

vkuzo · 2020-08-18T18:27:57Z

torch/nn/quantized/dynamic/modules/embeddingbag.py

+        else:
+            raise RuntimeError('Unsupported dtype on dynamic quantized embedding_bag!')
+
+    def forward(self, x):


is this just to satisfy the contract of nn.Module?

vkuzo · 2020-08-18T18:28:29Z

torch/nn/quantized/dynamic/modules/embeddingbag.py

+
+        if _weight is None:
+            scales = torch.ones(num_embeddings, dtype=torch.float)
+            zero_points = torch.ones(num_embeddings, dtype=torch.float)


optional: should this be zeros?

In this case it should be fine because we compute and store the bias as zero_points * scale * -1 so keeping it non-zero just makes sure we test pre-pack of some actual values.

torch/tensor.py

vkuzo · 2020-08-18T18:29:34Z

torch/tensor.py

                                    self.q_scale(),
                                    self.q_zero_point())
-            elif self.qscheme() == torch.per_channel_affine:
+            elif self.qscheme() == torch.per_channel_affine or self.qscheme() == torch.per_channel_affine_float_qparams:


Summary: Create quantized module that the user can use to perform embedding bag quantization The module uses the EmbeddingPackedParams to store the weights which can be serialized /deserialized using TorchBind custom classes (C++ get/setstate code) Following PR will add support for `from_float` to convert from float to quantized module Test Plan: python test/test_quantization.py TestDynamicQuantizedModule.test_embedding_bag_api Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D23167519](https://our.internmc.facebook.com/intern/diff/D23167519) [ghstack-poisoned]

facebook-github-bot · 2020-08-21T20:18:26Z

This pull request has been merged in 4db8ca1.

supriyar requested a review from apaszke as a code owner August 15, 2020 00:13

facebook-github-bot added the oncall: jit Add this issue/PR to JIT oncall triage queue label Aug 15, 2020

supriyar mentioned this pull request Aug 15, 2020

[quant] Make offsets an optional argument #43090

Closed

supriyar added 2 commits August 16, 2020 21:16

supriyar requested review from jerryzh168, raghuramank100, vkuzo and z-a-f August 17, 2020 21:33

supriyar mentioned this pull request Aug 17, 2020

[quant] Enable from_float for quantized Embedding_Bag #43176

Closed

vkuzo approved these changes Aug 18, 2020

View reviewed changes

supriyar requested a review from radkris-git August 18, 2020 20:55

This was referenced Aug 19, 2020

[quant] Add benchmarks for quantized embeddingbag module #43292

Closed

[quant] Add benchmarks for quantized embeddingbag module #43295

Closed

[quant] Add benchmarks for quantized embeddingbag module #43296

Closed

facebook-github-bot closed this in 4db8ca1 Aug 21, 2020

facebook-github-bot added the merged label Aug 21, 2020

supriyar mentioned this pull request Aug 21, 2020

[quant] Support set API for EmbeddingBag quantization #43433

Closed

facebook-github-bot deleted the gh/supriyar/160/head branch August 25, 2020 14:14

mruberry added the Merged label Oct 28, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[quant] Create nn.quantized.dynamic.EmbeddingBag #43088

[quant] Create nn.quantized.dynamic.EmbeddingBag #43088

Uh oh!

supriyar commented Aug 15, 2020 •

edited

Loading

Uh oh!

dr-ci bot commented Aug 15, 2020 •

edited

Loading

Uh oh!

vkuzo left a comment

Uh oh!

vkuzo Aug 18, 2020

Uh oh!

supriyar Aug 18, 2020

Uh oh!

vkuzo Aug 18, 2020

Uh oh!

vkuzo Aug 18, 2020

Uh oh!

supriyar Aug 18, 2020

Uh oh!

vkuzo Aug 18, 2020

Uh oh!

supriyar Aug 18, 2020

Uh oh!

Uh oh!

vkuzo Aug 18, 2020

Uh oh!

facebook-github-bot commented Aug 21, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

[quant] Create nn.quantized.dynamic.EmbeddingBag #43088

[quant] Create nn.quantized.dynamic.EmbeddingBag #43088

Uh oh!

Conversation

supriyar commented Aug 15, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dr-ci bot commented Aug 15, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💊 CI failures summary and remediations

🚧 1 ongoing upstream failure:

Uh oh!

vkuzo left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented Aug 21, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

supriyar commented Aug 15, 2020 •

edited

Loading

dr-ci bot commented Aug 15, 2020 •

edited

Loading