[quant][embedding qat] Support Embedding QAT via FX API #68296

b-koopman · 2021-11-12T22:59:48Z

Stack from ghstack:

Summary:

Support QAT workflow by using torch.fx QAT API. e.g. prepare_qat_fx and convert_fx.

Test Plan:

pytest test/quantization/fx/test_quantize_fx.py -v -k "test_qat_embedding_linear"

Reviewers:
supriyar, HDCharles

Subscribers:

Tasks:

Tags:

Differential Revision: D32404517

Summary: Support QAT workflow by using torch.fx QAT API. e.g. `prepare_qat_fx` and `convert_fx`. Test Plan: `pytest test/quantization/fx/test_quantize_fx.py -v -k "test_qat_embedding_linear"` Reviewers: supriyar, HDCharles Subscribers: Tasks: Tags: [ghstack-poisoned]

pytorch-probot · 2021-11-12T22:59:50Z

CI Flow Status

⚛️ CI Flow

Ruleset - Version: v1
Ruleset - File: https://github.com/pytorch/pytorch/blob/e1f5fa693fa3f98870476430580c80189a73c3ae/.github/generated-ciflow-ruleset.json
PR ciflow labels: ciflow/default

Workflows	Labels (bold enabled)	Status
Triggered Workflows
linux-bionic-cuda11.5-py3.6-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/default`, `ciflow/linux`	✅ triggered
linux-bionic-py3.6-clang9	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/noarch`, `ciflow/xla`	✅ triggered
linux-vulkan-bionic-py3.6-clang9	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/vulkan`	✅ triggered
linux-xenial-cuda11.3-py3.6-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/default`, `ciflow/linux`	✅ triggered
linux-xenial-py3-clang5-mobile-build	`ciflow/all`, `ciflow/default`, `ciflow/linux`, `ciflow/mobile`	✅ triggered
linux-xenial-py3-clang5-mobile-custom-build-static	`ciflow/all`, `ciflow/default`, `ciflow/linux`, `ciflow/mobile`	✅ triggered
linux-xenial-py3.6-clang7-asan	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/sanitizers`	✅ triggered
linux-xenial-py3.6-clang7-onnx	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/onnx`	✅ triggered
linux-xenial-py3.6-gcc5.4	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`	✅ triggered
linux-xenial-py3.6-gcc7	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`	✅ triggered
linux-xenial-py3.6-gcc7-bazel-test	`ciflow/all`, `ciflow/bazel`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`	✅ triggered
pytorch-linux-xenial-py3-clang5-android-ndk-r19c-gradle-custom-build-single	`ciflow/all`, `ciflow/android`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`	✅ triggered
pytorch-linux-xenial-py3-clang5-android-ndk-r19c-gradle-custom-build-single-full-jit	`ciflow/all`, `ciflow/android`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`	✅ triggered
win-vs2019-cpu-py3	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/win`	✅ triggered
win-vs2019-cuda11.3-py3	`ciflow/all`, `ciflow/cuda`, `ciflow/default`, `ciflow/win`	✅ triggered
Skipped Workflows
caffe2-linux-xenial-py3.6-gcc5.4	`ciflow/all`, `ciflow/cpu`, `ciflow/linux`	🚫 skipped
docker-builds	`ciflow/all`	🚫 skipped
ios-12-5-1-arm64	`ciflow/all`, `ciflow/ios`, `ciflow/macos`	🚫 skipped
ios-12-5-1-arm64-coreml	`ciflow/all`, `ciflow/ios`, `ciflow/macos`	🚫 skipped
ios-12-5-1-arm64-custom-ops	`ciflow/all`, `ciflow/ios`, `ciflow/macos`	🚫 skipped
ios-12-5-1-arm64-full-jit	`ciflow/all`, `ciflow/ios`, `ciflow/macos`	🚫 skipped
ios-12-5-1-arm64-metal	`ciflow/all`, `ciflow/ios`, `ciflow/macos`	🚫 skipped
ios-12-5-1-x86-64	`ciflow/all`, `ciflow/ios`, `ciflow/macos`	🚫 skipped
ios-12-5-1-x86-64-coreml	`ciflow/all`, `ciflow/ios`, `ciflow/macos`	🚫 skipped
ios-12-5-1-x86-64-full-jit	`ciflow/all`, `ciflow/ios`, `ciflow/macos`	🚫 skipped
libtorch-linux-bionic-cuda11.5-py3.6-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/libtorch`, `ciflow/linux`	🚫 skipped
libtorch-linux-xenial-cuda10.2-py3.6-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/libtorch`, `ciflow/linux`	🚫 skipped
libtorch-linux-xenial-cuda11.3-py3.6-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/libtorch`, `ciflow/linux`	🚫 skipped
linux-bionic-cuda10.2-py3.9-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/linux`, `ciflow/slow`	🚫 skipped
macos-10-15-py3-arm64	`ciflow/all`, `ciflow/macos`	🚫 skipped
macos-10-15-py3-lite-interpreter-x86-64	`ciflow/all`, `ciflow/macos`	🚫 skipped
macos-11-py3-x86-64	`ciflow/all`, `ciflow/macos`	🚫 skipped
parallelnative-linux-xenial-py3.6-gcc5.4	`ciflow/all`, `ciflow/cpu`, `ciflow/linux`	🚫 skipped
periodic-libtorch-linux-xenial-cuda11.1-py3.6-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/libtorch`, `ciflow/linux`, `ciflow/scheduled`	🚫 skipped
periodic-linux-xenial-cuda10.2-py3-gcc7-slow-gradcheck	`ciflow/all`, `ciflow/cuda`, `ciflow/linux`, `ciflow/scheduled`, `ciflow/slow`, `ciflow/slow-gradcheck`	🚫 skipped
periodic-linux-xenial-cuda11.1-py3.6-gcc7-debug	`ciflow/all`, `ciflow/cuda`, `ciflow/linux`, `ciflow/scheduled`	🚫 skipped
periodic-win-vs2019-cuda11.1-py3	`ciflow/all`, `ciflow/cuda`, `ciflow/scheduled`, `ciflow/win`	🚫 skipped

You can add a comment to the PR and tag @pytorchbot with the following commands:

# ciflow rerun, "ciflow/default" will always be added automatically
@pytorchbot ciflow rerun

# ciflow rerun with additional labels "-l <ciflow/label_name>", which is equivalent to adding these labels manually and trigger the rerun
@pytorchbot ciflow rerun -l ciflow/scheduled -l ciflow/slow

For more information, please take a look at the CI Flow Wiki.

facebook-github-bot · 2021-11-12T22:59:53Z

🔗 Helpful links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/68296
📄 Preview docs built from this PR
📄 Preview C++ docs built from this PR
🔧 Opt-in to CIFlow to control what jobs run on your PRs

💊 CI failures summary and remediations

As of commit e1f5fa6 (more details on the Dr. CI page):

💚 💚 Looks good so far! There are no failures yet. 💚 💚

This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

Summary: Support QAT workflow by using torch.fx QAT API. e.g. `prepare_qat_fx` and `convert_fx`. Test Plan: `pytest test/quantization/fx/test_quantize_fx.py -v -k "test_qat_embedding_linear"` Reviewers: supriyar, HDCharles Subscribers: Tasks: Tags: ghstack-source-id: ebf204e Pull Request resolved: #68296

b-koopman · 2021-11-12T23:00:52Z

@b-koopman has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Summary: Support QAT workflow by using torch.fx QAT API. e.g. `prepare_qat_fx` and `convert_fx`. Test Plan: `pytest test/quantization/fx/test_quantize_fx.py -v -k "test_qat_embedding_linear"` Reviewers: supriyar, HDCharles Subscribers: Tasks: Tags: Differential Revision: [D32404517](https://our.internmc.facebook.com/intern/diff/D32404517) [ghstack-poisoned]

Summary: Support QAT workflow by using torch.fx QAT API. e.g. `prepare_qat_fx` and `convert_fx`. Test Plan: `pytest test/quantization/fx/test_quantize_fx.py -v -k "test_qat_embedding_linear"` Reviewers: supriyar, HDCharles Subscribers: Tasks: Tags: ghstack-source-id: 49f0e8c Pull Request resolved: #68296

b-koopman · 2021-11-15T14:48:49Z

@b-koopman has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Summary: Support QAT workflow by using torch.fx QAT API. e.g. `prepare_qat_fx` and `convert_fx`. Test Plan: `pytest test/quantization/fx/test_quantize_fx.py -v -k "test_qat_embedding_linear"` Reviewers: supriyar, HDCharles Subscribers: Tasks: Tags: Differential Revision: [D32404517](https://our.internmc.facebook.com/intern/diff/D32404517) [ghstack-poisoned]

Summary: Support QAT workflow by using torch.fx QAT API. e.g. `prepare_qat_fx` and `convert_fx`. Test Plan: `pytest test/quantization/fx/test_quantize_fx.py -v -k "test_qat_embedding_linear"` Reviewers: supriyar, HDCharles Subscribers: Tasks: Tags: ghstack-source-id: 13eda3a Pull Request resolved: #68296

b-koopman · 2021-11-15T23:48:26Z

@b-koopman has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

supriyar

So to clarify, for Embedding QAT we don't need a custom mapping for prepare_qat_fx but we need one for convert_fx?
cc @jerryzh168

b-koopman · 2021-11-17T19:11:23Z

So to clarify, for Embedding QAT we don't need a custom mapping for prepare_qat_fx but we need one for convert_fx? cc @jerryzh168

Correct, this is implemented where nn.Embedding -> nn.qat.Embedding is implemented via quantization_patterns for the prepare, and therefore does not need the custom mapping.
https://github.com/pytorch/pytorch/pull/68296/files#diff-5770045cde7e1d2451524b7232e5c1201e0f4624fc16e86758834d390a6074d1R1163

However for nn.qat.Embedding -> nn.quantized.Embedding, in convert_fx, this requires a custom static mapping right now, since embedding QAT is not included in the default mappings (as is also currently needed for the eager QAT case)

Summary: Support QAT workflow by using torch.fx QAT API. e.g. `prepare_qat_fx` and `convert_fx`. Test Plan: `pytest test/quantization/fx/test_quantize_fx.py -v -k "test_qat_embedding_linear"` Reviewers: supriyar, HDCharles Subscribers: Tasks: Tags: Differential Revision: [D32404517](https://our.internmc.facebook.com/intern/diff/D32404517) [ghstack-poisoned]

Summary: Support QAT workflow by using torch.fx QAT API. e.g. `prepare_qat_fx` and `convert_fx`. Test Plan: `pytest test/quantization/fx/test_quantize_fx.py -v -k "test_qat_embedding_linear"` Reviewers: supriyar, HDCharles Subscribers: Tasks: Tags: ghstack-source-id: 84a4462 Pull Request resolved: #68296

b-koopman · 2021-11-19T15:09:12Z

@b-koopman has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Summary: Support QAT workflow by using torch.fx QAT API. e.g. `prepare_qat_fx` and `convert_fx`. Test Plan: `pytest test/quantization/fx/test_quantize_fx.py -v -k "test_qat_embedding_linear"` Reviewers: supriyar, HDCharles Subscribers: Tasks: Tags: Differential Revision: [D32404517](https://our.internmc.facebook.com/intern/diff/D32404517) [ghstack-poisoned]

Summary: Support QAT workflow by using torch.fx QAT API. e.g. `prepare_qat_fx` and `convert_fx`. Test Plan: `pytest test/quantization/fx/test_quantize_fx.py -v -k "test_qat_embedding_linear"` Reviewers: supriyar, HDCharles Subscribers: Tasks: Tags: ghstack-source-id: 5fba23b Pull Request resolved: #68296

Summary: Support QAT workflow by using torch.fx QAT API. e.g. `prepare_qat_fx` and `convert_fx`. Test Plan: `pytest test/quantization/fx/test_quantize_fx.py -v -k "test_qat_embedding_linear"` Reviewers: supriyar, HDCharles Subscribers: Tasks: Tags: Differential Revision: [D32404517](https://our.internmc.facebook.com/intern/diff/D32404517) [ghstack-poisoned]

Summary: Support QAT workflow by using torch.fx QAT API. e.g. `prepare_qat_fx` and `convert_fx`. Test Plan: `pytest test/quantization/fx/test_quantize_fx.py -v -k "test_qat_embedding_linear"` Reviewers: supriyar, HDCharles Subscribers: Tasks: Tags: ghstack-source-id: 5b6f6cc Pull Request resolved: #68296

b-koopman · 2021-12-01T13:49:54Z

@b-koopman has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

malfet · 2021-12-02T22:22:07Z

This broke MacOS test sanity see https://github.com/pytorch/pytorch/runs/4398281864?check_suite_focus=true#step:8:31 , reverting

facebook-github-bot · 2021-12-02T22:28:51Z

This pull request has been reverted by a0367f8. To re-land this change, follow these steps.

facebook-github-bot · 2022-01-06T04:07:56Z

This pull request has been reverted by a0367f8. To re-land this change, follow these steps.

pytorch-probot bot added the ciflow/default label Nov 12, 2021

b-koopman mentioned this pull request Nov 12, 2021

[quant][embedding qat] eager mode QAT for Embeddings #66429

Closed

facebook-github-bot added the cla signed label Nov 12, 2021

facebook-github-bot added the module: fx label Nov 12, 2021

b-koopman requested review from HDCharles and supriyar November 15, 2021 14:51

b-koopman linked an issue Nov 15, 2021 that may be closed by this pull request

[quant] Support QAT for Embedding/EmbeddingBag #61865

Closed

b-koopman requested a review from jingsh November 15, 2021 22:06

b-koopman mentioned this pull request Nov 15, 2021

[quant][embedding qat] Set FakeQuant zeropoint dtype matches observer #68390

Closed

supriyar approved these changes Nov 17, 2021

View reviewed changes

b-koopman mentioned this pull request Nov 18, 2021

[quant][embedding qat] Add benchmarks for QAT Embedding+EmbeddingBag #66560

Closed

b-koopman mentioned this pull request Nov 18, 2021

[quant][embedding QAT] Add Embedding op with fused FakeQuant #66985

Closed

b-koopman mentioned this pull request Nov 18, 2021

[quant][embedding qat] Add Benchmark for fused FakeQuant Embedding op #67945

Closed

b-koopman mentioned this pull request Nov 24, 2021

[quant][embedding qat] fused fakequant EmbeddingBag operator #68900

Closed

b-koopman mentioned this pull request Dec 1, 2021

[quant][embedding qat] Fix bug enforcing quant_min <= zero_point <= quant_max for float zeropoint #68852

Closed

b-koopman mentioned this pull request Dec 1, 2021

[quant][embdding qat] Add FX support for QAT EmbeddingBag #68121

Closed

jingsh approved these changes Dec 1, 2021

View reviewed changes

facebook-github-bot closed this in abda069 Dec 2, 2021

facebook-github-bot added the Reverted label Dec 2, 2021

suo mentioned this pull request Dec 3, 2021

[Meta] CI Revert Tracker #66178

Closed

facebook-github-bot deleted the gh/b-koopman/17/head branch December 6, 2021 15:17

[quant][embedding qat] Support Embedding QAT via FX API #68296

[quant][embedding qat] Support Embedding QAT via FX API #68296

Uh oh!

Conversation

b-koopman commented Nov 12, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-probot bot commented Nov 12, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚛️ CI Flow

Uh oh!

facebook-github-bot commented Nov 12, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful links

💊 CI failures summary and remediations

Uh oh!

b-koopman commented Nov 12, 2021

Uh oh!

b-koopman commented Nov 15, 2021

Uh oh!

b-koopman commented Nov 15, 2021

Uh oh!

supriyar left a comment

Choose a reason for hiding this comment

Uh oh!

b-koopman commented Nov 17, 2021

Uh oh!

b-koopman commented Nov 19, 2021

Uh oh!

b-koopman commented Dec 1, 2021

Uh oh!

malfet commented Dec 2, 2021

Uh oh!

facebook-github-bot commented Dec 2, 2021

Uh oh!

facebook-github-bot commented Jan 6, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

b-koopman commented Nov 12, 2021 •

edited

Loading

pytorch-probot bot commented Nov 12, 2021 •

edited

Loading

facebook-github-bot commented Nov 12, 2021 •

edited

Loading