[Quant][fx] Add lowering for Linear-Bn1d in QAT mode #73509

andrewor14 · 2022-02-28T15:31:43Z

Stack from ghstack (oldest at bottom):

-> [Quant][fx] Add lowering for Linear-Bn1d in QAT mode #73509

Summary: This adds functionality to lower reference models
involving the Linear-Bn1d pattern in FX QAT mode. This follows
#72431 and #72796, which add Linear-Bn1d fusion functionality
to eager QAT mode.

Test Plan: python test/test_quantization.py TestQuantizeFxOps.test_linear_module

Reviewers: jerryzh168, vkuzo

Subscribers: jerryzh168, vkuzo

Tasks: #66483

Differential Revision: D34591251

Summary: This adds functionality to lower reference models involving the Linear-Bn1d pattern in FX QAT mode. Test Plan: python test/test_quantization.py TestQuantizeFxOps.test_linear_module Reviewers: jerryzh168, vkuzo Subscribers: jerryzh168, vkuzo Tasks: #66483 [ghstack-poisoned]

pytorch-bot · 2022-02-28T15:31:47Z

CI Flow Status

⚛️ CI Flow

Ruleset - Version: v1
Ruleset - File: https://github.com/pytorch/pytorch/blob/61fe307a44432752fe3af830932ba32e8dc49dd1/.github/generated-ciflow-ruleset.json
PR ciflow labels: ciflow/default
Add ciflow labels to this PR to trigger more builds:

Workflows	Labels (bold enabled)	Status
Triggered Workflows
linux-binary-conda	`ciflow/binaries`, `ciflow/binaries_conda`, `ciflow/default`	✅ triggered
linux-binary-libtorch-cxx11-abi	`ciflow/binaries`, `ciflow/binaries_libtorch`, `ciflow/default`	✅ triggered
linux-binary-libtorch-pre-cxx11	`ciflow/binaries`, `ciflow/binaries_libtorch`, `ciflow/default`	✅ triggered
linux-binary-manywheel	`ciflow/binaries`, `ciflow/binaries_wheel`, `ciflow/default`	✅ triggered
linux-bionic-py3.7-clang9	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/noarch`, `ciflow/trunk`	✅ triggered
linux-bionic-rocm4.5-py3.7	`ciflow/all`, `ciflow/default`, `ciflow/linux`, `ciflow/rocm`, `ciflow/trunk`	✅ triggered
linux-docs	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/docs`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
linux-vulkan-bionic-py3.7-clang9	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`, `ciflow/vulkan`	✅ triggered
linux-xenial-cuda11.3-py3.7-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
linux-xenial-cuda11.3-py3.7-gcc7-bazel-test	`ciflow/all`, `ciflow/bazel`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
linux-xenial-py3-clang5-mobile-build	`ciflow/all`, `ciflow/default`, `ciflow/linux`, `ciflow/mobile`, `ciflow/trunk`	✅ triggered
linux-xenial-py3-clang5-mobile-custom-build-static	`ciflow/all`, `ciflow/default`, `ciflow/linux`, `ciflow/mobile`, `ciflow/trunk`	✅ triggered
linux-xenial-py3.7-clang7-asan	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/sanitizers`, `ciflow/trunk`	✅ triggered
linux-xenial-py3.7-clang7-onnx	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/onnx`, `ciflow/trunk`	✅ triggered
linux-xenial-py3.7-gcc5.4	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
linux-xenial-py3.7-gcc7	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
linux-xenial-py3.7-gcc7-no-ops	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
macos-arm64-binary-conda	`ciflow/binaries`, `ciflow/binaries_conda`, `ciflow/default`	✅ triggered
macos-arm64-binary-wheel	`ciflow/binaries`, `ciflow/binaries_wheel`, `ciflow/default`	✅ triggered
macos-binary-conda	`ciflow/binaries`, `ciflow/binaries_conda`, `ciflow/default`	✅ triggered
macos-binary-libtorch-cxx11-abi	`ciflow/binaries`, `ciflow/binaries_libtorch`, `ciflow/default`	✅ triggered
macos-binary-libtorch-pre-cxx11	`ciflow/binaries`, `ciflow/binaries_libtorch`, `ciflow/default`	✅ triggered
macos-binary-wheel	`ciflow/binaries`, `ciflow/binaries_wheel`, `ciflow/default`	✅ triggered
pytorch-linux-xenial-py3-clang5-android-ndk-r19c-gradle-custom-build-single	`ciflow/all`, `ciflow/android`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
pytorch-linux-xenial-py3-clang5-android-ndk-r19c-gradle-custom-build-single-full-jit	`ciflow/all`, `ciflow/android`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
win-vs2019-cpu-py3	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/trunk`, `ciflow/win`	✅ triggered
win-vs2019-cuda11.3-py3	`ciflow/all`, `ciflow/cuda`, `ciflow/default`, `ciflow/trunk`, `ciflow/win`	✅ triggered
windows-binary-libtorch-cxx11-abi	`ciflow/binaries`, `ciflow/binaries_libtorch`, `ciflow/default`	✅ triggered
windows-binary-libtorch-pre-cxx11	`ciflow/binaries`, `ciflow/binaries_libtorch`, `ciflow/default`	✅ triggered
windows-binary-wheel	`ciflow/binaries`, `ciflow/binaries_wheel`, `ciflow/default`	✅ triggered
Skipped Workflows
caffe2-linux-xenial-py3.7-gcc5.4	`ciflow/all`, `ciflow/cpu`, `ciflow/linux`, `ciflow/trunk`	🚫 skipped
docker-builds	`ciflow/all`, `ciflow/trunk`	🚫 skipped
ios-12-5-1-arm64	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/scheduled`	🚫 skipped
ios-12-5-1-arm64-coreml	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/scheduled`	🚫 skipped
ios-12-5-1-arm64-custom-ops	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/scheduled`	🚫 skipped
ios-12-5-1-arm64-metal	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/scheduled`	🚫 skipped
ios-12-5-1-x86-64	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
ios-12-5-1-x86-64-coreml	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
libtorch-linux-xenial-cuda10.2-py3.7-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/libtorch`, `ciflow/linux`, `ciflow/trunk`	🚫 skipped
libtorch-linux-xenial-cuda11.3-py3.7-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/libtorch`, `ciflow/linux`, `ciflow/trunk`	🚫 skipped
linux-bionic-cuda10.2-py3.9-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/linux`, `ciflow/slow`, `ciflow/trunk`	🚫 skipped
linux-docs-push	`ciflow/all`, `ciflow/cpu`, `ciflow/linux`, `ciflow/scheduled`	🚫 skipped
linux-xenial-cuda11.3-py3.7-gcc7-no-ops	`ciflow/all`, `ciflow/cuda`, `ciflow/linux`, `ciflow/trunk`	🚫 skipped
macos-10-15-py3-arm64	`ciflow/all`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
macos-10-15-py3-lite-interpreter-x86-64	`ciflow/all`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
macos-11-py3-x86-64	`ciflow/all`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
parallelnative-linux-xenial-py3.7-gcc5.4	`ciflow/all`, `ciflow/cpu`, `ciflow/linux`, `ciflow/trunk`	🚫 skipped
periodic-libtorch-linux-bionic-cuda11.5-py3.7-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/libtorch`, `ciflow/linux`, `ciflow/scheduled`	🚫 skipped
periodic-linux-bionic-cuda11.5-py3.7-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/linux`, `ciflow/scheduled`	🚫 skipped
periodic-linux-xenial-cuda10.2-py3-gcc7-slow-gradcheck	`ciflow/all`, `ciflow/cuda`, `ciflow/linux`, `ciflow/scheduled`, `ciflow/slow`, `ciflow/slow-gradcheck`	🚫 skipped
periodic-linux-xenial-cuda11.3-py3.7-gcc7-debug	`ciflow/all`, `ciflow/cuda`, `ciflow/linux`, `ciflow/scheduled`	🚫 skipped
periodic-win-vs2019-cuda11.5-py3	`ciflow/all`, `ciflow/cuda`, `ciflow/scheduled`, `ciflow/win`	🚫 skipped
pytorch-linux-xenial-py3-clang5-android-ndk-r19c-build	`ciflow/all`, `ciflow/android`, `ciflow/cpu`, `ciflow/linux`, `ciflow/trunk`	🚫 skipped
pytorch-xla-linux-bionic-py3.7-clang8	`ciflow/all`, `ciflow/cpu`, `ciflow/linux`, `ciflow/trunk`, `ciflow/xla`	🚫 skipped

facebook-github-bot · 2022-02-28T15:31:49Z

🔗 Helpful links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/73509
📄 Preview docs built from this PR
📄 Preview C++ docs built from this PR
🔧 Opt-in to CIFlow to control what jobs run on your PRs

💊 CI failures summary and remediations

As of commit 3909f2a (more details on the Dr. CI page):

💚 💚 Looks good so far! There are no failures yet. 💚 💚

This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

Summary: This adds functionality to lower reference models involving the Linear-Bn1d pattern in FX QAT mode. Test Plan: python test/test_quantization.py TestQuantizeFxOps.test_linear_module Reviewers: jerryzh168, vkuzo Subscribers: jerryzh168, vkuzo Tasks: #66483 ghstack-source-id: d0c2c3d Pull Request resolved: #73509

torch/nn/intrinsic/qat/modules/linear_fused.py

vkuzo

thanks for fixing this!

jerryzh168

looks good!

jerryzh168 · 2022-02-28T18:53:28Z

test/quantization/fx/test_quantize_fx.py

-        for model, quant_type in options:
-            self.checkGraphModeFxOp(
-                model, data, quant_type, quantized_nodes[quant_type])
+        for quant_type in self.all_quant_types:


thanks, this refactor looks very clean

Summary: This adds functionality to lower reference models involving the Linear-Bn1d pattern in FX QAT mode. This follows #72431 and #72796, which add Linear-Bn1d fusion functionality to eager QAT mode. Test Plan: python test/test_quantization.py TestQuantizeFxOps.test_linear_module Reviewers: jerryzh168, vkuzo Subscribers: jerryzh168, vkuzo Tasks: #66483 [ghstack-poisoned]

andrewor14 · 2022-03-02T21:07:08Z

@andrewor14 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Summary: This adds functionality to lower reference models involving the Linear-Bn1d pattern in FX QAT mode. Test Plan: python test/test_quantization.py TestQuantizeFxOps.test_linear_module Reviewers: jerryzh168, vkuzo Subscribers: jerryzh168, vkuzo Tasks: #66483 ghstack-source-id: 03832f1 Pull Request resolved: #73509

Summary: This adds functionality to lower reference models involving the Linear-Bn1d pattern in FX QAT mode. This follows #72431 and #72796, which add Linear-Bn1d fusion functionality to eager QAT mode. Test Plan: python test/test_quantization.py TestQuantizeFxOps.test_linear_module Reviewers: jerryzh168, vkuzo Subscribers: jerryzh168, vkuzo Tasks: #66483 Differential Revision: [D34591251](https://our.internmc.facebook.com/intern/diff/D34591251) [ghstack-poisoned]

Summary: This adds functionality to lower reference models involving the Linear-Bn1d pattern in FX QAT mode. Test Plan: python test/test_quantization.py TestQuantizeFxOps.test_linear_module Reviewers: jerryzh168, vkuzo Subscribers: jerryzh168, vkuzo Tasks: #66483 ghstack-source-id: 126b205 Pull Request resolved: #73509

andrewor14 · 2022-03-02T23:04:44Z

@andrewor14 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

torch/ao/quantization/fx/_lower_to_native_backend.py

jerryzh168 · 2022-03-03T00:05:14Z

looks like we do not have a test comparing against eager mode here for linear bn fusion, that is why there is no test failing. maybe we could add a eager mode test, like https://github.com/pytorch/pytorch/blob/master/test/quantization/fx/test_quantize_fx.py#L955

Summary: This adds functionality to lower reference models involving the Linear-Bn1d pattern in FX QAT mode. This follows #72431 and #72796, which add Linear-Bn1d fusion functionality to eager QAT mode. Test Plan: python test/test_quantization.py TestQuantizeFxOps.test_linear_module Reviewers: jerryzh168, vkuzo Subscribers: jerryzh168, vkuzo Tasks: #66483 Differential Revision: [D34591251](https://our.internmc.facebook.com/intern/diff/D34591251) [ghstack-poisoned]

Summary: This adds functionality to lower reference models involving the Linear-Bn1d pattern in FX QAT mode. Test Plan: python test/test_quantization.py TestQuantizeFxOps.test_linear_module Reviewers: jerryzh168, vkuzo Subscribers: jerryzh168, vkuzo Tasks: #66483 ghstack-source-id: 4962de7 Pull Request resolved: #73509

Summary: This adds functionality to lower reference models involving the Linear-Bn1d pattern in FX QAT mode. This follows #72431 and #72796, which add Linear-Bn1d fusion functionality to eager QAT mode. Test Plan: python test/test_quantization.py TestQuantizeFxOps.test_linear_module Reviewers: jerryzh168, vkuzo Subscribers: jerryzh168, vkuzo Tasks: #66483 Differential Revision: [D34591251](https://our.internmc.facebook.com/intern/diff/D34591251) [ghstack-poisoned]

Summary: This adds functionality to lower reference models involving the Linear-Bn1d pattern in FX QAT mode. Test Plan: python test/test_quantization.py TestQuantizeFxOps.test_linear_module Reviewers: jerryzh168, vkuzo Subscribers: jerryzh168, vkuzo Tasks: #66483 ghstack-source-id: 56e1bfa Pull Request resolved: #73509

andrewor14 · 2022-03-03T21:54:49Z

@andrewor14 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Summary: This adds functionality to lower reference models involving the Linear-Bn1d pattern in FX QAT mode. This follows #72431 and #72796, which add Linear-Bn1d fusion functionality to eager QAT mode. Test Plan: python test/test_quantization.py TestQuantizeFxOps.test_linear_module Reviewers: jerryzh168, vkuzo Subscribers: jerryzh168, vkuzo Tasks: #66483 Differential Revision: [D34591251](https://our.internmc.facebook.com/intern/diff/D34591251) [ghstack-poisoned]

andrewor14 · 2022-03-04T17:17:07Z

@andrewor14 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Summary: This adds functionality to lower reference models involving the Linear-Bn1d pattern in FX QAT mode. Test Plan: python test/test_quantization.py TestQuantizeFxOps.test_linear_module Reviewers: jerryzh168, vkuzo Subscribers: jerryzh168, vkuzo Tasks: #66483 ghstack-source-id: 6ec71d9 Pull Request resolved: #73509

Summary: This adds functionality to lower reference models involving the Linear-Bn1d pattern in FX QAT mode. This follows #72431 and #72796, which add Linear-Bn1d fusion functionality to eager QAT mode. Test Plan: python test/test_quantization.py TestQuantizeFxOps.test_linear_module Reviewers: jerryzh168, vkuzo Subscribers: jerryzh168, vkuzo Tasks: #66483 Differential Revision: [D34591251](https://our.internmc.facebook.com/intern/diff/D34591251) [ghstack-poisoned]

Summary: This adds functionality to lower reference models involving the Linear-Bn1d pattern in FX QAT mode. Test Plan: python test/test_quantization.py TestQuantizeFxOps.test_linear_module Reviewers: jerryzh168, vkuzo Subscribers: jerryzh168, vkuzo Tasks: #66483 ghstack-source-id: dd35e72 Pull Request resolved: #73509

andrewor14 · 2022-03-04T17:21:59Z

@andrewor14 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Summary: Pull Request resolved: #73509 This adds functionality to lower reference models involving the Linear-Bn1d pattern in FX QAT mode. This follows #72431 and #72796, which add Linear-Bn1d fusion functionality to eager QAT mode. Test Plan: python test/test_quantization.py TestQuantizeFxOps.test_linear_module Imported from OSS Reviewed By: dagitses Differential Revision: D34591251 fbshipit-source-id: 39144485f9954ee1830c8b414e724560fd7e47bf

Summary: Pull Request resolved: pytorch/pytorch#73509 This adds functionality to lower reference models involving the Linear-Bn1d pattern in FX QAT mode. This follows pytorch/pytorch#72431 and pytorch/pytorch#72796, which add Linear-Bn1d fusion functionality to eager QAT mode. Test Plan: python test/test_quantization.py TestQuantizeFxOps.test_linear_module Imported from OSS Reviewed By: dagitses Differential Revision: D34591251 fbshipit-source-id: 39144485f9954ee1830c8b414e724560fd7e47bf (cherry picked from commit b97a39b4d9df00e045fab4c01eca88e562ca2c02)

andrewor14 requested review from albanD and jbschlosser as code owners February 28, 2022 15:31

pytorch-bot bot added the ciflow/default label Feb 28, 2022

facebook-github-bot added the cla signed label Feb 28, 2022

facebook-github-bot added the module: fx label Feb 28, 2022

andrewor14 requested review from jerryzh168 and vkuzo and removed request for albanD, jbschlosser and vkuzo February 28, 2022 15:34

andrewor14 added release notes: quantization release notes category topic: improvements topic category labels Feb 28, 2022

vkuzo reviewed Feb 28, 2022

View reviewed changes

torch/nn/intrinsic/qat/modules/linear_fused.py Outdated Show resolved Hide resolved

vkuzo approved these changes Feb 28, 2022

View reviewed changes

jerryzh168 approved these changes Feb 28, 2022

View reviewed changes

jerryzh168 reviewed Feb 28, 2022

View reviewed changes

andrewor14 mentioned this pull request Mar 2, 2022

[Quant][fx] Add lowering for Linear-Bn1d in QAT mode #73680

Closed

andrewor14 mentioned this pull request Mar 2, 2022

[quant][fx] Add lowering support for qat and fused convs #73527

Closed

jerryzh168 reviewed Mar 2, 2022

View reviewed changes

torch/ao/quantization/fx/_lower_to_native_backend.py Outdated Show resolved Hide resolved

pytorchmergebot closed this in cedce3b Mar 7, 2022

andrewor14 deleted the gh/andrewor14/7/head branch March 7, 2022 16:11

WBobby mentioned this pull request Aug 17, 2022

Add ROCm5.2.3/AMDGPU support for PyTorch WBobby/pytorch#2

Closed

[Quant][fx] Add lowering for Linear-Bn1d in QAT mode #73509

[Quant][fx] Add lowering for Linear-Bn1d in QAT mode #73509

Uh oh!

Conversation

andrewor14 commented Feb 28, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Feb 28, 2022

⚛️ CI Flow

Uh oh!

facebook-github-bot commented Feb 28, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful links

💊 CI failures summary and remediations

Uh oh!

Uh oh!

vkuzo left a comment

Choose a reason for hiding this comment

Uh oh!

jerryzh168 left a comment

Choose a reason for hiding this comment

Uh oh!

jerryzh168 Feb 28, 2022

Choose a reason for hiding this comment

Uh oh!

andrewor14 commented Mar 2, 2022

Uh oh!

andrewor14 commented Mar 2, 2022

Uh oh!

Uh oh!

jerryzh168 commented Mar 3, 2022

Uh oh!

andrewor14 commented Mar 3, 2022

Uh oh!

andrewor14 commented Mar 4, 2022

Uh oh!

andrewor14 commented Mar 4, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

andrewor14 commented Feb 28, 2022 •

edited

Loading

facebook-github-bot commented Feb 28, 2022 •

edited

Loading