[Quant] Refactor reference module mapping #72755

terrychenism · 2022-02-11T23:31:42Z

Stack from ghstack (oldest at bottom):

-> [Quant] Refactor reference module mapping #72755

Summary:

Add is_reference flag in convert function
if is_reference = true, using the default reference module mapping
add linear reference module and test case

Test Plan:
python3 test/test_quantization.py TestQuantizeEagerOps.test_conv_transpose_2d

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: D34188856

Summary: Add is_refernece flag in convert function Test Plan: python3 test/test_quantization.py TestQuantizeEagerOps.test_conv_transpose_2d Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

pytorch-bot · 2022-02-11T23:31:46Z

CI Flow Status

⚛️ CI Flow

Ruleset - Version: v1
Ruleset - File: https://github.com/pytorch/pytorch/blob/385521535ee6ef08980dcaa3e636cfad829f6ccd/.github/generated-ciflow-ruleset.json
PR ciflow labels: ciflow/default
Add ciflow labels to this PR to trigger more builds:

Workflows	Labels (bold enabled)	Status
Triggered Workflows
linux-binary-conda	`ciflow/binaries`, `ciflow/binaries_conda`, `ciflow/default`	✅ triggered
linux-binary-libtorch-cxx11-abi	`ciflow/binaries`, `ciflow/binaries_libtorch`, `ciflow/default`	✅ triggered
linux-binary-libtorch-pre-cxx11	`ciflow/binaries`, `ciflow/binaries_libtorch`, `ciflow/default`	✅ triggered
linux-binary-manywheel	`ciflow/binaries`, `ciflow/binaries_wheel`, `ciflow/default`	✅ triggered
linux-bionic-py3.7-clang9	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/noarch`, `ciflow/trunk`, `ciflow/xla`	✅ triggered
linux-bionic-rocm4.5-py3.7	`ciflow/all`, `ciflow/default`, `ciflow/linux`, `ciflow/rocm`, `ciflow/trunk`	✅ triggered
linux-docs	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/docs`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
linux-vulkan-bionic-py3.7-clang9	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`, `ciflow/vulkan`	✅ triggered
linux-xenial-cuda11.3-py3.7-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
linux-xenial-cuda11.3-py3.7-gcc7-bazel-test	`ciflow/all`, `ciflow/bazel`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
linux-xenial-py3-clang5-mobile-build	`ciflow/all`, `ciflow/default`, `ciflow/linux`, `ciflow/mobile`, `ciflow/trunk`	✅ triggered
linux-xenial-py3-clang5-mobile-custom-build-static	`ciflow/all`, `ciflow/default`, `ciflow/linux`, `ciflow/mobile`, `ciflow/trunk`	✅ triggered
linux-xenial-py3.7-clang7-asan	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/sanitizers`, `ciflow/trunk`	✅ triggered
linux-xenial-py3.7-clang7-onnx	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/onnx`, `ciflow/trunk`	✅ triggered
linux-xenial-py3.7-gcc5.4	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
linux-xenial-py3.7-gcc7	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
linux-xenial-py3.7-gcc7-no-ops	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
macos-arm64-binary-conda	`ciflow/binaries`, `ciflow/binaries_conda`, `ciflow/default`	✅ triggered
macos-arm64-binary-wheel	`ciflow/binaries`, `ciflow/binaries_wheel`, `ciflow/default`	✅ triggered
macos-binary-conda	`ciflow/binaries`, `ciflow/binaries_conda`, `ciflow/default`	✅ triggered
macos-binary-libtorch-cxx11-abi	`ciflow/binaries`, `ciflow/binaries_libtorch`, `ciflow/default`	✅ triggered
macos-binary-libtorch-pre-cxx11	`ciflow/binaries`, `ciflow/binaries_libtorch`, `ciflow/default`	✅ triggered
macos-binary-wheel	`ciflow/binaries`, `ciflow/binaries_wheel`, `ciflow/default`	✅ triggered
pytorch-linux-xenial-py3-clang5-android-ndk-r19c-gradle-custom-build-single	`ciflow/all`, `ciflow/android`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
pytorch-linux-xenial-py3-clang5-android-ndk-r19c-gradle-custom-build-single-full-jit	`ciflow/all`, `ciflow/android`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
win-vs2019-cpu-py3	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/trunk`, `ciflow/win`	✅ triggered
win-vs2019-cuda11.3-py3	`ciflow/all`, `ciflow/cuda`, `ciflow/default`, `ciflow/trunk`, `ciflow/win`	✅ triggered
windows-binary-libtorch-cxx11-abi	`ciflow/binaries`, `ciflow/binaries_libtorch`, `ciflow/default`	✅ triggered
windows-binary-libtorch-pre-cxx11	`ciflow/binaries`, `ciflow/binaries_libtorch`, `ciflow/default`	✅ triggered
windows-binary-wheel	`ciflow/binaries`, `ciflow/binaries_wheel`, `ciflow/default`	✅ triggered
Skipped Workflows
caffe2-linux-xenial-py3.7-gcc5.4	`ciflow/all`, `ciflow/cpu`, `ciflow/linux`, `ciflow/trunk`	🚫 skipped
docker-builds	`ciflow/all`, `ciflow/trunk`	🚫 skipped
ios-12-5-1-arm64	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
ios-12-5-1-arm64-coreml	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
ios-12-5-1-arm64-custom-ops	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
ios-12-5-1-arm64-full-jit	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
ios-12-5-1-arm64-metal	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
ios-12-5-1-x86-64	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
ios-12-5-1-x86-64-coreml	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
ios-12-5-1-x86-64-full-jit	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
libtorch-linux-xenial-cuda10.2-py3.7-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/libtorch`, `ciflow/linux`, `ciflow/trunk`	🚫 skipped
libtorch-linux-xenial-cuda11.3-py3.7-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/libtorch`, `ciflow/linux`, `ciflow/trunk`	🚫 skipped
linux-bionic-cuda10.2-py3.9-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/linux`, `ciflow/slow`, `ciflow/trunk`	🚫 skipped
linux-docs-push	`ciflow/all`, `ciflow/cpu`, `ciflow/linux`, `ciflow/scheduled`	🚫 skipped
linux-xenial-cuda11.3-py3.7-gcc7-no-ops	`ciflow/all`, `ciflow/cuda`, `ciflow/linux`, `ciflow/trunk`	🚫 skipped
macos-10-15-py3-arm64	`ciflow/all`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
macos-10-15-py3-lite-interpreter-x86-64	`ciflow/all`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
macos-11-py3-x86-64	`ciflow/all`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
parallelnative-linux-xenial-py3.7-gcc5.4	`ciflow/all`, `ciflow/cpu`, `ciflow/linux`, `ciflow/trunk`	🚫 skipped
periodic-libtorch-linux-bionic-cuda11.5-py3.7-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/libtorch`, `ciflow/linux`, `ciflow/scheduled`	🚫 skipped
periodic-libtorch-linux-xenial-cuda11.1-py3.7-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/libtorch`, `ciflow/linux`, `ciflow/scheduled`	🚫 skipped
periodic-linux-bionic-cuda11.5-py3.7-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/linux`, `ciflow/scheduled`	🚫 skipped
periodic-linux-xenial-cuda10.2-py3-gcc7-slow-gradcheck	`ciflow/all`, `ciflow/cuda`, `ciflow/linux`, `ciflow/scheduled`, `ciflow/slow`, `ciflow/slow-gradcheck`	🚫 skipped
periodic-linux-xenial-cuda11.1-py3.7-gcc7-debug	`ciflow/all`, `ciflow/cuda`, `ciflow/linux`, `ciflow/scheduled`	🚫 skipped
periodic-win-vs2019-cuda11.1-py3	`ciflow/all`, `ciflow/cuda`, `ciflow/scheduled`, `ciflow/win`	🚫 skipped
periodic-win-vs2019-cuda11.5-py3	`ciflow/all`, `ciflow/cuda`, `ciflow/scheduled`, `ciflow/win`	🚫 skipped
pytorch-linux-xenial-py3-clang5-android-ndk-r19c-build	`ciflow/all`, `ciflow/android`, `ciflow/cpu`, `ciflow/linux`, `ciflow/trunk`	🚫 skipped

facebook-github-bot · 2022-02-11T23:31:48Z

🔗 Helpful links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/72755
📄 Preview docs built from this PR
📄 Preview C++ docs built from this PR
🔧 Opt-in to CIFlow to control what jobs run on your PRs

💊 CI failures summary and remediations

As of commit d2d938e (more details on the Dr. CI page):

💚 💚 Looks good so far! There are no failures yet. 💚 💚

This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

terrychenism · 2022-02-11T23:33:29Z

@terrychenism has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Summary: Add is_refernece flag in convert function Test Plan: python3 test/test_quantization.py TestQuantizeEagerOps.test_conv_transpose_2d Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D34188856](https://our.internmc.facebook.com/intern/diff/D34188856) [ghstack-poisoned]

Summary: Add is_refernece flag in convert function Test Plan: python3 test/test_quantization.py TestQuantizeEagerOps.test_conv_transpose_2d Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: ec4f83d Pull Request resolved: #72755

terrychenism · 2022-02-12T00:08:45Z

@terrychenism has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

jerryzh168 · 2022-02-15T17:58:11Z

torch/ao/quantization/quantize.py

+    if is_reference:
+        mapping = get_default_static_quant_reference_module_mappings()


I think we should do this in _convert, to keep everything in the same place

torch/nn/quantized/_reference/modules/linear.py

Summary: Add is_refernece flag in convert function Test Plan: python3 test/test_quantization.py TestQuantizeEagerOps.test_conv_transpose_2d Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D34188856](https://our.internmc.facebook.com/intern/diff/D34188856) [ghstack-poisoned]

Summary: Add is_refernece flag in convert function Test Plan: python3 test/test_quantization.py TestQuantizeEagerOps.test_conv_transpose_2d Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: daa3fab Pull Request resolved: #72755

terrychenism · 2022-02-15T18:32:07Z

@terrychenism has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Summary: Add is_refernece flag in convert function Test Plan: python3 test/test_quantization.py TestQuantizeEagerOps.test_conv_transpose_2d Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D34188856](https://our.internmc.facebook.com/intern/diff/D34188856) [ghstack-poisoned]

Summary: Add is_refernece flag in convert function Test Plan: python3 test/test_quantization.py TestQuantizeEagerOps.test_conv_transpose_2d Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 0599b4e Pull Request resolved: #72755

terrychenism · 2022-03-01T00:10:00Z

@terrychenism has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

jerryzh168 · 2022-03-04T22:18:32Z

torch/ao/quantization/quantize.py

        module, mapping=None, inplace=False, remove_qconfig=True,
-        convert_custom_config_dict=None):
+        is_reference=False, convert_custom_config_dict=None):
    r"""Converts submodules in input module to a different module according to `mapping`


maybe add some docs for is_reference flag?

jerryzh168

looks good, would be better to have a more detailed summary message as well. e.g. talk about the motivation and impact of the change and how the is_reference flag is implemented

jerryzh168 · 2022-03-04T22:20:49Z

torch/ao/quantization/quantize.py

+    if is_reference:
+        mapping = get_default_static_quant_reference_module_mappings()


actually not like this, this should go within the L530 branch I think

user should be able to overwrite the default mapping with explicit mappings, if user do not provide any mapping, we can provide a default mapping based on is_reference flag

jerryzh168

need to change the place for is_reference check in _convert function

Summary: Add is_refernece flag in convert function Test Plan: python3 test/test_quantization.py TestQuantizeEagerOps.test_conv_transpose_2d Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D34188856](https://our.internmc.facebook.com/intern/diff/D34188856) [ghstack-poisoned]

terrychenism · 2022-03-04T22:45:41Z

@terrychenism has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

jerryzh168 · 2022-03-04T22:49:26Z

torch/ao/quantization/quantize.py

    """
    if mapping is None:
        mapping = get_default_static_quant_module_mappings()
+        if is_reference:


nit: the following is a little bit better

if is_reference: ... else: ...

or just a single line with

mapping = xxx if is_reference else yyy

Summary: Add is_refernece flag in convert function Test Plan: python3 test/test_quantization.py TestQuantizeEagerOps.test_conv_transpose_2d Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D34188856](https://our.internmc.facebook.com/intern/diff/D34188856) [ghstack-poisoned]

Summary: Add is_refernece flag in convert function Test Plan: python3 test/test_quantization.py TestQuantizeEagerOps.test_conv_transpose_2d Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 34ce090 Pull Request resolved: #72755

terrychenism · 2022-03-04T22:55:23Z

@terrychenism has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Summary: 1. Add is_reference flag in convert function if is_reference = true, using the default reference module mapping 2. add linear reference module and test case Test Plan: python3 test/test_quantization.py TestQuantizeEagerOps.test_conv_transpose_2d Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D34188856](https://our.internmc.facebook.com/intern/diff/D34188856) [ghstack-poisoned]

Summary: Add is_refernece flag in convert function Test Plan: python3 test/test_quantization.py TestQuantizeEagerOps.test_conv_transpose_2d Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 2cb4552 Pull Request resolved: #72755

terrychenism · 2022-03-07T17:42:50Z

@terrychenism has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Summary: 1. Add is_reference flag in convert function if is_reference = true, using the default reference module mapping 2. add linear reference module and test case Test Plan: python3 test/test_quantization.py TestQuantizeEagerOps.test_conv_transpose_2d Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D34188856](https://our.internmc.facebook.com/intern/diff/D34188856) [ghstack-poisoned]

Summary: Add is_refernece flag in convert function Test Plan: python3 test/test_quantization.py TestQuantizeEagerOps.test_conv_transpose_2d Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 37383b5 Pull Request resolved: #72755

terrychenism · 2022-03-07T22:00:25Z

@terrychenism has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Summary: 1. Add is_reference flag in convert function if is_reference = true, using the default reference module mapping 2. add linear reference module and test case Test Plan: python3 test/test_quantization.py TestQuantizeEagerOps.test_conv_transpose_2d Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D34188856](https://our.internmc.facebook.com/intern/diff/D34188856) [ghstack-poisoned]

Summary: Add is_refernece flag in convert function Test Plan: python3 test/test_quantization.py TestQuantizeEagerOps.test_conv_transpose_2d Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 746020d Pull Request resolved: #72755

terrychenism · 2022-03-08T00:27:18Z

@terrychenism has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Summary: Pull Request resolved: #72755 Add is_refernece flag in convert function Test Plan: python3 test/test_quantization.py TestQuantizeEagerOps.test_conv_transpose_2d Imported from OSS Reviewed By: mruberry Differential Revision: D34188856 fbshipit-source-id: 291014a7b3b4d4b40ca0ca76a80711097dcc4b58

github-actions · 2022-03-08T06:48:48Z

Hey @terrychenism.
You've committed this PR, but it does not have both a 'release notes: ...' and 'topics: ...' label. Please add one of each to the PR. The 'release notes: ...' label should represent the part of PyTorch that this PR changes (fx, autograd, distributed, etc) and the 'topics: ...' label should represent the kind of PR it is (not user facing, new feature, bug fix, perf improvement, etc). The list of valid labels can be found here for the 'release notes: ...' and here for the 'topics: ...'.
For changes that are 'topic: not user facing' there is no need for a release notes label.

Summary: Pull Request resolved: pytorch/pytorch#72755 Add is_refernece flag in convert function Test Plan: python3 test/test_quantization.py TestQuantizeEagerOps.test_conv_transpose_2d Imported from OSS Reviewed By: mruberry Differential Revision: D34188856 fbshipit-source-id: 291014a7b3b4d4b40ca0ca76a80711097dcc4b58 (cherry picked from commit cfba3b8dc0373708712c0d847d590f0d587df002)

[Qunat] Refactor reference module mapping

3855215

Summary: Add is_refernece flag in convert function Test Plan: python3 test/test_quantization.py TestQuantizeEagerOps.test_conv_transpose_2d Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

terrychenism requested review from albanD and jbschlosser as code owners February 11, 2022 23:31

pytorch-bot bot added the ciflow/default label Feb 11, 2022

terrychenism mentioned this pull request Feb 11, 2022

[Quant] Add ConvTranspose reference module #72473

Closed

facebook-github-bot added the cla signed label Feb 11, 2022

terrychenism requested review from jerryzh168 and vkuzo and removed request for albanD and jbschlosser February 11, 2022 23:32

jerryzh168 reviewed Feb 15, 2022

View reviewed changes

torch/nn/quantized/_reference/modules/linear.py Show resolved Hide resolved

terrychenism changed the title ~~[Qunat] Refactor reference module mapping~~ [Quant] Refactor reference module mapping Feb 16, 2022

jerryzh168 reviewed Mar 4, 2022

View reviewed changes

jerryzh168 approved these changes Mar 4, 2022

View reviewed changes

jerryzh168 reviewed Mar 4, 2022

View reviewed changes

jerryzh168 requested changes Mar 4, 2022

View reviewed changes

jerryzh168 reviewed Mar 4, 2022

View reviewed changes

jerryzh168 approved these changes Mar 4, 2022

View reviewed changes

pytorchmergebot closed this in 4e6aefa Mar 8, 2022

facebook-github-bot deleted the gh/terrychenism/16/head branch March 11, 2022 15:17

WBobby mentioned this pull request Aug 17, 2022

Add ROCm5.2.3/AMDGPU support for PyTorch WBobby/pytorch#2

Closed

		if is_reference:
		mapping = get_default_static_quant_reference_module_mappings()

[Quant] Refactor reference module mapping #72755

[Quant] Refactor reference module mapping #72755

Uh oh!

Conversation

terrychenism commented Feb 11, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Feb 11, 2022

⚛️ CI Flow

Uh oh!

facebook-github-bot commented Feb 11, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful links

💊 CI failures summary and remediations

Uh oh!

terrychenism commented Feb 11, 2022

Uh oh!

terrychenism commented Feb 12, 2022

Uh oh!

jerryzh168 Feb 15, 2022

Choose a reason for hiding this comment

Uh oh!

Uh oh!

terrychenism commented Feb 15, 2022

Uh oh!

terrychenism commented Mar 1, 2022

Uh oh!

jerryzh168 Mar 4, 2022

Choose a reason for hiding this comment

Uh oh!

jerryzh168 left a comment

Choose a reason for hiding this comment

Uh oh!

jerryzh168 Mar 4, 2022

Choose a reason for hiding this comment

Uh oh!

jerryzh168 Mar 4, 2022

Choose a reason for hiding this comment

Uh oh!

jerryzh168 left a comment

Choose a reason for hiding this comment

Uh oh!

terrychenism commented Mar 4, 2022

Uh oh!

jerryzh168 Mar 4, 2022

Choose a reason for hiding this comment

Uh oh!

jerryzh168 Mar 4, 2022

Choose a reason for hiding this comment

Uh oh!

terrychenism commented Mar 4, 2022

Uh oh!

terrychenism commented Mar 7, 2022

Uh oh!

terrychenism commented Mar 7, 2022

Uh oh!

terrychenism commented Mar 8, 2022

Uh oh!

github-actions bot commented Mar 8, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

terrychenism commented Feb 11, 2022 •

edited

Loading

facebook-github-bot commented Feb 11, 2022 •

edited

Loading