[AO][bugfix] Fixing FX QAT but for untraceable modules #74277

HDCharles · 2022-03-16T01:53:20Z

Stack from ghstack (oldest at bottom):

-> [AO][bugfix] Fixing FX QAT but for untraceable modules #74277

Summary: see issue: #74240

this fixes that issue by skipping the children of untraceable modules during
propagate_qconfig. This required extending said function to take the
prepare_custom_config_dict as an optional argument.

Test Plan:python test/test_quantization.py
python test/test_quantization.py TestQuantizeFx.test_qat_skip_untraced

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: D34916074

Summary: see issue: #74240 this fixes that issue by setting qconfig=None for children of untraceable modules. New functionality was needed in QuantizationTracer to distinguish these modules from custom and standalone modules which do not need such treatment. Test Plan:python test/test_quantization.py python test/test_quantization.py TestQuantizeFx.test_qat_skip_untraced Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

pytorch-bot · 2022-03-16T01:53:24Z

CI Flow Status

⚛️ CI Flow

Ruleset - Version: v1
Ruleset - File: https://github.com/pytorch/pytorch/blob/7bd42d32dba20ee62038ef19d151a7eaf9ae9d82/.github/generated-ciflow-ruleset.json
PR ciflow labels: ciflow/default
Add ciflow labels to this PR to trigger more builds:

Workflows	Labels (bold enabled)	Status
Triggered Workflows
linux-binary-conda	`ciflow/binaries`, `ciflow/binaries_conda`, `ciflow/default`	✅ triggered
linux-binary-libtorch-cxx11-abi	`ciflow/all`, `ciflow/binaries`, `ciflow/binaries_libtorch`, `ciflow/default`, `ciflow/trunk`	✅ triggered
linux-binary-libtorch-pre-cxx11	`ciflow/all`, `ciflow/binaries`, `ciflow/binaries_libtorch`, `ciflow/default`, `ciflow/trunk`	✅ triggered
linux-binary-manywheel	`ciflow/all`, `ciflow/binaries`, `ciflow/binaries_wheel`, `ciflow/default`, `ciflow/trunk`	✅ triggered
linux-bionic-py3.7-clang9	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/noarch`, `ciflow/trunk`	✅ triggered
linux-bionic-rocm4.5-py3.7	`ciflow/all`, `ciflow/default`, `ciflow/linux`, `ciflow/rocm`, `ciflow/trunk`	✅ triggered
linux-docs	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/docs`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
linux-vulkan-bionic-py3.7-clang9	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`, `ciflow/vulkan`	✅ triggered
linux-xenial-cuda11.3-py3.7-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
linux-xenial-cuda11.3-py3.7-gcc7-bazel-test	`ciflow/all`, `ciflow/bazel`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
linux-xenial-py3-clang5-mobile-build	`ciflow/all`, `ciflow/default`, `ciflow/linux`, `ciflow/mobile`, `ciflow/trunk`	✅ triggered
linux-xenial-py3-clang5-mobile-custom-build-static	`ciflow/all`, `ciflow/default`, `ciflow/linux`, `ciflow/mobile`, `ciflow/trunk`	✅ triggered
linux-xenial-py3.7-clang7-asan	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/sanitizers`, `ciflow/trunk`	✅ triggered
linux-xenial-py3.7-clang7-onnx	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/onnx`, `ciflow/trunk`	✅ triggered
linux-xenial-py3.7-gcc5.4	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
linux-xenial-py3.7-gcc5.4-mobile-lightweight-dispatch-build	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/libtorch`, `ciflow/linux`, `ciflow/mobile`, `ciflow/trunk`	✅ triggered
linux-xenial-py3.7-gcc7	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
linux-xenial-py3.7-gcc7-no-ops	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
macos-arm64-binary-conda	`ciflow/binaries`, `ciflow/binaries_conda`, `ciflow/default`	✅ triggered
macos-arm64-binary-wheel	`ciflow/binaries`, `ciflow/binaries_wheel`, `ciflow/default`	✅ triggered
macos-binary-conda	`ciflow/binaries`, `ciflow/binaries_conda`, `ciflow/default`	✅ triggered
macos-binary-libtorch-cxx11-abi	`ciflow/binaries`, `ciflow/binaries_libtorch`, `ciflow/default`	✅ triggered
macos-binary-libtorch-pre-cxx11	`ciflow/binaries`, `ciflow/binaries_libtorch`, `ciflow/default`	✅ triggered
macos-binary-wheel	`ciflow/binaries`, `ciflow/binaries_wheel`, `ciflow/default`	✅ triggered
pytorch-linux-xenial-py3-clang5-android-ndk-r19c-gradle-custom-build-single	`ciflow/all`, `ciflow/android`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
pytorch-linux-xenial-py3-clang5-android-ndk-r19c-gradle-custom-build-single-full-jit	`ciflow/all`, `ciflow/android`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
win-vs2019-cpu-py3	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/trunk`, `ciflow/win`	✅ triggered
win-vs2019-cuda11.3-py3	`ciflow/all`, `ciflow/cuda`, `ciflow/default`, `ciflow/trunk`, `ciflow/win`	✅ triggered
windows-binary-conda	`ciflow/binaries`, `ciflow/binaries_conda`, `ciflow/default`	✅ triggered
windows-binary-libtorch-debug	`ciflow/all`, `ciflow/binaries`, `ciflow/binaries_libtorch`, `ciflow/default`, `ciflow/trunk`	✅ triggered
windows-binary-libtorch-release	`ciflow/all`, `ciflow/binaries`, `ciflow/binaries_libtorch`, `ciflow/default`, `ciflow/trunk`	✅ triggered
windows-binary-wheel	`ciflow/all`, `ciflow/binaries`, `ciflow/binaries_wheel`, `ciflow/default`, `ciflow/trunk`	✅ triggered
Skipped Workflows
caffe2-linux-xenial-py3.7-gcc5.4	`ciflow/all`, `ciflow/cpu`, `ciflow/linux`, `ciflow/trunk`	🚫 skipped
docker-builds	`ciflow/all`, `ciflow/trunk`	🚫 skipped
ios-12-5-1-arm64	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/scheduled`	🚫 skipped
ios-12-5-1-arm64-coreml	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/scheduled`	🚫 skipped
ios-12-5-1-arm64-custom-ops	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/scheduled`	🚫 skipped
ios-12-5-1-arm64-metal	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/scheduled`	🚫 skipped
ios-12-5-1-x86-64	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
ios-12-5-1-x86-64-coreml	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
libtorch-linux-xenial-cuda10.2-py3.7-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/libtorch`, `ciflow/linux`, `ciflow/trunk`	🚫 skipped
libtorch-linux-xenial-cuda11.3-py3.7-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/libtorch`, `ciflow/linux`, `ciflow/trunk`	🚫 skipped
linux-bionic-cuda10.2-py3.9-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/linux`, `ciflow/slow`, `ciflow/trunk`	🚫 skipped
linux-bionic-rocm4.5-py3.7-distributed	`ciflow/all`, `ciflow/linux`, `ciflow/rocm`, `ciflow/trunk`	🚫 skipped
linux-docs-push	`ciflow/all`, `ciflow/cpu`, `ciflow/linux`, `ciflow/scheduled`	🚫 skipped
linux-xenial-cuda11.3-py3.7-gcc7-no-ops	`ciflow/all`, `ciflow/cuda`, `ciflow/linux`, `ciflow/trunk`	🚫 skipped
macos-10-15-py3-arm64	`ciflow/all`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
macos-10-15-py3-lite-interpreter-x86-64	`ciflow/all`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
macos-11-py3-x86-64	`ciflow/all`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
parallelnative-linux-xenial-py3.7-gcc5.4	`ciflow/all`, `ciflow/cpu`, `ciflow/linux`, `ciflow/trunk`	🚫 skipped
periodic-libtorch-linux-bionic-cuda11.5-py3.7-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/libtorch`, `ciflow/linux`, `ciflow/scheduled`	🚫 skipped
periodic-linux-bionic-cuda11.5-py3.7-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/linux`, `ciflow/scheduled`	🚫 skipped
periodic-linux-xenial-cuda10.2-py3-gcc7-slow-gradcheck	`ciflow/all`, `ciflow/cuda`, `ciflow/linux`, `ciflow/scheduled`, `ciflow/slow`, `ciflow/slow-gradcheck`	🚫 skipped
periodic-linux-xenial-cuda11.3-py3.7-gcc7-debug	`ciflow/all`, `ciflow/cuda`, `ciflow/linux`, `ciflow/scheduled`	🚫 skipped
periodic-win-vs2019-cuda11.5-py3	`ciflow/all`, `ciflow/cuda`, `ciflow/scheduled`, `ciflow/win`	🚫 skipped
pytorch-linux-xenial-py3-clang5-android-ndk-r19c-build	`ciflow/all`, `ciflow/android`, `ciflow/cpu`, `ciflow/linux`, `ciflow/trunk`	🚫 skipped
pytorch-xla-linux-bionic-py3.7-clang8	`ciflow/all`, `ciflow/cpu`, `ciflow/linux`, `ciflow/trunk`, `ciflow/xla`	🚫 skipped

facebook-github-bot · 2022-03-16T01:53:25Z

🔗 Helpful links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/74277
↩️ [fb-only] Re-run with SSH instructions
Need help or want to give feedback on the CI? Visit our office hours

💊 CI failures summary and remediations

As of commit c1c6d59 (more details on the Dr. CI page):

1/1 failures introduced in this PR

🕵️ 1 new failure recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

pull / win-vs2019-cuda11.3-py3 / build (1/1)

Step: "Build" (full log | diagnosis details | 🔁 rerun)

2022-03-30T02:52:16.7907745Z CMake Error: Error...ariable not set, cmake may not be built correctly.

2022-03-30T02:52:16.3725798Z CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
2022-03-30T02:52:16.3845522Z CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
2022-03-30T02:52:16.3846114Z Missing variable is:
2022-03-30T02:52:16.3846531Z CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
2022-03-30T02:52:16.3978213Z CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
2022-03-30T02:52:16.3978747Z Missing variable is:
2022-03-30T02:52:16.3979143Z CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
2022-03-30T02:52:16.4033725Z CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
2022-03-30T02:52:16.4034267Z Missing variable is:
2022-03-30T02:52:16.4034647Z CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
2022-03-30T02:52:16.7907745Z CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
2022-03-30T02:52:16.7908395Z Missing variable is:
2022-03-30T02:52:16.7908813Z CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
2022-03-30T02:52:17.0153190Z -- Generating done
2022-03-30T02:52:17.1501925Z CMake Warning:
2022-03-30T02:52:17.1502529Z   Manually-specified variables were not used by the project:
2022-03-30T02:52:17.1503047Z 
2022-03-30T02:52:17.1503546Z     BUILD_ENVIRONMENT
2022-03-30T02:52:17.1503859Z     BUILD_TYPE
2022-03-30T02:52:17.1504163Z     BUILD_WHEEL
2022-03-30T02:52:17.1504359Z

This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

Summary: see issue: #74240 this fixes that issue by setting qconfig=None for children of untraceable modules. New functionality was needed in QuantizationTracer to distinguish these modules from custom and standalone modules which do not need such treatment. Test Plan:python test/test_quantization.py python test/test_quantization.py TestQuantizeFx.test_qat_skip_untraced Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

Summary: see issue: #74240 this fixes that issue by setting qconfig=None for children of untraceable modules. New functionality was needed in QuantizationTracer to distinguish these modules from custom and standalone modules which do not need such treatment. Test Plan:python test/test_quantization.py python test/test_quantization.py TestQuantizeFx.test_qat_skip_untraced Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: c5b17c6 Pull Request resolved: #74277

HDCharles · 2022-03-16T03:28:29Z

@HDCharles has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

jerryzh168 · 2022-03-16T18:09:28Z

torch/ao/quantization/quantize_fx.py

+        ):
+            for name, child in m.named_children():
+                if not hasattr(child, "qconfig"):
+                    child.qconfig = None  # type: ignore[assignment]


I feel ideally we should set qconfig in one place, to keep code easier to maintain and reason about, can we do this in prepare?

Summary: see issue: #74240 this fixes that issue by setting qconfig=None for children of untraceable modules. New functionality was needed in QuantizationTracer to distinguish these modules from custom and standalone modules which do not need such treatment. Test Plan:python test/test_quantization.py python test/test_quantization.py TestQuantizeFx.test_qat_skip_untraced Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D34916074](https://our.internmc.facebook.com/intern/diff/D34916074) [ghstack-poisoned]

HDCharles · 2022-03-22T17:57:18Z

@HDCharles has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Summary: see issue: #74240 this fixes that issue by skipping the children of untraceable modules during propagate_qconfig. This required extending said function to take the prepare_custom_config_dict as an optional argument. Test Plan:python test/test_quantization.py python test/test_quantization.py TestQuantizeFx.test_qat_skip_untraced Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D34916074](https://our.internmc.facebook.com/intern/diff/D34916074) [ghstack-poisoned]

Summary: see issue: #74240 this fixes that issue by skipping the children of untraceable modules during propagate_qconfig. This required extending said function to take the prepare_custom_config_dict as an optional argument. Test Plan:python test/test_quantization.py python test/test_quantization.py TestQuantizeFx.test_qat_skip_untraced Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 3bdfc8f Pull Request resolved: #74277

HDCharles · 2022-03-22T19:39:27Z

@HDCharles has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

torch/ao/quantization/quantize.py

vkuzo · 2022-03-22T20:05:09Z

torch/ao/quantization/quantize.py

+            )

-def propagate_qconfig_(module, qconfig_dict=None):
+def propagate_qconfig_(module, qconfig_dict=None, prepare_custom_config_dict=None):


could we add the arg to the docstring too?

vkuzo

looks great!

jerryzh168 · 2022-03-22T20:52:34Z

looks great, do we want to add support for custom module as well in a followup PR?

Summary: see issue: #74240 this fixes that issue by skipping the children of untraceable modules during propagate_qconfig. This required extending said function to take the prepare_custom_config_dict as an optional argument. Test Plan:python test/test_quantization.py python test/test_quantization.py TestQuantizeFx.test_qat_skip_untraced Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D34916074](https://our.internmc.facebook.com/intern/diff/D34916074) [ghstack-poisoned]

Summary: see issue: #74240 this fixes that issue by skipping the children of untraceable modules during propagate_qconfig. This required extending said function to take the prepare_custom_config_dict as an optional argument. Test Plan:python test/test_quantization.py python test/test_quantization.py TestQuantizeFx.test_qat_skip_untraced Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 2c04383 Pull Request resolved: #74277

HDCharles · 2022-03-29T21:05:26Z

@HDCharles has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Summary: see issue: #74240 this fixes that issue by skipping the children of untraceable modules during propagate_qconfig. This required extending said function to take the prepare_custom_config_dict as an optional argument. Test Plan:python test/test_quantization.py python test/test_quantization.py TestQuantizeFx.test_qat_skip_untraced Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D34916074](https://our.internmc.facebook.com/intern/diff/D34916074) [ghstack-poisoned]

Summary: see issue: #74240 this fixes that issue by skipping the children of untraceable modules during propagate_qconfig. This required extending said function to take the prepare_custom_config_dict as an optional argument. Test Plan:python test/test_quantization.py python test/test_quantization.py TestQuantizeFx.test_qat_skip_untraced Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: a908c34 Pull Request resolved: #74277

HDCharles · 2022-03-30T02:08:40Z

@HDCharles has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Summary: see issue: #74240 this fixes that issue by skipping the children of untraceable modules during propagate_qconfig. This required extending said function to take the prepare_custom_config_dict as an optional argument. Test Plan:python test/test_quantization.py python test/test_quantization.py TestQuantizeFx.test_qat_skip_untraced Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D34916074](https://our.internmc.facebook.com/intern/diff/D34916074) [ghstack-poisoned]

Summary: see issue: #74240 this fixes that issue by skipping the children of untraceable modules during propagate_qconfig. This required extending said function to take the prepare_custom_config_dict as an optional argument. Test Plan:python test/test_quantization.py python test/test_quantization.py TestQuantizeFx.test_qat_skip_untraced Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: baa5c02 Pull Request resolved: #74277

HDCharles · 2022-03-30T02:40:10Z

@HDCharles has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Summary: Pull Request resolved: #74277 see issue: #74240 this fixes that issue by skipping the children of untraceable modules during propagate_qconfig. This required extending said function to take the prepare_custom_config_dict as an optional argument. Test Plan: python test/test_quantization.py python test/test_quantization.py TestQuantizeFx.test_qat_skip_untraced Imported from OSS Reviewed By: vkuzo Differential Revision: D34916074 fbshipit-source-id: 11caba2cbf78566fb51adf698b01bbba0275de28

github-actions · 2022-03-30T15:09:25Z

Hey @HDCharles.
You've committed this PR, but it does not have both a 'release notes: ...' and 'topics: ...' label. Please add one of each to the PR. The 'release notes: ...' label should represent the part of PyTorch that this PR changes (fx, autograd, distributed, etc) and the 'topics: ...' label should represent the kind of PR it is (not user facing, new feature, bug fix, perf improvement, etc). The list of valid labels can be found here for the 'release notes: ...' and here for the 'topics: ...'.
For changes that are 'topic: not user facing' there is no need for a release notes label.

pytorch-bot bot added the ciflow/default label Mar 16, 2022

facebook-github-bot added the cla signed label Mar 16, 2022

facebook-github-bot added the module: fx label Mar 16, 2022

jerryzh168 reviewed Mar 16, 2022

View reviewed changes

HDCharles requested review from jerryzh168 and vkuzo March 22, 2022 19:57

vkuzo reviewed Mar 22, 2022

View reviewed changes

torch/ao/quantization/quantize.py Show resolved Hide resolved

vkuzo reviewed Mar 22, 2022

View reviewed changes

vkuzo approved these changes Mar 22, 2022

View reviewed changes

suo removed the ciflow/default label Mar 22, 2022

pytorchmergebot closed this in bf091f7 Mar 30, 2022

facebook-github-bot deleted the gh/HDCharles/61/head branch April 3, 2022 14:16

WBobby mentioned this pull request Aug 17, 2022

Add ROCm5.2.3/AMDGPU support for PyTorch WBobby/pytorch#2

Closed

[AO][bugfix] Fixing FX QAT but for untraceable modules #74277

[AO][bugfix] Fixing FX QAT but for untraceable modules #74277

Uh oh!

Conversation

HDCharles commented Mar 16, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Mar 16, 2022

⚛️ CI Flow

Uh oh!

facebook-github-bot commented Mar 16, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful links

💊 CI failures summary and remediations

🕵️ 1 new failure recognized by patterns

pull / win-vs2019-cuda11.3-py3 / build (1/1)

Uh oh!

HDCharles commented Mar 16, 2022

Uh oh!

jerryzh168 Mar 16, 2022

Choose a reason for hiding this comment

Uh oh!

HDCharles commented Mar 22, 2022

Uh oh!

HDCharles commented Mar 22, 2022

Uh oh!

Uh oh!

vkuzo Mar 22, 2022

Choose a reason for hiding this comment

Uh oh!

vkuzo left a comment

Choose a reason for hiding this comment

Uh oh!

jerryzh168 commented Mar 22, 2022

Uh oh!

HDCharles commented Mar 29, 2022

Uh oh!

HDCharles commented Mar 30, 2022

Uh oh!

HDCharles commented Mar 30, 2022

Uh oh!

github-actions bot commented Mar 30, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

HDCharles commented Mar 16, 2022 •

edited

Loading

facebook-github-bot commented Mar 16, 2022 •

edited

Loading