[quant] Attach qconfig to all modules #42576

jerryzh168 · 2020-08-05T01:32:04Z

Stack from ghstack:

[quant] Attach qconfig to all modules #42576 [quant] Attach qconfig to all modules

Summary:
Previously we have qconfig propagate list and we only attach qconfig for modules
in the list, this works when everything is quantized in the form of module.
but now we are expanding quantization for functional/torch ops, we'll need to attach qconfig
to all modules

Test Plan:
regression tests

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: D22939453

Summary: Previously we have qconfig propagate list and we only attach qconfig for modules in the list, this works when everything is quantized in the form of module. but now we are expanding quantization for functional/torch ops, we'll need to attach qconfig to all modules Test Plan: Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

Summary: Previously we have qconfig propagate list and we only attach qconfig for modules in the list, this works when everything is quantized in the form of module. but now we are expanding quantization for functional/torch ops, we'll need to attach qconfig to all modules Test Plan: Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 7b89e56 Pull Request resolved: #42576

dr-ci · 2020-08-05T02:28:39Z

💊 CI failures summary and remediations

As of commit 998586a (more details on the Dr. CI page):

3/3 failures possibly* introduced in this PR
- 2/3 non-CircleCI failure(s)

🕵️ 1 new failure recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

pytorch_windows_vs2019_py36_cuda11.0_build (1/1)

Step: "Build" (full log | diagnosis details | 🔁 rerun)

FAILED: caffe2/CMakeFiles/torch_cpu.dir/operators/hard_sigmoid_op.cc.obj

caused by: Failed to read response header
caused by: failed to fill whole buffer
TION /MD /O2 /Ob2 /DNDEBUG /w /bigobj -DNDEBUG -DCUDA_HAS_FP16=1 -DUSE_GCC_GET_CPUID -DUSE_AVX -DUSE_AVX2 -DTH_HAVE_THREAD /EHsc /DNOMINMAX /wd4267 /wd4251 /wd4522 /wd4838 /wd4305 /wd4244 /wd4190 /wd4101 /wd4996 /wd4275 /bigobj -O2 -openmp:experimental -DCAFFE2_BUILD_MAIN_LIB -DONNX_BUILD_MAIN_LIB -std:c++14 /showIncludes /Focaffe2\CMakeFiles\torch_cpu.dir\operators\gather_fused_8bit_rowwise_op.cc.obj /Fdcaffe2\CMakeFiles\torch_cpu.dir\ /FS -c ..\caffe2\operators\gather_fused_8bit_rowwise_op.cc 
FAILED: caffe2/CMakeFiles/torch_cpu.dir/operators/gather_fused_8bit_rowwise_op.cc.obj  
TION /MD /O2 /Ob2 /DNDEBUG /w /bigobj -DNDEBUG -DCUDA_HAS_FP16=1 -DUSE_GCC_GET_CPUID -DUSE_AVX -DUSE_AVX2 -DTH_HAVE_THREAD /EHsc /DNOMINMAX /wd4267 /wd4251 /wd4522 /wd4838 /wd4305 /wd4244 /wd4190 /wd4101 /wd4996 /wd4275 /bigobj -O2 -openmp:experimental -DCAFFE2_BUILD_MAIN_LIB -DONNX_BUILD_MAIN_LIB -std:c++14 /showIncludes /Focaffe2\CMakeFiles\torch_cpu.dir\operators\gather_fused_8bit_rowwise_op.cc.obj /Fdcaffe2\CMakeFiles\torch_cpu.dir\ /FS -c ..\caffe2\operators\gather_fused_8bit_rowwise_op.cc 
error: failed to execute compile
caused by: error reading compile response from server
caused by: Failed to read response header
caused by: failed to fill whole buffer
ION -DHAVE_AVX2_CPU_DEFINITION /MD /O2 /Ob2 /DNDEBUG /w /bigobj -DNDEBUG -DCUDA_HAS_FP16=1 -DUSE_GCC_GET_CPUID -DUSE_AVX -DUSE_AVX2 -DTH_HAVE_THREAD /EHsc /DNOMINMAX /wd4267 /wd4251 /wd4522 /wd4838 /wd4305 /wd4244 /wd4190 /wd4101 /wd4996 /wd4275 /bigobj -O2 -openmp:experimental -DCAFFE2_BUILD_MAIN_LIB -DONNX_BUILD_MAIN_LIB -std:c++14 /showIncludes /Focaffe2\CMakeFiles\torch_cpu.dir\operators\hard_sigmoid_op.cc.obj /Fdcaffe2\CMakeFiles\torch_cpu.dir\ /FS -c ..\caffe2\operators\hard_sigmoid_op.cc 
FAILED: caffe2/CMakeFiles/torch_cpu.dir/operators/hard_sigmoid_op.cc.obj  
ION -DHAVE_AVX2_CPU_DEFINITION /MD /O2 /Ob2 /DNDEBUG /w /bigobj -DNDEBUG -DCUDA_HAS_FP16=1 -DUSE_GCC_GET_CPUID -DUSE_AVX -DUSE_AVX2 -DTH_HAVE_THREAD /EHsc /DNOMINMAX /wd4267 /wd4251 /wd4522 /wd4838 /wd4305 /wd4244 /wd4190 /wd4101 /wd4996 /wd4275 /bigobj -O2 -openmp:experimental -DCAFFE2_BUILD_MAIN_LIB -DONNX_BUILD_MAIN_LIB -std:c++14 /showIncludes /Focaffe2\CMakeFiles\torch_cpu.dir\operators\hard_sigmoid_op.cc.obj /Fdcaffe2\CMakeFiles\torch_cpu.dir\ /FS -c ..\caffe2\operators\hard_sigmoid_op.cc 
error: failed to execute compile
caused by: error reading compile response from server
caused by: Failed to read response header
caused by: failed to fill whole buffer
ninja: build stopped: subcommand failed. 
-- Building version 1.7.0a0+998586a 
ect\build\win_tmp\bin\randomtemp.exe -DCUDNN_LIBRARY=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0\lib\x64 -DJAVA_HOME=C:\Program Files\OpenJDK\jdk-12.0.2 -DNUMPY_INCLUDE_DIR=C:\Jenkins\Miniconda3\lib\site-packages\numpy\core\include -DPYTHON_EXECUTABLE=C:\Jenkins\Miniconda3\python.exe -DPYTHON_INCLUDE_DIR=C:\Jenkins\Miniconda3\include -DPYTHON_LIBRARY=C:\Jenkins\Miniconda3/libs/python36.lib -DTORCH_BUILD_VERSION=1.7.0a0+998586a -DUSE_CUDA=1 -DUSE_NUMPY=True C:\Users\circleci\project 
cmake --build . --target install --config Release -- -j 16 
Traceback (most recent call last):

ci.pytorch.org: 2 failed

Failed: pr/caffe2-pytorch-linux-xenial-rocm3.5.1-py3.6-test
Failed: pr/pytorch-linux-xenial-rocm3.5.1-py3.6

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker or post in the (internal) Dr. CI Users group.

See how this bot performed.

This comment has been revised 48 times.

z-a-f · 2020-08-05T19:35:52Z

torch/quantization/quantize.py


-    if type(module) in white_list:
-        module.qconfig = module_qconfig
+    module.qconfig = module_qconfig


How do we make sure only the ones that are allowed are getting the qconfig?

yeah, I realized that, just updated the diff. Ideally we don't have people using this "white_list" API, otherwise we'll need to make changes to callsite.

The numeric suite tests might be impacted by this change, please check if they are passing prior to landing.

vkuzo · 2020-08-06T00:39:55Z

btw, what's the context for this change? Is this for GraphModule, for adding finer grain control such as "don't quantize stateless aten op X", something else?

Summary: Previously we have qconfig propagate list and we only attach qconfig for modules in the list, this works when everything is quantized in the form of module. but now we are expanding quantization for functional/torch ops, we'll need to attach qconfig to all modules Test Plan: regression tests Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D22939453](https://our.internmc.facebook.com/intern/diff/D22939453) [ghstack-poisoned]

Summary: Previously we have qconfig propagate list and we only attach qconfig for modules in the list, this works when everything is quantized in the form of module. but now we are expanding quantization for functional/torch ops, we'll need to attach qconfig to all modules Test Plan: Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 01614db Pull Request resolved: #42576

jerryzh168 · 2020-08-06T00:59:35Z

btw, what's the context for this change? Is this for GraphModule, for adding finer grain control such as "don't quantize stateless aten op X", something else?

Yeah this is for graphmodule, we need to attach qconfig to the parent_module to be able to quantize the ops in the forward of that module. for example:

class Linear(torch.nn.Module):
	def __init__(self, weight):
		super().__init__()

	def forward(self, x, weight):
		return F.linear(x, weight)

torch/quantization/quantize.py

Summary: Previously we have qconfig propagate list and we only attach qconfig for modules in the list, this works when everything is quantized in the form of module. but now we are expanding quantization for functional/torch ops, we'll need to attach qconfig to all modules Test Plan: regression tests Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D22939453](https://our.internmc.facebook.com/intern/diff/D22939453) [ghstack-poisoned]

Summary: Previously we have qconfig propagate list and we only attach qconfig for modules in the list, this works when everything is quantized in the form of module. but now we are expanding quantization for functional/torch ops, we'll need to attach qconfig to all modules Test Plan: Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 38b8608 Pull Request resolved: #42576

Summary: Previously we have qconfig propagate list and we only attach qconfig for modules in the list, this works when everything is quantized in the form of module. but now we are expanding quantization for functional/torch ops, we'll need to attach qconfig to all modules Test Plan: regression tests Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D22939453](https://our.internmc.facebook.com/intern/diff/D22939453) [ghstack-poisoned]

Summary: Previously we have qconfig propagate list and we only attach qconfig for modules in the list, this works when everything is quantized in the form of module. but now we are expanding quantization for functional/torch ops, we'll need to attach qconfig to all modules Test Plan: Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 545c486 Pull Request resolved: #42576

raghuramank100 · 2020-08-07T22:42:47Z

torch/quantization/quantize.py

    if hasattr(module, 'qconfig') and module.qconfig is not None and \
-       len(module._modules) == 0 and not isinstance(module, torch.nn.Sequential):
+       len(module._modules) == 0 and not isinstance(module, torch.nn.Sequential) \
+       and type(module) in DEFAULT_QCONFIG_PROPAGATE_WHITE_LIST:


We should be using white_list instead of DEFAULT_QCONFIG_PROPAGATE_WHITE_LIST

raghuramank100

Had a few questions. Thanks!

Summary: Previously we have qconfig propagate list and we only attach qconfig for modules in the list, this works when everything is quantized in the form of module. but now we are expanding quantization for functional/torch ops, we'll need to attach qconfig to all modules Test Plan: regression tests Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D22939453](https://our.internmc.facebook.com/intern/diff/D22939453) [ghstack-poisoned]

Summary: Previously we have qconfig propagate list and we only attach qconfig for modules in the list, this works when everything is quantized in the form of module. but now we are expanding quantization for functional/torch ops, we'll need to attach qconfig to all modules Test Plan: Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: b8287c7 Pull Request resolved: #42576

Summary: Previously we have qconfig propagate list and we only attach qconfig for modules in the list, this works when everything is quantized in the form of module. but now we are expanding quantization for functional/torch ops, we'll need to attach qconfig to all modules Test Plan: regression tests Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D22939453](https://our.internmc.facebook.com/intern/diff/D22939453) [ghstack-poisoned]

Summary: Previously we have qconfig propagate list and we only attach qconfig for modules in the list, this works when everything is quantized in the form of module. but now we are expanding quantization for functional/torch ops, we'll need to attach qconfig to all modules Test Plan: Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 4629ad4 Pull Request resolved: #42576

facebook-github-bot · 2020-08-12T04:25:16Z

This pull request has been merged in ac93d45.

jerryzh168 requested review from raghuramank100, supriyar, vkuzo and z-a-f August 5, 2020 01:32

z-a-f reviewed Aug 5, 2020

View reviewed changes

vkuzo approved these changes Aug 6, 2020

View reviewed changes

torch/quantization/quantize.py Outdated Show resolved Hide resolved

raghuramank100 reviewed Aug 7, 2020

View reviewed changes

raghuramank100 suggested changes Aug 7, 2020

View reviewed changes

raghuramank100 approved these changes Aug 7, 2020

View reviewed changes

facebook-github-bot closed this in ac93d45 Aug 12, 2020

facebook-github-bot added the merged label Aug 12, 2020

jerryzh168 mentioned this pull request Aug 13, 2020

whitelist keyword to quantization.prepare is implemented incorrectly #41633

Closed

facebook-github-bot deleted the gh/jerryzh168/388/head branch August 15, 2020 14:17

mruberry added the Merged label Oct 28, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[quant] Attach qconfig to all modules #42576

[quant] Attach qconfig to all modules #42576

Uh oh!

jerryzh168 commented Aug 5, 2020 •

edited

Loading

Uh oh!

dr-ci bot commented Aug 5, 2020 •

edited

Loading

Uh oh!

z-a-f Aug 5, 2020

Uh oh!

jerryzh168 Aug 6, 2020

Uh oh!

raghuramank100 Aug 7, 2020

Uh oh!

vkuzo commented Aug 6, 2020

Uh oh!

jerryzh168 commented Aug 6, 2020

Uh oh!

Uh oh!

raghuramank100 Aug 7, 2020

Uh oh!

raghuramank100 left a comment

Uh oh!

facebook-github-bot commented Aug 12, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

[quant] Attach qconfig to all modules #42576

[quant] Attach qconfig to all modules #42576

Uh oh!

Conversation

jerryzh168 commented Aug 5, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dr-ci bot commented Aug 5, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💊 CI failures summary and remediations

🕵️ 1 new failure recognized by patterns

pytorch_windows_vs2019_py36_cuda11.0_build (1/1)

ci.pytorch.org: 2 failed

Uh oh!

z-a-f Aug 5, 2020

Choose a reason for hiding this comment

Uh oh!

jerryzh168 Aug 6, 2020

Choose a reason for hiding this comment

Uh oh!

raghuramank100 Aug 7, 2020

Choose a reason for hiding this comment

Uh oh!

vkuzo commented Aug 6, 2020

Uh oh!

jerryzh168 commented Aug 6, 2020

Uh oh!

Uh oh!

raghuramank100 Aug 7, 2020

Choose a reason for hiding this comment

Uh oh!

raghuramank100 left a comment

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented Aug 12, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

jerryzh168 commented Aug 5, 2020 •

edited

Loading

dr-ci bot commented Aug 5, 2020 •

edited

Loading