quant: add q_batchnorm_1d op #42491
Closed
quant: add q_batchnorm_1d op #42491
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Summary: Hooks up quantized batchnorm_1d to the quantized_bn kernel. Eager mode hookup will be in a future PR, and graph mode should work after this PR. Note: currently the implementation is ~2x slower on the benchmark than q_batch_norm2d because we convert back to contiguous memory format at the end, since channels_last is only defined for rank >= 4. If further optimization is needed, that can be a separate PR (will need the NHWC folks to see if there is a workaround). Context: There have been both internal and external requests for various quantized BN1d use cases. Test Plan: ``` python test/test_quantization.py TestQuantizedOps.test_batch_norm_1d_2d_3d python test/test_quantization.py TestQuantizedOps.test_batch_norm_1d_2d_3d_relu python test/test_quantization.py TestQuantizeJitOps.test_qbatch_norm // performance: // https://gist.github.com/vkuzo/73a07c0f24c05f5804990d9ebfaecf5e ``` Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]
💊 CI failures summary and remediationsAs of commit 322626b (more details on the Dr. CI page):
ci.pytorch.org: 1 failedThis comment was automatically generated by Dr. CI (expand for details).Follow this link to opt-out of these comments for your Pull Requests.Please report bugs/suggestions on the GitHub issue tracker or post in the (internal) Dr. CI Users group. This comment has been revised 10 times. |
Summary: Hooks up quantized batchnorm_1d to the quantized_bn kernel. Eager mode hookup will be in a future PR, and graph mode should work after this PR. Note: currently the implementation is ~2x slower on the benchmark than q_batch_norm2d because we convert back to contiguous memory format at the end, since channels_last is only defined for rank >= 4. If further optimization is needed, that can be a separate PR (will need the NHWC folks to see if there is a workaround). Meanwhile, having this is better than not having anything. Context: There have been both internal and external requests for various quantized BN1d use cases. Test Plan: ``` python test/test_quantization.py TestQuantizedOps.test_batch_norm_1d_2d_3d python test/test_quantization.py TestQuantizedOps.test_batch_norm_1d_2d_3d_relu python test/test_quantization.py TestQuantizeJitOps.test_qbatch_norm // performance: // https://gist.github.com/vkuzo/73a07c0f24c05f5804990d9ebfaecf5e ``` Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]
vkuzo
added a commit
that referenced
this pull request
Aug 3, 2020
Summary: Hooks up quantized batchnorm_1d to the quantized_bn kernel. Eager mode hookup will be in a future PR, and graph mode should work after this PR. Note: currently the implementation is ~2x slower on the benchmark than q_batch_norm2d because we convert back to contiguous memory format at the end, since channels_last is only defined for rank >= 4. If further optimization is needed, that can be a separate PR (will need the NHWC folks to see if there is a workaround). Context: There have been both internal and external requests for various quantized BN1d use cases. Test Plan: ``` python test/test_quantization.py TestQuantizedOps.test_batch_norm_1d_2d_3d python test/test_quantization.py TestQuantizedOps.test_batch_norm_1d_2d_3d_relu python test/test_quantization.py TestQuantizeJitOps.test_qbatch_norm // performance: // https://gist.github.com/vkuzo/73a07c0f24c05f5804990d9ebfaecf5e ``` Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: e8890a9 Pull Request resolved: #42491
vkuzo
commented
Aug 4, 2020
jerryzh168
reviewed
Aug 4, 2020
|
|
||
| @skipIfNoFBGEMM | ||
| def test_batch_norm2d_relu(self): | ||
| def test_batch_norm_1d_2d_3d_relu(self): |
Contributor
There was a problem hiding this comment.
nit: maybe just test_batch_norm_relu and add dimensions in the docstring if needed.
jerryzh168
reviewed
Aug 4, 2020
|
|
||
| @skipIfNoFBGEMM | ||
| def test_batch_norm3d(self): | ||
| def test_batch_norm_1d_2d_3d(self): |
Summary: Hooks up quantized batchnorm_1d to the quantized_bn kernel. Eager mode hookup will be in a future PR, and graph mode should work after this PR. Note: currently the implementation is ~2x slower on the benchmark than q_batch_norm2d because we convert back to contiguous memory format at the end, since channels_last is only defined for rank >= 4. If further optimization is needed, that can be a separate PR (will need the NHWC folks to see if there is a workaround). Meanwhile, having this is better than not having anything. Context: There have been both internal and external requests for various quantized BN1d use cases. Test Plan: ``` python test/test_quantization.py TestQuantizedOps.test_batch_norm_1d_2d_3d python test/test_quantization.py TestQuantizedOps.test_batch_norm_1d_2d_3d_relu python test/test_quantization.py TestQuantizeJitOps.test_qbatch_norm // performance: // https://gist.github.com/vkuzo/73a07c0f24c05f5804990d9ebfaecf5e ``` Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D22926254](https://our.internmc.facebook.com/intern/diff/D22926254) [ghstack-poisoned]
Contributor
|
This pull request has been merged in 50f0d2b. |
MauiDesign
pushed a commit
to MauiDesign/PyTorchPyTorch
that referenced
this pull request
Aug 16, 2020
Summary: Hooks up quantized batchnorm_1d to the quantized_bn kernel. Eager mode hookup will be in a future PR, and graph mode should work after this PR. Note: currently the implementation is ~2x slower on the benchmark than q_batch_norm2d because we convert back to contiguous memory format at the end, since channels_last is only defined for rank >= 4. If further optimization is needed, that can be a separate PR (will need the NHWC folks to see if there is a workaround). Context: There have been both internal and external requests for various quantized BN1d use cases. Test Plan: ``` python test/test_quantization.py TestQuantizedOps.test_batch_norm_1d_2d_3d python test/test_quantization.py TestQuantizedOps.test_batch_norm_1d_2d_3d_relu python test/test_quantization.py TestQuantizeJitOps.test_qbatch_norm // performance: // https://gist.github.com/vkuzo/73a07c0f24c05f5804990d9ebfaecf5e ``` Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 8c1e75b Pull Request resolved: pytorch/pytorch#42491
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Stack from ghstack:
Summary:
Hooks up quantized batchnorm_1d to the quantized_bn kernel. Eager mode
hookup will be in a future PR, and graph mode should work after this PR.
Note: currently the implementation is ~2x slower on the benchmark than q_batch_norm2d
because we convert back to contiguous memory format at the end, since
channels_last is only defined for rank >= 4. If further optimization is
needed, that can be a separate PR (will need the NHWC folks to see if
there is a workaround). Meanwhile, having this is better than not having anything.
Context: There have been both internal and external requests for various
quantized BN1d use cases.
Test Plan:
Reviewers:
Subscribers:
Tasks:
Tags:
Differential Revision: D22926254