[ONNX] Add pass that fuses Conv and BatchNormalization #40547

KsenijaS · 2020-06-25T01:27:08Z

Add pass that fuses Conv and Batchnormalization nodes into one node Conv.
This pass is only applied in inference mode (training is None or TrainingMode.Eval).
Since this pass needs access to param_dict it is written outside peephole file where these kind of passes (fusing multiple nodes into one) is usually placed.

This PR also adds wrapper skipIfNoEmbed to skip debug_embed_params test:
Pass that fuses Conv and Batchnorm changes the params of resnet model and parameters of onnx and pytorch model won't match. Since parameters are not matching, debug_embed_params test for test_resnet will fail and that is expected, therefore debug_embed_params test for test_resnet should be skipped.

dr-ci · 2020-06-25T01:30:44Z

💊 CI failures summary and remediations

As of commit 2b1f1f8 (more details on the Dr. CI page):

1/1 failures possibly* introduced in this PR
- 1/1 non-CircleCI failure(s)

ci.pytorch.org: 1 failed

Failed: pr/pytorch-linux-xenial-rocm3.5.1-py3.6

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker or post in the (internal) Dr. CI Users group.

See how this bot performed.

This comment has been revised 77 times.

torch/csrc/jit/python/init.cpp

torch/csrc/jit/passes/onnx/initializer_peephole.cpp

test/onnx/test_utility_funs.py

test/onnx/test_pytorch_onnx_onnxruntime.py

neginraoof · 2020-06-30T18:47:19Z

@KsenijaS Could you please resolve the conflicts and take a looks at Caffe2 and model test failures?
Thanks

test/onnx/test_utility_funs.py

torch/onnx/utils.py

torch/csrc/jit/passes/onnx/initializer_based_peephole.cpp

torch/onnx/utils.py

torch/csrc/jit/passes/onnx/initializer_based_peephole.h

torch/csrc/jit/passes/onnx/eval_peephole.cpp

KsenijaS · 2020-07-12T19:50:17Z

test/onnx/test_pytorch_onnx_caffe2.py

        return func(self)
    return wrapper

+def skipIfNoEmbed(func):


@houseroad parameters for onnx and pytorch won't match for resnet model when fusing Conv and Batchnorm in the eval mode, and debug_embed_params test will fail because of that . One solution is to create a wrapper to skip debug_embed_params test. What do you think of this solution?

Could you point me to where it fails?

torch/csrc/jit/passes/onnx/eval_peephole.cpp

neginraoof

Thanks.
Would be great if @houseroad could take a look at the update in Caffe2 tests.

neginraoof · 2020-07-15T16:43:48Z

torch/csrc/jit/passes/onnx/eval_peephole.cpp

+      auto origconvNode = *it;
+      auto epsilon = bnNode->f(attr::epsilon);
+      auto w_conv_value = getValues(origconvNode, valsToParamsMap);
+      if (w_conv_value.size() < 1 ||


The logic here could be stated more clearly.
If inputs are 3, size of w_conv_value should be 2
If inputs are 2, size of w_conv_value should be 1
Continue if none of the above.

KsenijaS · 2020-07-15T17:39:14Z

@houseroad can you please take a look? Thanks

houseroad

Overall it looks good to me, didn't carefully check the fuse logic. Left some inline comments.

houseroad · 2020-07-16T06:39:26Z

test/onnx/test_pytorch_onnx_caffe2.py

        return func(self)
    return wrapper

+def skipIfNoEmbed(func):


Could you point me to where it fails?

houseroad · 2020-07-16T06:41:12Z

test/onnx/test_utility_funs.py

+        for node in graph.nodes():
+            assert node.kind() != "onnx::BatchNormalization"
+
+    def test_conv_bn(self):


Why do we want to add model execution test here? The test_utility_funs.py suppose to only contain checks on the model structure.

we are comparing ort outputs with and without training (without pass/ with pass) and in pytorch_onnx_onnxruntime we are comparing pytorch outputs with ort outputs

houseroad · 2020-07-16T06:42:00Z

test/onnx/test_utility_funs.py

+        ort_outs2 = ort_sess.run(None, ort_inputs)
+        [np.testing.assert_allclose(ort_out1, ort_out2, atol=1e-7, rtol=0.001) for ort_out1, ort_out2 in zip(ort_outs1, ort_outs2)]
+
+    def test_multiple_conv_bn(self):


same here, can we move them to test_pytorch_onnx_onnxruntime.py?

same as before we are comparing ort outputs with pass with ort outputs without the pass.

torch/csrc/jit/passes/onnx/eval_peephole.h

KsenijaS · 2020-07-16T20:37:21Z

@houseroad Traceback for test resnet with fuse conv-bn pass is:
FAIL: test_resnet (main.TestCaffe2Backend_opset9)

Traceback (most recent call last):
File "test/onnx/test_pytorch_onnx_caffe2.py", line 518, in test_resnet
state_dict=state_dict, atol=1e-5)
File "test/onnx/test_pytorch_onnx_caffe2.py", line 219, in run_model_test
operator_export_type=operator_export_type)
File "test/onnx/test_pytorch_onnx_caffe2.py", line 170, in run_debug_test
np.testing.assert_almost_equal(x.data.cpu().numpy(), y, decimal=3)
File "/home/ksenija/anaconda3/envs/pytorch/lib/python3.7/site-packages/numpy/testing/_private/utils.py", line 600, in assert_almost_equal
return assert_array_almost_equal(actual, desired, decimal, err_msg)
File "/home/ksenija/anaconda3/envs/pytorch/lib/python3.7/site-packages/numpy/testing/_private/utils.py", line 1064, in assert_array_almost_equal
precision=decimal)
File "/home/ksenija/anaconda3/envs/pytorch/lib/python3.7/site-packages/numpy/testing/_private/utils.py", line 785, in assert_array_compare
flagged = func_assert_same_pos(x, y, func=isnan, hasval='nan')
File "/home/ksenija/anaconda3/envs/pytorch/lib/python3.7/site-packages/numpy/testing/_private/utils.py", line 761, in func_assert_same_pos
raise AssertionError(msg)
AssertionError:
Arrays are not almost equal to 3 decimals

x and y nan location mismatch:
x: array([[-0.979, 0.093, -0.469, ..., -2.597, -0.888, 1.386],
[-0.582, -0.08 , -0.224, ..., -2.605, -0.825, 1.359]],
dtype=float32)
y: array([[nan, nan, nan, ..., nan, nan, nan],
[nan, nan, nan, ..., nan, nan, nan]], dtype=float32)

KsenijaS · 2020-07-20T15:19:35Z

@houseroad do you have more questions?

facebook-github-bot

@bzinodev has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

KsenijaS · 2020-07-22T18:58:54Z

@bzinodev can this PR be merged? Thanks!

facebook-github-bot · 2020-07-22T22:20:28Z

@bzinodev merged this pull request in af5d0bf.

KsenijaS added 4 commits June 23, 2020 17:08

add fuse conv batchnorm pass

5167912

add tests

c13fce7

update pass

c499daa

add tests

c8294fb

KsenijaS requested a review from apaszke as a code owner June 25, 2020 01:27

clang format

1af7295

facebook-github-bot added the oncall: jit Add this issue/PR to JIT oncall triage queue label Jun 25, 2020

pytorchbot added the open source label Jun 25, 2020

gchanan requested a review from houseroad June 25, 2020 20:19

gchanan added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Jun 25, 2020

neginraoof reviewed Jun 25, 2020

View reviewed changes

test/onnx/test_utility_funs.py Show resolved Hide resolved

neginraoof reviewed Jun 25, 2020

View reviewed changes

test/onnx/test_pytorch_onnx_onnxruntime.py Outdated Show resolved Hide resolved

KsenijaS added 6 commits June 29, 2020 09:37

add tests

f26d34e

bach->batch

196d448

rename

671e161

add test

2f03a90

fix merge conflict

c7dafe5

remove pbuildparam function

0a1773c

KsenijaS added 3 commits July 1, 2020 18:56

add test

b092cb2

fix merge conflict

15ad4de

add buildParamMap func

a888941

BowenBao reviewed Jul 6, 2020

View reviewed changes

test/onnx/test_utility_funs.py Show resolved Hide resolved

KsenijaS added 3 commits July 7, 2020 10:06

fix caffe2 tests

ea2d8f6

update the pass

037b0f6

Merge https://github.com/pytorch/pytorch into conv_bn

6cccd8c

BowenBao reviewed Jul 7, 2020

View reviewed changes

torch/csrc/jit/passes/onnx/initializer_based_peephole.h Outdated Show resolved Hide resolved

KsenijaS added 6 commits July 8, 2020 20:28

clang format

61f3c70

remove print

8bd86bd

fix flake8 errors

bc95285

Merge https://github.com/pytorch/pytorch into conv_bn

ca2b5f5

clang format

f755055

remove test

ebac284

BowenBao reviewed Jul 9, 2020

View reviewed changes

KsenijaS added 2 commits July 9, 2020 20:19

address reviewr's comments

255dc3d

add wraper skipIfNoEmbed to skip debug_embed_params test

fc7b10d

KsenijaS commented Jul 12, 2020

View reviewed changes

BowenBao reviewed Jul 14, 2020

View reviewed changes

torch/csrc/jit/passes/onnx/eval_peephole.cpp Show resolved Hide resolved

KsenijaS added 2 commits July 14, 2020 15:45

check inputs

d36a2ca

Merge https://github.com/pytorch/pytorch into conv_bn

2b1f1f8

neginraoof approved these changes Jul 15, 2020

View reviewed changes

houseroad reviewed Jul 16, 2020

View reviewed changes

bzinodev self-requested a review July 20, 2020 18:38

bzinodev approved these changes Jul 20, 2020

View reviewed changes

facebook-github-bot reviewed Jul 20, 2020

View reviewed changes

facebook-github-bot closed this in af5d0bf Jul 22, 2020

facebook-github-bot added the merged label Jul 22, 2020

mruberry added the Merged label Oct 28, 2020

shinh mentioned this pull request Dec 11, 2020

[ONNX] Cannot turn off Conv+BN fusion in eval mode #49226

Closed

kwmaeng91 mentioned this pull request Jun 9, 2023

How is BatchNorm layers handled? facebookresearch/CrypTen#472

Closed

AymenBOUGUERRA mentioned this pull request Aug 7, 2023

Yolov7 pruned model does not detect anything? VainF/Torch-Pruning#186

Open

[ONNX] Add pass that fuses Conv and BatchNormalization #40547

[ONNX] Add pass that fuses Conv and BatchNormalization #40547

Uh oh!

Conversation

KsenijaS commented Jun 25, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dr-ci bot commented Jun 25, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💊 CI failures summary and remediations

ci.pytorch.org: 1 failed

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

neginraoof commented Jun 30, 2020

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

neginraoof left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

KsenijaS commented Jul 15, 2020

Uh oh!

houseroad left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

KsenijaS commented Jul 16, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

KsenijaS commented Jul 20, 2020

Uh oh!

facebook-github-bot left a comment

Choose a reason for hiding this comment

Uh oh!

KsenijaS commented Jul 22, 2020

Uh oh!

facebook-github-bot commented Jul 22, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

KsenijaS commented Jun 25, 2020 •

edited

Loading

dr-ci bot commented Jun 25, 2020 •

edited

Loading

KsenijaS commented Jul 16, 2020 •

edited

Loading