JIT pass for add relu fusion. #39343

kimishpatel · 2020-06-01T18:27:41Z

Stack from ghstack:

Add add_relu fusion pass to optimize_for_mobile. #40252 Add add_relu fusion pass to optimize_for_mobile.
JIT pass for add relu fusion. #39343 JIT pass for add relu fusion.
Add fused add_relu op. #39342 Add fused add_relu op.
[Vec256][neon] Add neon backend for vec256 #39341 [Vec256][neon] Add neon backend for vec256
Add benchmark for add op. #40059 Add benchmark for add op.

Summary:
Building on top of previous PR that adds fused add_relu op, this PR adds
a JIT pass to transform input graph to find all fusable instancs of add

relu and fuses them.

Test Plan:
python test/test_jit.py TestJit.test_add_relu_fusion

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: D21822396

Summary: Building on top of previous PR that adds fused add_relu op, this PR adds a JIT pass to transform input graph to find all fusable instancs of add + relu and fuses them. Test Plan: python test/test_jit.py TestJit.test_add_relu_fusion Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

dr-ci · 2020-06-01T18:53:11Z

💊 CI failures summary and remediations

As of commit b2628b4 (more details on the Dr. CI page):

💚 💚 Looks good so far! There are no failures yet. 💚 💚

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker or post in the (internal) Dr. CI Users group.

See how this bot performed.

This comment has been revised 53 times.

Summary: Building on top of previous PR that adds fused add_relu op, this PR adds a JIT pass to transform input graph to find all fusable instancs of add + relu and fuses them. Test Plan: python test/test_jit.py TestJit.test_add_relu_fusion Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D21822396](https://our.internmc.facebook.com/intern/diff/D21822396) [ghstack-poisoned]

Summary: Building on top of previous PR that adds fused add_relu op, this PR adds a JIT pass to transform input graph to find all fusable instancs of add + relu and fuses them. Test Plan: python test/test_jit.py TestJit.test_add_relu_fusion Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 8a90fef Pull Request resolved: #39343

Summary: Building on top of previous PR that adds fused add_relu op, this PR adds a JIT pass to transform input graph to find all fusable instancs of add + relu and fuses them. Test Plan: python test/test_jit.py TestJit.test_add_relu_fusion Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D21822396](https://our.internmc.facebook.com/intern/diff/D21822396) [ghstack-poisoned]

Summary: Building on top of previous PR that adds fused add_relu op, this PR adds a JIT pass to transform input graph to find all fusable instancs of add + relu and fuses them. Test Plan: python test/test_jit.py TestJit.test_add_relu_fusion Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 55d97dc Pull Request resolved: #39343

Summary: Building on top of previous PR that adds fused add_relu op, this PR adds a JIT pass to transform input graph to find all fusable instancs of add + relu and fuses them. Test Plan: python test/test_jit.py TestJit.test_add_relu_fusion Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D21822396](https://our.internmc.facebook.com/intern/diff/D21822396) [ghstack-poisoned]

dreiss · 2020-06-25T23:29:01Z

test/test_jit.py

+        a = a * -10
+        a = a + 5


I think this was out of habit for quantized stuff. I was trying to bias values of tensor rather than uniformly distributed between 0 and 1 to be able to catch error which are not easily canceled out. In this specific instance it may not matter.

dreiss · 2020-06-25T23:31:07Z

test/test_jit.py

+        new_res = m(a, b)
+        FileCheck().check_not("aten::add(") \
+            .check_not("aten::relu_(") \
+            .check("aten::add_relu_(") \


Why is this add_relu_? Shouldn't it be add_relu, since the original operand was not modified? Doesn't the optimized model mutate a, whereas the original did not?

so add is not inplace but relu_ is. So it made sense to replace add + relu_ with add_relu_.

dreiss · 2020-06-25T23:32:43Z

test/test_jit.py

+            .check_not("aten::relu(") \
+            .check("aten::add_relu_(") \
+            .run(m.graph)
+        torch.testing.assert_allclose(orig_res, new_res)


Should also test that both models properly mutate a. Above, should probably test that they both don't mutate.

dreiss · 2020-06-25T23:35:27Z

torch/csrc/jit/passes/fuse_relu.cpp

+  // inplace. since in the graph, it does not seem
+  // that output of add will be really used anywhere else.


I don't agree with this, since you're now mutating one of the addends, rather than the sum.

Hmm. I think you are right. I need to check a graph like this:

def m(a, b, c): tmp = torch.add(a, b) x = self.relu_(tmp) d = torch.add(a, c) return x + d

If add + relu_ is replaced with add_relu_ then a is mutated and if a.add(c) is executed afterwards that is wrong. This transformation adds a new alias. In the original graph tmp = torch.add(a, b) a and tmp dont' alias but replacing that with tmp = torch.add_relu_(a, b) now I have made a and tmp alias each other.
I will test and fix this.

@dreiss I fixed this and also added exactly this test that I mentioned above. Without fixing it, the test fails, after the fix it passes.

Summary: Building on top of previous PR that adds fused add_relu op, this PR adds a JIT pass to transform input graph to find all fusable instancs of add + relu and fuses them. Test Plan: python test/test_jit.py TestJit.test_add_relu_fusion Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D21822396](https://our.internmc.facebook.com/intern/diff/D21822396) [ghstack-poisoned]

Summary: Building on top of previous PR that adds fused add_relu op, this PR adds a JIT pass to transform input graph to find all fusable instancs of add + relu and fuses them. Test Plan: python test/test_jit.py TestJit.test_add_relu_fusion Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: f35fc72 Pull Request resolved: #39343

Summary: Building on top of previous PR that adds fused add_relu op, this PR adds a JIT pass to transform input graph to find all fusable instancs of add + relu and fuses them. Test Plan: python test/test_jit.py TestJit.test_add_relu_fusion Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D21822396](https://our.internmc.facebook.com/intern/diff/D21822396) [ghstack-poisoned]

facebook-github-bot · 2020-07-10T00:18:29Z

This pull request has been merged in c5dcf05.

kimishpatel requested a review from apaszke as a code owner June 1, 2020 18:27

This was referenced Jun 1, 2020

[Vec256][neon] Add neon backend for vec256 #39341

Closed

Add fused add_relu op. #39342

Closed

facebook-github-bot added the oncall: jit Add this issue/PR to JIT oncall triage queue label Jun 1, 2020

kimishpatel requested review from AshkanAliabadi and dreiss June 1, 2020 18:49

kimishpatel mentioned this pull request Jun 15, 2020

Add benchmark for add op. #40059

Closed

kimishpatel mentioned this pull request Jun 18, 2020

Add add_relu fusion pass to optimize_for_mobile. #40252

Closed

kimishpatel added 2 commits June 19, 2020 11:50

dreiss requested changes Jun 25, 2020

View reviewed changes

kimishpatel requested a review from dreiss June 26, 2020 16:24

dreiss approved these changes Jul 6, 2020

View reviewed changes

kimishpatel added 2 commits July 8, 2020 16:03

facebook-github-bot closed this in c5dcf05 Jul 9, 2020

facebook-github-bot added the merged label Jul 10, 2020

facebook-github-bot deleted the gh/kimishpatel/29/head branch July 13, 2020 14:19

mruberry added the Merged label Oct 28, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

JIT pass for add relu fusion. #39343

JIT pass for add relu fusion. #39343

Uh oh!

kimishpatel commented Jun 1, 2020 •

edited

Loading

Uh oh!

dr-ci bot commented Jun 1, 2020 •

edited

Loading

Uh oh!

dreiss Jun 25, 2020

Uh oh!

kimishpatel Jun 25, 2020

Uh oh!

dreiss Jun 25, 2020

Uh oh!

kimishpatel Jun 25, 2020

Uh oh!

dreiss Jun 25, 2020

Uh oh!

dreiss Jun 25, 2020

Uh oh!

kimishpatel Jun 26, 2020 •

edited

Loading

Uh oh!

kimishpatel Jun 26, 2020

Uh oh!

facebook-github-bot commented Jul 10, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

		// inplace. since in the graph, it does not seem
		// that output of add will be really used anywhere else.

JIT pass for add relu fusion. #39343

JIT pass for add relu fusion. #39343

Uh oh!

Conversation

kimishpatel commented Jun 1, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dr-ci bot commented Jun 1, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💊 CI failures summary and remediations

Uh oh!

dreiss Jun 25, 2020

Choose a reason for hiding this comment

Uh oh!

kimishpatel Jun 25, 2020

Choose a reason for hiding this comment

Uh oh!

dreiss Jun 25, 2020

Choose a reason for hiding this comment

Uh oh!

kimishpatel Jun 25, 2020

Choose a reason for hiding this comment

Uh oh!

dreiss Jun 25, 2020

Choose a reason for hiding this comment

Uh oh!

dreiss Jun 25, 2020

Choose a reason for hiding this comment

Uh oh!

kimishpatel Jun 26, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kimishpatel Jun 26, 2020

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented Jul 10, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

kimishpatel commented Jun 1, 2020 •

edited

Loading

dr-ci bot commented Jun 1, 2020 •

edited

Loading

kimishpatel Jun 26, 2020 •

edited

Loading