[Vulkan] Add Vulkan Rewrite to Transfer Inputs and Outputs to Vulkan and CPU Backends Respectively #87432

salilsdesai · 2022-10-21T01:04:22Z

Stack from ghstack (oldest at bottom):

-> [Vulkan] Add Vulkan Rewrite to Transfer Inputs and Outputs to Vulkan and CPU Backends Respectively #87432
[Vulkan] Add support for Optimization Blocklist to Vulkan Rewrite #87431

With this change, we don't have to manually invoke transferring input and output backends when we run vulkan models.

Graph rewrite code based off of:

32efff4#diff-a473bddb458dc24225866a45092d6eca064eddd256245d93020e48e216eee4d5R160-R179

Differential Revision: D39519168

NOTE FOR REVIEWERS: This PR has internal Meta-specific changes or comments, please review them on Phabricator!

…and CPU Backends Respectively With this change, we don't have to manually invoke transferring input and output backends when we run vulkan models. Graph rewrite code based off of: - 32efff4#diff-a473bddb458dc24225866a45092d6eca064eddd256245d93020e48e216eee4d5R160-R179 Differential Revision: [D39519168](https://our.internmc.facebook.com/intern/diff/D39519168/) **NOTE FOR REVIEWERS**: This PR has internal Meta-specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D39519168/)! [ghstack-poisoned]

pytorch-bot · 2022-10-21T01:04:24Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/87432

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit our office hours

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 Failures, 2 Pending

As of commit cc16ec3:

The following jobs have failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

…and CPU Backends Respectively With this change, we don't have to manually invoke transferring input and output backends when we run vulkan models. Graph rewrite code based off of: - 32efff4#diff-a473bddb458dc24225866a45092d6eca064eddd256245d93020e48e216eee4d5R160-R179 Differential Revision: [D39519168](https://our.internmc.facebook.com/intern/diff/D39519168/) **NOTE FOR REVIEWERS**: This PR has internal Meta-specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D39519168/)! ghstack-source-id: 171128918 Pull Request resolved: #87432

digantdesai

LGTM

albanD · 2022-10-21T14:26:36Z

test/test_public_bindings.py

            "unify_type_list",
            "vitals_enabled",
-
+            "VULKAN_AUTOMATIC_GPU_TRANSFER",


Do you actually want this to be public API as torch.VULKAN_AUTOMATIC_GPU_TRANSFER ?
Is it properly documented as well?

… to Vulkan and CPU Backends Respectively" With this change, we don't have to manually invoke transferring input and output backends when we run vulkan models. Graph rewrite code based off of: - 32efff4#diff-a473bddb458dc24225866a45092d6eca064eddd256245d93020e48e216eee4d5R160-R179 Differential Revision: [D39519168](https://our.internmc.facebook.com/intern/diff/D39519168/) **NOTE FOR REVIEWERS**: This PR has internal Meta-specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D39519168/)! [ghstack-poisoned]

…and CPU Backends Respectively Pull Request resolved: #87432 With this change, we don't have to manually invoke transferring input and output backends when we run vulkan models. Graph rewrite code based off of: - 32efff4#diff-a473bddb458dc24225866a45092d6eca064eddd256245d93020e48e216eee4d5R160-R179 ghstack-source-id: 171174616 Differential Revision: [D39519168](https://our.internmc.facebook.com/intern/diff/D39519168/) **NOTE FOR REVIEWERS**: This PR has internal Meta-specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D39519168/)!

… to Vulkan and CPU Backends Respectively" With this change, we don't have to manually invoke transferring input and output backends when we run vulkan models. Graph rewrite code based off of: - 32efff4#diff-a473bddb458dc24225866a45092d6eca064eddd256245d93020e48e216eee4d5R160-R179 Differential Revision: [D39519168](https://our.internmc.facebook.com/intern/diff/D39519168/) **NOTE FOR REVIEWERS**: This PR has internal Meta-specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D39519168/)! [ghstack-poisoned]

…and CPU Backends Respectively Pull Request resolved: #87432 With this change, we don't have to manually invoke transferring input and output backends when we run vulkan models. Graph rewrite code based off of: - 32efff4#diff-a473bddb458dc24225866a45092d6eca064eddd256245d93020e48e216eee4d5R160-R179 ghstack-source-id: 171282664 Differential Revision: [D39519168](https://our.internmc.facebook.com/intern/diff/D39519168/) **NOTE FOR REVIEWERS**: This PR has internal Meta-specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D39519168/)!

… to Vulkan and CPU Backends Respectively" With this change, we don't have to manually invoke transferring input and output backends when we run vulkan models. Graph rewrite code based off of: - 32efff4#diff-a473bddb458dc24225866a45092d6eca064eddd256245d93020e48e216eee4d5R160-R179 Differential Revision: [D39519168](https://our.internmc.facebook.com/intern/diff/D39519168/) **NOTE FOR REVIEWERS**: This PR has internal Meta-specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D39519168/)! [ghstack-poisoned]

…and CPU Backends Respectively Pull Request resolved: #87432 With this change, we don't have to manually invoke transferring input and output backends when we run vulkan models. Graph rewrite code based off of: - 32efff4#diff-a473bddb458dc24225866a45092d6eca064eddd256245d93020e48e216eee4d5R160-R179 ghstack-source-id: 171357604 Differential Revision: [D39519168](https://our.internmc.facebook.com/intern/diff/D39519168/) **NOTE FOR REVIEWERS**: This PR has internal Meta-specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D39519168/)!

… to Vulkan and CPU Backends Respectively" With this change, we don't have to manually invoke transferring input and output backends when we run vulkan models. Graph rewrite code based off of: - 32efff4#diff-a473bddb458dc24225866a45092d6eca064eddd256245d93020e48e216eee4d5R160-R179 Differential Revision: [D39519168](https://our.internmc.facebook.com/intern/diff/D39519168/) **NOTE FOR REVIEWERS**: This PR has internal Meta-specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D39519168/)! [ghstack-poisoned]

…and CPU Backends Respectively Pull Request resolved: #87432 With this change, we don't have to manually invoke transferring input and output backends when we run vulkan models. Graph rewrite code based off of: - 32efff4#diff-a473bddb458dc24225866a45092d6eca064eddd256245d93020e48e216eee4d5R160-R179 ghstack-source-id: 171521248 Differential Revision: [D39519168](https://our.internmc.facebook.com/intern/diff/D39519168/) **NOTE FOR REVIEWERS**: This PR has internal Meta-specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D39519168/)!

… to Vulkan and CPU Backends Respectively" With this change, we don't have to manually invoke transferring input and output backends when we run vulkan models. Graph rewrite code based off of: - 32efff4#diff-a473bddb458dc24225866a45092d6eca064eddd256245d93020e48e216eee4d5R160-R179 Differential Revision: [D39519168](https://our.internmc.facebook.com/intern/diff/D39519168/) **NOTE FOR REVIEWERS**: This PR has internal Meta-specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D39519168/)! [ghstack-poisoned]

…and CPU Backends Respectively Pull Request resolved: #87432 With this change, we don't have to manually invoke transferring input and output backends when we run vulkan models. Graph rewrite code based off of: - 32efff4#diff-a473bddb458dc24225866a45092d6eca064eddd256245d93020e48e216eee4d5R160-R179 ghstack-source-id: 171715102 Differential Revision: [D39519168](https://our.internmc.facebook.com/intern/diff/D39519168/) **NOTE FOR REVIEWERS**: This PR has internal Meta-specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D39519168/)!

… to Vulkan and CPU Backends Respectively" With this change, we don't have to manually invoke transferring input and output backends when we run vulkan models. Graph rewrite code based off of: - 32efff4#diff-a473bddb458dc24225866a45092d6eca064eddd256245d93020e48e216eee4d5R160-R179 Differential Revision: [D39519168](https://our.internmc.facebook.com/intern/diff/D39519168/) **NOTE FOR REVIEWERS**: This PR has internal Meta-specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D39519168/)! [ghstack-poisoned]

…and CPU Backends Respectively Pull Request resolved: #87432 With this change, we don't have to manually invoke transferring input and output backends when we run vulkan models. Graph rewrite code based off of: - 32efff4#diff-a473bddb458dc24225866a45092d6eca064eddd256245d93020e48e216eee4d5R160-R179 ghstack-source-id: 171919059 Differential Revision: [D39519168](https://our.internmc.facebook.com/intern/diff/D39519168/) **NOTE FOR REVIEWERS**: This PR has internal Meta-specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D39519168/)!

… to Vulkan and CPU Backends Respectively" With this change, we don't have to manually invoke transferring input and output backends when we run vulkan models. Graph rewrite code based off of: - 32efff4#diff-a473bddb458dc24225866a45092d6eca064eddd256245d93020e48e216eee4d5R160-R179 Differential Revision: [D39519168](https://our.internmc.facebook.com/intern/diff/D39519168/) **NOTE FOR REVIEWERS**: This PR has internal Meta-specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D39519168/)! [ghstack-poisoned]

…and CPU Backends Respectively Pull Request resolved: #87432 @bypass-github-export-checks With this change, we don't have to manually invoke transferring input and output backends when we run vulkan models. Graph rewrite code based off of: - 32efff4#diff-a473bddb458dc24225866a45092d6eca064eddd256245d93020e48e216eee4d5R160-R179 ghstack-source-id: 171972016 Differential Revision: [D39519168](https://our.internmc.facebook.com/intern/diff/D39519168/) **NOTE FOR REVIEWERS**: This PR has internal Meta-specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D39519168/)!

… to Vulkan and CPU Backends Respectively" With this change, we don't have to manually invoke transferring input and output backends when we run vulkan models. Graph rewrite code based off of: - 32efff4#diff-a473bddb458dc24225866a45092d6eca064eddd256245d93020e48e216eee4d5R160-R179 Differential Revision: [D39519168](https://our.internmc.facebook.com/intern/diff/D39519168/) **NOTE FOR REVIEWERS**: This PR has internal Meta-specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D39519168/)! [ghstack-poisoned]

…and CPU Backends Respectively Pull Request resolved: #87432 @bypass-github-export-checks With this change, we don't have to manually invoke transferring input and output backends when we run vulkan models. Graph rewrite code based off of: - 32efff4#diff-a473bddb458dc24225866a45092d6eca064eddd256245d93020e48e216eee4d5R160-R179 ghstack-source-id: 172070804 Differential Revision: [D39519168](https://our.internmc.facebook.com/intern/diff/D39519168/) **NOTE FOR REVIEWERS**: This PR has internal Meta-specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D39519168/)!

facebook-github-bot · 2022-10-31T14:16:17Z

@pytorchbot merge

(Initiating merge automatically since Phabricator Diff has merged)

pytorchmergebot · 2022-10-31T14:18:40Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

github-actions · 2022-10-31T14:19:20Z

Hey @salilsdesai.
You've committed this PR, but it does not have both a 'release notes: ...' and 'topics: ...' label. Please add one of each to the PR. The 'release notes: ...' label should represent the part of PyTorch that this PR changes (fx, autograd, distributed, etc) and the 'topics: ...' label should represent the kind of PR it is (not user facing, new feature, bug fix, perf improvement, etc). The list of valid labels can be found here for the 'release notes: ...' and here for the 'topics: ...'.
For changes that are 'topic: not user facing' there is no need for a release notes label.

…and CPU Backends Respectively (pytorch#87432) With this change, we don't have to manually invoke transferring input and output backends when we run vulkan models. Graph rewrite code based off of: - pytorch@32efff4#diff-a473bddb458dc24225866a45092d6eca064eddd256245d93020e48e216eee4d5R160-R179 Differential Revision: [D39519168](https://our.internmc.facebook.com/intern/diff/D39519168/) **NOTE FOR REVIEWERS**: This PR has internal Meta-specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D39519168/)! Pull Request resolved: pytorch#87432 Approved by: https://github.com/mcr229, https://github.com/digantdesai

salilsdesai · 2022-12-16T22:28:22Z

For sample model

class TestModel(torch.nn.Module):
    def forward(self, x):
        return x + x

Before Transfer

graph(%self : __torch__.___torch_mangle_2.TestModel,
      %x.1 : Tensor):
  %2 : int = prim::Constant[value=1]()
  %3 : Tensor = aten::add(%x.1, %x.1, %2)
  return (%3)

After Transfer

graph(%self : __torch__.___torch_mangle_2.TestModel,
      %x.1 : Tensor):
  %4 : str = prim::Constant[value="vulkan"]()
  %5 : str = prim::Constant[value="vulkan"]()
  %6 : NoneType = prim::Constant()
  %7 : NoneType = prim::Constant()
  %8 : NoneType = prim::Constant()
  %9 : NoneType = prim::Constant()
  %10 : bool = prim::Constant[value=0]()
  %11 : bool = prim::Constant[value=0]()
  %12 : NoneType = prim::Constant()
  %13 : str = prim::Constant[value="vulkan"]()
  %14 : str = prim::Constant[value="vulkan"]()
  %15 : str = prim::Constant[value="vulkan"]()
  %16 : str = prim::Constant[value="vulkan"]()
  %17 : str = prim::Constant[value="vulkan"]()
  %18 : Device = aten::device(%17)
  %19 : str = prim::Constant[value="vulkan"]()
  %20 : NoneType = prim::Constant()
  %21 : NoneType = prim::Constant()
  %22 : NoneType = prim::Constant()
  %23 : NoneType = prim::Constant()
  %24 : bool = prim::Constant[value=0]()
  %25 : bool = prim::Constant[value=0]()
  %26 : NoneType = prim::Constant()
  %27 : str = prim::Constant[value="vulkan"]()
  %28 : str = prim::Constant[value="vulkan"]()
  %29 : Device = aten::device(%28)
  %30 : NoneType = prim::Constant()
  %31 : bool = prim::Constant[value=0]()
  %32 : bool = prim::Constant[value=0]()
  %33 : Tensor = aten::to(%x.1, %29, %30, %31, %32)
  %2 : int = prim::Constant[value=1]()
  %3 : Tensor = aten::add(%33, %33, %2)
  %34 : str = prim::Constant[value="cpu"]()
  %35 : str = prim::Constant[value="cpu"]()
  %36 : NoneType = prim::Constant()
  %37 : NoneType = prim::Constant()
  %38 : NoneType = prim::Constant()
  %39 : NoneType = prim::Constant()
  %40 : bool = prim::Constant[value=0]()
  %41 : bool = prim::Constant[value=0]()
  %42 : NoneType = prim::Constant()
  %43 : str = prim::Constant[value="cpu"]()
  %44 : str = prim::Constant[value="cpu"]()
  %45 : str = prim::Constant[value="cpu"]()
  %46 : str = prim::Constant[value="cpu"]()
  %47 : str = prim::Constant[value="cpu"]()
  %48 : Device = aten::device(%47)
  %49 : str = prim::Constant[value="cpu"]()
  %50 : NoneType = prim::Constant()
  %51 : NoneType = prim::Constant()
  %52 : NoneType = prim::Constant()
  %53 : NoneType = prim::Constant()
  %54 : bool = prim::Constant[value=0]()
  %55 : bool = prim::Constant[value=0]()
  %56 : NoneType = prim::Constant()
  %57 : str = prim::Constant[value="cpu"]()
  %58 : str = prim::Constant[value="cpu"]()
  %59 : Device = aten::device(%58)
  %60 : NoneType = prim::Constant()
  %61 : bool = prim::Constant[value=0]()
  %62 : bool = prim::Constant[value=0]()
  %63 : Tensor = aten::to(%3, %59, %60, %61, %62)
  return (%63)

After final dead code elimination

graph(%self : __torch__.___torch_mangle_2.TestModel,
      %x.1 : Tensor):
  %65 : Device = prim::Constant[value="cpu"]()
  %2 : int = prim::Constant[value=1]()
  %31 : bool = prim::Constant[value=0]()
  %30 : NoneType = prim::Constant()
  %64 : Device = prim::Constant[value="vulkan"]()
  %33 : Tensor = aten::to(%x.1, %64, %30, %31, %31)
  %3 : Tensor = aten::add(%33, %33, %2)
  %63 : Tensor = aten::to(%3, %65, %30, %31, %31)
  return (%63)

salilsdesai requested a review from albanD as a code owner October 21, 2022 01:04

pytorch-bot bot added the release notes: jit release notes category label Oct 21, 2022

salilsdesai mentioned this pull request Oct 21, 2022

[Vulkan] Add support for Optimization Blocklist to Vulkan Rewrite #87431

Closed

mcr229 approved these changes Oct 21, 2022

View reviewed changes

pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Oct 21, 2022

digantdesai self-requested a review October 21, 2022 01:10

digantdesai approved these changes Oct 21, 2022

View reviewed changes

albanD requested changes Oct 21, 2022

View reviewed changes

pytorchmergebot added the Merged label Oct 31, 2022

pytorchmergebot closed this in df1cc0e Oct 31, 2022

facebook-github-bot deleted the gh/salilsdesai/7/head branch June 8, 2023 18:42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Vulkan] Add Vulkan Rewrite to Transfer Inputs and Outputs to Vulkan and CPU Backends Respectively #87432

[Vulkan] Add Vulkan Rewrite to Transfer Inputs and Outputs to Vulkan and CPU Backends Respectively #87432

Uh oh!

salilsdesai commented Oct 21, 2022 •

edited

Loading

Uh oh!

pytorch-bot bot commented Oct 21, 2022 •

edited

Loading

Uh oh!

digantdesai left a comment

Uh oh!

albanD Oct 21, 2022

Uh oh!

facebook-github-bot commented Oct 31, 2022

Uh oh!

pytorchmergebot commented Oct 31, 2022

Uh oh!

github-actions bot commented Oct 31, 2022

Uh oh!

salilsdesai commented Dec 16, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

[Vulkan] Add Vulkan Rewrite to Transfer Inputs and Outputs to Vulkan and CPU Backends Respectively #87432

[Vulkan] Add Vulkan Rewrite to Transfer Inputs and Outputs to Vulkan and CPU Backends Respectively #87432

Uh oh!

Conversation

salilsdesai commented Oct 21, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Oct 21, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/87432

❌ 2 Failures, 2 Pending

Uh oh!

digantdesai left a comment

Choose a reason for hiding this comment

Uh oh!

albanD Oct 21, 2022

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented Oct 31, 2022

Uh oh!

pytorchmergebot commented Oct 31, 2022

Merge started

Uh oh!

github-actions bot commented Oct 31, 2022

Uh oh!

salilsdesai commented Dec 16, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

salilsdesai commented Oct 21, 2022 •

edited

Loading

pytorch-bot bot commented Oct 21, 2022 •

edited

Loading