[dtensor][5/N] add cached propagator for TP #90734

wanchaol · 2022-12-13T01:01:36Z

Stack from ghstack (oldest at bottom):

This PR adds a cached propagator for TP use, it caches the sharding
prop decision for the same input sharding on an operator. This could
improve eager mode performance.

Differential Revision: D42876249

This PR adds a cached propagator for TP use, it caches the sharding prop decision for the same input sharding on an operator. This could improve eager mode performance. [ghstack-poisoned]

pytorch-bot · 2022-12-13T01:01:39Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/90734

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 Failures

As of commit c74fec7:

NEW FAILURES - The following jobs have failed:

win-vs2019-cuda11.6-py3 / test (functorch, 1, 1, windows.g5.4xlarge.nvidia.gpu)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

This PR adds a cached propagator for TP use, it caches the sharding prop decision for the same input sharding on an operator. This could improve eager mode performance. [ghstack-poisoned]

XilunWu

thx for enabling _CachingPropagator in TP api.

This PR adds a cached propagator for TP use, it caches the sharding prop decision for the same input sharding on an operator. This could improve eager mode performance. [ghstack-poisoned]

wanchaol · 2023-01-31T02:39:43Z

@wanchaol has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

This PR adds a cached propagator for TP use, it caches the sharding prop decision for the same input sharding on an operator. This could improve eager mode performance. Differential Revision: [D42876249](https://our.internmc.facebook.com/intern/diff/D42876249) [ghstack-poisoned]

wanchaol · 2023-01-31T16:16:04Z

@wanchaol has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

This PR adds a cached propagator for TP use, it caches the sharding prop decision for the same input sharding on an operator. This could improve eager mode performance. Differential Revision: [D42876249](https://our.internmc.facebook.com/intern/diff/D42876249) [ghstack-poisoned]

fduwjj

Thanks for making this happens! Just a small QQ. Unblock for now.

fduwjj · 2023-01-31T18:41:04Z

torch/distributed/tensor/parallel/api.py


+# switch the DTensor propagator to use the caching propagator to speed up
+# the TP eager execution time.
+DTensor._propagator = _CachingPropagator(DTensor._propagator.op_to_rules)


Can we have a flag to switch this on/off. Or we want to enable it by default?

I feel we should turn this on by default, otherwise eager mode perf would be very bad, especially once we implement more complicated sharding prop. This caching logic is relatively safe as it only change the one with the exact same input specs.

facebook-github-bot · 2023-02-01T05:02:27Z

@pytorchbot merge

(Initiating merge automatically since Phabricator Diff has merged)

pytorchmergebot · 2023-02-01T05:04:04Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

seemethere · 2023-02-01T21:56:09Z

@wanchaol has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

How are these linked to the incorrect diff that was landed? From what I understand the correct diff for this is D42888015

cc @bigfootjon

bigfootjon · 2023-02-01T23:45:04Z

Ping me on workchat and I'll take a look tomorrow

[dtensor][4/N] add cached propagator for TP

27bd069

This PR adds a cached propagator for TP use, it caches the sharding prop decision for the same input sharding on an operator. This could improve eager mode performance. [ghstack-poisoned]

wanchaol requested review from H-Huang, awgu, kwen2501, mrshenli, pritamdamania87, rohan-varma and zhaojuanmao as code owners December 13, 2022 01:01

Update on "[dtensor][4/N] add cached propagator for TP"

7132c86

This PR adds a cached propagator for TP use, it caches the sharding prop decision for the same input sharding on an operator. This could improve eager mode performance. [ghstack-poisoned]

wanchaol added the release notes: distributed (dtensor) release notes category label Dec 20, 2022

wanchaol added 6 commits December 29, 2022 19:43

Update on "[dtensor][4/N] add cached propagator for TP"

358b4ee

This PR adds a cached propagator for TP use, it caches the sharding prop decision for the same input sharding on an operator. This could improve eager mode performance. [ghstack-poisoned]

Update on "[dtensor][4/N] add cached propagator for TP"

489cbe1

This PR adds a cached propagator for TP use, it caches the sharding prop decision for the same input sharding on an operator. This could improve eager mode performance. [ghstack-poisoned]

Update on "[dtensor][4/N] add cached propagator for TP"

15942f6

This PR adds a cached propagator for TP use, it caches the sharding prop decision for the same input sharding on an operator. This could improve eager mode performance. [ghstack-poisoned]

Update on "[dtensor][4/N] add cached propagator for TP"

006e5ed

This PR adds a cached propagator for TP use, it caches the sharding prop decision for the same input sharding on an operator. This could improve eager mode performance. [ghstack-poisoned]

Update on "[dtensor][4/N] add cached propagator for TP"

ad35923

This PR adds a cached propagator for TP use, it caches the sharding prop decision for the same input sharding on an operator. This could improve eager mode performance. [ghstack-poisoned]

Update on "[dtensor][4/N] add cached propagator for TP"

7a20f1f

This PR adds a cached propagator for TP use, it caches the sharding prop decision for the same input sharding on an operator. This could improve eager mode performance. [ghstack-poisoned]

wanchaol mentioned this pull request Jan 6, 2023

[dtensor][2/N] add __repr__ to placements #91785

Closed

Update on "[dtensor][5/N] add cached propagator for TP"

21ebe94

This PR adds a cached propagator for TP use, it caches the sharding prop decision for the same input sharding on an operator. This could improve eager mode performance. [ghstack-poisoned]

wanchaol changed the title ~~[dtensor][4/N] add cached propagator for TP~~ [dtensor][5/N] add cached propagator for TP Jan 6, 2023

wanchaol requested review from XilunWu, aazzolini and fduwjj and removed request for pritamdamania87 January 6, 2023 23:07

Update on "[dtensor][5/N] add cached propagator for TP"

1a8faf2

This PR adds a cached propagator for TP use, it caches the sharding prop decision for the same input sharding on an operator. This could improve eager mode performance. [ghstack-poisoned]

XilunWu approved these changes Jan 18, 2023

View reviewed changes

wanchaol added 4 commits January 19, 2023 00:39

Update on "[dtensor][5/N] add cached propagator for TP"

1cc5c3d

This PR adds a cached propagator for TP use, it caches the sharding prop decision for the same input sharding on an operator. This could improve eager mode performance. [ghstack-poisoned]

Update on "[dtensor][5/N] add cached propagator for TP"

e08730e

This PR adds a cached propagator for TP use, it caches the sharding prop decision for the same input sharding on an operator. This could improve eager mode performance. [ghstack-poisoned]

Update on "[dtensor][5/N] add cached propagator for TP"

3f8b5a6

This PR adds a cached propagator for TP use, it caches the sharding prop decision for the same input sharding on an operator. This could improve eager mode performance. [ghstack-poisoned]

Update on "[dtensor][5/N] add cached propagator for TP"

43e53be

This PR adds a cached propagator for TP use, it caches the sharding prop decision for the same input sharding on an operator. This could improve eager mode performance. [ghstack-poisoned]

This was referenced Jan 24, 2023

add numpy typing plugin to mypy config #92930

Closed

[dtensor][8/N] switch DeviceMesh to use numpy array for devices #92931

Closed

Update on "[dtensor][5/N] add cached propagator for TP"

1b0dc4b

This PR adds a cached propagator for TP use, it caches the sharding prop decision for the same input sharding on an operator. This could improve eager mode performance. [ghstack-poisoned]

wanchaol mentioned this pull request Jan 26, 2023

[dtensor][7/N] remove backend in with_comms #93040

Closed

wanchaol added 6 commits January 26, 2023 04:05

Update on "[dtensor][5/N] add cached propagator for TP"

6efe682

This PR adds a cached propagator for TP use, it caches the sharding prop decision for the same input sharding on an operator. This could improve eager mode performance. [ghstack-poisoned]

Update on "[dtensor][5/N] add cached propagator for TP"

8dfd28a

This PR adds a cached propagator for TP use, it caches the sharding prop decision for the same input sharding on an operator. This could improve eager mode performance. [ghstack-poisoned]

Update on "[dtensor][5/N] add cached propagator for TP"

bb38d89

This PR adds a cached propagator for TP use, it caches the sharding prop decision for the same input sharding on an operator. This could improve eager mode performance. [ghstack-poisoned]

Update on "[dtensor][5/N] add cached propagator for TP"

ca2b14f

This PR adds a cached propagator for TP use, it caches the sharding prop decision for the same input sharding on an operator. This could improve eager mode performance. [ghstack-poisoned]

Update on "[dtensor][5/N] add cached propagator for TP"

009a245

This PR adds a cached propagator for TP use, it caches the sharding prop decision for the same input sharding on an operator. This could improve eager mode performance. [ghstack-poisoned]

Update on "[dtensor][5/N] add cached propagator for TP"

197d2f7

This PR adds a cached propagator for TP use, it caches the sharding prop decision for the same input sharding on an operator. This could improve eager mode performance. [ghstack-poisoned]

wanchaol added 2 commits January 31, 2023 08:26

fduwjj approved these changes Jan 31, 2023

View reviewed changes

pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Feb 1, 2023

pytorchmergebot added the Merged label Feb 1, 2023

pytorchmergebot closed this in 9a56997 Feb 1, 2023

facebook-github-bot deleted the gh/wanchaol/238/head branch June 8, 2023 19:07

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[dtensor][5/N] add cached propagator for TP #90734

[dtensor][5/N] add cached propagator for TP #90734

Uh oh!

wanchaol commented Dec 13, 2022 •

edited

Loading

Uh oh!

pytorch-bot bot commented Dec 13, 2022 •

edited

Loading

Uh oh!

XilunWu left a comment

Uh oh!

wanchaol commented Jan 31, 2023

Uh oh!

wanchaol commented Jan 31, 2023

Uh oh!

fduwjj left a comment

Uh oh!

fduwjj Jan 31, 2023

Uh oh!

wanchaol Jan 31, 2023

Uh oh!

facebook-github-bot commented Feb 1, 2023

Uh oh!

pytorchmergebot commented Feb 1, 2023

Uh oh!

seemethere commented Feb 1, 2023 •

edited

Loading

Uh oh!

bigfootjon commented Feb 1, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

[dtensor][5/N] add cached propagator for TP #90734

[dtensor][5/N] add cached propagator for TP #90734

Uh oh!

Conversation

wanchaol commented Dec 13, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Dec 13, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/90734

❌ 1 Failures

Uh oh!

XilunWu left a comment

Choose a reason for hiding this comment

Uh oh!

wanchaol commented Jan 31, 2023

Uh oh!

wanchaol commented Jan 31, 2023

Uh oh!

fduwjj left a comment

Choose a reason for hiding this comment

Uh oh!

fduwjj Jan 31, 2023

Choose a reason for hiding this comment

Uh oh!

wanchaol Jan 31, 2023

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented Feb 1, 2023

Uh oh!

pytorchmergebot commented Feb 1, 2023

Merge started

Uh oh!

seemethere commented Feb 1, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bigfootjon commented Feb 1, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

wanchaol commented Dec 13, 2022 •

edited

Loading

pytorch-bot bot commented Dec 13, 2022 •

edited

Loading

seemethere commented Feb 1, 2023 •

edited

Loading