Move private forward grad mode helpers to torch.autograd.forward_ad #90240
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Motivation - These were previously defined in functorch. They are not functorch-specific, so I'm moving them to torch.autograd.forward_ad and the autograd python bindings. - I need this to avoid some of my cyclic import problems. Should these be public APIs? Probably. Though this needs discussion, so punting it to the future. Test Plan: - moved the tests of these from test/functorch/test_eager_transforms.py to test/test_autograd.py [ghstack-poisoned]
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/90240
Note: Links to docs will display an error until the docs builds have been completed. ❌ 1 FailuresAs of commit fb114c3: The following jobs have failed:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
This was referenced Dec 6, 2022
…orward_ad" Motivation - These were previously defined in functorch. They are not functorch-specific, so I'm moving them to torch.autograd.forward_ad and the autograd python bindings. - I need this to avoid some of my cyclic import problems. Should these be public APIs? Probably. Though this needs discussion, so punting it to the future. Test Plan: - moved the tests of these from test/functorch/test_eager_transforms.py to test/test_autograd.py [ghstack-poisoned]
…orward_ad" Motivation - These were previously defined in functorch. They are not functorch-specific, so I'm moving them to torch.autograd.forward_ad and the autograd python bindings. - I need this to avoid some of my cyclic import problems. Should these be public APIs? Probably. Though this needs discussion, so punting it to the future. Test Plan: - moved the tests of these from test/functorch/test_eager_transforms.py to test/test_autograd.py [ghstack-poisoned]
…orward_ad" Motivation - These were previously defined in functorch. They are not functorch-specific, so I'm moving them to torch.autograd.forward_ad and the autograd python bindings. - I need this to avoid some of my cyclic import problems. Should these be public APIs? Probably. Though this needs discussion, so punting it to the future. Test Plan: - moved the tests of these from test/functorch/test_eager_transforms.py to test/test_autograd.py [ghstack-poisoned]
…orward_ad" Motivation - These were previously defined in functorch. They are not functorch-specific, so I'm moving them to torch.autograd.forward_ad and the autograd python bindings. - I need this to avoid some of my cyclic import problems. Should these be public APIs? Probably. Though this needs discussion, so punting it to the future. Test Plan: - moved the tests of these from test/functorch/test_eager_transforms.py to test/test_autograd.py [ghstack-poisoned]
…orward_ad" Motivation - These were previously defined in functorch. They are not functorch-specific, so I'm moving them to torch.autograd.forward_ad and the autograd python bindings. - I need this to avoid some of my cyclic import problems. Should these be public APIs? Probably. Though this needs discussion, so punting it to the future. Test Plan: - moved the tests of these from test/functorch/test_eager_transforms.py to test/test_autograd.py [ghstack-poisoned]
soulitzer
reviewed
Dec 12, 2022
torch/autograd/forward_ad.py
Outdated
| # Private helper function to enable or disable fwd grad. | ||
| # If you're a user and want to use this, please file an issue to discuss the use case. | ||
| @contextlib.contextmanager | ||
| def _enable_fwd_grad(enabled=True): |
Contributor
There was a problem hiding this comment.
Maybe we should be consistent with the API for backward AD. This function looks like it is analogous to _set_grad_enabled, so we should name it _set_fwd_grad_enabled and don't have a default value. (If it were analogous to enable_grad, it shouldn't take any arguments and always sets it to True.)
…orward_ad" Motivation - These were previously defined in functorch. They are not functorch-specific, so I'm moving them to torch.autograd.forward_ad and the autograd python bindings. - I need this to avoid some of my cyclic import problems. Should these be public APIs? Probably. Though this needs discussion, so punting it to the future. Test Plan: - moved the tests of these from test/functorch/test_eager_transforms.py to test/test_autograd.py [ghstack-poisoned]
…orward_ad" Motivation - These were previously defined in functorch. They are not functorch-specific, so I'm moving them to torch.autograd.forward_ad and the autograd python bindings. - I need this to avoid some of my cyclic import problems. Should these be public APIs? Probably. Though this needs discussion, so punting it to the future. Test Plan: - moved the tests of these from test/functorch/test_eager_transforms.py to test/test_autograd.py [ghstack-poisoned]
soulitzer
approved these changes
Dec 13, 2022
Contributor
Author
|
@pytorchbot merge -f "test failures look unrelated" |
Collaborator
Merge startedYour change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Collaborator
|
Comments like pytorch/torch/csrc/autograd/autograd_meta.cpp Line 275 in 5c133c5 |
11 tasks
Contributor
Author
zou3519
added a commit
that referenced
this pull request
Dec 13, 2022
Addresses #90240 (comment) This was the only comment I found after grepping the codebase, but please let me know if I missed others. Test Plan: - code reading [ghstack-poisoned]
zou3519
added a commit
that referenced
this pull request
Dec 13, 2022
Addresses #90240 (comment) This was the only comment I found after grepping the codebase, but please let me know if I missed others. Test Plan: - code reading ghstack-source-id: 7d94a9d Pull Request resolved: #90790
zou3519
added a commit
that referenced
this pull request
Dec 16, 2022
…being used in custom Function" Addresses #90240 (comment) This was the only comment I found after grepping the codebase, but please let me know if I missed others. Test Plan: - code reading [ghstack-poisoned]
zou3519
added a commit
that referenced
this pull request
Dec 16, 2022
…stom Function" Addresses #90240 (comment) This was the only comment I found after grepping the codebase, but please let me know if I missed others. Test Plan: - code reading [ghstack-poisoned]
pytorchmergebot
pushed a commit
that referenced
this pull request
Dec 17, 2022
…on (#90790) Addresses #90240 (comment) This was the only comment I found after grepping the codebase, but please let me know if I missed others. Test Plan: - code reading Pull Request resolved: #90790 Approved by: https://github.com/soulitzer
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Stack from ghstack:
Motivation
functorch-specific, so I'm moving them to torch.autograd.forward_ad and
the autograd python bindings.
Should these be public APIs? Probably. Though this needs discussion, so
punting it to the future.
Test Plan:
to test/test_autograd.py