Skip to content

Make Adadelta,Adagrad & Adamax differentiable #86096

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
emcastillo wants to merge 4 commits into from
Closed

Make Adadelta,Adagrad & Adamax differentiable #86096

emcastillo wants to merge 4 commits into from

Conversation

@emcastillo
Copy link
Collaborator

@emcastillo emcastillo commented Oct 3, 2022

Continuing the differentiable optimizers support

@emcastillo emcastillo requested a review from albanD as a code owner October 3, 2022 05:38
@pytorch-bot
Copy link

pytorch-bot bot commented Oct 3, 2022 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/86096

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit d9e91d8:
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@emcastillo emcastillo added module: optimizer Related to torch.optim topic: improvements topic category release notes: nn release notes category and removed cla signed labels Oct 3, 2022
@emcastillo emcastillo changed the title Make AdaDelta differentiable Make Adadelta & Adagrad differentiable Oct 3, 2022
@mikaylagawarecki mikaylagawarecki added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Oct 3, 2022
@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Oct 4, 2022

/easycla

As part of the transition to the PyTorch Foundation, this project now requires contributions be covered under the new CLA. See #85559 for additional details.

This comment will trigger a new check of this PR. If you are already covered, you will simply see a new "EasyCLA" check that passes. If you are not covered, a bot will leave a new comment with a link to sign.

@linux-foundation-easycla
Copy link

linux-foundation-easycla bot commented Oct 4, 2022 •
edited
Loading

CLA Signed

The committers listed above are authorized under a signed CLA.

@emcastillo emcastillo force-pushed the differentiable-adadelta branch from 6fca7eb to 54475d5 Compare October 4, 2022 00:07
@emcastillo emcastillo changed the title Make Adadelta & Adagrad differentiable Make Adadelta,Adagrad & Adamax differentiable Oct 4, 2022
@emcastillo emcastillo force-pushed the differentiable-adadelta branch from b1fb0a1 to 69a875d Compare October 4, 2022 02:59
@emcastillo emcastillo mentioned this pull request Oct 4, 2022
Closed
janeyx99
janeyx99 reviewed Oct 5, 2022
View reviewed changes
torch/optim/adamax.py Outdated
Copy link
Contributor

@janeyx99 janeyx99 Oct 5, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the distinction here to maintain functional-ism/no mutations?

Copy link
Collaborator Author

@emcastillo emcastillo Oct 6, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is to avoid an in-place modification of exp_inf that will trigger autograd errors :)

@janeyx99
Copy link
Contributor

janeyx99 commented Oct 5, 2022

@emcastillo this change looks good overall! Could you sign the CLA + rebase your PR to make the pr_sanity_check green?

The XLA failure looks unrelated, so rerunning to make sure that's the case.

@emcastillo
Copy link
Collaborator Author

emcastillo commented Oct 6, 2022 •
edited
Loading

Yeah, I am waiting for my company to fill the corporate CLA, so it will take a few days :)
Thanks!!

@janeyx99
Copy link
Contributor

janeyx99 commented Oct 6, 2022

could you rebase locally to see if you could clear the other two CI failures? you can also comment “@pytorchbot rebase” to rebase onto our stable viable/strict branch

@emcastillo emcastillo force-pushed the differentiable-adadelta branch from 69a875d to ed11691 Compare October 6, 2022 05:38
@janeyx99
Copy link
Contributor

janeyx99 commented Oct 6, 2022

Nice! This PR would be good to go once your company gets the CLA done.

@pytorch-bot pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Oct 6, 2022
@emcastillo emcastillo force-pushed the differentiable-adadelta branch from ed11691 to d9e91d8 Compare October 12, 2022 08:27
@emcastillo
Copy link
Collaborator Author

emcastillo commented Oct 12, 2022

@pytorchbot merge

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Oct 12, 2022

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

pytorchmergebot pushed a commit that referenced this pull request Oct 17, 2022
@pytorchmergebot
Make AdamW, NAdam & RAdam differentiable (#86183)
1b43883 
Blocked by #86096
Pull Request resolved: #86183
Approved by: https://github.com/albanD
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@janeyx99 janeyx99 janeyx99 approved these changes

@albanD albanD Awaiting requested review from albanD

Assignees

No one assigned

Labels

ciflow/trunk Trigger trunk jobs on your pull request cla signed Merged module: optimizer Related to torch.optim open source release notes: nn release notes category topic: improvements topic category triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Projects

torch.nn/optim
Status: Done

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@emcastillo @facebook-github-bot @janeyx99 @pytorchmergebot @pytorchbot @mikaylagawarecki