Allow Tensor-likes in torch.autograd.gradcheck #45732

hameerabbasi · 2020-10-02T11:12:26Z

Re-do of #43877.

albanD

Looks good.
Note that another option can be to leave the current default and just say:

if True, check if undefined output grads are supported and treated as zeros. This flag is ignored for non-``Tensor`` outputs.

torch/autograd/gradcheck.py

hameerabbasi · 2020-10-08T14:14:42Z

Note that another option can be to leave the current default and just say:

I'd prefer not to ignore a flag when it is set:

>>> import this
...
Explicit is better than implicit.
...

albanD

I don't think the doc you mention is accurate. You actually default to True if all inputs are Tensors and False otherwise.
I think there is an issue with that:

This will disable this for other functions that pass arguments like (t1, t2, float_eps) and I'm pretty sure we have some of these.
We don't have to disable this if any is non-Tensor. We just don't want to use this for non-Tensor objects.

Do you think that modifying the line below to:

outputs_to_check = [[torch._C._functions.UndefinedGrad()(o) for o in _differentiable_outputs(func(*tupled_inputs)) if isinstance(o, torch.Tensor)]]

Would work?

And you can change the doc accordingly to mention that the flag only affect Tensor objects.

hameerabbasi · 2020-10-08T15:30:33Z

Do you think that modifying the line below to:
outputs_to_check = [[torch._C._functions.UndefinedGrad()(o) for o in _differentiable_outputs(func(*tupled_inputs)) if isinstance(o, torch.Tensor)]]
Would work?

And you can change the doc accordingly to mention that the flag only affect Tensor objects.

That's a fair point; done!

albanD

LGTM
Just one small comment.

Out of curiosity. Does this work with gradgradcheck as well?

torch/autograd/gradcheck.py

hameerabbasi · 2020-10-08T16:43:55Z

Out of curiosity. Does this work with gradgradcheck as well?

It does, added a test.

albanD

Perfect!
Thanks for the updates.

facebook-github-bot

@albanD has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

dr-ci · 2020-10-09T06:23:45Z

💊 CI failures summary and remediations

As of commit c93a341 (more details on the Dr. CI page):

1/1 failures introduced in this PR

🕵️ 1 new failure recognized by patterns

The following CI failures do not appear to be due to upstream breakages (reran 1 job to discount flakiness):

pytorch_macos_10_13_py3_build (1/1)

Step: "Update Homebrew" (full log | diagnosis details | 🔁 rerun) <confirmed not flaky by 2 failures>

fatal: Could not read from remote repository.

remote: Total 60 (delta 51), reused 21 (delta 17), pack-reused 0         
Unpacking objects:  96% (58/60) Unpacking objects:  98% (59/60) Unpacking objects: 100% (60/60) Unpacking objects: 100% (60/60), 13.00 KiB | 302.00 KiB/s, done. 
From ssh://github.com/Homebrew/homebrew-cask-versions 
 + f127a2be3...8d1d26f57 master     -> origin/master  (forced update) 
+ git reset --hard origin/master 
HEAD is now at 8d1d26f57 Update microsoft-edge-beta from 86.0.622.36 to 86.0.622.38 (#9741) 
+ for path in '$(find /usr/local/Homebrew -type d -name .git)' 
+ cd /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/.git/.. 
+ git fetch --depth=1 origin 
Connection to github.com closed by remote host.  
fatal: Could not read from remote repository. 
 
Please make sure you have the correct access rights 
and the repository exists.

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker or post in the (internal) Dr. CI Users group.

See how this bot performed.

This comment has been revised 20 times.

…to gradcheck-override

hameerabbasi · 2020-10-09T13:59:00Z

The remaining test failure appears to be a persistent network issue, and unrelated to this PR.

facebook-github-bot

@albanD has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

facebook-github-bot · 2020-10-09T19:47:48Z

@albanD merged this pull request in f8b3af2.

facebook-github-bot · 2020-10-09T19:47:55Z

@albanD merged this pull request in f8b3af2.

hameerabbasi mentioned this pull request Oct 2, 2020

torch.autograd.gradcheck support for Tensor-like types (__torch_function__). #42942

Closed

pytorchbot added the open source label Oct 2, 2020

VitalyFedyunin requested a review from albanD October 5, 2020 23:11

albanD added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Oct 6, 2020

hameerabbasi closed this Oct 8, 2020

hameerabbasi force-pushed the gradcheck-override branch from a7d151a to 9679e1a Compare October 8, 2020 13:25

hameerabbasi added the module: numpy Related to numpy support, and also numpy compatibility of our operators label Oct 8, 2020

hameerabbasi deleted the gradcheck-override branch October 8, 2020 13:26

hameerabbasi restored the gradcheck-override branch October 8, 2020 13:27

hameerabbasi reopened this Oct 8, 2020

hameerabbasi force-pushed the gradcheck-override branch from 7f05c68 to 4162da8 Compare October 8, 2020 13:32

hameerabbasi requested a review from apaszke as a code owner October 8, 2020 13:32

hameerabbasi force-pushed the gradcheck-override branch from 4162da8 to fada1d4 Compare October 8, 2020 13:48

albanD reviewed Oct 8, 2020

View reviewed changes

torch/autograd/gradcheck.py Outdated Show resolved Hide resolved

hameerabbasi force-pushed the gradcheck-override branch from fada1d4 to 2ef0683 Compare October 8, 2020 14:13

albanD reviewed Oct 8, 2020

View reviewed changes

hameerabbasi force-pushed the gradcheck-override branch 2 times, most recently from 5b828ef to fac5b0e Compare October 8, 2020 15:29

albanD reviewed Oct 8, 2020

View reviewed changes

torch/autograd/gradcheck.py Outdated Show resolved Hide resolved

hameerabbasi force-pushed the gradcheck-override branch from fac5b0e to 00de60a Compare October 8, 2020 16:40

Allow Tensor-likes in gradcheck and gradgradcheck.

1b463ae

hameerabbasi force-pushed the gradcheck-override branch from 00de60a to 1b463ae Compare October 8, 2020 16:43

albanD approved these changes Oct 8, 2020

View reviewed changes

facebook-github-bot reviewed Oct 8, 2020

View reviewed changes

hameerabbasi requested a review from anjali411 October 9, 2020 05:35

hameerabbasi mentioned this pull request Oct 9, 2020

Complex gradcheck logic #43208

Closed

7 tasks

Fix mypy error.

123e2b2

hameerabbasi added 2 commits October 9, 2020 02:11

Fix up tests/mypy.

eeeaf92

Merge branch 'viable/strict' of https://github.com/pytorch/pytorch in…

c93a341

…to gradcheck-override

hameerabbasi requested a review from albanD October 9, 2020 13:25

facebook-github-bot reviewed Oct 9, 2020

View reviewed changes

albanD approved these changes Oct 9, 2020

View reviewed changes

facebook-github-bot closed this in f8b3af2 Oct 9, 2020

facebook-github-bot added the Merged label Oct 9, 2020

hameerabbasi deleted the gradcheck-override branch October 12, 2020 14:10

Allow Tensor-likes in torch.autograd.gradcheck #45732

Allow Tensor-likes in torch.autograd.gradcheck #45732

Uh oh!

Conversation

hameerabbasi commented Oct 2, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

albanD left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

hameerabbasi commented Oct 8, 2020

Uh oh!

albanD left a comment

Choose a reason for hiding this comment

Uh oh!

hameerabbasi commented Oct 8, 2020

Uh oh!

albanD left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

hameerabbasi commented Oct 8, 2020

Uh oh!

albanD left a comment

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot left a comment

Choose a reason for hiding this comment

Uh oh!

dr-ci bot commented Oct 9, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💊 CI failures summary and remediations

🕵️ 1 new failure recognized by patterns

pytorch_macos_10_13_py3_build (1/1)

Uh oh!

hameerabbasi commented Oct 9, 2020

Uh oh!

facebook-github-bot left a comment

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented Oct 9, 2020

Uh oh!

facebook-github-bot commented Oct 9, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

hameerabbasi commented Oct 2, 2020 •

edited

Loading

dr-ci bot commented Oct 9, 2020 •

edited

Loading