Skip to content

Enable torch.cuda.nccl typechecking #45344

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
malfet wants to merge 1 commit into from
Closed

Enable torch.cuda.nccl typechecking #45344

malfet wants to merge 1 commit into from

Conversation

@malfet
Copy link
Contributor

@malfet malfet commented Sep 25, 2020

Fixes #45336

@malfet malfet added module: typing Related to mypy type annotations triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module labels Sep 25, 2020
@malfet malfet requested review from ezyang and walterddr September 25, 2020 17:38
facebook-github-bot
facebook-github-bot reviewed Sep 25, 2020
View reviewed changes
Copy link
Contributor

@facebook-github-bot facebook-github-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@malfet has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

walterddr
walterddr reviewed Sep 25, 2020
View reviewed changes
walterddr
walterddr reviewed Sep 25, 2020
View reviewed changes
torch/cuda/nccl.py Outdated
Copy link
Contributor

@walterddr walterddr Sep 25, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

was wondering why do we have this given we also have outputs as optional argument

@malfet malfet force-pushed the malfet/enable-torch.cuda.nccl-typechecking branch from aa56bb7 to 2f2a9f1 Compare September 25, 2020 20:58
facebook-github-bot
facebook-github-bot reviewed Sep 25, 2020
View reviewed changes
Copy link
Contributor

@facebook-github-bot facebook-github-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@malfet has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@malfet malfet force-pushed the malfet/enable-torch.cuda.nccl-typechecking branch from 2f2a9f1 to 8bdaac7 Compare September 25, 2020 21:30
facebook-github-bot
facebook-github-bot reviewed Sep 25, 2020
View reviewed changes
Copy link
Contributor

@facebook-github-bot facebook-github-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@malfet has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@dr-ci
Copy link

dr-ci bot commented Sep 25, 2020 •
edited
Loading

💊 CI failures summary and remediations

As of commit 8bdaac7 (more details on the Dr. CI page):

🕵️ 1 new failure recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

See CircleCI build pytorch_macos_10_13_py3_test (1/1)

Step: "Test" (full log | diagnosis details | 🔁 rerun)

Sep 25 23:41:18 /Users/[E request_callback_no_python.cpp:562] Received error while processing request type 2: RuntimeError: Can not pickle torch.futures.Future 
Sep 25 23:41:18 [E request_callback_no_python.cpp:562] Received error while processing request type 2: RuntimeError: Can not pickle torch.futures.Future 
Sep 25 23:41:18  
Sep 25 23:41:18 At: 
Sep 25 23:41:18   /Users/distiller/workspace/miniconda3/lib/python3.7/site-packages/torch/distributed/rpc/internal.py(94): serialize 
Sep 25 23:41:18   /Users/distiller/workspace/miniconda3/lib/python3.7/site-packages/torch/distributed/rpc/internal.py(146): serialize 
Sep 25 23:41:18  
Sep 25 23:41:18 [E request_callback_no_python.cpp:562] Received error while processing request type 2: RuntimeError: Can not pickle torch.futures.Future 
Sep 25 23:41:18  
Sep 25 23:41:18 At: 
Sep 25 23:41:18   /Users/distiller/workspace/miniconda3/lib/python3.7/site-packages/torch/distributed/rpc/internal.py(94): serialize 
Sep 25 23:41:18   /Users/[E request_callback_no_python.cpp:562] Received error while processing request type 2: RuntimeError: Can not pickle torch.futures.Future 
Sep 25 23:41:18  
Sep 25 23:41:18 At: 
Sep 25 23:41:18   /Users/distiller/workspace/miniconda3/lib/python3.7/site-packages/torch/distributed/rpc/internal.py(94): serialize 
Sep 25 23:41:18   /Users/distiller/workspace/miniconda3/lib/python3.7/site-packages/torch/distributed/rpc/internal.py(146): serialize 
Sep 25 23:41:18  
Sep 25 23:41:18 distiller/workspace/miniconda3/lib/python3.7/site-packages/torch/distributed/rpc/internal.py(146): serialize 
Sep 25 23:41:18  
Sep 25 23:41:18 ok (1.592s) 
Sep 25 23:41:19   test_return_future_remote (__main__.ProcessGroupRpcTestWithSpawn) ... RPC was initialized with the PROCESS_GROUP backend which is deprecated and slated to be removed and superseded by the TENSORPIPE backend. It is recommended to migrate to the TENSORPIPE backend. 
Sep 25 23:41:19 RPC was initialized with the PROCESS_GROUP backend which is deprecated and slated to be removed and superseded by the TENSORPIPE backend. It is recommended to migrate to the TENSORPIPE backend.

🚧 1 fixed upstream failure:

These were probably caused by upstream breakages that were already fixed.

Please rebase on the viable/strict branch (expand for instructions)

If your commit is newer than viable/strict, you can try basing on an older, stable commit:

git fetch https://github.com/pytorch/pytorch viable/strict
git rebase --onto FETCH_HEAD $(git merge-base origin/master HEAD)

If your commit is older than viable/strict:

git fetch https://github.com/pytorch/pytorch viable/strict
git rebase FETCH_HEAD

Check out the recency history of this "viable master" tracking branch.

This comment was automatically generated by Dr. CI (expand for details).Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker or post in the (internal) Dr. CI Users group.

See how this bot performed.

This comment has been revised 3 times.

@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Sep 26, 2020

@malfet merged this pull request in 8ab2ad3.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@facebook-github-bot facebook-github-bot facebook-github-bot left review comments

@ezyang ezyang Awaiting requested review from ezyang

+1 more reviewer

@walterddr walterddr walterddr approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

Merged module: typing Related to mypy type annotations triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Enable torch.cuda.nccl typecheck during CI

4 participants

@malfet @facebook-github-bot @walterddr @mruberry