Add remote_parameters() into RemoteModule class. #43906
Conversation
wayi1
requested review from
apaszke,
mrshenli,
pietern,
pritamdamania87 and
zhaojuanmao
as code owners
|
This pull request was exported from Phabricator. Differential Revision: D23399586 |
Codecov Report
@@ Coverage Diff @@
## master #43906 +/- ##
==========================================
- Coverage 69.35% 69.33% -0.02%
==========================================
Files 381 381
Lines 47321 47341 +20
==========================================
+ Hits 32820 32826 +6
- Misses 14501 14515 +14
Continue to review full report at Codecov.
|
💊 CI failures summary and remediationsAs of commit ff2d095 (more details on the Dr. CI page):
Extra GitHub checks: 1 failed
ci.pytorch.org: 1 failedcodecov.io: 1 failed
This comment was automatically generated by Dr. CI (expand for details).Follow this link to opt-out of these comments for your Pull Requests.Please report bugs/suggestions on the GitHub issue tracker or post in the (internal) Dr. CI Users group. This comment has been revised 17 times. |
pritamdamania87
left a comment
pritamdamania87
left a comment
There was a problem hiding this comment.
Overall approach looks good, lets add some test coverage :)
torch/testing/_internal/distributed/nn/api/remote_module_test.py
Outdated
Show resolved
Hide resolved
|
This pull request was exported from Phabricator. Differential Revision: D23399586 |
2 similar comments
|
This pull request was exported from Phabricator. Differential Revision: D23399586 |
|
This pull request was exported from Phabricator. Differential Revision: D23399586 |
Summary: Pull Request resolved: #43906 This method returns a list of RRefs of remote parameters that can be fed into the DistributedOptimizer. Original PR issue: RemoteModule enhancements #40550 Test Plan: buck test caffe2/test/distributed/rpc:process_group_agent -- RemoteModule Differential Revision: D23399586 fbshipit-source-id: add1a925924c946759ab08656d2150a863c6df95
|
This pull request was exported from Phabricator. Differential Revision: D23399586 |
|
This pull request has been merged in 8b17fd2. |
| Returns: | ||
| A list of RRefs to remote module parameters. | ||
| """ | ||
| return rpc.rpc_sync(self.on, _param_rrefs, args=(self.module_rref, recurse)) |
There was a problem hiding this comment.
Just curious, would it be good to support an async API for this as well? It could be useful if users want to retrieve the parameters for several remote modules and an async API would allow us to do this concurrently.
5 participants
Summary:
This method returns a list of RRefs of remote parameters that can be fed into the DistributedOptimizer.
Still trying to add a unit test.
Test Plan: buck test caffe2/test/distributed/rpc:process_group_agent -- RemoteModule
Differential Revision: D23399586