[torch deploy] Add -rdynamic option explicitly to CMakeLists.txt #75461
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Summary: This flag is needed in the OSS example and it wasn't clear that it was needed because it wasn't explicitly used when linking to torch deploy for the tests. Since torch deploy builds the tests using `python setup.py develop`, Pytorch actually sets this flag in the `CMAKE_EXE_LINKER_FLAGS` variable somewhere along the build. I had to print out all the variables used in the pytorch build to realize that I did not have this flag set in my OSS torch deploy example. I think having it explicit for the tests makes it clear which flags are actually necessary in an open source environment. **What is -rdynamic?** This flag (also known as `--export-dynamic` at the linker level) signals that the library it is targeting should export its symbols to the dynamic table. In doing so, shared libraries that are opened using `dlopen` (which is what torch deploy uses to launch subinterpreters: https://www.internalfb.com/code/fbsource/[ff6d5cfcc2b3]/xplat/caffe2/torch/csrc/deploy/deploy.cpp?lines=254), will be able to reference symbols defined in the modules that launched them. Without this flag, the symbols from the `torch::deploy` library that the subinterpreters need, remain `undefined`. This leads to runtime errors like: `/tmp/torch_XYZ: undefined symbol - torch::deploy::Etc.` `/tmp/torch_XYZ` is a subinterpreter. Test Plan: Ran the tests in an OSS environment `python torch/csrc/deploy/example/generate_examples.py` `./build/bin/test_deploy` Differential Revision: D35477135 fbshipit-source-id: 10b5bfb6f652f1703128d3b0c82b3785f8186558
Contributor
🔗 Helpful links
💊 CI failures summary and remediationsAs of commit 868e9b6 (more details on the Dr. CI page): 💚 💚 Looks good so far! There are no failures yet. 💚 💚 This comment was automatically generated by Dr. CI (expand for details).Please report bugs/suggestions to the (internal) Dr. CI Users group. |
Contributor
|
This pull request was exported from Phabricator. Differential Revision: D35477135 |
PaliC
approved these changes
Apr 8, 2022
facebook-github-bot
pushed a commit
that referenced
this pull request
Apr 8, 2022
) Summary: Pull Request resolved: #75461 This flag is needed in the OSS example and it wasn't clear that it was needed because it wasn't explicitly used when linking to torch deploy for the tests. Since torch deploy builds the tests using `python setup.py develop`, Pytorch actually sets this flag in the `CMAKE_EXE_LINKER_FLAGS` variable somewhere along the build. I had to print out all the variables used in the pytorch build to realize that I did not have this flag set in my OSS torch deploy example. I think having it explicit for the tests makes it clear which flags are actually necessary in an open source environment. **What is -rdynamic?** This flag (also known as `--export-dynamic` at the linker level) signals that the library it is targeting should export its symbols to the dynamic table. In doing so, shared libraries that are opened using `dlopen` (which is what torch deploy uses to launch subinterpreters: https://www.internalfb.com/code/fbsource/[ff6d5cfcc2b3]/xplat/caffe2/torch/csrc/deploy/deploy.cpp?lines=254), will be able to reference symbols defined in the modules that launched them. Without this flag, the symbols from the `torch::deploy` library that the subinterpreters need, remain `undefined`. This leads to runtime errors like: `/tmp/torch_XYZ: undefined symbol - torch::deploy::Etc.` `/tmp/torch_XYZ` is a subinterpreter. Test Plan: Ran the tests in an OSS environment `python torch/csrc/deploy/example/generate_examples.py` `./build/bin/test_deploy` Reviewed By: PaliC Differential Revision: D35477135 fbshipit-source-id: 30bd2b9fadd36b2a32066a52cda5b746d597e99f
Contributor
|
Hey @s4ayub. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
This flag is needed in the OSS example and it wasn't clear that it was needed because it wasn't explicitly used when linking to torch deploy for the tests.
Since torch deploy builds the tests using
python setup.py develop, Pytorch actually sets this flag in theCMAKE_EXE_LINKER_FLAGSvariable somewhere along the build. I had to print out all the variables used in the pytorch build to realize that I did not have this flag set in my OSS torch deploy example.I think having it explicit for the tests makes it clear which flags are actually necessary in an open source environment.
What is -rdynamic?
This flag (also known as
--export-dynamicat the linker level) signals that the library it is targeting should export its symbols to the dynamic table. In doing so, shared libraries that are opened usingdlopen(which is what torch deploy uses to launch subinterpreters: https://www.internalfb.com/code/fbsource/[ff6d5cfcc2b3]/xplat/caffe2/torch/csrc/deploy/deploy.cpp?lines=254), will be able to reference symbols defined in the modules that launched them.Without this flag, the symbols from the
torch::deploylibrary that the subinterpreters need, remainundefined.This leads to runtime errors like:
/tmp/torch_XYZ: undefined symbol - torch::deploy::Etc./tmp/torch_XYZis a subinterpreter.Test Plan:
Ran the tests in an OSS environment
python torch/csrc/deploy/example/generate_examples.py./build/bin/test_deployDifferential Revision: D35477135