WIP: Reproducibility note #11329
WIP: Reproducibility note #11329
Conversation
themightyoarfish
requested review from
apaszke,
colesbury,
ezyang,
gchanan,
soumith,
ssnl,
zdevito and
zou3519
as code owners
|
I think there are known reproducibility issues with distributed (search the forums). |
|
Ok then the note should be amended, but would you agree that there should be some documentation on it? At least on a single machine this would do the trick, and finding this information was really hard. |
|
@themightyoarfish Please revert the submodule change. Moreover, it might be good to mention that reproducibility is not guaranteed across versions and/or architectures. E.g., a RNG on CUDA definitely behaves different than the RNG on CPU even though they use same seed; and when we optimize a sampling kernel in a release, the generated numbers might be different (although distribution is the same). |
- Add caveat that reproducibility is not guaranteed across platforms or releases
themightyoarfish
force-pushed
the
reproducibility-note
branch
from
8b1ade4 to
83c28e6
Compare
|
I have fixed the submodule change hopefully and clarified the note. I wasn't intending to communicate that it's possible to be reproducible across platforms or releases or anything; the info is intended for users trying to make their runs reproducible. |
README.md
Outdated
| To build documentation in various formats, you will need Sphinx. | ||
|
|
||
| ``` | ||
| conda install sphinx sphinx_rtd_theme |
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
facebook-github-bot
left a comment
facebook-github-bot
left a comment
There was a problem hiding this comment.
ezyang is landing this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
…nn (#11434) Summary: Ping ezyang This addresses your comment in #114. Strangely, when running the doc build (`make html`) none of my changes are actually showing, could you point out what I'm doing wrong? Once #11329 is merged it might make sense to link to the reproducibility note everywhere. Pull Request resolved: #11434 Differential Revision: D9751208 Pulled By: ezyang fbshipit-source-id: cc672472449564ff099323c39603e8ff2b2d35c9
|
Seems like we have a typo in the note |
6 participants
This adds a Note on making experiments reproducible.
It also adds Instructions for building the Documentation to
README.md. Please ping if I missed any requirements.I'm not sure what to do about the submodule changes. Please advise.