torch.utils.data.Dataloader: documentation about RNG state consumption (#22540)

InnovArul · facebook-github-bot · commit 43d36415b980 · 2019-07-08T08:22:04.000-07:00
Summary: the outcome from the pytorch forum issue: https://discuss.pytorch.org/t/dataloader-problem-problem-arises-when-shuffle-true/45631 The discussion is here: #20749 Pull Request resolved: #22540 Differential Revision: D16131777 Pulled By: ezyang fbshipit-source-id: 566deda1b44dc7fae54250e9b508d120851a2848
diff --git a/docs/source/data.rst b/docs/source/data.rst
@@ -332,12 +332,11 @@ are compatible with Windows while using multi-process data loading:
 Randomness in multi-process data loading
 """"""""""""""""""""""""""""""""""""""""""
 
-By default, each worker will have its PyTorch seed set to
-``base_seed + worker_id``, where ``base_seed`` is a long generated
-by main process using its RNG. However, seeds for other libraries
-may be duplicated upon initializing workers (w.g., NumPy), causing
-each worker to return identical random numbers. (See
-:ref:`this section <dataloader-workers-random-seed>` in FAQ.)
+By default, each worker will have its PyTorch seed set to ``base_seed + worker_id``,
+where ``base_seed`` is a long generated by main process using its RNG (thereby, 
+consuming a RNG state mandatorily). However, seeds for other libraries may be 
+duplicated upon initializing workers (w.g., NumPy), causing each worker to return 
+identical random numbers. (See :ref:`this section <dataloader-workers-random-seed>` in FAQ.).
 
 In :attr:`worker_init_fn`, you may access the PyTorch seed set for each worker
 with either :func:`torch.utils.data.get_worker_info().seed <torch.utils.data.get_worker_info>`
diff --git a/torch/utils/data/dataloader.py b/torch/utils/data/dataloader.py
@@ -23,6 +23,7 @@
 # aspect.
 default_collate = _utils.collate.default_collate
 
+
 class _DatasetKind(object):
     Map = 0
     Iterable = 1
@@ -39,6 +40,7 @@ class _InfiniteConstantSampler(Sampler):
     r"""Analogous to ``itertools.repeat(None, None)``.
     Used as sampler for :class:`~torch.utils.data.IterableDataset`.
     """
+
     def __init__(self):
         super(_InfiniteConstantSampler, self).__init__(None)
 
