Clarify cycliclr param docs (#20880)

sampepose · facebook-github-bot · commit 082936f03335 · 2019-05-28T12:07:47.000-07:00
Summary: Pull Request resolved: #20880 This clarifies how the momentum parameters should be used. Reviewed By: soumith Differential Revision: D15482450 fbshipit-source-id: e3649a38876c5912cb101d8e404abca7c3431766
diff --git a/torch/optim/lr_scheduler.py b/torch/optim/lr_scheduler.py
@@ -498,16 +498,21 @@ class CyclicLR(_LRScheduler):
         cycle_momentum (bool): If ``True``, momentum is cycled inversely
             to learning rate between 'base_momentum' and 'max_momentum'.
             Default: True
-        base_momentum (float or list): Initial momentum which is the
-            lower boundary in the cycle for each parameter group.
+        base_momentum (float or list): Lower momentum boundaries in the cycle
+            for each parameter group. Note that momentum is cycled inversely
+            to learning rate; at the peak of a cycle, momentum is
+            'base_momentum' and learning rate is 'max_lr'.
             Default: 0.8
         max_momentum (float or list): Upper momentum boundaries in the cycle
             for each parameter group. Functionally,
             it defines the cycle amplitude (max_momentum - base_momentum).
             The momentum at any cycle is the difference of max_momentum
             and some scaling of the amplitude; therefore
             base_momentum may not actually be reached depending on
-            scaling function. Default: 0.9
+            scaling function. Note that momentum is cycled inversely
+            to learning rate; at the start of a cycle, momentum is 'max_momentum'
+            and learning rate is 'base_lr'
+            Default: 0.9
         last_epoch (int): The index of the last batch. This parameter is used when
             resuming a training job. Since `step()` should be invoked after each
             batch instead of after each epoch, this number represents the total
