Update on "[optim][adamw] default to foreach when CUDA + differentiable=False"

janeyx99 · janeyx99 · commit b78c9b27d443 · 2023-01-17T20:03:46.000Z
cc ezyang gchanan

[ghstack-poisoned]
diff --git a/torch/optim/adamw.py b/torch/optim/adamw.py
@@ -64,7 +64,7 @@ class AdamW(Optimizer):
             minimizing (default: False)
         foreach (bool, optional): whether foreach implementation of optimizer is used.
             If unspecified by the user (so foreach is None), we will try to use foreach
-            over the for-loop implementation on CUDA, since it is usually significantly 
+            over the for-loop implementation on CUDA, since it is usually significantly
             more performant. (default: None)
         capturable (bool, optional): whether this instance is safe to capture in a CUDA graph.
             Passing True can impair ungraphed performance, so if you don't intend to
