Fix the ELU formula in the docs (#43764)

mariosasko · facebook-github-bot · commit cfba33bde3d6 · 2020-09-14T14:01:56.000-07:00
Summary: Fixes #43389. This PR replaces the old ELU formula from the docs that yields wrong results for negative alphas with the new one that fixes the issue and relies on the cases notation which makes the formula more straightforward. Pull Request resolved: #43764 Reviewed By: ailzhang Differential Revision: D23425532 Pulled By: albanD fbshipit-source-id: d0931996e5667897d926ba4fc7a8cc66e8a66837
diff --git a/torch/nn/modules/activation.py b/torch/nn/modules/activation.py
@@ -442,7 +442,10 @@ class ELU(Module):
     r"""Applies the element-wise function:
 
     .. math::
-        \text{ELU}(x) = \max(0,x) + \min(0, \alpha * (\exp(x) - 1))
+        \text{ELU}(x) = \begin{cases}
+        x, & \text{ if } x > 0\\
+        \alpha * (\exp(x) - 1), & \text{ if } x \leq 0
+        \end{cases}
 
     Args:
         alpha: the :math:`\alpha` value for the ELU formulation. Default: 1.0
