🐛 Bug

As the original paper(https://arxiv.org/pdf/1711.05101.pdf, green boxes) shows

the formula of applying weight decay to Adam should be
\theta_t = (1 - \lambda) * \theta_{t - 1} or
\theta_t = (1 - {schedule multiplier} * \lambda) * \theta_{t - 1}.

But, AdamW implementation in master branch (https://github.com/pytorch/pytorch/blob/master/torch/optim/adamw.py#L73) applies weight decay with learning rate.

\theta_t = (1 - \lambda * {learning rate}) * \theta_{t - 1}

Expected behavior

I think this line(

cc @vincentqb