As reported here https://discuss.pytorch.org/t/bug-of-nn-embedding-when-sparse-true-and-padding-idx-is-set/9382 with repro.

When sparse=True, padding_idx should probably error out (as the easiest solution).
Alternatively, the gradients for padding_idx indices should be zeroed out explicitly in the sparse=True backward codepath.