Gaussian Error Linear Unit (GELU) has become really mainstream with models such as BERT using it. But nn.modules.activation doesn't have a GELU implementation. Feature request to expand the arsenal of activation functions available right off the bat in Pytorch :)

I could work on it, to get it merged upstream if assigned.