Issue description

I noticed similar issue has been reported in #7999 and fixed in pytorch 0.4.1 #8033

However, when using Module ReLU, the problem still exists.

Code example

import torch
import torch.nn as nn
a = torch.Tensor(1).fill_(float('nan'))
relu = nn.ReLU()
print(relu(a))  # => 0

System Info

PyTorch version: 0.4.1
Is debug build: No
CUDA used to build PyTorch: 9.0.176

OS: Ubuntu 16.04.5 LTS
GCC version: (Ubuntu 5.4.0-6ubuntu1~16.04.10) 5.4.0 20160609
CMake version: version 3.5.1

Python version: 3.6
Is CUDA available: Yes
CUDA runtime version: 8.0.61
GPU models and configuration:
GPU 0: GeForce GTX 1080 Ti
GPU 1: GeForce GTX 1080 Ti
GPU 2: GeForce GTX 1080 Ti

Nvidia driver version: 396.37
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.5.1.10
/usr/lib/x86_64-linux-gnu/libcudnn.so.6.0.21
/usr/lib/x86_64-linux-gnu/libcudnn_static.a
/usr/local/MATLAB/R2016b/bin/glnxa64/libcudnn.so.4.0.7
/usr/local/cuda-8.0/lib64/libcudnn.so.7.0.5
/usr/local/cuda-8.0/lib64/libcudnn_static.a
/usr/local/cuda-9.0/lib64/libcudnn.so.7.1.4
/usr/local/cuda-9.0/lib64/libcudnn_static.a
/usr/local/cuda-9.2/lib64/libcudnn.so.7.1.4
/usr/local/cuda-9.2/lib64/libcudnn_static.a

Versions of relevant libraries:
[pip] numpy (1.14.3)
[pip] numpydoc (0.8.0)
[pip] torch (0.4.1)
[pip] torchvision (0.2.1)
[conda] pytorch 0.4.1 py36_cuda9.0.176_cudnn7.1.2_1 pytorch
[conda] torchvision 0.2.1 py36_1 pytorch