-
Notifications
You must be signed in to change notification settings - Fork 26.3k
Description
Issue description
torch.nn.MSELoss seems to get inaccurate calculation when size_average=True and reduce=True
Code example
For tensors of 0.1's and 0's, the desired output should be 0.01. (0.1 - 0)^2 = 0.01.
I can get correct result by code below:
In [10]: preds = torch.ones(5, 68, 64, 64) / 10
In [11]: res = torch.pow((preds - labels), 2)
In [12]: torch.mean(res)
Out[12]:
tensor(1.00000e-02 *
1.0000)
However, if I do something like below, I get inaccurate results:
In [16]: criterion = torch.nn.MSELoss()
In [17]: criterion(labels, preds)
Out[17]:
tensor(1.00000e-03 *
9.8371)
But when I turned off size_average and reduce and calculate mean manually, I get desired output:
In [21]: criterion = torch.nn.MSELoss(False, False)
In [22]: torch.sum(criterion(labels, preds)) / (5*68*64*64)
Out[22]:
tensor(1.00000e-02 *
1.0000)
Or:
In [23]: torch.mean(criterion(labels, preds))
Out[23]:
tensor(1.00000e-02 *
1.0000)
I kind of know that you cannot get exactly number when you using float datatype, but 0.01 and 0.00983 seems to be too off for me.
I am not quite sure if it is intended to be like that or I am not using it correctly.
System Info
PyTorch version: 0.4.0
Is debug build: No
CUDA used to build PyTorch: 9.0.176
OS: Ubuntu 16.04.3 LTS
GCC version: (Ubuntu 5.4.0-6ubuntu1~16.04.9) 5.4.0 20160609
CMake version: version 3.5.1
Python version: 2.7
Is CUDA available: Yes
CUDA runtime version: Could not collect
GPU models and configuration:
GPU 0: GeForce GTX 1080 Ti
GPU 1: GeForce GTX 1080 Ti
GPU 2: GeForce GTX 1080 Ti
Nvidia driver version: 387.34
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.7.1.2
/usr/lib/x86_64-linux-gnu/libcudnn_static_v7.a
/usr/local/cuda-9.0/targets/x86_64-linux/lib/libcudnn.so
/usr/local/cuda-9.0/targets/x86_64-linux/lib/libcudnn.so.7
/usr/local/cuda-9.0/targets/x86_64-linux/lib/libcudnn.so.7.1.4
/usr/local/cuda-9.0/targets/x86_64-linux/lib/libcudnn_static.a
Versions of relevant libraries:
[pip] numpy (1.14.5)
[pip] torch (0.4.0)
[pip] torchvision (0.2.1)
[conda] Could not collect