Skip to content

Incorrect calculation for torch.nn.MSELoss #8710

@protossw512

Description

@protossw512

Issue description

torch.nn.MSELoss seems to get inaccurate calculation when size_average=True and reduce=True

Code example

For tensors of 0.1's and 0's, the desired output should be 0.01. (0.1 - 0)^2 = 0.01.
I can get correct result by code below:

In [10]: preds = torch.ones(5, 68, 64, 64) / 10
In [11]: res = torch.pow((preds - labels), 2)
In [12]: torch.mean(res)
Out[12]: 
tensor(1.00000e-02 *
       1.0000)

However, if I do something like below, I get inaccurate results:

In [16]: criterion = torch.nn.MSELoss()
In [17]: criterion(labels, preds)
Out[17]: 
tensor(1.00000e-03 *
       9.8371)

But when I turned off size_average and reduce and calculate mean manually, I get desired output:

In [21]: criterion = torch.nn.MSELoss(False, False)
In [22]: torch.sum(criterion(labels, preds)) / (5*68*64*64)
Out[22]: 
tensor(1.00000e-02 *
       1.0000)

Or:

In [23]: torch.mean(criterion(labels, preds))
Out[23]: 
tensor(1.00000e-02 *
       1.0000)

I kind of know that you cannot get exactly number when you using float datatype, but 0.01 and 0.00983 seems to be too off for me.
I am not quite sure if it is intended to be like that or I am not using it correctly.

System Info

PyTorch version: 0.4.0
Is debug build: No
CUDA used to build PyTorch: 9.0.176

OS: Ubuntu 16.04.3 LTS
GCC version: (Ubuntu 5.4.0-6ubuntu1~16.04.9) 5.4.0 20160609
CMake version: version 3.5.1

Python version: 2.7
Is CUDA available: Yes
CUDA runtime version: Could not collect
GPU models and configuration:
GPU 0: GeForce GTX 1080 Ti
GPU 1: GeForce GTX 1080 Ti
GPU 2: GeForce GTX 1080 Ti

Nvidia driver version: 387.34
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.7.1.2
/usr/lib/x86_64-linux-gnu/libcudnn_static_v7.a
/usr/local/cuda-9.0/targets/x86_64-linux/lib/libcudnn.so
/usr/local/cuda-9.0/targets/x86_64-linux/lib/libcudnn.so.7
/usr/local/cuda-9.0/targets/x86_64-linux/lib/libcudnn.so.7.1.4
/usr/local/cuda-9.0/targets/x86_64-linux/lib/libcudnn_static.a

Versions of relevant libraries:
[pip] numpy (1.14.5)
[pip] torch (0.4.0)
[pip] torchvision (0.2.1)
[conda] Could not collect

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions