Skip to content

Specifying pos_weight in F.binary_cross_entropy_with_logits may lead to NaN #13773

@jatentaki

Description

@jatentaki

🐛 Bug

The weight of positive examples should default to 1.. However, there are inputs for which the result is finite without specifying pos_weight and nan with pos_weight=torch.tensor([1.]).

To Reproduce

Steps to reproduce the behavior:

import torch, math 
import torch.nn.functional as F

logits = torch.tensor([0., -120.])
target = torch.tensor([0., 1.])

# passes
assert math.isfinite(
    F.binary_cross_entropy_with_logits(logits, target).item()
)

pos_weight = torch.tensor([1., 1.])

# fails
assert math.isfinite(
    F.binary_cross_entropy_with_logits(logits, target, pos_weight=pos_weight).item()
)

Note that for moderate inputs (-10 instead of -120) this works as expected

Expected behavior

Both cases should lead to the same result, because the latter is just explicitly passing the default value.

Environment

PyTorch version: 1.0.0.dev20181108
Is debug build: No
CUDA used to build PyTorch: None

OS: Ubuntu 18.04.1 LTS
GCC version: (Ubuntu 7.3.0-27ubuntu1~18.04) 7.3.0
CMake version: version 3.10.2

Python version: 3.6
Is CUDA available: No
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA

Versions of relevant libraries:
[pip] Could not collect
[conda] pytorch-nightly-cpu 1.0.0.dev20181108 py3.6_cpu_0 pytorch
[conda] torch-dimcheck 0.0.1
[conda] torchvision 0.2.1

Additional context

I find the same issue when running on GPU

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions