reduce=False --> No error for invalid labels #5296
Description
Claim:
When I use an invalid label with F.cross_entropy with reduce=False, I don't receive an error message. I do receive an error message when reduce=True.
Code:
import torch
from torch.autograd import Variable
import torch.nn.functional as F
pred = [[0.99, 0.01]]
target = [[2]] # An invalid label. Valid labels: [0, 1]
preds = Variable(torch.FloatTensor([pred]))
targets = Variable(torch.LongTensor([target]))
num_classes = preds.size(-1)
preds_flat = preds.view(-1, num_classes)
targets_flat = targets.view(-1)
for _ in range(3):
loss = F.cross_entropy(preds_flat, targets_flat, reduce=False)
print("loss:", loss.data[0])Result:
loss: -2.103348379023373e-06
loss: -2.802596928649634e-45
loss: -2.0069783204235137e-06
No error message received.
Expected error message:
RuntimeError: Assertion `cur_target >= 0 && cur_target < n_classes' failed. at /opt/conda/conda-bld/pytorch_1512387374934/work/torch/lib/THNN/generic/ClassNLLCriterion.c:87Context:
I was using a PyTorch implementation of SSD loss, accidentally used an invalid label, and got extremely large losses and extremely small losses, and inconsistently so.
Environment:
- OS: Ubuntu 16.04 on Docker and host
- PyTorch version: 0.3.0.post4
- How you installed PyTorch: conda
- Python version: 3.6.3