System Information

OpenCV version: 5.0.0-pre
Operating System / Platform: Ubuntu 22.04
Compiler & compiler version: GCC 11.4.0
Python version: 3.10.12

Detailed description

cv2.dnn.readNetFromONNX(..., engine=cv2.dnn.ENGINE_NEW) crashes with an assertion failure when running any Swin Transformer based model (SwinIR, BiRefNet, etc.).

Root cause - ONNX Slice with empty range:

The ONNX Slice operator allows start > end for a positive step, producing an empty output tensor (size 0 on that axis). This is valid per the ONNX spec.

Why Swin Transformer triggers this:

Swin implements shifted-window attention via a cyclic shift on the feature map,then slices it into window partitions. After the shift, boundary windows produce zero-size regions on certain axes - exactly these empty-range Slice ops. Every Swin-based model hits this path on every forward pass.

Where it crashes:
slice2_layer.cpp::getOutShape() computes outsz < 0 for this case and hits:

  OpenCV(5.0.0-pre) Error: Assertion failed (outsz >= 0) in getOutShape,
  file modules/dnn/src/layers/slice2_layer.cpp, line 124

ENGINE_ORT handles the same model correctly. Only ENGINE_NEW is affected.

Steps to reproduce

Steps to reproduce:

import cv2
import numpy as np

# Model: SwinIR x4 GAN
# Download: https://huggingface.co/rocca/swin-ir-onnx/resolve/main/003_realSR_BSRGAN_DFO_s64w8_SwinIR-M_x4_GAN.onnx

MODEL = "003_realSR_BSRGAN_DFO_s64w8_SwinIR-M_x4_GAN.onnx"

net = cv2.dnn.readNetFromONNX(MODEL, engine=cv2.dnn.ENGINE_NEW)
blob = np.random.rand(1, 3, 64, 64).astype(np.float32)
net.setInput(blob)
out = net.forward()  # crashes here

OpenCV(5.0.0-pre) Error: Assertion failed (outsz >= 0) in getOutShape,
file modules/dnn/src/layers/slice2_layer.cpp, line 124

Root cause: getOutShape() does not handle the valid empty-slice case. allEnds[axis] is also set before the outsz clamp - if the assert is bypassed without fixing this, forward() uses a garbage end value and causes SEGFAULT.

Fix in slice2_layer.cpp:

// replace CV_Assert(outsz >= 0) with:
if (outsz < 0) {
    outsz = 0; 
    end = start;  // empty range - forward loop never executes
}
if (allEnds)
    allEnds[axis] = end;   // must be AFTER clamp
outShape[axis] = outsz;

Affects all Swin Transformer backbones: SwinIR, SwinV2, BiRefNet etc.

Test model: https://huggingface.co/rocca/swin-ir-onnx

Issue submission checklist

• I report the issue, it's not a question
• I checked the problem with documentation, FAQ, open issues, forum.opencv.org, Stack Overflow, etc and have not found any solution
• I updated to the latest OpenCV version and the issue is still there
• There is reproducer code and related data files (videos, images, onnx, etc)