this is bringing us back to mkldnn not being called with dilation. Should this PR be landed on top of the one that allows convolutions with dilation to use mkldnn?

enable it at #40483

what is convolution behavior wrt mkl or dense inputs? Similar to cases you posted above for linear and bn.

After #40610 torch.convolution won't be calling mkldnn in a fair number of cases, so you still need to call mkldnn_convolution here.

@ngimel, I don't think need to call mkldnn_convolution here, if input and weight are MKLDNN tensor, it will call mkldnn_convolution directly, see

@ngimel

Is it enough to check only for mkldnn inputs? Currently the following (which, as far as I understand, sends regular tensor to mkldnn convolution) works:

x = torch.randn(N, C, 56, 56, dtype=torch.float)#.to_mkldnn()
conv2d = torch.nn.Conv2d(in_channels=C,
                            out_channels=M,
                            kernel_size=3,
                            padding=1)
mkldnn_conv2d = mkldnn_utils.to_mkldnn(copy.deepcopy(conv2d))
with torch.backends.mkldnn.flags(enabled=True):
    y_mkldnn = mkldnn_conv2d(x)#.to_dense()

Same question for all other functions - you added checks only for mkldnn inputs.

@ngimel , for convolution, this input check can be removed, see

1. torch.backends.mkldnn.flags(enabled=True), input and weight are MKLNDN tensor, it is ok.
2. torch.backends.mkldnn.flags(enabled=True), input is dense tensor, and weight are MKLNDN tensor, will call MKLDNN path, there will make check to make sure input and weight have same type: see
   

   pytorch/aten/src/ATen/native/Convolution.cpp

   Lines 746 to 748 in 6468bc4

   	TORCH_CHECK(input.options().type_equal(weight.options()),
   	"Input type (", input.toString(), ") and weight type (", weight.toString(),
   	") should be the same");

   , so this will report a error: Input type (torch.FloatTensor) and weight type (Mkldnntorch.FloatTensor) should be the same.
3. torch.backends.mkldnn.flags(enabled=False), input and weight are MKLNDN tensor, will call this path
   

   pytorch/aten/src/ATen/native/Convolution.cpp

   Lines 801 to 802 in 6468bc4

   	output = at::_convolution_nogroup(
   	input.contiguous(), weight, bias, params.stride, params.padding, params.dilation, params.transposed, params.output_padding);

   , which not support MKLDNN inputs. So this will report a error: "opaque tensors do not have is_contiguous".
4. torch.backends.mkldnn.flags(enabled=False), input is dense tensor, weight are MKLNDN tensor, also call path
   

   pytorch/aten/src/ATen/native/Convolution.cpp

   Lines 801 to 802 in 6468bc4

   	output = at::_convolution_nogroup(
   	input.contiguous(), weight, bias, params.stride, params.padding, params.dilation, params.transposed, params.output_padding);

   
   this will report a error: Cannot access data pointer of Tensor that doesn't have storage.

For linear:

x = torch.randn(56, 56, dtype=torch.float)
linear = torch.nn.Linear(56, 56 bias=True)
mkldnn_linear = mkldnn_utils.to_mkldnn(copy.deepcopy(linear))
with torch.backends.mkldnn.flags(enabled=True):
    y_mkldnn = mkldnn_linear(x)

1. torch.backends.mkldnn.flags(enabled=True), input and weight are MKLDNN tensor, it can works.
2. torch.backends.mkldnn.flags(enabled=True), input is dense tensor, weight are MKLDNN tensor, it can works, because we will first transfer input to MLDNN tensor in

   pytorch/torch/utils/mkldnn.py

   Line 30 in bdc0019

   x_mkldnn = x if x.is_mkldnn else x.to_mkldnn()

   .
3. torch.backends.mkldnn.flags(enabled=False), input and weight are MKLDNN tensor, it can report error: MKLDNN linear module can't run when mkldnn disabled
4. torch.backends.mkldnn.flags(enabled=False), input is dense tensor, weight are MKLDNN tensor, it can also report error: MKLDNN linear module can't run when mkldnn disabled.

For BatchNorm,

x = torch.randn(1, 3, 56, 56, dtype=torch.float)
bn = torch.nn.BatchNorm2d(3).float().train(False)
mkldnn_bn = mkldnn_utils.to_mkldnn(copy.deepcopy(bn))
with torch.backends.mkldnn.flags(enabled=True):
    y_mkldnn = bn(x)

1. torch.backends.mkldnn.flags(enabled=True), input and weight are MKLDNN tensor, it can works.
2. torch.backends.mkldnn.flags(enabled=True), input is dense tensor, weight are MKLDNN tensor, it report a error:mkldnn_tensor.is_mkldnn() INTERNAL ASSERT FAILED at "../aten/src/ATen/native/mkldnn/MKLDNNCommon.cpp":56, please report a bug to PyTorch. mkldnn_to_dense expects MKL-DNN tensor input.
3. torch.backends.mkldnn.flags(enabled=False), input and weight are MKLDNN tensor, it will report a error:MKLDNN batch_norm module can't run when mkldnn is disabled.
4. torch.backends.mkldnn.flags(enabled=False), input is dense tensor, weight are MKLDNN tensor, it will report a error:mkldnn_tensor.is_mkldnn() INTERNAL ASSERT FAILED at "../aten/src/ATen/native/mkldnn/MKLDNNCommon.cpp":56, please report a bug to PyTorch. mkldnn_to_dense expects MKL-DNN tensor input.

There should not be INTERNAL_ASSERTs here, right? Is it a real pytorch bug that you guys have to fix, or should the error message be different. Couple additional questions

1. why is mkldnn_to_dense attempted here, is it when converting output of the operation to dense tensor?
2. Why is behavior here different than in linear, where case 2 works, and case 4 errors out as expected, with expected error message?

For the first one, it call in

For the second: the case 2 can be works for linear is that we always convert dense tensor to MKLDNN tensor for input, see