at::empty(0, ...

This actually trips up experienced PyTorch developers all the time. If we take a look at empty:

the size comes from here:

which is computed using

And if we run the following program:

std::cout << prod_intlist({}) << std::endl;  // prints 1
std::cout << prod_intlist(0) << std::endl;   // prints 0

it will print 1, 0. It is kinda confusing that an empty initializer list produces a tensor with one element and putting a zero in produces a tensor with no elements, but I try to remember it by recalling that zero specifies the size.

Oh wait. You're immediately populating the value into result, not resizing it (which I assumed since it's such a common pattern). You're doing everything right.

(Although what about empty(1, ... for readability?)

empty({}) produces 0-dim tensor with 1 element, empty({1}) produces 1-dim tensor with 1 element. vdot has to return 0-dim, so empty({}) is correct

Edited.

To fix the XLA issue and simplify the code just always do:

y_np = y.cpu().numpy()

There's no harm in calling .cpu() on a CPU tensor.

Would you also add tests for:

• dot and vdot getting arguments of different dtypes, incorrect number of dimensions, and mismatched devices (this test is a little weird to write in the device generic framework, but you can check that your current device is a cuda device and then create a tensor on that device + a cpu tensor to do it)
• after the out variant is added, including mismatched out dtype
• after the method variant is added, you can add a test here, too:

https://github.com/pytorch/pytorch/pull/43004/files#r470634289

What about something like:

"Computes the dot product (inner product) of input and the complex conjugate of other."

And not having to explain in a note how vdot is distinct from dot?

I think we should include the note in the description because it brings more clarity to the description.

"Computes the dot product (inner product) of two tensors. The vdot(a, b) function handles complex numbers differently than dot(a, b). If the first argument is complex the complex conjugate of the first argument is used for the calculation of the dot product."

May still need to inline this note per @anjali411's feedback. I'll let her have final say over doc string.

@kshitij12345 nit -- can you change the (new line) formatting to:
"Computes the dot product (inner product) of two tensors. The vdot(a, b) function
handles complex numbers differently than dot(a, b). If the first argument is complex
the complex conjugate of the first argument is used for the calculation of the dot product."

Done. Thanks!

>>> np.vdot(np.array([2]), np.array([2+3j]))
(4+6j)
>>> np.vdot(np.array([2+3j]), np.array([2]))
(4-6j)

maybe it's worth adding a comment that we only call dot_cuda when self is not complex because we want the above mentioned behavior.

nvm I guess we throw an error for input tensors of different dtype:

>>> torch.dot(torch.tensor([1j]), torch.tensor([2]))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: dot : expected both vectors to have same dtype, but found ComplexFloat and Long

I think we should add type promotion for dot and vdot to better align with numpy.

cc. @mruberry

Type promotion would be nice but I think it's OK if it's not in this first PR. Type promotion is a little tricky to implement today when not using TensorIterator.

Yeah. I was thinking about it. But since the plan is to use existing dot for real types, wasn't sure as to how to go about having type promotion.

Also other place where it diverges from numpy is that numpy operator supports broadcasting while dot doesn't. We can add broadcasting logic before passing the inputs to dot. What do you feel about it?

I would file follow-up issues for broadcasting + type promotion. We may even want to add architecture to support doing these things easily outside of TensorIterator.

numpy vdot does not support broadcasting, and we don't try to follow numpy dot behavior:

In [5]: np.vdot([2,3], [3])                                                                                                                                                                                                         
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-5-6192cfa76e6f> in <module>
----> 1 np.vdot([2,3], [3])

<__array_function__ internals> in vdot(*args, **kwargs)

ValueError: cannot reshape array of size 1 into shape (2,)

@mruberry yeah that sounds good. out of curiosity -- what's tricky about implementing type promotion when not using TensorIterator?

@kshitij12345

if (!self.is_complex()) {
    return dot_cuda(self, other);
}

will give us the desired behavior for (real, complex) dot product when the type promotion is enabled, so we should still add a note documenting that. This is so that when the type promotion is enabled in future, we have the behavior already documented.

Type promotion challenges:

• sometimes have to be careful to preserve your inputs (luckily not the case here)
• compute the result type
• cast inputs to the result type
• validate safe casting to out
• copy to out (if necessary)
• write a custom test for your op's type promotion behavior

It's not the end of the world. In this case we would want to change the behavior of dot, too, so it seems separable.