here might be a good places to talk about holomorphic functions. Maybe the "upshot" section of https://jax.readthedocs.io/en/latest/notebooks/autodiff_cookbook.html#Complex-numbers-and-differentiation

should we just add it to the autograd doc and link that doc here?

I don't see any reason why this wouldn't work

If distributed doesn't work, it doesn't for any good reason, and we should fix it (shouldn't be hard)

Synced with @mrshenli and we will have to add mappings between complex and float dtypes for data transfer (because for eg. nccl doesn't support complex dtypes) and add some tests to ensure that it works correctly. These changes will be included as part of 1.7 because it's too last minute to add it in 1.6.

Vec256 is a bit internal and I probably wouldn't mention it in a user doc like this

Also ComplexFloat and ComplexDouble haven't been defined yet, but they also sound internal.

You can just say something like "Operations involving complex numbers in PyTorch are optimized to use vectorized assembly instructions and specialized kernels (e.g. LAPACK, CuBlas)."

I'm not really sure why you want to compare users writing their own functions -- why would they do that?

nit: this sentence describes the current state, but it's going to be out of date soon after you write it (i.e., we'll have complex support and torchaudio won't have hacks. You can just say something like "Complex Tensors are provided..."

nit: it's a little wordier, but "Tensors of complex dtype" is more descriptive/accurate.

nit: I believe "workaround" is two words when used as a verb.

Also ComplexFloat and ComplexDouble haven't been defined yet, but they also sound internal.

You can just say something like "Operations involving complex numbers in PyTorch are optimized to use vectorized assembly instructions and specialized kernels (e.g. LAPACK, CuBlas)."

I'm not really sure why you want to compare users writing their own functions -- why would they do that?

are there any factory functions that dont support complex tensors?

torch.arange, torch.range, torch.linspace, torch.logspace, torch.quantize_per_tensor, torch.quantize_per_channel

do you think it's better to just have a list of the factory functions not supported instead?

I think the core of this should go in the autograd section and you should link it from here.

so should we move everything mentioned here (including the example) to autograd mechanics as well?
cc. @albanD

I think it is already in the autograd doc as well.
The list explaining what you get for different type of functions is good to have here I think.
Not sure what the code sample brings though.

Note that you use j in the autograd part of the doc. Maybe you want to make that consistent.

I'm not sure that just saying "view functions" makes it clear to the user that these are views. Maybe you want to insist on it a bit more as it is fairly important.

...:func:torch.view_as_complex and - :func:torch.view_as_real functions which involve zero copy. does that look better?

Or add a new setence: Note that these functions don't perform any copy and return a view of the input Tensor.

We might want to double check the conjugate here!
Also if this is true, we might want to update our optimizers to do that when the parameters are complex

yeah that's correct! the gradient definition that we are using does specify that right?

If it's already in the gradient definition, no need to do it again here.

bullet list this?

This seems like it's underselling the feature. The goal is to not just replace existing workarounds, right?

I would consider introducing these existing workarounds in its own section later. Maybe a section like: "adapting existing code"?

This is really about acquiring just the real or imaginary part of the tensor, but the title seems like it's about the attributes themselves

sorry I am not sure what you mean. can you provide an alternate suggestion?

looks like a copy paste bug tensortensor

fixed! (also updated the example to make it more readable)

nit: there still feels like there is some tense confusion here. Part of it is that there are plenty of libraries, TODAY, that represent complex as float tensors. So it should be present tense. Consider something like, "Tensors of complex dtypes provide a more natural user experience for working with complex numbers, compared with maintaining float tensors with shape :math:(..., 2) where the last dimension contains the real and imaginary values."

nit: You probably don't actually need this sentence

removed!

nit: backtick the code fragments

not sure if it's meaningful to say "evaluated as if it were holomorphic"; what does that even mean?

ha I meant to say the gradient evaluation for non-holomorphic functions are done assuming the input function is holomorphic. I'll re-phrase to make it clear

Can you consolidate all the historical information in this section?

Yeah however I think some historical context in the introduction sets a better stage to truly justify "more natural user experience" and "faster and more memory efficient than operations on float tensors mimicking them".

has anyone made the argument that mimicking complex numbers is a better user experience? faster? I haven't personally come across those arguments, my impression is that everyone recognizes the previous behavior as a hack.

torch.sum(a.conj() * b)) ?

synced offline: remove the code block from autograd section until we have a clear definition of functions that can be used with view_as_real and view_as_complex and for which we can ensure correct gradient computation.