It's not clear what "same result" means here

For holomorphic functions, the gradient can be fully represented with complex numbers due to the Cauchy-Riemann equations

Suppose we have a function F: C -> C which we can decompose into functions u and v which compute the real and imaginary parts of the function:

for function F at (x, y)

In PyTorch, the VJP is mostly what we care about, as it is the computation performed when we do backwards mode automatic differentiation. Notice that d and i are negated in the formula above.

I might suggest absorbing "How are the JVP and VJP defined for :math:R^2 -> C and :math:C -> R^2 functions?" section into this one. The structure then is "Here is the general definition", and then "Here is a particular example"

I just felt it would be cleaner to have them in separate sections primarily because not everyone looking to get information about autograd for complex functions would be interested in cross domain functions

of the imaginary part of the function (v(x, y) above) are discarded (e.g., this is equivalent to dropping the imaginary part of the loss before performing a backwards). To get the gradient with respect to the imaginary components of the function, you must explicitly specify gradient=torch.tensor(1j))

yeah that's easier to read. updated!
not sure what you mean by:

To get the gradient with respect to the imaginary components of the function, you must explicitly specify gradient=torch.tensor(1j))

do you mean to say for any other desired behavior, specify the grad_out accordingly?

I'm still a little confused about the function of this section. Are we trying to explain why a conjugation occurs when we define the vjp for view_as_complex and view_as_rule? If so, it feels like it would be more direct if we directly talked about that particular case.

The goal here was to specify the formulas being used in view_as_real and view_as_complex yeah and then also generally give people an idea on how they can define their own backward for similar functions or others.

I think you have mixed use of i and 1j throughout, you should update to use a single one all over. And maybe at the beginning add a quick "(in this document, we use blah to define the imaginary number)".

You're still mixing j and 1j, also I can't see where you defined it?

nit the VJP you define above is not this function here.
The one you define above is directly the linear mapping (because you hard coded the input in space V directly into J).

oh yeah good catch updated!

How was this updated?

the "output of", I imagine

Not sure what this paragraph brings?
Why do you need the conjugation?
If the thesis contains all the details, maybe just replace this paragraph by a sentence pointing to the thesis for more details.

you need conjugation because the vectors from the dual space as explained below

I think this is missing one extra step:
The backward in pytorch does not compute a jacobian.

I think you want something that mentions that for R->R and C->R, backward() (with no argument) computes the full gradient of the function.
But for C->C functions, this is not case...

Yeah agreed but we are not saying the backward is computing the grad. We are just commenting on the Jacobian that's used to summarize that why the gradient for holomorphic functions can still be represented a s a complex number

Ok, then I guess the title of the section is what confused me here.

Why bold and not in math here?

Is \partial_0 defined?
Maybe \frac{\partial u(x, y)}{\partial x} ?

How was this updated?

I don't think it is clear what you mean by "the first vector in the output" here ?

hmm okay rewrote it. let me know if that looks better

I am still unsure of the value of this paragraph. It seems to justify the conjugation by saying that we need the output to be in C. But if we just want the output to be in C, we could do without the conjugation no?
Is the justification here that the mapping we use from C* to C is defined based on the standard dot product on C. And so it maps vectors by doing hermitian transpose.

Reading more about this, it feels like the "pure" C function definition does not hold.
And to be able to get quantities that match the gradients for holomorphic functions and provide sensible direction for use in gradient descent algorithms, modified definitions are needed.
Unfortunately, these definitions represent the full information about the derivatives of a general complex function using twice as many elements as a regular R -> R gradient we are used to.
And so we cannot expect to get these "extended gradients" in our framework where the .grad field is hard-coded to be the same size as the Tensor it represent the gradients of.

What do you mean by "pure" C function definition does not hold?

What I call "pure C function definition" here is what happens if you apply the derivation definition (as a limit) from real functions to complex functions: you get derivatives only for holomorphic functions.

:math:`F: V -> W`

?

Based on offline sync with @albanD, I removed this section to avoid providing fuzzy or possibly incorrect explanation

it's probably better to explicitly say that this is Python pseudocode. For example, you couldn't actually use * in a real PyTorch program as that would give you pointwise multiplication, not matrix product. And the bracket syntax means something else in Python, that is also not intended here either.

Or even better, make this real code using the real PyTorch operations. Then you don't have to explain the pseudocode syntax.

I updated this section to just use math notation instead. It's simpler to read and avoids the confusion

What I call "pure C function definition" here is what happens if you apply the derivation definition (as a limit) from real functions to complex functions: you get derivatives only for holomorphic functions.