[Inductor] support mixed dtype in the native_layer_norm_backward meta function#159830
[Inductor] support mixed dtype in the native_layer_norm_backward meta function#159830markc-614 wants to merge 3 commits intopytorch:mainfrom
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/159830
Note: Links to docs will display an error until the docs builds have been completed. ⏳ No Failures, 1 PendingAs of commit b257578 with merge base df4ebdd ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
markc-614
force-pushed
the
mashaobin/inductor/fix-ln-decompostion
branch
2 times, most recently
from
b10aba6 to
8da3c4f
Compare
markc-614
force-pushed
the
mashaobin/inductor/fix-ln-decompostion
branch
from
8da3c4f to
51fa8ee
Compare
|
@pytorchbot label "topic: not user facing" |
markc-614
changed the title
markc-614
changed the title
|
@masnesral please help to review this, thank you |
jerryzh168
added
the
triaged
|
@eellison @ysiraichi please help to review this, thank you |
| _maybe_cast(d_input, input.dtype), | ||
| _maybe_cast(d_weight, input.dtype), | ||
| _maybe_cast(d_bias, input.dtype), | ||
| _maybe_cast(d_weight, weight.dtype if weight is not None else None), |
There was a problem hiding this comment.
Yes it is correct.
It's because the weight and bias are optional. But when they're not there, both these gradients will be None anyways so no cast is needed.
| _maybe_cast(d_input, input.dtype), | ||
| _maybe_cast(d_weight, input.dtype), | ||
| _maybe_cast(d_bias, input.dtype), | ||
| _maybe_cast(d_weight, weight.dtype if weight is not None else None), |
There was a problem hiding this comment.
Yes it is correct.
It's because the weight and bias are optional. But when they're not there, both these gradients will be None anyways so no cast is needed.
|
@markc-614 can you rebase on top of the latest main branch so we get CI signal please? |
|
Also @markc-614 I think it would be great to test the privateuse1 integration in compile a bit more in core to make it more stable. |
markc-614
force-pushed
the
mashaobin/inductor/fix-ln-decompostion
branch
from
6fe8bc7 to
72d0714
Compare
markc-614
requested review from
XuehaiPan,
eqy,
janeyx99 and
mikaylagawarecki
as code owners
markc-614
requested review from
digantdesai,
fmassa,
jbschlosser,
jianyuh and
soulitzer
as code owners
markc-614
force-pushed
the
mashaobin/inductor/fix-ln-decompostion
branch
from
1d01742 to
0ea1463
Compare
@albanD done |
@albanD do we need to add test cases to torch_openreg? |
@markc-614 These changes aren't specific to PrivateUse1, they apply to the general logic for all accelerators in PyTorch, right? Also, Any other suggestions @albanD :D |
|
@fffrog Yes, exactly. |
Thank you, it would be even better if you could modify the title of this PR and remove the "for privateuse1" suffix. |
markc-614
changed the title
markc-614
changed the title
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
… function (pytorch#159830) Fixes pytorch#159829 Pull Request resolved: pytorch#159830 Approved by: https://github.com/albanD
… function (pytorch#159830) Fixes pytorch#159829 Pull Request resolved: pytorch#159830 Approved by: https://github.com/albanD
… function (pytorch#159830) Fixes pytorch#159829 Pull Request resolved: pytorch#159830 Approved by: https://github.com/albanD
7 participants
Fixes #159829