[AOTInductor] Simplified AOTInductor interface and model class #110411
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/110411
Note: Links to docs will display an error until the docs builds have been completed. ✅ You can merge normally! (1 Unrelated Failure)As of commit c80ee86 with merge base 31d6358 ( UNSTABLE - The following job failed but was likely due to flakiness present on trunk and has been marked as unstable:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
This pull request was exported from Phabricator. Differential Revision: D49835430 |
chenyang78
force-pushed
the
export-D49835430
branch
from
2f875e3 to
a80203c
Compare
|
This pull request was exported from Phabricator. Differential Revision: D49835430 |
chenyang78
force-pushed
the
export-D49835430
branch
from
a80203c to
f540a33
Compare
|
This pull request was exported from Phabricator. Differential Revision: D49835430 |
torch/_inductor/codegen/wrapper.py
Outdated
There was a problem hiding this comment.
Since you are working on simplification, let me ask this question: do we really need to maintain the name and dtype for inputs and outputs? If we do, then should we make them as members of AOTInductorModelContainer, as they are the same for the model instances?
There was a problem hiding this comment.
I think we'd better keep input/output names, because the order of inputs and outputs kept in the .so might not be the same as the order of inputs and outputs prepared by the user. Having an API that returns the name at an index would allow the user to create correct mappings. We don't really need dtype. Let me remove it.
Regarding where to keep input/output name info, I think attaching them to the model class may be better, because the user may not use the model container via the new APIs AOTInductorModelCreate and AOTInductorModelRun.
…ch#110411) Summary: This PR removed several APIs from the AOTInductor interface, which are not used by the client. It also simplified AOTInductor's model class by removing the dim info for input/output tensors. We included dim info before to return max output shapes, which was used by the client to allocate memory for output tensors. Now, we allocate output tensor memory from the .so so that we don't need to maintain such information any more. The deletion of dim info from the model class also simplified the codegen quite a bit. Test Plan: ci Reviewed By: khabinov Differential Revision: D49835430
chenyang78
force-pushed
the
export-D49835430
branch
from
f540a33 to
1bce442
Compare
|
This pull request was exported from Phabricator. Differential Revision: D49835430 |
chenyang78
force-pushed
the
export-D49835430
branch
from
1bce442 to
c80ee86
Compare
|
This pull request was exported from Phabricator. Differential Revision: D49835430 |
|
@pytorchbot merge (Initiating merge automatically since Phabricator Diff has merged) |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
6 participants
Summary:
This PR removed several APIs from the AOTInductor interface,
which are not used by the client.
It also simplified AOTInductor's model class by removing
the dim info for input/output tensors. We included dim info
before to return max output shapes, which was used by the client
to allocate memory for output tensors. Now, we allocate output
tensor memory from the .so so that we don't need to maintain
such information any more. The deletion of dim info from
the model class also simplified the codegen quite a bit.
Test Plan: ci
Reviewed By: khabinov
Differential Revision: D49835430
cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @kadeng @muchulee8 @aakhundov @ColinPeppler