[ONNX] Expose the testing module#162495
Conversation
Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
pytorch-bot
bot
added
the
release notes: onnx
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/162495
Note: Links to docs will display an error until the docs builds have been completed. ✅ You can merge normally! (1 Unrelated Failure)As of commit 280a075 with merge base c0142f5 ( FLAKY - The following job failed but was likely due to flakiness present on trunk:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
justinchuby
added
module: onnx
|
What is the motivation? |
|
I think it is becoming increasingly useful for users to validate their models when we are making dynamo=True as the default. As an example, we are using it in prototypes: https://github.com/jambayk/torch-onnx-models/blob/main/test/rms_norm_test.py |
Is it like a checker? If so, we can just embed it into the exporter? What do we expect our users do when they find their onnx_program failing to assertion? |
|
In the example above, we can use this to compare the exported model with a reference ep: jambayk/torch-onnx-models#1
I would prefer separating it, because this way it is more flexible (we can control and manipulate the onnx_program, for example by updating the onnx ir model first before asserting). |
But this sounds like we don't guarantee the validity of the exported model from our API. |
| onnx_export | ||
| onnx_ops | ||
| onnx_verification | ||
| onnx_testing |
There was a problem hiding this comment.
Is there a place that we can notify the users where they might want to use this to validate their current onnx_program? right now, I am thinking probably it's for advanced users who do customized model surgery (although I feel like they won't do it in onnx program, they probably directly use onnx ir or model proto instead.)?
There was a problem hiding this comment.
I am not entirely sure what the alternative would be?
Maybe we can frame it as "if you want to use custom ops etc. you can use this to verify it"? |
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
* Created a new module `torch/onnx/testing.py` that exposes the `assert_onnx_program` function for testing exported ONNX models. * Updated the ONNX documentation (`docs/source/onnx.md`) to include `onnx_testing` in the list of relevant modules. Pull Request resolved: pytorch#162495 Approved by: https://github.com/titaiwangms, https://github.com/xadupre
* Created a new module `torch/onnx/testing.py` that exposes the `assert_onnx_program` function for testing exported ONNX models. * Updated the ONNX documentation (`docs/source/onnx.md`) to include `onnx_testing` in the list of relevant modules. Pull Request resolved: pytorch#162495 Approved by: https://github.com/titaiwangms, https://github.com/xadupre
* Created a new module `torch/onnx/testing.py` that exposes the `assert_onnx_program` function for testing exported ONNX models. * Updated the ONNX documentation (`docs/source/onnx.md`) to include `onnx_testing` in the list of relevant modules. Pull Request resolved: pytorch#162495 Approved by: https://github.com/titaiwangms, https://github.com/xadupre
* Created a new module `torch/onnx/testing.py` that exposes the `assert_onnx_program` function for testing exported ONNX models. * Updated the ONNX documentation (`docs/source/onnx.md`) to include `onnx_testing` in the list of relevant modules. Pull Request resolved: pytorch#162495 Approved by: https://github.com/titaiwangms, https://github.com/xadupre
5 participants
torch/onnx/testing.pythat exposes theassert_onnx_programfunction for testing exported ONNX models.docs/source/onnx.md) to includeonnx_testingin the list of relevant modules.cc @titaiwangms