Repository navigation
Conversation
github-actions
Bot
added
documentation
bowang007
changed the title
Generate bounded candidate batches and backend tuning variants through the existing evaluator. Search decisions follow measured results, keep progress toward baseline targets, and reserve final validation for alternative finalists. Checkpoint unfinished searches, remeasure restored candidates, and expose optional compilation diagnostics. Extend the public API, replay CLI, packaged instructions, and runnable example while preserving single-candidate providers. Validation: 481 region regression tests including real GPU integration; 72 focused controller tests and 71 policy tests after follow-up changes; all targeted pre-commit hooks passed.
There was a problem hiding this comment.
can we add cuteDSL if possible?
There was a problem hiding this comment.
@narendasan Our automatic kernel generation system only support cuda tile, triton and cuda python now. That's the reason why such is not included in this PR. But we could add it as another option anytime soon
There was a problem hiding this comment.
Can stuff like this piggy back on infra we built for kernels? Like we can walk through the whole custom op registration process the same way. Just break before building the plugin and test in eager?
There was a problem hiding this comment.
Do you know if there is a good way to invoke this as part of the compilation process instead of a stand alone app?
| class Model(torch.nn.Module): | ||
| def __init__(self, generator): | ||
| super().__init__() | ||
| self.generator = generator |
There was a problem hiding this comment.
this is quite invasive, you need to change the constructor of the model to use this feature
There was a problem hiding this comment.
Ideally for training mode / eager execution this code is non invasive, transparently passes through and only gets invoked under torch_tensorrt.compile if possible
|
|
||
| def forward(self, x, bias, scale): | ||
| x = x + 1.0 | ||
| with fuse_in_plugin(generator=self.generator, name="bias_relu_scale"): |
There was a problem hiding this comment.
I feel like it would be great to be able to pass a dataloader with real data to validate against
Description
This feature introduces agentic kernel generation for the region that user specifies.
Type of change
Please delete options that are not relevant and/or add your own.
Checklist: