Skip to content

fix: Allow full model compilation with collection inputs (input_signature) #1656

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
peri044 merged 2 commits into from
Mar 30, 2023
Merged

fix: Allow full model compilation with collection inputs (input_signature) #1656

peri044 merged 2 commits into from
Mar 30, 2023

Conversation

@gs-olive
Copy link
Contributor

@gs-olive gs-olive commented Feb 9, 2023 •
edited
Loading

Description

  • Allow users to specify full model compilation when using input_signature, which allows for complex collection-based inputs
  • Enable "psuedo-partitioning" phase for input collections as well as output collections
  • Update OutputIsCollection to include dictionary outputs, and add function InputIsCollection to detect collection-based inputs during graph compilation
  • Remove automatic fallback for collection pack/unpack operations when using input_signature argument
  • Add collections tests to ensure full compilation is respected for input and output collections

Fixes #1602
Fixes #1293
Addresses #1595

Type of change

  • Bug fix
  • New feature

Checklist:

  • [ x ] My code follows the style guidelines of this project (You can use the linters)
  • [ x ] I have performed a self-review of my own code
  • [ x ] I have commented my code, particularly in hard-to-understand areas and hacks
  • [ x ] I have made corresponding changes to the documentation
  • [ x ] I have added tests to verify my fix or my feature
  • [ x ] New and existing unit tests pass locally with my changes
  • [ x ] I have added the relevant labels to my PR in so that relevant reviewers are notified

@gs-olive gs-olive self-assigned this Feb 9, 2023
@github-actions github-actions bot added component: api [Python] Issues re: Python API component: api [C++] Issues re: C++ API component: conversion Issues re: Conversion stage component: core Issues re: The core compiler component: lowering Issues re: The lowering / preprocessing passes component: partitioning component: tests Issues re: Tests labels Feb 9, 2023
@github-actions github-actions bot requested a review from narendasan February 9, 2023 23:27
@Christina-Young-NVIDIA
Copy link

Christina-Young-NVIDIA commented Feb 28, 2023

Pending PR #1599 .

@gs-olive gs-olive force-pushed the input_signature_full_compilation branch from a219e05 to 71ac294 Compare March 14, 2023 20:07
@gs-olive gs-olive marked this pull request as ready for review March 14, 2023 20:08
@gs-olive gs-olive mentioned this pull request Mar 21, 2023
Closed
peri044
peri044 approved these changes Mar 27, 2023
View reviewed changes
Copy link
Collaborator

@peri044 peri044 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@peri044
Copy link
Collaborator

peri044 commented Mar 27, 2023 •
edited
Loading

@gs-olive Can you add collection example usage to our README https://github.com/pytorch/TensorRT#python ? And mention a line or two about how we can send collection inputs for an example model. You can add it there and also here https://pytorch.org/TensorRT/getting_started/getting_started_with_python_api.html (source)

gs-olive added 2 commits March 28, 2023 18:41
@gs-olive
fix: Allow full model compilation with collection Inputs
619b9a0 
- Allow users to specify full model compilation when using
`input_signature`, which allows for complex collection-based inputs
- Enable "psuedo-partitioning" phase for input collections as well as
output collections
- Update `OutputIsCollection` to include dictionary outputs, and add
function `InputIsCollection` to detect collection-based inputs during
graph compilation
- Remove automatic fallback for collection pack/unpack operations when
using `input_signature` argument
- Add collections tests to ensure full compilation is respected for
input and output collections
@gs-olive
chore: Add samples of input_signature usage to docs
985f6a2 
- Add documentation to `README` for usage of input signature
- Add documentation to "Getting Started" page for usage of input
signature
@gs-olive gs-olive force-pushed the input_signature_full_compilation branch from 71ac294 to 985f6a2 Compare March 29, 2023 03:44
@github-actions github-actions bot added the documentation Improvements or additions to documentation label Mar 29, 2023
@gs-olive gs-olive requested a review from peri044 March 29, 2023 03:45
peri044
peri044 approved these changes Mar 30, 2023
View reviewed changes
Copy link
Collaborator

@peri044 peri044 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@peri044 peri044 merged commit 5a45f6b into pytorch:main Mar 30, 2023
@gs-olive gs-olive deleted the input_signature_full_compilation branch March 30, 2023 22:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@peri044 peri044 peri044 approved these changes

@narendasan narendasan Awaiting requested review from narendasan

Assignees

@gs-olive gs-olive

Labels

cla signed component: api [C++] Issues re: C++ API component: api [Python] Issues re: Python API component: conversion Issues re: Conversion stage component: core Issues re: The core compiler component: lowering Issues re: The lowering / preprocessing passes component: partitioning component: tests Issues re: Tests documentation Improvements or additions to documentation

Projects

None yet

Milestone

No milestone

4 participants

@gs-olive @Christina-Young-NVIDIA @peri044 @facebook-github-bot