Skip to content

Extend _save_for_mobile and _load_for_mobile to work with flatbuffer format #74594

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
pavithranrao wants to merge 7 commits into from
Closed

Extend _save_for_mobile and _load_for_mobile to work with flatbuffer format #74594

pavithranrao wants to merge 7 commits into from

Conversation

@pavithranrao
Copy link
Contributor

@pavithranrao pavithranrao commented Mar 23, 2022 •
edited
Loading

Extend _save_for_mobile and _load_for_mobile to work with flatbuffer format

Providing an option to write flatbuffer format for mobile models

Stack from ghstack (oldest at bottom):

Original commit changeset: 780dfb6fd6ba

Original Phabricator Diff: D34805092

Differential Revision: D35067157

NOTE FOR REVIEWERS: This PR has internal Facebook specific changes or comments, please review them on Phabricator!

@pavithranrao
Back out "Revert D34805092: Extend _save_for_mobile and _load_for_mob…
dd35af4 
…ile to support flatbuffer format; Default format is pickle + Change buck targets to support `only pickle` and `pickle + flatbuffer` for migration"

Original commit changeset: 780dfb6fd6ba

Original Phabricator Diff: D34805092

Differential Revision: [D35067157](https://our.internmc.facebook.com/intern/diff/D35067157/)

**NOTE FOR REVIEWERS**: This PR has internal Facebook specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D35067157/)!

[ghstack-poisoned]
@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Mar 23, 2022 •
edited
Loading

🔗 Helpful links

💊 CI failures summary and remediations

As of commit a9eaac0 (more details on the Dr. CI page):

💚 💚 Looks good so far! There are no failures yet. 💚 💚

This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

@facebook-github-bot facebook-github-bot added cla signed oncall: jit Add this issue/PR to JIT oncall triage queue labels Mar 23, 2022
pavithranrao added a commit that referenced this pull request Mar 23, 2022
@pavithranrao
Back out "Revert D34805092: Extend _save_for_mobile and _load_for_mob…
90b8f66 
…ile to support flatbuffer format; Default format is pickle + Change buck targets to support `only pickle` and `pickle + flatbuffer` for migration"

Original commit changeset: 780dfb6fd6ba

Original Phabricator Diff: D34805092

Differential Revision: [D35067157](https://our.internmc.facebook.com/intern/diff/D35067157/)

**NOTE FOR REVIEWERS**: This PR has internal Facebook specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D35067157/)!

ghstack-source-id: 151961923
Pull Request resolved: #74594
@pavithranrao pavithranrao requested review from dbort and iseeyuan March 23, 2022 01:04
@pavithranrao pavithranrao changed the title Back out "Revert D34805092: Extend _save_for_mobile and _load_for_mobile to support flatbuffer format; Default format is pickle + Change buck targets to support only pickle and pickle + flatbuffer for migration" Extend _save_for_mobile and _load_for_mobile to work with flatbuffer format Mar 23, 2022
@pavithranrao
Update on "Extend _save_for_mobile and _load_for_mobile to work with …
1f12e47 
…flatbuffer format"

Original commit changeset: 780dfb6fd6ba

Original Phabricator Diff: D34805092

Differential Revision: [D35067157](https://our.internmc.facebook.com/intern/diff/D35067157/)

**NOTE FOR REVIEWERS**: This PR has internal Facebook specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D35067157/)!

[ghstack-poisoned]
pavithranrao added a commit that referenced this pull request Mar 23, 2022
@pavithranrao
Back out "Revert D34805092: Extend _save_for_mobile and _load_for_mob…
1eca6c4 
…ile to support flatbuffer format; Default format is pickle + Change buck targets to support `only pickle` and `pickle + flatbuffer` for migration"

Pull Request resolved: #74594

Original commit changeset: 780dfb6fd6ba

Original Phabricator Diff: D34805092
ghstack-source-id: 151974085

Differential Revision: [D35067157](https://our.internmc.facebook.com/intern/diff/D35067157/)

**NOTE FOR REVIEWERS**: This PR has internal Facebook specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D35067157/)!
@pavithranrao
Update on "Extend _save_for_mobile and _load_for_mobile to work with …
e7faf51 
…flatbuffer format"

Original commit changeset: 780dfb6fd6ba

Original Phabricator Diff: D34805092

Differential Revision: [D35067157](https://our.internmc.facebook.com/intern/diff/D35067157/)

**NOTE FOR REVIEWERS**: This PR has internal Facebook specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D35067157/)!

[ghstack-poisoned]
pavithranrao added a commit that referenced this pull request Mar 23, 2022
@pavithranrao
Back out "Revert D34805092: Extend _save_for_mobile and _load_for_mob…
c3e0734 
…ile to support flatbuffer format; Default format is pickle + Change buck targets to support `only pickle` and `pickle + flatbuffer` for migration"

Pull Request resolved: #74594

Original commit changeset: 780dfb6fd6ba

Original Phabricator Diff: D34805092
ghstack-source-id: 151976950

Differential Revision: [D35067157](https://our.internmc.facebook.com/intern/diff/D35067157/)

**NOTE FOR REVIEWERS**: This PR has internal Facebook specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D35067157/)!
@pavithranrao
Update on "Extend _save_for_mobile and _load_for_mobile to work with …
a36ef45 
…flatbuffer format"

Original commit changeset: 780dfb6fd6ba

Original Phabricator Diff: D34805092

Differential Revision: [D35067157](https://our.internmc.facebook.com/intern/diff/D35067157/)

**NOTE FOR REVIEWERS**: This PR has internal Facebook specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D35067157/)!

[ghstack-poisoned]
@pavithranrao
Update on "Extend _save_for_mobile and _load_for_mobile to work with …
b80867d 
…flatbuffer format"

Original commit changeset: 780dfb6fd6ba

Original Phabricator Diff: D34805092

Differential Revision: [D35067157](https://our.internmc.facebook.com/intern/diff/D35067157/)

**NOTE FOR REVIEWERS**: This PR has internal Facebook specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D35067157/)!

[ghstack-poisoned]
pavithranrao added a commit that referenced this pull request Mar 23, 2022
@pavithranrao
Back out "Revert D34805092: Extend _save_for_mobile and _load_for_mob…
ecebb03 
…ile to support flatbuffer format; Default format is pickle + Change buck targets to support `only pickle` and `pickle + flatbuffer` for migration"

Pull Request resolved: #74594

Original commit changeset: 780dfb6fd6ba

Original Phabricator Diff: D34805092
ghstack-source-id: 152013732

Differential Revision: [D35067157](https://our.internmc.facebook.com/intern/diff/D35067157/)

**NOTE FOR REVIEWERS**: This PR has internal Facebook specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D35067157/)!
@pavithranrao pavithranrao added oncall: jit Add this issue/PR to JIT oncall triage queue and removed oncall: jit Add this issue/PR to JIT oncall triage queue labels Mar 23, 2022
@pavithranrao
Update on "Extend _save_for_mobile and _load_for_mobile to work with …
9fd6e49 
…flatbuffer format"

Original commit changeset: 780dfb6fd6ba

Original Phabricator Diff: D34805092

Differential Revision: [D35067157](https://our.internmc.facebook.com/intern/diff/D35067157/)

**NOTE FOR REVIEWERS**: This PR has internal Facebook specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D35067157/)!

[ghstack-poisoned]
@pavithranrao
Update on "Extend _save_for_mobile and _load_for_mobile to work with …
a9eaac0 
…flatbuffer format"

Original commit changeset: 780dfb6fd6ba

Original Phabricator Diff: D34805092

Differential Revision: [D35067157](https://our.internmc.facebook.com/intern/diff/D35067157/)

**NOTE FOR REVIEWERS**: This PR has internal Facebook specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D35067157/)!

[ghstack-poisoned]
pavithranrao added a commit that referenced this pull request Mar 23, 2022
@pavithranrao
Back out "Revert D34805092: Extend _save_for_mobile and _load_for_mob…
9eef5d8 
…ile to support flatbuffer format; Default format is pickle + Change buck targets to support `only pickle` and `pickle + flatbuffer` for migration"

Pull Request resolved: #74594

Original commit changeset: 780dfb6fd6ba

Original Phabricator Diff: D34805092
ghstack-source-id: 152044801

Differential Revision: [D35067157](https://our.internmc.facebook.com/intern/diff/D35067157/)

**NOTE FOR REVIEWERS**: This PR has internal Facebook specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D35067157/)!
facebook-github-bot pushed a commit that referenced this pull request Mar 24, 2022
@pavithranrao @facebook-github-bot
Back out "Revert D34805092: Extend _save_for_mobile and _load_for_mob…
f738069 
…ile to support flatbuffer format; Default format is pickle + Change buck targets to support `only pickle` and `pickle + flatbuffer` for migration" (#74594)

Summary:
Pull Request resolved: #74594

Extending `_save_for_mobile` and `_load_for_mobile` to support faltbuffer format with additional optional argument which is set to pick pickle by default.

Adding new binary target with suffix `_pickle_and_flatbuffer` to help migration.

Size test in D34909502 shows the size has regressed by ~40K but after removing pickle and comparing lite_predictors we have ~120K size measure that we will achieve when deprecating pickle and moving to flatbuffer

**BEFORE:**

```lang=mermaid
graph TD;
    torch_core-->torch_mobile_deserialize;

    torch_mobile_core-->torch_mobile_deserialize;

    jit_module_saving-->torch_core;
    jit_module_saving-->torch_mobile_core;

    torch_mobile_deserialize-->caffe2_serialize;
    torch_mobile_deserialize-->torch_mobile_module;

    caffe2_serialize-->miniz;

    flatbuffer_loader-->mobile_bytecode;
    flatbuffer_serializer-->mobile_bytecode;

    mobile_bytecode-->flatbuffer_2.0;

    flatbuffer_loader-->torch_mobile_module;
    flatbuffer_serializer-->torch_mobile_module;
```

**AFTER:**
```lang=mermaid
graph TD;
    torch_core-->torch_mobile_deserialize;

    torch_mobile_core-->torch_mobile_deserialize;

    jit_module_saving-->torch_core;
    jit_module_saving-->torch_mobile_core;

    torch_mobile_deserialize-->caffe2_serialize;
    torch_mobile_deserialize-->torch_mobile_module;

    caffe2_serialize-->miniz;

    flatbuffer_loader-->mobile_bytecode;
    flatbuffer_serializer-->mobile_bytecode;

    mobile_bytecode-->flatbuffer_2.0;

    torch_mobile_deserialize_pickle_and_flatbuffer-->|new| flatbuffer_loader;
    torch_mobile_deserialize_pickle_and_flatbuffer-->|new| torch_mobile_deserialize;
    torch_mobile_core_pickle_and_flatbuffer-->|new| torch_mobile_deserialize_pickle_and_flatbuffer;
    torch_core_pickle_and_flatbuffer-->|new| torch_mobile_deserialize_pickle_and_flatbuffer;

    jit_module_saving_pickle_and_flatbuffer-->|new| torch_core_pickle_and_flatbuffer;
    jit_module_saving_pickle_and_flatbuffer-->|new| torch_mobile_core_pickle_and_flatbuffer;

    flatbuffer_serializer-->torch_mobile_module;

    jit_module_saving_pickle_and_flatbuffer-->|new|jit_module_saving;
    jit_module_saving_pickle_and_flatbuffer-->|new|flatbuffer_serializer;

    flatbuffer_loader-->torch_mobile_module;
```

Original commit changeset: 780dfb6fd6ba

Original Phabricator Diff: D34805092 (284b2b7)
ghstack-source-id: 152044801

(Note: this ignores all push blocking failures!)

Test Plan:
CI

```
~/fbsource/fbcode] cd ~/fbsource/fbcode/ && buck test -c fbcode.caffe2_enable_flatbuffer=1 //caffe2/test/cpp/jit:jit  -- FlatbufferTest.ExtraFiles
Parsing buck files: finished in 0.9 sec
Building: finished in 5.3 sec (100%) 12992/54304 jobs, 0/54304 updated
  Total time: 6.2 sec
More details at https://www.internalfb.com/intern/buck/build/2b387fff-f813-4cfa-b53f-eb2378630d4e
BUILD SUCCEEDED
Tpx test run coordinator for Facebook. See https://fburl.com/tpx for details.
Running with tpx session id: f93a84d6-e7ce-41a0-a97f-0ef3fa6d199d
Trace available for this run at /tmp/tpx-20220323-134108.766518-f93a84d6-e7ce-41a0-a97f-0ef3fa6d199d/trace.log
RemoteExecution session id: reSessionID-f93a84d6-e7ce-41a0-a97f-0ef3fa6d199d-tpx
Started reporting to test run: https://www.internalfb.com/intern/testinfra/testrun/4503599723101693
    ✓ ListingSuccess: caffe2/test/cpp/jit:jit : 486 tests discovered (19.122)
    ✓ Pass: caffe2/test/cpp/jit:jit - FlatbufferTest.ExtraFiles (0.187)
Summary
  Pass: 1
  ListingSuccess: 1
If you need help understanding your runs, please follow the wiki: https://fburl.com/posting_in_tpx_users
Finished test run: https://www.internalfb.com/intern/testinfra/testrun/4503599723101693
```

Similar Build Deps Dags

```
[pavithran@devvm5216.vll0 /data/users/pavithran/fbsource] buck query 'allpaths(//xplat/caffe2:torch_mobile_all_ops_pickle_and_flatbuffer, //xplat/caffe2:torch_mobile_deserialize_pickle_and_flatbuffer)' --output-format dot-compact  | pastry
P486770901: https://www.internalfb.com/intern/paste/P486770901/

[pavithran@devvm5216.vll0 /data/users/pavithran/fbsource] buck query 'allpaths(//xplat/caffe2:torch_mobile_all_ops, //xplat/caffe2:torch_mobile_deserialize)' --output-format dot-compact  | pastry
P486771278: https://www.internalfb.com/intern/paste/P486771278/
```

pickle_and_flatbuffer: https://www.internalfb.com/intern/dgw/graph/?build_id=P486770901
pickle: https://www.internalfb.com/intern/dgw/graph/?build_id=P486771278

Reviewed By: iseeyuan

Differential Revision: D35067157

fbshipit-source-id: 9044259c17a2e0da79bd6aedb28efbdfd57e23e0
@github-actions
Copy link
Contributor

github-actions bot commented Mar 24, 2022

Hey @pavithranrao.
You've committed this PR, but it does not have both a 'release notes: ...' and 'topics: ...' label. Please add one of each to the PR. The 'release notes: ...' label should represent the part of PyTorch that this PR changes (fx, autograd, distributed, etc) and the 'topics: ...' label should represent the kind of PR it is (not user facing, new feature, bug fix, perf improvement, etc). The list of valid labels can be found here for the 'release notes: ...' and here for the 'topics: ...'.
For changes that are 'topic: not user facing' there is no need for a release notes label.

This was referenced Mar 25, 2022
Closed
Closed
shahofblah pushed a commit that referenced this pull request Mar 25, 2022
@pavithranrao @shahofblah
Back out "Revert D34805092: Extend _save_for_mobile and _load_for_mob…
d0b7214 
…ile to support flatbuffer format; Default format is pickle + Change buck targets to support `only pickle` and `pickle + flatbuffer` for migration" (#74594)

Summary:
Pull Request resolved: #74594

Extending `_save_for_mobile` and `_load_for_mobile` to support faltbuffer format with additional optional argument which is set to pick pickle by default.

Adding new binary target with suffix `_pickle_and_flatbuffer` to help migration.

Size test in D34909502 shows the size has regressed by ~40K but after removing pickle and comparing lite_predictors we have ~120K size measure that we will achieve when deprecating pickle and moving to flatbuffer

**BEFORE:**

```lang=mermaid
graph TD;
    torch_core-->torch_mobile_deserialize;

    torch_mobile_core-->torch_mobile_deserialize;

    jit_module_saving-->torch_core;
    jit_module_saving-->torch_mobile_core;

    torch_mobile_deserialize-->caffe2_serialize;
    torch_mobile_deserialize-->torch_mobile_module;

    caffe2_serialize-->miniz;

    flatbuffer_loader-->mobile_bytecode;
    flatbuffer_serializer-->mobile_bytecode;

    mobile_bytecode-->flatbuffer_2.0;

    flatbuffer_loader-->torch_mobile_module;
    flatbuffer_serializer-->torch_mobile_module;
```

**AFTER:**
```lang=mermaid
graph TD;
    torch_core-->torch_mobile_deserialize;

    torch_mobile_core-->torch_mobile_deserialize;

    jit_module_saving-->torch_core;
    jit_module_saving-->torch_mobile_core;

    torch_mobile_deserialize-->caffe2_serialize;
    torch_mobile_deserialize-->torch_mobile_module;

    caffe2_serialize-->miniz;

    flatbuffer_loader-->mobile_bytecode;
    flatbuffer_serializer-->mobile_bytecode;

    mobile_bytecode-->flatbuffer_2.0;

    torch_mobile_deserialize_pickle_and_flatbuffer-->|new| flatbuffer_loader;
    torch_mobile_deserialize_pickle_and_flatbuffer-->|new| torch_mobile_deserialize;
    torch_mobile_core_pickle_and_flatbuffer-->|new| torch_mobile_deserialize_pickle_and_flatbuffer;
    torch_core_pickle_and_flatbuffer-->|new| torch_mobile_deserialize_pickle_and_flatbuffer;

    jit_module_saving_pickle_and_flatbuffer-->|new| torch_core_pickle_and_flatbuffer;
    jit_module_saving_pickle_and_flatbuffer-->|new| torch_mobile_core_pickle_and_flatbuffer;

    flatbuffer_serializer-->torch_mobile_module;

    jit_module_saving_pickle_and_flatbuffer-->|new|jit_module_saving;
    jit_module_saving_pickle_and_flatbuffer-->|new|flatbuffer_serializer;

    flatbuffer_loader-->torch_mobile_module;
```

Original commit changeset: 780dfb6fd6ba

Original Phabricator Diff: D34805092 (284b2b7)
ghstack-source-id: 152044801

(Note: this ignores all push blocking failures!)

Test Plan:
CI

```
~/fbsource/fbcode] cd ~/fbsource/fbcode/ && buck test -c fbcode.caffe2_enable_flatbuffer=1 //caffe2/test/cpp/jit:jit  -- FlatbufferTest.ExtraFiles
Parsing buck files: finished in 0.9 sec
Building: finished in 5.3 sec (100%) 12992/54304 jobs, 0/54304 updated
  Total time: 6.2 sec
More details at https://www.internalfb.com/intern/buck/build/2b387fff-f813-4cfa-b53f-eb2378630d4e
BUILD SUCCEEDED
Tpx test run coordinator for Facebook. See https://fburl.com/tpx for details.
Running with tpx session id: f93a84d6-e7ce-41a0-a97f-0ef3fa6d199d
Trace available for this run at /tmp/tpx-20220323-134108.766518-f93a84d6-e7ce-41a0-a97f-0ef3fa6d199d/trace.log
RemoteExecution session id: reSessionID-f93a84d6-e7ce-41a0-a97f-0ef3fa6d199d-tpx
Started reporting to test run: https://www.internalfb.com/intern/testinfra/testrun/4503599723101693
    ✓ ListingSuccess: caffe2/test/cpp/jit:jit : 486 tests discovered (19.122)
    ✓ Pass: caffe2/test/cpp/jit:jit - FlatbufferTest.ExtraFiles (0.187)
Summary
  Pass: 1
  ListingSuccess: 1
If you need help understanding your runs, please follow the wiki: https://fburl.com/posting_in_tpx_users
Finished test run: https://www.internalfb.com/intern/testinfra/testrun/4503599723101693
```

Similar Build Deps Dags

```
[pavithran@devvm5216.vll0 /data/users/pavithran/fbsource] buck query 'allpaths(//xplat/caffe2:torch_mobile_all_ops_pickle_and_flatbuffer, //xplat/caffe2:torch_mobile_deserialize_pickle_and_flatbuffer)' --output-format dot-compact  | pastry
P486770901: https://www.internalfb.com/intern/paste/P486770901/

[pavithran@devvm5216.vll0 /data/users/pavithran/fbsource] buck query 'allpaths(//xplat/caffe2:torch_mobile_all_ops, //xplat/caffe2:torch_mobile_deserialize)' --output-format dot-compact  | pastry
P486771278: https://www.internalfb.com/intern/paste/P486771278/
```

pickle_and_flatbuffer: https://www.internalfb.com/intern/dgw/graph/?build_id=P486770901
pickle: https://www.internalfb.com/intern/dgw/graph/?build_id=P486771278

Reviewed By: iseeyuan

Differential Revision: D35067157

fbshipit-source-id: 9044259c17a2e0da79bd6aedb28efbdfd57e23e0
(cherry picked from commit f738069)
@facebook-github-bot facebook-github-bot deleted the gh/pavithranrao/53/head branch March 28, 2022 14:17
@WBobby WBobby mentioned this pull request Aug 17, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@iseeyuan iseeyuan iseeyuan approved these changes

@dbort dbort Awaiting requested review from dbort

Assignees

No one assigned

Labels

cla signed oncall: jit Add this issue/PR to JIT oncall triage queue

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@pavithranrao @facebook-github-bot @iseeyuan