Update generated headers and clang format. #73810

qihqi · 2022-03-04T20:46:46Z

title:

Test plan: CI

pytorch-bot · 2022-03-04T20:46:50Z

CI Flow Status

⚛️ CI Flow

Ruleset - Version: v1
Ruleset - File: https://github.com/qihqi/pytorch/blob/d0ba3870427ffb1fd6256fdab80c580064bfdb80/.github/generated-ciflow-ruleset.json
PR ciflow labels: ciflow/default
Add ciflow labels to this PR to trigger more builds:

Workflows	Labels (bold enabled)	Status
Triggered Workflows
linux-binary-conda	`ciflow/binaries`, `ciflow/binaries_conda`, `ciflow/default`	✅ triggered
linux-binary-libtorch-cxx11-abi	`ciflow/binaries`, `ciflow/binaries_libtorch`, `ciflow/default`	✅ triggered
linux-binary-libtorch-pre-cxx11	`ciflow/binaries`, `ciflow/binaries_libtorch`, `ciflow/default`	✅ triggered
linux-binary-manywheel	`ciflow/binaries`, `ciflow/binaries_wheel`, `ciflow/default`	✅ triggered
linux-bionic-py3.7-clang9	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/noarch`, `ciflow/trunk`	✅ triggered
linux-bionic-rocm4.5-py3.7	`ciflow/all`, `ciflow/default`, `ciflow/linux`, `ciflow/rocm`, `ciflow/trunk`	✅ triggered
linux-docs	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/docs`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
linux-vulkan-bionic-py3.7-clang9	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`, `ciflow/vulkan`	✅ triggered
linux-xenial-cuda11.3-py3.7-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
linux-xenial-cuda11.3-py3.7-gcc7-bazel-test	`ciflow/all`, `ciflow/bazel`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
linux-xenial-py3-clang5-mobile-build	`ciflow/all`, `ciflow/default`, `ciflow/linux`, `ciflow/mobile`, `ciflow/trunk`	✅ triggered
linux-xenial-py3-clang5-mobile-custom-build-static	`ciflow/all`, `ciflow/default`, `ciflow/linux`, `ciflow/mobile`, `ciflow/trunk`	✅ triggered
linux-xenial-py3.7-clang7-asan	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/sanitizers`, `ciflow/trunk`	✅ triggered
linux-xenial-py3.7-clang7-onnx	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/onnx`, `ciflow/trunk`	✅ triggered
linux-xenial-py3.7-gcc5.4	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
linux-xenial-py3.7-gcc7	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
linux-xenial-py3.7-gcc7-no-ops	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
macos-arm64-binary-conda	`ciflow/binaries`, `ciflow/binaries_conda`, `ciflow/default`	✅ triggered
macos-arm64-binary-wheel	`ciflow/binaries`, `ciflow/binaries_wheel`, `ciflow/default`	✅ triggered
macos-binary-conda	`ciflow/binaries`, `ciflow/binaries_conda`, `ciflow/default`	✅ triggered
macos-binary-libtorch-cxx11-abi	`ciflow/binaries`, `ciflow/binaries_libtorch`, `ciflow/default`	✅ triggered
macos-binary-libtorch-pre-cxx11	`ciflow/binaries`, `ciflow/binaries_libtorch`, `ciflow/default`	✅ triggered
macos-binary-wheel	`ciflow/binaries`, `ciflow/binaries_wheel`, `ciflow/default`	✅ triggered
pytorch-linux-xenial-py3-clang5-android-ndk-r19c-gradle-custom-build-single	`ciflow/all`, `ciflow/android`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
pytorch-linux-xenial-py3-clang5-android-ndk-r19c-gradle-custom-build-single-full-jit	`ciflow/all`, `ciflow/android`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
win-vs2019-cpu-py3	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/trunk`, `ciflow/win`	✅ triggered
win-vs2019-cuda11.3-py3	`ciflow/all`, `ciflow/cuda`, `ciflow/default`, `ciflow/trunk`, `ciflow/win`	✅ triggered
windows-binary-libtorch-cxx11-abi	`ciflow/binaries`, `ciflow/binaries_libtorch`, `ciflow/default`	✅ triggered
windows-binary-libtorch-pre-cxx11	`ciflow/binaries`, `ciflow/binaries_libtorch`, `ciflow/default`	✅ triggered
windows-binary-wheel	`ciflow/binaries`, `ciflow/binaries_wheel`, `ciflow/default`	✅ triggered
Skipped Workflows
caffe2-linux-xenial-py3.7-gcc5.4	`ciflow/all`, `ciflow/cpu`, `ciflow/linux`, `ciflow/trunk`	🚫 skipped
docker-builds	`ciflow/all`, `ciflow/trunk`	🚫 skipped
ios-12-5-1-arm64	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/scheduled`	🚫 skipped
ios-12-5-1-arm64-coreml	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/scheduled`	🚫 skipped
ios-12-5-1-arm64-custom-ops	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/scheduled`	🚫 skipped
ios-12-5-1-arm64-metal	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/scheduled`	🚫 skipped
ios-12-5-1-x86-64	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
ios-12-5-1-x86-64-coreml	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
libtorch-linux-xenial-cuda10.2-py3.7-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/libtorch`, `ciflow/linux`, `ciflow/trunk`	🚫 skipped
libtorch-linux-xenial-cuda11.3-py3.7-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/libtorch`, `ciflow/linux`, `ciflow/trunk`	🚫 skipped
linux-bionic-cuda10.2-py3.9-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/linux`, `ciflow/slow`, `ciflow/trunk`	🚫 skipped
linux-docs-push	`ciflow/all`, `ciflow/cpu`, `ciflow/linux`, `ciflow/scheduled`	🚫 skipped
linux-xenial-cuda11.3-py3.7-gcc7-no-ops	`ciflow/all`, `ciflow/cuda`, `ciflow/linux`, `ciflow/trunk`	🚫 skipped
macos-10-15-py3-arm64	`ciflow/all`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
macos-10-15-py3-lite-interpreter-x86-64	`ciflow/all`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
macos-11-py3-x86-64	`ciflow/all`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
parallelnative-linux-xenial-py3.7-gcc5.4	`ciflow/all`, `ciflow/cpu`, `ciflow/linux`, `ciflow/trunk`	🚫 skipped
periodic-libtorch-linux-bionic-cuda11.5-py3.7-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/libtorch`, `ciflow/linux`, `ciflow/scheduled`	🚫 skipped
periodic-linux-bionic-cuda11.5-py3.7-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/linux`, `ciflow/scheduled`	🚫 skipped
periodic-linux-xenial-cuda10.2-py3-gcc7-slow-gradcheck	`ciflow/all`, `ciflow/cuda`, `ciflow/linux`, `ciflow/scheduled`, `ciflow/slow`, `ciflow/slow-gradcheck`	🚫 skipped
periodic-linux-xenial-cuda11.3-py3.7-gcc7-debug	`ciflow/all`, `ciflow/cuda`, `ciflow/linux`, `ciflow/scheduled`	🚫 skipped
periodic-win-vs2019-cuda11.5-py3	`ciflow/all`, `ciflow/cuda`, `ciflow/scheduled`, `ciflow/win`	🚫 skipped
pytorch-linux-xenial-py3-clang5-android-ndk-r19c-build	`ciflow/all`, `ciflow/android`, `ciflow/cpu`, `ciflow/linux`, `ciflow/trunk`	🚫 skipped
pytorch-xla-linux-bionic-py3.7-clang8	`ciflow/all`, `ciflow/cpu`, `ciflow/linux`, `ciflow/trunk`, `ciflow/xla`	🚫 skipped

facebook-github-bot · 2022-03-04T20:46:51Z

🔗 Helpful links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/73810
📄 Preview docs built from this PR
↩️ [fb-only] Re-run with SSH instructions
🔧 Opt-in to CIFlow to control what jobs run on your PRs

💊 CI failures summary and remediations

As of commit ac790b2 (more details on the Dr. CI page):

1/1 failures introduced in this PR

1 failure not recognized by patterns:

Job	Step	Action
^{win-vs2019-cuda11.3-py3 / test (default, 2, 2, windows.8xlarge.nvidia.gpu)}	^Unknown	🔁 rerun

This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

facebook-github-bot · 2022-03-04T20:47:16Z

@qihqi has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot · 2022-03-07T19:29:25Z

This pull request was exported from Phabricator. Differential Revision: D34652217

facebook-github-bot · 2022-03-07T21:34:27Z

This pull request was exported from Phabricator. Differential Revision: D34652217

facebook-github-bot · 2022-03-07T21:42:16Z

This pull request was exported from Phabricator. Differential Revision: D34652217

facebook-github-bot · 2022-03-09T01:18:02Z

This pull request was exported from Phabricator. Differential Revision: D34652217

facebook-github-bot · 2022-03-09T01:52:24Z

This pull request was exported from Phabricator. Differential Revision: D34652217

facebook-github-bot · 2022-03-09T18:51:57Z

This pull request was exported from Phabricator. Differential Revision: D34652217

iseeyuan · 2022-03-09T15:57:36Z

test/cpp/jit/test_flatbuffer.cpp

Does the constant Tensors share the same data in the mobile module?

iseeyuan · 2022-03-09T18:51:34Z

test/cpp/jit/test_save_load.cpp

Suggested change

static Module roundtrip(const Module& m) {

static Module roundtripThroughMobile(const Module& m) {

iseeyuan · 2022-03-09T18:54:21Z

test/cpp/jit/test_save_load.cpp

Could we also save the extra files to mobile module to have a more complete roundtrip?

extra files is orthogonal.
The full roundtrip is
jit::Module -> (ivalue, source, constant) -> flatbuffer -> (ivalue, source, constant) -> jit::Module.
This test only tests first first and last step:
jit::Module -> (ivalue, source, constant)->jit::Module.

A test with extrafiles would be better aimed at testing torch::jit::load when it handles flatbuffer.

gmagogsfm

Awesome, thanks for working out a solution so quickly.

gmagogsfm · 2022-03-09T21:23:28Z

test/jit/test_save_load.py

gmagogsfm · 2022-03-09T21:25:14Z

tools/linter/clang_format_all.py

This sounds like a more independent change that can be spun off into its own PR?

And maybe we can introduce an exclude_list instead of hard-coding it as a special case here.

gmagogsfm · 2022-03-09T21:28:29Z

test/jit/test_save_load.py

Do we want to eventually allow torch.jit.load to load a FlatBuffer module with JIT source code?

gmagogsfm · 2022-03-09T22:16:16Z

torch/csrc/jit/mobile/flatbuffer_loader.cpp

Nit:

Header that corresponds to current cpp file should be first #include, not others.

gmagogsfm · 2022-03-09T22:51:41Z

torch/csrc/jit/serialization/flatbuffer_serializer.cpp

Should we consider add these additional arguments to end of argument list so that we don't accidentaly break anyone that's passing extra_files as the third positional argument?

gmagogsfm · 2022-03-09T22:59:13Z

torch/csrc/jit/serialization/mobile_bytecode.fbs

What's the difference between version and operator_version? The first one refers to ByteCodeVersion today?

gmagogsfm · 2022-03-09T22:59:58Z

torch/csrc/jit/serialization/mobile_bytecode_generated.h

Are a lot of changes in this file the result of upgrading flatbuffer version?

gmagogsfm · 2022-03-09T23:14:24Z

torch/csrc/jit/serialization/import.h

Why do we need to emphasize it is JIT module when this is in torch::jit?

gmagogsfm · 2022-03-09T23:17:30Z

torch/jit/_serialization.py

Is it possible for FlatBuffer dependency to be missing in OSS? If so, then we could probably have a dedicated error message for OSS

facebook-github-bot · 2022-03-10T04:47:18Z

This pull request was exported from Phabricator. Differential Revision: D34652217

facebook-github-bot · 2022-03-10T18:32:34Z

This pull request was exported from Phabricator. Differential Revision: D34652217

facebook-github-bot · 2022-03-10T19:01:23Z

This pull request was exported from Phabricator. Differential Revision: D34652217

facebook-github-bot · 2022-03-10T21:21:09Z

This pull request was exported from Phabricator. Differential Revision: D34652217

qihqi · 2022-03-10T21:26:32Z

@gmagogsfm @iseeyuan as suggested I split the diff to make the generated header / clangformat change first. I'll publish another PR with the JIT source stuff.

iseeyuan

LGTM. Please make sure the size with JIT module would be on par with the pickle format, where constants are shared and the overhead of saving JIT module is small.

Summary: Update flatbuffer generated header and add it to ignore for clang format Pull Request resolved: pytorch#73810 Test Plan: CI Reviewed By: iseeyuan Differential Revision: D34652217 Pulled By: qihqi fbshipit-source-id: 9a7daac921da7bae1bdd761942c2489c6902453a

facebook-github-bot · 2022-03-11T18:37:12Z

This pull request was exported from Phabricator. Differential Revision: D34652217

Summary: Update flatbuffer generated header and add it to ignore for clang format Pull Request resolved: #73810 Test Plan: CI Reviewed By: iseeyuan Differential Revision: D34652217 Pulled By: qihqi fbshipit-source-id: fe281afd25d618d2e4852d6b76b813e2fbee0ddc

github-actions · 2022-03-11T20:22:31Z

Hey @qihqi.
You've committed this PR, but it does not have both a 'release notes: ...' and 'topics: ...' label. Please add one of each to the PR. The 'release notes: ...' label should represent the part of PyTorch that this PR changes (fx, autograd, distributed, etc) and the 'topics: ...' label should represent the kind of PR it is (not user facing, new feature, bug fix, perf improvement, etc). The list of valid labels can be found here for the 'release notes: ...' and here for the 'topics: ...'.
For changes that are 'topic: not user facing' there is no need for a release notes label.

pytorch-bot bot added the ciflow/default label Mar 4, 2022

facebook-github-bot added the cla signed label Mar 4, 2022

facebook-github-bot added the oncall: jit Add this issue/PR to JIT oncall triage queue label Mar 4, 2022

qihqi force-pushed the flatbuffer1 branch from d0ba387 to 4493478 Compare March 7, 2022 19:29

qihqi force-pushed the flatbuffer1 branch from 4493478 to 75f8f8c Compare March 7, 2022 21:34

qihqi force-pushed the flatbuffer1 branch from 75f8f8c to e11cb79 Compare March 7, 2022 21:42

qihqi force-pushed the flatbuffer1 branch from e11cb79 to 68219ae Compare March 9, 2022 01:18

qihqi force-pushed the flatbuffer1 branch from 68219ae to 56de7d2 Compare March 9, 2022 01:52

iseeyuan requested review from dbort, gmagogsfm and pavithranrao March 9, 2022 15:23

qihqi force-pushed the flatbuffer1 branch from 56de7d2 to d5671d8 Compare March 9, 2022 18:51

iseeyuan reviewed Mar 9, 2022

View reviewed changes

gmagogsfm reviewed Mar 9, 2022

View reviewed changes

qihqi force-pushed the flatbuffer1 branch from d5671d8 to 1f27a6e Compare March 10, 2022 02:02

qihqi force-pushed the flatbuffer1 branch from 1f27a6e to 2150f91 Compare March 10, 2022 04:47

qihqi force-pushed the flatbuffer1 branch from 2150f91 to 071113e Compare March 10, 2022 18:32

qihqi force-pushed the flatbuffer1 branch from 071113e to 1a7c20b Compare March 10, 2022 19:01

qihqi force-pushed the flatbuffer1 branch from 1a7c20b to bdde8f8 Compare March 10, 2022 21:21

qihqi changed the title ~~Add jit sources into flatbuffer bundle~~ Update generated headers and clang format. Mar 10, 2022

qihqi requested review from gmagogsfm and iseeyuan March 10, 2022 21:26

iseeyuan approved these changes Mar 11, 2022

View reviewed changes

qihqi force-pushed the flatbuffer1 branch from bdde8f8 to ac790b2 Compare March 11, 2022 18:37

pytorchmergebot closed this in 3e556ef Mar 11, 2022

qihqi self-assigned this Mar 11, 2022

qihqi added the release notes: jit release notes category label Mar 11, 2022

WBobby mentioned this pull request Aug 17, 2022

Add ROCm5.2.3/AMDGPU support for PyTorch WBobby/pytorch#2

Closed

	static Module roundtrip(const Module& m) {
	static Module roundtripThroughMobile(const Module& m) {

Update generated headers and clang format. #73810

Update generated headers and clang format. #73810

Uh oh!

Conversation

qihqi commented Mar 4, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Mar 4, 2022

⚛️ CI Flow

Uh oh!

facebook-github-bot commented Mar 4, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful links

💊 CI failures summary and remediations

1 failure not recognized by patterns:

Uh oh!

facebook-github-bot commented Mar 4, 2022

Uh oh!

facebook-github-bot commented Mar 7, 2022

Uh oh!

facebook-github-bot commented Mar 7, 2022

Uh oh!

facebook-github-bot commented Mar 7, 2022

Uh oh!

facebook-github-bot commented Mar 9, 2022

Uh oh!

facebook-github-bot commented Mar 9, 2022

Uh oh!

facebook-github-bot commented Mar 9, 2022

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gmagogsfm left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented Mar 10, 2022

Uh oh!

facebook-github-bot commented Mar 10, 2022

Uh oh!

facebook-github-bot commented Mar 10, 2022

Uh oh!

facebook-github-bot commented Mar 10, 2022

Uh oh!

qihqi commented Mar 10, 2022

Uh oh!

iseeyuan left a comment

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented Mar 11, 2022

Uh oh!

qihqi commented Mar 4, 2022 •

edited

Loading

facebook-github-bot commented Mar 4, 2022 •

edited

Loading