Skip to content

Conversation

@PaliC
Copy link
Contributor

@PaliC PaliC commented Mar 16, 2022

Stack from ghstack (oldest at bottom):

Now instead of spitting out a NotImplemented Error when the Package Exporter finds an object for a mocked module, it combines these errors with the rest of the Packaging Errors of PackageExporter to get something like

torch.package.package_exporter.PackagingError: 
* Module was mocked out, but is still being used in the package.Please intern or extern the mocked modules if objects are supposed to be inthe package.
    package_a
      Context: Object(s) '['PackageASubpackageObject']' from module package_a was mocked out during packaging but is being used in resource - obj.pkl in package obj. 
    package_a.subpackage
      Context: Object(s) '['PackageASubpackageObject']' from module package_a.subpackage was mocked out during packaging but is being used in resource - obj.pkl in package obj. 

This makes it significantly easier to fix mocked object errors as they all should appear at once.

Differential Revision: D34951973

…rs with the rest of the errors in PackageExporter

[ghstack-poisoned]
PaliC added a commit that referenced this pull request Mar 16, 2022
…rs with the rest of the errors in PackageExporter

ghstack-source-id: f1672f7
Pull Request resolved: #74315
@PaliC PaliC requested review from aivanou, d4l3k, kiukchung and suo March 16, 2022 18:13
@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Mar 16, 2022

🔗 Helpful links

💊 CI failures summary and remediations

As of commit f7b4555 (more details on the Dr. CI page):


  • 1/1 failures introduced in this PR

🕵️ 1 new failure recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

See GitHub Actions build linux-xenial-py3.7-clang7-asan / test (default, 2, 3, linux.2xlarge) (1/1)

Step: "Test" (full log | diagnosis details | 🔁 rerun)

2022-03-17T05:46:59.6587283Z RuntimeError: test_binary_ufuncs failed!
2022-03-17T05:46:59.6437391Z     #61 0x55e506e12554 in _start (/opt/conda/bin/python3.7+0x1d7554)
2022-03-17T05:46:59.6437561Z 
2022-03-17T05:46:59.6437697Z AddressSanitizer can not provide additional info.
2022-03-17T05:46:59.6438909Z SUMMARY: AddressSanitizer: BUS /build/llvm-toolchain-7-7.1.0~svn353565/projects/compiler-rt/lib/sanitizer_common/sanitizer_libc.cc:153 
2022-03-17T05:46:59.6439872Z ==1040==ABORTING
2022-03-17T05:46:59.6578719Z Traceback (most recent call last):
2022-03-17T05:46:59.6579173Z   File "test/run_test.py", line 1049, in <module>
2022-03-17T05:46:59.6583599Z     main()
2022-03-17T05:46:59.6583825Z   File "test/run_test.py", line 1027, in main
2022-03-17T05:46:59.6586990Z     raise RuntimeError(err_message)
2022-03-17T05:46:59.6587283Z RuntimeError: test_binary_ufuncs failed!
2022-03-17T05:47:00.0393766Z + cleanup
2022-03-17T05:47:00.0393988Z + retcode=1
2022-03-17T05:47:00.0394146Z + set +x
2022-03-17T05:47:00.0443207Z ##[error]Process completed with exit code 1.
2022-03-17T05:47:00.0495277Z ##[group]Run # Ensure the working directory gets chowned back to the current user
2022-03-17T05:47:00.0495641Z �[36;1m# Ensure the working directory gets chowned back to the current user�[0m
2022-03-17T05:47:00.0495967Z �[36;1mdocker run --rm -v "$(pwd)":/v -w /v "${ALPINE_IMAGE}" chown -R "$(id -u):$(id -g)" .�[0m
2022-03-17T05:47:00.0982242Z shell: /usr/bin/bash -e {0}
2022-03-17T05:47:00.0982443Z env:
2022-03-17T05:47:00.0982668Z   BUILD_ENVIRONMENT: linux-xenial-py3.7-clang7-asan

This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

@PaliC
Copy link
Contributor Author

PaliC commented Mar 16, 2022

@PaliC has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@pytorch-bot
Copy link

pytorch-bot bot commented Mar 16, 2022

CI Flow Status

⚛️ CI Flow

Ruleset - Version: v1
Ruleset - File: https://github.com/pytorch/pytorch/blob/b59401c0a34a37eb9baae3a87761e189d47567b9/.github/generated-ciflow-ruleset.json
PR ciflow labels: ciflow/default
Add ciflow labels to this PR to trigger more builds:

Workflows Labels (bold enabled) Status
Triggered Workflows
linux-binary-conda ciflow/binaries, ciflow/binaries_conda, ciflow/default ✅ triggered
linux-binary-libtorch-cxx11-abi ciflow/all, ciflow/binaries, ciflow/binaries_libtorch, ciflow/default, ciflow/trunk ✅ triggered
linux-binary-libtorch-pre-cxx11 ciflow/all, ciflow/binaries, ciflow/binaries_libtorch, ciflow/default, ciflow/trunk ✅ triggered
linux-binary-manywheel ciflow/all, ciflow/binaries, ciflow/binaries_wheel, ciflow/default, ciflow/trunk ✅ triggered
linux-bionic-py3.7-clang9 ciflow/all, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/noarch, ciflow/trunk ✅ triggered
linux-bionic-rocm4.5-py3.7 ciflow/all, ciflow/default, ciflow/linux, ciflow/rocm, ciflow/trunk ✅ triggered
linux-docs ciflow/all, ciflow/cpu, ciflow/default, ciflow/docs, ciflow/linux, ciflow/trunk ✅ triggered
linux-vulkan-bionic-py3.7-clang9 ciflow/all, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/trunk, ciflow/vulkan ✅ triggered
linux-xenial-cuda11.3-py3.7-gcc7 ciflow/all, ciflow/cuda, ciflow/default, ciflow/linux, ciflow/trunk ✅ triggered
linux-xenial-cuda11.3-py3.7-gcc7-bazel-test ciflow/all, ciflow/bazel, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/trunk ✅ triggered
linux-xenial-py3-clang5-mobile-build ciflow/all, ciflow/default, ciflow/linux, ciflow/mobile, ciflow/trunk ✅ triggered
linux-xenial-py3-clang5-mobile-custom-build-static ciflow/all, ciflow/default, ciflow/linux, ciflow/mobile, ciflow/trunk ✅ triggered
linux-xenial-py3.7-clang7-asan ciflow/all, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/sanitizers, ciflow/trunk ✅ triggered
linux-xenial-py3.7-clang7-onnx ciflow/all, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/onnx, ciflow/trunk ✅ triggered
linux-xenial-py3.7-gcc5.4 ciflow/all, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/trunk ✅ triggered
linux-xenial-py3.7-gcc5.4-mobile-lightweight-dispatch-build ciflow/all, ciflow/cpu, ciflow/default, ciflow/libtorch, ciflow/linux, ciflow/mobile, ciflow/trunk ✅ triggered
linux-xenial-py3.7-gcc7 ciflow/all, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/trunk ✅ triggered
linux-xenial-py3.7-gcc7-no-ops ciflow/all, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/trunk ✅ triggered
macos-arm64-binary-conda ciflow/binaries, ciflow/binaries_conda, ciflow/default ✅ triggered
macos-arm64-binary-wheel ciflow/binaries, ciflow/binaries_wheel, ciflow/default ✅ triggered
macos-binary-conda ciflow/binaries, ciflow/binaries_conda, ciflow/default ✅ triggered
macos-binary-libtorch-cxx11-abi ciflow/binaries, ciflow/binaries_libtorch, ciflow/default ✅ triggered
macos-binary-libtorch-pre-cxx11 ciflow/binaries, ciflow/binaries_libtorch, ciflow/default ✅ triggered
macos-binary-wheel ciflow/binaries, ciflow/binaries_wheel, ciflow/default ✅ triggered
pytorch-linux-xenial-py3-clang5-android-ndk-r19c-gradle-custom-build-single ciflow/all, ciflow/android, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/trunk ✅ triggered
pytorch-linux-xenial-py3-clang5-android-ndk-r19c-gradle-custom-build-single-full-jit ciflow/all, ciflow/android, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/trunk ✅ triggered
win-vs2019-cpu-py3 ciflow/all, ciflow/cpu, ciflow/default, ciflow/trunk, ciflow/win ✅ triggered
win-vs2019-cuda11.3-py3 ciflow/all, ciflow/cuda, ciflow/default, ciflow/trunk, ciflow/win ✅ triggered
windows-binary-conda ciflow/binaries, ciflow/binaries_conda, ciflow/default ✅ triggered
windows-binary-libtorch-debug ciflow/all, ciflow/binaries, ciflow/binaries_libtorch, ciflow/default, ciflow/trunk ✅ triggered
windows-binary-libtorch-release ciflow/all, ciflow/binaries, ciflow/binaries_libtorch, ciflow/default, ciflow/trunk ✅ triggered
windows-binary-wheel ciflow/all, ciflow/binaries, ciflow/binaries_wheel, ciflow/default, ciflow/trunk ✅ triggered
Skipped Workflows
caffe2-linux-xenial-py3.7-gcc5.4 ciflow/all, ciflow/cpu, ciflow/linux, ciflow/trunk 🚫 skipped
docker-builds ciflow/all, ciflow/trunk 🚫 skipped
ios-12-5-1-arm64 ciflow/all, ciflow/ios, ciflow/macos, ciflow/scheduled 🚫 skipped
ios-12-5-1-arm64-coreml ciflow/all, ciflow/ios, ciflow/macos, ciflow/scheduled 🚫 skipped
ios-12-5-1-arm64-custom-ops ciflow/all, ciflow/ios, ciflow/macos, ciflow/scheduled 🚫 skipped
ios-12-5-1-arm64-metal ciflow/all, ciflow/ios, ciflow/macos, ciflow/scheduled 🚫 skipped
ios-12-5-1-x86-64 ciflow/all, ciflow/ios, ciflow/macos, ciflow/trunk 🚫 skipped
ios-12-5-1-x86-64-coreml ciflow/all, ciflow/ios, ciflow/macos, ciflow/trunk 🚫 skipped
libtorch-linux-xenial-cuda10.2-py3.7-gcc7 ciflow/all, ciflow/cuda, ciflow/libtorch, ciflow/linux, ciflow/trunk 🚫 skipped
libtorch-linux-xenial-cuda11.3-py3.7-gcc7 ciflow/all, ciflow/cuda, ciflow/libtorch, ciflow/linux, ciflow/trunk 🚫 skipped
linux-bionic-cuda10.2-py3.9-gcc7 ciflow/all, ciflow/cuda, ciflow/linux, ciflow/slow, ciflow/trunk 🚫 skipped
linux-bionic-rocm4.5-py3.7-distributed ciflow/all, ciflow/linux, ciflow/rocm, ciflow/trunk 🚫 skipped
linux-docs-push ciflow/all, ciflow/cpu, ciflow/linux, ciflow/scheduled 🚫 skipped
linux-xenial-cuda11.3-py3.7-gcc7-no-ops ciflow/all, ciflow/cuda, ciflow/linux, ciflow/trunk 🚫 skipped
macos-10-15-py3-arm64 ciflow/all, ciflow/macos, ciflow/trunk 🚫 skipped
macos-10-15-py3-lite-interpreter-x86-64 ciflow/all, ciflow/macos, ciflow/trunk 🚫 skipped
macos-11-py3-x86-64 ciflow/all, ciflow/macos, ciflow/trunk 🚫 skipped
parallelnative-linux-xenial-py3.7-gcc5.4 ciflow/all, ciflow/cpu, ciflow/linux, ciflow/trunk 🚫 skipped
periodic-libtorch-linux-bionic-cuda11.5-py3.7-gcc7 ciflow/all, ciflow/cuda, ciflow/libtorch, ciflow/linux, ciflow/scheduled 🚫 skipped
periodic-linux-bionic-cuda11.5-py3.7-gcc7 ciflow/all, ciflow/cuda, ciflow/linux, ciflow/scheduled 🚫 skipped
periodic-linux-xenial-cuda10.2-py3-gcc7-slow-gradcheck ciflow/all, ciflow/cuda, ciflow/linux, ciflow/scheduled, ciflow/slow, ciflow/slow-gradcheck 🚫 skipped
periodic-linux-xenial-cuda11.3-py3.7-gcc7-debug ciflow/all, ciflow/cuda, ciflow/linux, ciflow/scheduled 🚫 skipped
periodic-win-vs2019-cuda11.5-py3 ciflow/all, ciflow/cuda, ciflow/scheduled, ciflow/win 🚫 skipped
pytorch-linux-xenial-py3-clang5-android-ndk-r19c-build ciflow/all, ciflow/android, ciflow/cpu, ciflow/linux, ciflow/trunk 🚫 skipped
pytorch-xla-linux-bionic-py3.7-clang8 ciflow/all, ciflow/cpu, ciflow/linux, ciflow/trunk, ciflow/xla 🚫 skipped

facebook-github-bot pushed a commit that referenced this pull request Mar 17, 2022
…rs with the rest of the errors in PackageExporter (#74315)

Summary:
Pull Request resolved: #74315

Now instead of spitting out a NotImplemented Error when the Package Exporter finds an object for a mocked module, it combines these errors with the rest of the Packaging Errors of PackageExporter to get something like

```
torch.package.package_exporter.PackagingError:
* Module was mocked out, but is still being used in the package.Please intern or extern the mocked modules if objects are supposed to be inthe package.
    package_a
      Context: Object(s) '['PackageASubpackageObject']' from module package_a was mocked out during packaging but is being used in resource - obj.pkl in package obj.
    package_a.subpackage
      Context: Object(s) '['PackageASubpackageObject']' from module package_a.subpackage was mocked out during packaging but is being used in resource - obj.pkl in package obj.
```

This makes it significantly easier to fix mocked object errors as they all should appear at once.

Test Plan: Imported from OSS

Reviewed By: aivanou

Differential Revision: D34932200

Pulled By: PaliC

fbshipit-source-id: 7f12bd88dbfbad974fd04b5dcaba3203b5c68a04
@github-actions
Copy link
Contributor

Hey @PaliC.
You've committed this PR, but it does not have both a 'release notes: ...' and 'topics: ...' label. Please add one of each to the PR. The 'release notes: ...' label should represent the part of PyTorch that this PR changes (fx, autograd, distributed, etc) and the 'topics: ...' label should represent the kind of PR it is (not user facing, new feature, bug fix, perf improvement, etc). The list of valid labels can be found here for the 'release notes: ...' and here for the 'topics: ...'.
For changes that are 'topic: not user facing' there is no need for a release notes label.

@suo
Copy link
Member

suo commented Mar 17, 2022

Sadly, this PR broke lint. I am reverting

@suo
Copy link
Member

suo commented Mar 17, 2022

GHA had an outage this morning, if you look CI didn't run on this PR at all

@facebook-github-bot
Copy link
Contributor

This pull request has been reverted by 577bf04. To re-land this change, please open another pull request, assignthe same reviewers, fix the CI failures that caused the revert and make sure that the failing CI runs on the PR by applying the proper ciflow label (e.g., ciflow/trunk).

@PaliC PaliC reopened this Mar 17, 2022
…object errors with the rest of the errors in PackageExporter"


Now instead of spitting out a NotImplemented Error when the Package Exporter finds an object for a mocked module, it combines these errors with the rest of the Packaging Errors of PackageExporter to get something like 

```
torch.package.package_exporter.PackagingError: 
* Module was mocked out, but is still being used in the package.Please intern or extern the mocked modules if objects are supposed to be inthe package.
    package_a
      Context: Object(s) '['PackageASubpackageObject']' from module package_a was mocked out during packaging but is being used in resource - obj.pkl in package obj. 
    package_a.subpackage
      Context: Object(s) '['PackageASubpackageObject']' from module package_a.subpackage was mocked out during packaging but is being used in resource - obj.pkl in package obj. 
```

This makes it significantly easier to fix mocked object errors as they all should appear at once.

[ghstack-poisoned]
PaliC added a commit that referenced this pull request Mar 17, 2022
…rs with the rest of the errors in PackageExporter

ghstack-source-id: 59e099c
Pull Request resolved: #74315
@PaliC
Copy link
Contributor Author

PaliC commented Mar 17, 2022

@PaliC has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

facebook-github-bot pushed a commit that referenced this pull request Mar 17, 2022
…rs with the rest of the errors in PackageExporter (#74315)

Summary:
Pull Request resolved: #74315

Now instead of spitting out a NotImplemented Error when the Package Exporter finds an object for a mocked module, it combines these errors with the rest of the Packaging Errors of PackageExporter to get something like

```
torch.package.package_exporter.PackagingError:
* Module was mocked out, but is still being used in the package.Please intern or extern the mocked modules if objects are supposed to be inthe package.
    package_a
      Context: Object(s) '['PackageASubpackageObject']' from module package_a was mocked out during packaging but is being used in resource - obj.pkl in package obj.
    package_a.subpackage
      Context: Object(s) '['PackageASubpackageObject']' from module package_a.subpackage was mocked out during packaging but is being used in resource - obj.pkl in package obj.
```

This makes it significantly easier to fix mocked object errors as they all should appear at once.

Test Plan: Imported from OSS

Reviewed By: d4l3k

Differential Revision: D34951973

Pulled By: PaliC

fbshipit-source-id: 01ee4ba3767967ef9a9bcd69ad86362ebc100b2d
@github-actions
Copy link
Contributor

Hey @PaliC.
You've committed this PR, but it does not have both a 'release notes: ...' and 'topics: ...' label. Please add one of each to the PR. The 'release notes: ...' label should represent the part of PyTorch that this PR changes (fx, autograd, distributed, etc) and the 'topics: ...' label should represent the kind of PR it is (not user facing, new feature, bug fix, perf improvement, etc). The list of valid labels can be found here for the 'release notes: ...' and here for the 'topics: ...'.
For changes that are 'topic: not user facing' there is no need for a release notes label.

@facebook-github-bot facebook-github-bot deleted the gh/PaliC/148/head branch March 21, 2022 14:17
@suo suo mentioned this pull request Mar 22, 2022
shahofblah pushed a commit that referenced this pull request Mar 25, 2022
…rs with the rest of the errors in PackageExporter (#74315)

Summary:
Pull Request resolved: #74315

Now instead of spitting out a NotImplemented Error when the Package Exporter finds an object for a mocked module, it combines these errors with the rest of the Packaging Errors of PackageExporter to get something like

```
torch.package.package_exporter.PackagingError:
* Module was mocked out, but is still being used in the package.Please intern or extern the mocked modules if objects are supposed to be inthe package.
    package_a
      Context: Object(s) '['PackageASubpackageObject']' from module package_a was mocked out during packaging but is being used in resource - obj.pkl in package obj.
    package_a.subpackage
      Context: Object(s) '['PackageASubpackageObject']' from module package_a.subpackage was mocked out during packaging but is being used in resource - obj.pkl in package obj.
```

This makes it significantly easier to fix mocked object errors as they all should appear at once.

Test Plan: Imported from OSS

Reviewed By: d4l3k

Differential Revision: D34951973

Pulled By: PaliC

fbshipit-source-id: 01ee4ba3767967ef9a9bcd69ad86362ebc100b2d
(cherry picked from commit 900edd2)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants