Skip to content

Conversation

@atalman
Copy link
Contributor

@atalman atalman commented Mar 15, 2022

Mitigates #72368

As per discussion here #72368
Some ASAN tests take much longer then same tests that not run under ASAN:

test_fn_gradgrad_pca_lowrank_cpu_float64 (__main__.TestGradientsCPU) ... ok (60.780s)
test_fn_gradgrad_svd_cpu_complex128 (__main__.TestGradientsCPU) ... ok (69.131s)
test_inplace_gradgrad_cumprod_cpu_complex128 (__main__.TestGradientsCPU) ... ok (211.554s)
test_variant_consistency_jit_diff_cpu_complex64 (__main__.TestJitCPU) ... ok (67.640s)
2022-03-12T21:46:25.3026906Z   test_variant_consistency_jit_linalg_solve_triangular_cpu_float32 (__main__.TestJitCPU) ... ok (125.208s)
2022-03-12T21:48:58.0469092Z   test_variant_consistency_jit_linalg_svd_cpu_complex64 (__main__.TestJitCPU) ... ok (152.744s)
2022-03-12T21:50:11.6688335Z   test_variant_consistency_jit_linalg_svd_cpu_float32 (__main__.TestJitCPU) ... ok (73.622s)
2022-03-12T21:54:44.5263321Z   test_variant_consistency_jit_lu_solve_cpu_complex64 (__main__.TestJitCPU) ... ok (102.051s)
2022-03-12T21:55:35.6167891Z   test_variant_consistency_jit_lu_solve_cpu_float32 (__main__.TestJitCPU) ... ok (51.090s)
2022-03-12T22:00:58.4220662Z   test_variant_consistency_jit_nanquantile_cpu_float32 (__main__.TestJitCPU) ... ok (47.142s)
2022-03-12T22:12:25.8979944Z   test_variant_consistency_jit_nn_functional_max_pool1d_cpu_float32 (__main__.TestJitCPU) ... ok (494.579s)
2022-03-12T22:32:45.9750642Z   test_variant_consistency_jit_nn_functional_max_pool2d_cpu_float32 (__main__.TestJitCPU) ... ok (1220.077s)
2022-03-12T22:40:31.3121960Z   test_variant_consistency_jit_nn_functional_max_pool3d_cpu_float32 (__main__.TestJitCPU) ... ok (465.337s)
2022-03-12T22:41:56.5711967Z   test_variant_consistency_jit_nn_functional_pad_circular_cpu_complex64 (__main__.TestJitCPU) ... ok (58.542s)
2022-03-12T22:45:48.7048047Z   test_variant_consistency_jit_nn_functional_pad_constant_cpu_complex64 (__main__.TestJitCPU) ... ok (232.128s)
2022-03-12T22:47:49.1422719Z   test_variant_consistency_jit_nn_functional_pad_constant_cpu_float32 (__main__.TestJitCPU) ... ok (120.437s)
2022-03-12T22:48:48.9686822Z   test_variant_consistency_jit_nn_functional_pad_reflect_cpu_complex64 (__main__.TestJitCPU) ... ok (59.826s)
2022-03-12T22:49:49.2502012Z   test_variant_consistency_jit_nn_functional_pad_replicate_cpu_complex64 (__main__.TestJitCPU) ... ok (60.272s)
2022-03-12T22:51:02.5255728Z   test_variant_consistency_jit_nn_functional_poisson_nll_loss_cpu_float32 (__main__.TestJitCPU) ... ok (73.208s)
2022-03-12T22:54:23.8291811Z   test_variant_consistency_jit_norm_cpu_complex64 (__main__.TestJitCPU) ... ok (136.107s)
2022-03-12T22:55:33.0800761Z   test_variant_consistency_jit_norm_cpu_float32 (__main__.TestJitCPU) ... ok (69.251s)
2022-03-12T22:57:50.4699741Z   test_variant_consistency_jit_ormqr_cpu_complex64 (__main__.TestJitCPU) ... ok (105.720s)
2022-03-12T22:58:46.2191192Z   test_variant_consistency_jit_ormqr_cpu_float32 (__main__.TestJitCPU) ... ok (55.749s)
2022-03-12T23:01:40.5424782Z   test_variant_consistency_jit_prod_cpu_complex64 (__main__.TestJitCPU) ... ok (89.440s)
2022-03-12T23:03:20.2300845Z   test_variant_consistency_jit_put_cpu_complex64 (__main__.TestJitCPU) ... ok (55.004s)
2022-03-12T23:05:34.2481242Z   test_variant_consistency_jit_qr_cpu_complex64 (__main__.TestJitCPU) ... ok (106.490s)
2022-03-12T23:06:26.1268335Z   test_variant_consistency_jit_qr_cpu_float32 (__main__.TestJitCPU) ... ok (51.879s)
2022-03-12T23:07:51.5261184Z   test_variant_consistency_jit_quantile_cpu_float32 (__main__.TestJitCPU) ... ok (85.399s)
test_variant_consistency_jit_sort_cpu_float32 (__main__.TestJitCPU) ... ok (84.378s)
2022-03-12T23:23:48.6314435Z   test_variant_consistency_jit_sum_cpu_complex64 (__main__.TestJitCPU) ... ok (55.706s)
2022-03-12T23:24:19.2219967Z   test_variant_consistency_jit_sum_cpu_float32 (__main__.TestJitCPU) ... ok (30.590s)
2022-03-12T23:36:45.9809917Z   test_variant_consistency_jit_svd_cpu_complex64 (__main__.TestJitCPU) ... ok (746.744s)
2022-03-12T23:42:42.7827088Z   test_variant_consistency_jit_svd_cpu_float32 (__main__.TestJitCPU) ... ok (356.802s)
2022-03-12T23:47:12.7248896Z   test_variant_consistency_jit_tile_cpu_complex64 (__main__.TestJitCPU) ... ok (85.721s) 

This PR ignores ASAN and periodic job for slow job calculation.

Tested by printing the matched jobs rather then continue:

python export_slow_test.py 
Overwriting existent file: .pytorch-slow-tests.json
linux-xenial-py3.7-clang7-asan-test
linux-xenial-py3.7-clang7-asan-test
linux-xenial-py3.7-clang7-asan-test
periodic-linux-xenial-cuda10.2-py3-gcc7-slow-gradcheck-test
periodic-linux-xenial-cuda10.2-py3-gcc7-slow-gradcheck-test

@pytorch-bot
Copy link

pytorch-bot bot commented Mar 15, 2022

CI Flow Status

⚛️ CI Flow

Ruleset - Version: v1
Ruleset - File: https://github.com/atalman/pytorch/blob/88c18becec86d0c0d6988eb81bc522292a919867/.github/generated-ciflow-ruleset.json
PR ciflow labels: ciflow/default
Add ciflow labels to this PR to trigger more builds:

Workflows Labels (bold enabled) Status
Triggered Workflows
linux-binary-conda ciflow/binaries, ciflow/binaries_conda, ciflow/default ✅ triggered
linux-binary-libtorch-cxx11-abi ciflow/all, ciflow/binaries, ciflow/binaries_libtorch, ciflow/default, ciflow/trunk ✅ triggered
linux-binary-libtorch-pre-cxx11 ciflow/all, ciflow/binaries, ciflow/binaries_libtorch, ciflow/default, ciflow/trunk ✅ triggered
linux-binary-manywheel ciflow/all, ciflow/binaries, ciflow/binaries_wheel, ciflow/default, ciflow/trunk ✅ triggered
linux-bionic-py3.7-clang9 ciflow/all, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/noarch, ciflow/trunk ✅ triggered
linux-bionic-rocm4.5-py3.7 ciflow/all, ciflow/default, ciflow/linux, ciflow/rocm, ciflow/trunk ✅ triggered
linux-docs ciflow/all, ciflow/cpu, ciflow/default, ciflow/docs, ciflow/linux, ciflow/trunk ✅ triggered
linux-vulkan-bionic-py3.7-clang9 ciflow/all, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/trunk, ciflow/vulkan ✅ triggered
linux-xenial-cuda11.3-py3.7-gcc7 ciflow/all, ciflow/cuda, ciflow/default, ciflow/linux, ciflow/trunk ✅ triggered
linux-xenial-cuda11.3-py3.7-gcc7-bazel-test ciflow/all, ciflow/bazel, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/trunk ✅ triggered
linux-xenial-py3-clang5-mobile-build ciflow/all, ciflow/default, ciflow/linux, ciflow/mobile, ciflow/trunk ✅ triggered
linux-xenial-py3-clang5-mobile-custom-build-static ciflow/all, ciflow/default, ciflow/linux, ciflow/mobile, ciflow/trunk ✅ triggered
linux-xenial-py3.7-clang7-asan ciflow/all, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/sanitizers, ciflow/trunk ✅ triggered
linux-xenial-py3.7-clang7-onnx ciflow/all, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/onnx, ciflow/trunk ✅ triggered
linux-xenial-py3.7-gcc5.4 ciflow/all, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/trunk ✅ triggered
linux-xenial-py3.7-gcc5.4-mobile-lightweight-dispatch-build ciflow/all, ciflow/cpu, ciflow/default, ciflow/libtorch, ciflow/linux, ciflow/mobile, ciflow/trunk ✅ triggered
linux-xenial-py3.7-gcc7 ciflow/all, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/trunk ✅ triggered
linux-xenial-py3.7-gcc7-no-ops ciflow/all, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/trunk ✅ triggered
macos-arm64-binary-conda ciflow/binaries, ciflow/binaries_conda, ciflow/default ✅ triggered
macos-arm64-binary-wheel ciflow/binaries, ciflow/binaries_wheel, ciflow/default ✅ triggered
macos-binary-conda ciflow/binaries, ciflow/binaries_conda, ciflow/default ✅ triggered
macos-binary-libtorch-cxx11-abi ciflow/binaries, ciflow/binaries_libtorch, ciflow/default ✅ triggered
macos-binary-libtorch-pre-cxx11 ciflow/binaries, ciflow/binaries_libtorch, ciflow/default ✅ triggered
macos-binary-wheel ciflow/binaries, ciflow/binaries_wheel, ciflow/default ✅ triggered
pytorch-linux-xenial-py3-clang5-android-ndk-r19c-gradle-custom-build-single ciflow/all, ciflow/android, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/trunk ✅ triggered
pytorch-linux-xenial-py3-clang5-android-ndk-r19c-gradle-custom-build-single-full-jit ciflow/all, ciflow/android, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/trunk ✅ triggered
win-vs2019-cpu-py3 ciflow/all, ciflow/cpu, ciflow/default, ciflow/trunk, ciflow/win ✅ triggered
win-vs2019-cuda11.3-py3 ciflow/all, ciflow/cuda, ciflow/default, ciflow/trunk, ciflow/win ✅ triggered
windows-binary-conda ciflow/binaries, ciflow/binaries_conda, ciflow/default ✅ triggered
windows-binary-libtorch-debug ciflow/all, ciflow/binaries, ciflow/binaries_libtorch, ciflow/default, ciflow/trunk ✅ triggered
windows-binary-libtorch-release ciflow/all, ciflow/binaries, ciflow/binaries_libtorch, ciflow/default, ciflow/trunk ✅ triggered
windows-binary-wheel ciflow/all, ciflow/binaries, ciflow/binaries_wheel, ciflow/default, ciflow/trunk ✅ triggered
Skipped Workflows
caffe2-linux-xenial-py3.7-gcc5.4 ciflow/all, ciflow/cpu, ciflow/linux, ciflow/trunk 🚫 skipped
docker-builds ciflow/all, ciflow/trunk 🚫 skipped
ios-12-5-1-arm64 ciflow/all, ciflow/ios, ciflow/macos, ciflow/scheduled 🚫 skipped
ios-12-5-1-arm64-coreml ciflow/all, ciflow/ios, ciflow/macos, ciflow/scheduled 🚫 skipped
ios-12-5-1-arm64-custom-ops ciflow/all, ciflow/ios, ciflow/macos, ciflow/scheduled 🚫 skipped
ios-12-5-1-arm64-metal ciflow/all, ciflow/ios, ciflow/macos, ciflow/scheduled 🚫 skipped
ios-12-5-1-x86-64 ciflow/all, ciflow/ios, ciflow/macos, ciflow/trunk 🚫 skipped
ios-12-5-1-x86-64-coreml ciflow/all, ciflow/ios, ciflow/macos, ciflow/trunk 🚫 skipped
libtorch-linux-xenial-cuda10.2-py3.7-gcc7 ciflow/all, ciflow/cuda, ciflow/libtorch, ciflow/linux, ciflow/trunk 🚫 skipped
libtorch-linux-xenial-cuda11.3-py3.7-gcc7 ciflow/all, ciflow/cuda, ciflow/libtorch, ciflow/linux, ciflow/trunk 🚫 skipped
linux-bionic-cuda10.2-py3.9-gcc7 ciflow/all, ciflow/cuda, ciflow/linux, ciflow/slow, ciflow/trunk 🚫 skipped
linux-bionic-rocm4.5-py3.7-distributed ciflow/all, ciflow/linux, ciflow/rocm, ciflow/trunk 🚫 skipped
linux-docs-push ciflow/all, ciflow/cpu, ciflow/linux, ciflow/scheduled 🚫 skipped
linux-xenial-cuda11.3-py3.7-gcc7-no-ops ciflow/all, ciflow/cuda, ciflow/linux, ciflow/trunk 🚫 skipped
macos-10-15-py3-arm64 ciflow/all, ciflow/macos, ciflow/trunk 🚫 skipped
macos-10-15-py3-lite-interpreter-x86-64 ciflow/all, ciflow/macos, ciflow/trunk 🚫 skipped
macos-11-py3-x86-64 ciflow/all, ciflow/macos, ciflow/trunk 🚫 skipped
parallelnative-linux-xenial-py3.7-gcc5.4 ciflow/all, ciflow/cpu, ciflow/linux, ciflow/trunk 🚫 skipped
periodic-libtorch-linux-bionic-cuda11.5-py3.7-gcc7 ciflow/all, ciflow/cuda, ciflow/libtorch, ciflow/linux, ciflow/scheduled 🚫 skipped
periodic-linux-bionic-cuda11.5-py3.7-gcc7 ciflow/all, ciflow/cuda, ciflow/linux, ciflow/scheduled 🚫 skipped
periodic-linux-xenial-cuda10.2-py3-gcc7-slow-gradcheck ciflow/all, ciflow/cuda, ciflow/linux, ciflow/scheduled, ciflow/slow, ciflow/slow-gradcheck 🚫 skipped
periodic-linux-xenial-cuda11.3-py3.7-gcc7-debug ciflow/all, ciflow/cuda, ciflow/linux, ciflow/scheduled 🚫 skipped
periodic-win-vs2019-cuda11.5-py3 ciflow/all, ciflow/cuda, ciflow/scheduled, ciflow/win 🚫 skipped
pytorch-linux-xenial-py3-clang5-android-ndk-r19c-build ciflow/all, ciflow/android, ciflow/cpu, ciflow/linux, ciflow/trunk 🚫 skipped
pytorch-xla-linux-bionic-py3.7-clang8 ciflow/all, ciflow/cpu, ciflow/linux, ciflow/trunk, ciflow/xla 🚫 skipped

@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Mar 15, 2022

🔗 Helpful links

💊 CI failures summary and remediations

As of commit a22d5c6 (more details on the Dr. CI page):


💚 💚 Looks good so far! There are no failures yet. 💚 💚


This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

@atalman atalman requested a review from malfet March 15, 2022 20:17
Copy link
Contributor

@janeyx99 janeyx99 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

@facebook-github-bot
Copy link
Contributor

@atalman has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@seemethere
Copy link
Member

Should we rebase this PR? Looks like there's a lot of failing signal here

@atalman
Copy link
Contributor Author

atalman commented Mar 15, 2022

Should we rebase this PR? Looks like there's a lot of failing signal here

Done!

@atalman atalman closed this Mar 15, 2022
@atalman atalman reopened this Mar 15, 2022
Copy link
Contributor

@malfet malfet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but please rebase on the stable branch

@facebook-github-bot
Copy link
Contributor

@atalman has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot pushed a commit that referenced this pull request Mar 16, 2022
Summary:
Mitigates #72368

As per discussion here  #72368
Some ASAN tests take much longer then same tests that not run under ASAN:

```
test_fn_gradgrad_pca_lowrank_cpu_float64 (__main__.TestGradientsCPU) ... ok (60.780s)
test_fn_gradgrad_svd_cpu_complex128 (__main__.TestGradientsCPU) ... ok (69.131s)
test_inplace_gradgrad_cumprod_cpu_complex128 (__main__.TestGradientsCPU) ... ok (211.554s)
test_variant_consistency_jit_diff_cpu_complex64 (__main__.TestJitCPU) ... ok (67.640s)
2022-03-12T21:46:25.3026906Z   test_variant_consistency_jit_linalg_solve_triangular_cpu_float32 (__main__.TestJitCPU) ... ok (125.208s)
2022-03-12T21:48:58.0469092Z   test_variant_consistency_jit_linalg_svd_cpu_complex64 (__main__.TestJitCPU) ... ok (152.744s)
2022-03-12T21:50:11.6688335Z   test_variant_consistency_jit_linalg_svd_cpu_float32 (__main__.TestJitCPU) ... ok (73.622s)
2022-03-12T21:54:44.5263321Z   test_variant_consistency_jit_lu_solve_cpu_complex64 (__main__.TestJitCPU) ... ok (102.051s)
2022-03-12T21:55:35.6167891Z   test_variant_consistency_jit_lu_solve_cpu_float32 (__main__.TestJitCPU) ... ok (51.090s)
2022-03-12T22:00:58.4220662Z   test_variant_consistency_jit_nanquantile_cpu_float32 (__main__.TestJitCPU) ... ok (47.142s)
2022-03-12T22:12:25.8979944Z   test_variant_consistency_jit_nn_functional_max_pool1d_cpu_float32 (__main__.TestJitCPU) ... ok (494.579s)
2022-03-12T22:32:45.9750642Z   test_variant_consistency_jit_nn_functional_max_pool2d_cpu_float32 (__main__.TestJitCPU) ... ok (1220.077s)
2022-03-12T22:40:31.3121960Z   test_variant_consistency_jit_nn_functional_max_pool3d_cpu_float32 (__main__.TestJitCPU) ... ok (465.337s)
2022-03-12T22:41:56.5711967Z   test_variant_consistency_jit_nn_functional_pad_circular_cpu_complex64 (__main__.TestJitCPU) ... ok (58.542s)
2022-03-12T22:45:48.7048047Z   test_variant_consistency_jit_nn_functional_pad_constant_cpu_complex64 (__main__.TestJitCPU) ... ok (232.128s)
2022-03-12T22:47:49.1422719Z   test_variant_consistency_jit_nn_functional_pad_constant_cpu_float32 (__main__.TestJitCPU) ... ok (120.437s)
2022-03-12T22:48:48.9686822Z   test_variant_consistency_jit_nn_functional_pad_reflect_cpu_complex64 (__main__.TestJitCPU) ... ok (59.826s)
2022-03-12T22:49:49.2502012Z   test_variant_consistency_jit_nn_functional_pad_replicate_cpu_complex64 (__main__.TestJitCPU) ... ok (60.272s)
2022-03-12T22:51:02.5255728Z   test_variant_consistency_jit_nn_functional_poisson_nll_loss_cpu_float32 (__main__.TestJitCPU) ... ok (73.208s)
2022-03-12T22:54:23.8291811Z   test_variant_consistency_jit_norm_cpu_complex64 (__main__.TestJitCPU) ... ok (136.107s)
2022-03-12T22:55:33.0800761Z   test_variant_consistency_jit_norm_cpu_float32 (__main__.TestJitCPU) ... ok (69.251s)
2022-03-12T22:57:50.4699741Z   test_variant_consistency_jit_ormqr_cpu_complex64 (__main__.TestJitCPU) ... ok (105.720s)
2022-03-12T22:58:46.2191192Z   test_variant_consistency_jit_ormqr_cpu_float32 (__main__.TestJitCPU) ... ok (55.749s)
2022-03-12T23:01:40.5424782Z   test_variant_consistency_jit_prod_cpu_complex64 (__main__.TestJitCPU) ... ok (89.440s)
2022-03-12T23:03:20.2300845Z   test_variant_consistency_jit_put_cpu_complex64 (__main__.TestJitCPU) ... ok (55.004s)
2022-03-12T23:05:34.2481242Z   test_variant_consistency_jit_qr_cpu_complex64 (__main__.TestJitCPU) ... ok (106.490s)
2022-03-12T23:06:26.1268335Z   test_variant_consistency_jit_qr_cpu_float32 (__main__.TestJitCPU) ... ok (51.879s)
2022-03-12T23:07:51.5261184Z   test_variant_consistency_jit_quantile_cpu_float32 (__main__.TestJitCPU) ... ok (85.399s)
test_variant_consistency_jit_sort_cpu_float32 (__main__.TestJitCPU) ... ok (84.378s)
2022-03-12T23:23:48.6314435Z   test_variant_consistency_jit_sum_cpu_complex64 (__main__.TestJitCPU) ... ok (55.706s)
2022-03-12T23:24:19.2219967Z   test_variant_consistency_jit_sum_cpu_float32 (__main__.TestJitCPU) ... ok (30.590s)
2022-03-12T23:36:45.9809917Z   test_variant_consistency_jit_svd_cpu_complex64 (__main__.TestJitCPU) ... ok (746.744s)
2022-03-12T23:42:42.7827088Z   test_variant_consistency_jit_svd_cpu_float32 (__main__.TestJitCPU) ... ok (356.802s)
2022-03-12T23:47:12.7248896Z   test_variant_consistency_jit_tile_cpu_complex64 (__main__.TestJitCPU) ... ok (85.721s)
```
-------------

This PR ignores ASAN and periodic job for slow job calculation.

Tested  by printing the matched jobs rather then continue:

```
python export_slow_test.py
Overwriting existent file: .pytorch-slow-tests.json
linux-xenial-py3.7-clang7-asan-test
linux-xenial-py3.7-clang7-asan-test
linux-xenial-py3.7-clang7-asan-test
periodic-linux-xenial-cuda10.2-py3-gcc7-slow-gradcheck-test
periodic-linux-xenial-cuda10.2-py3-gcc7-slow-gradcheck-test
```

Pull Request resolved: #74253

Reviewed By: janeyx99

Differential Revision: D34906097

Pulled By: atalman

fbshipit-source-id: 48103f970e0c4e8683aebd8f78b7d7546fdb35d4
@github-actions
Copy link
Contributor

Hey @atalman.
You've committed this PR, but it does not have both a 'release notes: ...' and 'topics: ...' label. Please add one of each to the PR. The 'release notes: ...' label should represent the part of PyTorch that this PR changes (fx, autograd, distributed, etc) and the 'topics: ...' label should represent the kind of PR it is (not user facing, new feature, bug fix, perf improvement, etc). The list of valid labels can be found here for the 'release notes: ...' and here for the 'topics: ...'.
For changes that are 'topic: not user facing' there is no need for a release notes label.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants