Add BFloat16 support for logsigmoid, hardsigmoid, hardshrink, softshrink, hardswish and softplus on CPU #63134

CaoE · 2021-08-12T06:30:58Z

Add BFloat16 support for logsigmoid, hardsigmoid, hardshrink, softshrink, hardswish and softplus on CPU, and optimize the performance of softshrink.

facebook-github-bot · 2021-08-12T06:31:03Z

🔗 Helpful links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/63134
📄 Preview docs built from this PR
📄 Preview C++ docs built from this PR
🔧 Opt-in to CIFlow to control what jobs run on your PRs

💊 CI failures summary and remediations

As of commit 2b830af (more details on the Dr. CI page):

1/1 failures introduced in this PR

🕵️ 1 new failure recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

linux-bionic-rocm4.5-py3.7 / test (distributed, 1, 1, linux.rocm.gpu) (1/1)

Step: "Unknown" (full log | diagnosis details | 🔁 rerun)

2022-03-02T13:06:04.3339840Z RuntimeError: hello

2022-03-02T13:06:04.3329514Z -- Process 0 terminated with the following error:
2022-03-02T13:06:04.3330263Z Traceback (most recent call last):
2022-03-02T13:06:04.3331744Z   File "/opt/conda/lib/python3.7/site-packages/torch/multiprocessing/spawn.py", line 69, in _wrap
2022-03-02T13:06:04.3332610Z     fn(i, *args)
2022-03-02T13:06:04.3333939Z   File "/opt/conda/lib/python3.7/site-packages/torch/distributed/elastic/multiprocessing/api.py", line 369, in _wrap
2022-03-02T13:06:04.3334910Z     ret = record(fn)(*args_)
2022-03-02T13:06:04.3336334Z   File "/opt/conda/lib/python3.7/site-packages/torch/distributed/elastic/multiprocessing/errors/__init__.py", line 345, in wrapper
2022-03-02T13:06:04.3337342Z     return f(*args, **kwargs)
2022-03-02T13:06:04.3338273Z   File "/var/lib/jenkins/pytorch/test/distributed/elastic/multiprocessing/api_test.py", line 138, in echo2
2022-03-02T13:06:04.3339204Z     raise RuntimeError(msg)
2022-03-02T13:06:04.3339840Z RuntimeError: hello
2022-03-02T13:06:04.3340195Z 
2022-03-02T13:06:04.3647809Z ok (3.199s)
2022-03-02T13:06:04.3670267Z   test_function_with_tensor (__main__.StartProcessesTest) ... ok (0.002s)
2022-03-02T13:06:04.3684477Z   test_invalid_log_dir (__main__.StartProcessesTest) ... ok (0.002s)
2022-03-02T13:06:04.3747779Z   test_multiprocess_context_close (__main__.StartProcessesTest) ... Closing process 2449 via signal SIGTERM
2022-03-02T13:06:04.3802488Z ok (0.011s)
2022-03-02T13:06:04.3841647Z   test_multiprocessing_context_poll_raises_exception (__main__.StartProcessesTest) ... failed (exitcode: -1) local_rank: 0 (pid: 123) of fn: echo0 (start_method: spawn)
2022-03-02T13:06:04.3842896Z Traceback (most recent call last):
2022-03-02T13:06:04.3844315Z   File "/opt/conda/lib/python3.7/site-packages/torch/distributed/elastic/multiprocessing/api.py", line 453, in _poll
2022-03-02T13:06:04.3845564Z     self._pc.join(-1)

This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

VitalyFedyunin · 2021-08-15T16:44:33Z

Please add benchmarks comparing float and bfloat kernels for various input sizes.
Please rebase
Windows build error might be relevant (check after rebase)

CaoE · 2021-08-18T15:15:14Z

Rebased @VitalyFedyunin.
Single core performance is tested on Xeon(R) Platinum 8180 @2.5 Ghz.

Single socket (28 cores) performance is tested on Xeon(R) Platinum 8180 @2.5 Ghz.

codecov · 2021-08-27T06:31:56Z

Codecov Report

Merging #63134 (12cf17d) into master (feefc94) will increase coverage by 0.26%.
The diff coverage is n/a.

❗ Current head 12cf17d differs from pull request most recent head 1f7f3ab. Consider uploading reports for the commit 1f7f3ab to get more accurate results

@@            Coverage Diff             @@
##           master   #63134      +/-   ##
==========================================
+ Coverage   66.37%   66.64%   +0.26%     
==========================================
  Files         739      707      -32     
  Lines       94299    92346    -1953     
==========================================
- Hits        62595    61540    -1055     
+ Misses      31704    30806     -898

pytorch-probot · 2021-10-08T05:04:27Z

CI Flow Status

⚛️ CI Flow

Ruleset - Version: v1
Ruleset - File: https://github.com/CaoE/pytorch/blob/1f7f3ab2577a13216c04a105f5790291f12c0158/.github/generated-ciflow-ruleset.json
PR ciflow labels: ciflow/default

Workflows	Labels (bold enabled)	Status
Triggered Workflows
linux-bionic-py3.7-clang9	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/noarch`, `ciflow/trunk`	✅ triggered
linux-docs	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/docs`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
linux-vulkan-bionic-py3.7-clang9	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`, `ciflow/vulkan`	✅ triggered
linux-xenial-cuda11.3-py3.7-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
linux-xenial-cuda11.3-py3.7-gcc7-bazel-test	`ciflow/all`, `ciflow/bazel`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
linux-xenial-py3-clang5-mobile-build	`ciflow/all`, `ciflow/default`, `ciflow/linux`, `ciflow/mobile`, `ciflow/trunk`	✅ triggered
linux-xenial-py3-clang5-mobile-custom-build-static	`ciflow/all`, `ciflow/default`, `ciflow/linux`, `ciflow/mobile`, `ciflow/trunk`	✅ triggered
linux-xenial-py3.7-clang7-asan	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/sanitizers`, `ciflow/trunk`	✅ triggered
linux-xenial-py3.7-clang7-onnx	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/onnx`, `ciflow/trunk`	✅ triggered
linux-xenial-py3.7-gcc5.4	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
linux-xenial-py3.7-gcc7	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
linux-xenial-py3.7-gcc7-no-ops	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
pytorch-linux-xenial-py3-clang5-android-ndk-r19c-gradle-custom-build-single	`ciflow/all`, `ciflow/android`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
pytorch-linux-xenial-py3-clang5-android-ndk-r19c-gradle-custom-build-single-full-jit	`ciflow/all`, `ciflow/android`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
win-vs2019-cpu-py3	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/trunk`, `ciflow/win`	✅ triggered
win-vs2019-cuda11.3-py3	`ciflow/all`, `ciflow/cuda`, `ciflow/default`, `ciflow/trunk`, `ciflow/win`	✅ triggered
Skipped Workflows
caffe2-linux-xenial-py3.7-gcc5.4	`ciflow/all`, `ciflow/cpu`, `ciflow/linux`, `ciflow/trunk`	🚫 skipped
docker-builds	`ciflow/all`, `ciflow/trunk`	🚫 skipped
ios-12-5-1-arm64	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
ios-12-5-1-arm64-coreml	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
ios-12-5-1-arm64-custom-ops	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
ios-12-5-1-arm64-full-jit	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
ios-12-5-1-arm64-metal	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
ios-12-5-1-x86-64	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
ios-12-5-1-x86-64-coreml	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
ios-12-5-1-x86-64-full-jit	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
libtorch-linux-xenial-cuda10.2-py3.7-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/libtorch`, `ciflow/linux`, `ciflow/trunk`	🚫 skipped
libtorch-linux-xenial-cuda11.3-py3.7-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/libtorch`, `ciflow/linux`, `ciflow/trunk`	🚫 skipped
linux-binary-conda	`ciflow/binaries`, `ciflow/binaries/conda`	🚫 skipped
linux-binary-libtorch-cxx11-abi	`ciflow/binaries`, `ciflow/binaries/libtorch`	🚫 skipped
linux-binary-libtorch-pre-cxx11	`ciflow/binaries`, `ciflow/binaries/libtorch`	🚫 skipped
linux-binary-manywheel	`ciflow/binaries`, `ciflow/binaries/wheel`	🚫 skipped
linux-bionic-cuda10.2-py3.9-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/linux`, `ciflow/slow`, `ciflow/trunk`	🚫 skipped
linux-bionic-py3.6-clang9	`ciflow/xla`	🚫 skipped
linux-docs-push	`ciflow/all`, `ciflow/cpu`, `ciflow/linux`, `ciflow/scheduled`	🚫 skipped
linux-xenial-cuda11.3-py3.7-gcc7-no-ops	`ciflow/all`, `ciflow/cuda`, `ciflow/linux`, `ciflow/trunk`	🚫 skipped
macos-10-15-py3-arm64	`ciflow/all`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
macos-10-15-py3-lite-interpreter-x86-64	`ciflow/all`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
macos-11-py3-x86-64	`ciflow/all`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
parallelnative-linux-xenial-py3.7-gcc5.4	`ciflow/all`, `ciflow/cpu`, `ciflow/linux`, `ciflow/trunk`	🚫 skipped
periodic-libtorch-linux-bionic-cuda11.5-py3.7-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/libtorch`, `ciflow/linux`, `ciflow/scheduled`	🚫 skipped
periodic-libtorch-linux-xenial-cuda11.1-py3.7-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/libtorch`, `ciflow/linux`, `ciflow/scheduled`	🚫 skipped
periodic-linux-bionic-cuda11.5-py3.7-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/linux`, `ciflow/scheduled`	🚫 skipped
periodic-linux-xenial-cuda10.2-py3-gcc7-slow-gradcheck	`ciflow/all`, `ciflow/cuda`, `ciflow/linux`, `ciflow/scheduled`, `ciflow/slow`, `ciflow/slow-gradcheck`	🚫 skipped
periodic-linux-xenial-cuda11.1-py3.7-gcc7-debug	`ciflow/all`, `ciflow/cuda`, `ciflow/linux`, `ciflow/scheduled`	🚫 skipped
periodic-win-vs2019-cuda11.1-py3	`ciflow/all`, `ciflow/cuda`, `ciflow/scheduled`, `ciflow/win`	🚫 skipped
periodic-win-vs2019-cuda11.5-py3	`ciflow/all`, `ciflow/cuda`, `ciflow/scheduled`, `ciflow/win`	🚫 skipped
pytorch-linux-xenial-py3-clang5-android-ndk-r19c-build	`ciflow/all`, `ciflow/android`, `ciflow/cpu`, `ciflow/linux`, `ciflow/trunk`	🚫 skipped

You can add a comment to the PR and tag @pytorchbot with the following commands:

# ciflow rerun, "ciflow/default" will always be added automatically
@pytorchbot ciflow rerun

# ciflow rerun with additional labels "-l <ciflow/label_name>", which is equivalent to adding these labels manually and trigger the rerun
@pytorchbot ciflow rerun -l ciflow/scheduled -l ciflow/slow

For more information, please take a look at the CI Flow Wiki.

CaoE · 2021-10-19T01:58:28Z

Hi @VitalyFedyunin
Thanks for your suggestions and explanation. PR is rebased, could you please review it?

CaoE · 2021-12-28T04:57:22Z

Hi @VitalyFedyunin, could you please review it ? Thank you.

pytorchbot · 2022-01-24T03:02:58Z

Build started for merge commit.

…ink, hardswish, and softplus on CPU

facebook-github-bot · 2022-03-15T16:58:58Z

@frank-wei has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

…ink, hardswish and softplus on CPU (#63134) Summary: Add BFloat16 support for logsigmoid, hardsigmoid, hardshrink, softshrink, hardswish and softplus on CPU, and optimize the performance of softshrink. Pull Request resolved: #63134 Reviewed By: yinghai Differential Revision: D34897992 Pulled By: frank-wei fbshipit-source-id: 4c778f5271d6fa54dd78158258941def8d9252f5

facebook-github-bot added the cla signed label Aug 12, 2021

pytorchbot added the open source label Aug 12, 2021

CaoE force-pushed the bf16_nn4 branch 3 times, most recently from 1a97e8f to 06774a8 Compare August 13, 2021 05:27

CaoE changed the title ~~Add BFloat16 support for logsigmoid, hardsigmoid, hardshrink, softshrink, and hardswish on CPU~~ Add BFloat16 support for logsigmoid, hardsigmoid, hardshrink, softshrink, hardswish, and softplus on CPU Aug 13, 2021

CaoE force-pushed the bf16_nn4 branch from 06774a8 to d802c48 Compare August 13, 2021 05:34

VitalyFedyunin self-requested a review August 15, 2021 16:44

CaoE force-pushed the bf16_nn4 branch from d802c48 to 181b775 Compare August 16, 2021 07:33

astaff added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Aug 16, 2021

CaoE force-pushed the bf16_nn4 branch from 181b775 to e558b09 Compare August 19, 2021 02:56

CaoE force-pushed the bf16_nn4 branch 3 times, most recently from 343e903 to 55c7313 Compare August 27, 2021 02:39

CaoE changed the title ~~Add BFloat16 support for logsigmoid, hardsigmoid, hardshrink, softshrink, hardswish, and softplus on CPU~~ Add BFloat16 support for logsigmoid, hardsigmoid, hardshrink, softshrink, hardswish, softplus and elu on CPU Aug 27, 2021

CaoE closed this Aug 31, 2021

CaoE reopened this Sep 2, 2021

CaoE force-pushed the bf16_nn4 branch 3 times, most recently from 423a004 to 12cf17d Compare September 7, 2021 07:05

CaoE changed the title ~~Add BFloat16 support for logsigmoid, hardsigmoid, hardshrink, softshrink, hardswish, softplus and elu on CPU~~ Add BFloat16 support for logsigmoid, hardsigmoid, hardshrink, softshrink, hardswish and softplus on CPU Sep 7, 2021

jgong5 mentioned this pull request Sep 24, 2021

[RFC] Extend Autocast to CPU/CUDA with BF16 data type #55374

Open

CaoE force-pushed the bf16_nn4 branch 2 times, most recently from 9c7be87 to 52285d1 Compare September 27, 2021 08:13

CaoE force-pushed the bf16_nn4 branch from 52285d1 to 0bea331 Compare October 8, 2021 05:04

pytorch-probot bot added the ciflow/default label Oct 8, 2021

CaoE force-pushed the bf16_nn4 branch from 0bea331 to e63d4cb Compare October 18, 2021 03:06

CaoE force-pushed the bf16_nn4 branch from e63d4cb to 85d5770 Compare November 2, 2021 02:45

CaoE force-pushed the bf16_nn4 branch 2 times, most recently from 3bec7d6 to 4468b97 Compare November 15, 2021 06:18

CaoE force-pushed the bf16_nn4 branch 2 times, most recently from 7985dde to daba39a Compare December 17, 2021 06:51

CaoE force-pushed the bf16_nn4 branch 2 times, most recently from f1267f0 to 6698e91 Compare December 21, 2021 01:32

CaoE force-pushed the bf16_nn4 branch from 6698e91 to 317f488 Compare December 28, 2021 09:53

CaoE force-pushed the bf16_nn4 branch from 317f488 to ddb4368 Compare January 5, 2022 06:04

CaoE force-pushed the bf16_nn4 branch 2 times, most recently from 1f7f3ab to 9fc52bc Compare January 24, 2022 03:02

CaoE added 2 commits March 2, 2022 19:52

add BFloat16 support for logsigmoid, hardsigmoid, hardshrink, softshr…

46ece24

…ink, hardswish, and softplus on CPU

optimize the performance of softshrink on CPU

2b830af

CaoE force-pushed the bf16_nn4 branch from 9fc52bc to 2b830af Compare March 2, 2022 11:53

frank-wei self-requested a review March 15, 2022 17:01

suo removed the ciflow/default label Mar 22, 2022

frank-wei approved these changes Apr 4, 2022

View reviewed changes

pytorchmergebot closed this in 77c7a50 Apr 4, 2022

frank-wei added the intel This tag is for PR from Intel label Jun 3, 2022

WBobby mentioned this pull request Aug 17, 2022

Add ROCm5.2.3/AMDGPU support for PyTorch WBobby/pytorch#2

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add BFloat16 support for logsigmoid, hardsigmoid, hardshrink, softshrink, hardswish and softplus on CPU #63134

Add BFloat16 support for logsigmoid, hardsigmoid, hardshrink, softshrink, hardswish and softplus on CPU #63134

Uh oh!

CaoE commented Aug 12, 2021 •

edited

Loading

Uh oh!

facebook-github-bot commented Aug 12, 2021 •

edited

Loading

Uh oh!

VitalyFedyunin commented Aug 15, 2021

Uh oh!

CaoE commented Aug 18, 2021 •

edited

Loading

Uh oh!

codecov bot commented Aug 27, 2021 •

edited

Loading

Uh oh!

pytorch-probot bot commented Oct 8, 2021 •

edited

Loading

⚛️ CI Flow

Uh oh!

CaoE commented Oct 19, 2021 •

edited

Loading

Uh oh!

CaoE commented Dec 28, 2021

Uh oh!

pytorchbot commented Jan 24, 2022

Uh oh!

facebook-github-bot commented Mar 15, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Add BFloat16 support for logsigmoid, hardsigmoid, hardshrink, softshrink, hardswish and softplus on CPU #63134

Add BFloat16 support for logsigmoid, hardsigmoid, hardshrink, softshrink, hardswish and softplus on CPU #63134

Uh oh!

Conversation

CaoE commented Aug 12, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

facebook-github-bot commented Aug 12, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful links

💊 CI failures summary and remediations

🕵️ 1 new failure recognized by patterns

linux-bionic-rocm4.5-py3.7 / test (distributed, 1, 1, linux.rocm.gpu) (1/1)

Uh oh!

VitalyFedyunin commented Aug 15, 2021

Uh oh!

CaoE commented Aug 18, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Aug 27, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

pytorch-probot bot commented Oct 8, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚛️ CI Flow

Uh oh!

CaoE commented Oct 19, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

CaoE commented Dec 28, 2021

Uh oh!

pytorchbot commented Jan 24, 2022

Uh oh!

facebook-github-bot commented Mar 15, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

CaoE commented Aug 12, 2021 •

edited

Loading

facebook-github-bot commented Aug 12, 2021 •

edited

Loading

CaoE commented Aug 18, 2021 •

edited

Loading

codecov bot commented Aug 27, 2021 •

edited

Loading

pytorch-probot bot commented Oct 8, 2021 •

edited

Loading

CaoE commented Oct 19, 2021 •

edited

Loading