Skip to content

Let logspace support bfloat16 on both CPU and CUDA #44675

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
xuhdev wants to merge 1 commit into from
Closed

Let logspace support bfloat16 on both CPU and CUDA #44675

xuhdev wants to merge 1 commit into from

Conversation

@xuhdev
Copy link
Collaborator

@xuhdev xuhdev commented Sep 15, 2020

No description provided.

@xuhdev xuhdev requested a review from mruberry September 15, 2020 00:13
@xuhdev xuhdev changed the title Let logspace support bfloat16 and half (CUDA only) Let logspace support bfloat16 on both CPU and CUDA Sep 15, 2020
@dr-ci
Copy link

dr-ci bot commented Sep 15, 2020 •
edited
Loading

💊 CI failures summary and remediations

As of commit e28bb6d (more details on the Dr. CI page):

✅ None of the CI failures appear to be your fault 💚

❄️ 1 failure tentatively classified as flaky

but reruns have not yet been triggered to confirm:

See CircleCI build pytorch_linux_xenial_cuda10_2_cudnn7_py3_gcc7_test (1/1)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun) ❄️

Sep 15 03:55:37 ConnectionResetError: [Errno 104] Connection reset by peer 
Sep 15 03:55:37   File "/opt/conda/lib/python3.6/multiprocessing/connection.py", line 455, in accept 
Sep 15 03:55:37     deliver_challenge(c, self._authkey) 
Sep 15 03:55:37   File "/opt/conda/lib/python3.6/multiprocessing/connection.py", line 722, in deliver_challenge 
Sep 15 03:55:37     response = connection.recv_bytes(256)        # reject large message 
Sep 15 03:55:37   File "/opt/conda/lib/python3.6/multiprocessing/connection.py", line 216, in recv_bytes 
Sep 15 03:55:37     buf = self._recv_bytes(maxlength) 
Sep 15 03:55:37   File "/opt/conda/lib/python3.6/multiprocessing/connection.py", line 407, in _recv_bytes 
Sep 15 03:55:37     buf = self._recv(4) 
Sep 15 03:55:37   File "/opt/conda/lib/python3.6/multiprocessing/connection.py", line 379, in _recv 
Sep 15 03:55:37     chunk = read(handle, remaining) 
Sep 15 03:55:37 ConnectionResetError: [Errno 104] Connection reset by peer 
Sep 15 03:55:37 /opt/conda/lib/python3.6/multiprocessing/semaphore_tracker.py:143: UserWarning: semaphore_tracker: There appear to be 14 leaked semaphores to clean up at shutdown 
Sep 15 03:55:37   len(cache)) 
Sep 15 03:55:40 Process ErrorTrackingProcess-380: 
Sep 15 03:55:40 Traceback (most recent call last): 
Sep 15 03:55:40   File "/opt/conda/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap 
Sep 15 03:55:40     self.run() 
Sep 15 03:55:40   File "/var/lib/jenkins/workspace/test/test_dataloader.py", line 361, in run 
Sep 15 03:55:40     super(ErrorTrackingProcess, self).run() 
Sep 15 03:55:40   File "/opt/conda/lib/python3.6/multiprocessing/process.py", line 93, in run 
Sep 15 03:55:40     self._target(*self._args, **self._kwargs)
This comment was automatically generated by Dr. CI (expand for details).Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker or post in the (internal) Dr. CI Users group.

See how this bot performed.

This comment has been revised 1 time.

@ngimel ngimel added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Sep 15, 2020
mruberry
mruberry reviewed Sep 15, 2020
View reviewed changes
test/test_tensor_creation_ops.py
@unittest.skipIf(not TEST_NUMPY, "NumPy not found")
@precisionOverride({torch.float: 1e-6, torch.double: 1e-10})
@dtypes(torch.float, torch.double)
@dtypes(*torch.testing.get_all_fp_dtypes(include_half=False, include_bfloat16=False))
Copy link
Collaborator

@mruberry mruberry Sep 15, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is OK but you're not testing bfloat16 in this PR. You may need to add a separate, "easier" tests that compares bfloat16 vs. NumPy's logspace in fp32.

Copy link
Collaborator Author

@xuhdev xuhdev Sep 15, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to linspace, I think this test simply test its consistency with NumPy. The modification of the decorator of test_logspace below adds bfloat16 into the test.

Copy link
Collaborator

@mruberry mruberry Sep 15, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK; that seems reasonable.

mruberry
mruberry approved these changes Sep 15, 2020
View reviewed changes
Copy link
Collaborator

@mruberry mruberry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool!

facebook-github-bot
facebook-github-bot reviewed Sep 15, 2020
View reviewed changes
Copy link
Contributor

@facebook-github-bot facebook-github-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mruberry has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@xuhdev xuhdev deleted the logspace-types branch September 17, 2020 21:25
@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Sep 17, 2020

@mruberry merged this pull request in a153eaf.

xuzhao9 pushed a commit that referenced this pull request Sep 18, 2020
@xuhdev @xuzhao9
Let logspace support bfloat16 on both CPU and CUDA (#44675)
1b81d09 
Summary: Pull Request resolved: #44675

Reviewed By: ngimel

Differential Revision: D23710801

Pulled By: mruberry

fbshipit-source-id: 12d8e56f41bb635b500e89aaaf5df86a1795eb72
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@facebook-github-bot facebook-github-bot facebook-github-bot left review comments

@mruberry mruberry mruberry approved these changes

Assignees

No one assigned

Labels

Merged open source triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@xuhdev @facebook-github-bot @mruberry @ngimel @pytorchbot