Update torch.set_default_dtype doc #41263

anjali411 · 2020-07-10T16:54:17Z

Stack from ghstack:

Update torch.set_default_dtype doc #41263 Update torch.set_default_dtype doc

Differential Revision: D22482989

[ghstack-poisoned]

gchanan · 2020-07-10T17:05:40Z

torch/__init__.py

    Example::
-
-        >>> torch.tensor([1.2, 3]).dtype           # initial default for floating point is torch.float32
+        >>> # initial default for floating point is torch.float


nit: this isn't a huge deal, but I think the previous wording was slightly better, because it matches what is returned for the next line. As it is now, people will wonder why torch.float and torch.float32 don't match exactly.

torch/__init__.py

gchanan · 2020-07-10T17:08:27Z

torch/__init__.py

+        >>> # initial default for floating point is torch.cfloat
+        >>> torch.tensor([1.2, 3j]).dtype
+        torch.complex64
+        >>> torch.set_default_tensor_type(torch.DoubleTensor)


I don't think we want to document this -- the type interface isn't complete (e.g. there's no quantized, XLA) -- it's really logically deprecated.

yeah that makes sense: updated to torch.set_default_dtype(torch.float64)

gchanan · 2020-07-10T17:12:02Z

torch/__init__.py

-    used as default floating point type for type inference in
-    :func:`torch.tensor`.
+    r"""Sets the default floating point dtype to :attr:`d`.
+    This type is also used:


this says "also used" but it's sort of defined by how it's used, i.e. "sets the default floating point dtype" doesn't really mean anything outside of how it' s used. I'd just change this to something like:
"This dtype is used as:"

gchanan · 2020-07-10T17:16:27Z

torch/__init__.py

+    1. As default floating point type for type inference in :func:`torch.tensor`.
+    2. To determine the default complex dtype. The default complex dtype is set to
+       ``torch.complex128`` if default floating point tensor type is ``torch.float64``,
+       else it's set to ``torch.complex64``


nit: else -> otherwise.

is it not true that this is the corresponding complex dtype for the float default?

that's true

gchanan · 2020-07-10T17:32:33Z

torch/__init__.py

-    :func:`torch.tensor`.
+    r"""Sets the default floating point dtype to :attr:`d`.
+    This type is also used:
+    1. As default floating point type for type inference in :func:`torch.tensor`.


this could be a little clearer (I know it's coming from the prior writeup). But "default floating point type" isn't really defined yet. Maybe something like:
"As the dtype inferred for python floats" (and similar for complex).

[ghstack-poisoned]

gchanan · 2020-07-10T17:58:30Z

torch/__init__.py

+    r"""Sets the default floating point dtype to :attr:`d`.
+    This dtype is:
+    1. The inferred dtype for python floats in :func:`torch.tensor`.
+    2. Used to determine the default complex dtype. The default complex dtype is set to


same issue with 1 -- this should say something like "the inferred dtype for python complex numbers"

does this sound better?

Used to infer dtype for python complex numbers. The default complex dtype is set to ``torch.complex128`` if default floating point dtype is ``torch.float64``, otherwise it's set to ``torch.complex64``

gchanan · 2020-07-10T17:59:16Z

torch/__init__.py

+    1. The inferred dtype for python floats in :func:`torch.tensor`.
+    2. Used to determine the default complex dtype. The default complex dtype is set to
+       ``torch.complex128`` if default floating point tensor type is ``torch.float64``,
+       otherwise it's set to ``torch.complex64``


I'm curious: why did you word it this way instead of saying something about the corresponding float dtype?

The only reason why I didn't it frame it that way is because we don't have a complex dtype corresponding to torch.bfloat16, torch.float16 (technically we do but the support for ComplexHalf is almost non-existing)

[ghstack-poisoned]

ghstack-source-id: 736011b Pull Request resolved: #41263

dr-ci · 2020-07-11T00:02:52Z

💊 CI failures summary and remediations

As of commit 2030496 (more details on the Dr. CI page):

2/2 failures possibly* introduced in this PR
- 2/2 non-CircleCI failure(s)

ci.pytorch.org: 2 failed

Failed: pr/pytorch-linux-xenial-rocm3.5.1-py3.6
Failed: pr/caffe2-pytorch-linux-xenial-rocm3.5.1-py3.6-test

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker or post in the (internal) Dr. CI Users group.

See how this bot performed.

This comment has been revised 1 time.

mruberry · 2020-07-11T07:01:59Z

torch/__init__.py

+    This dtype is:
+    1. The inferred dtype for python floats in :func:`torch.tensor`.
+    2. Used to infer dtype for python complex numbers. The default complex dtype is set to
+       ``torch.complex128`` if default floating point dtype is ``torch.float64``,


"if the default..."

mruberry · 2020-07-11T07:04:40Z

torch/__init__.py

-    :func:`torch.tensor`.
+    r"""Sets the default floating point dtype to :attr:`d`.
+    This dtype is:
+    1. The inferred dtype for python floats in :func:`torch.tensor`.


This isn't quite right since it's the inferred type for Python floats in other contexts, too:

t = torch.tensor(5) torch.set_default_dtype(torch.double) (t + 5.).dtype : torch.float64 torch.full((2,), 2.).dtype : torch.float64

I think this falls under the torch.tensor case since when you add a scalar to a tensor, we first make a zero dim tensor containing the scalar (hence the dtype of this new tensor in this case will be torch.float64

That's true in the first case but also an implementation detail I wouldn't expect a user to be be aware of. In the second case I don't think torch.tensor() is involved?

mruberry · 2020-07-11T07:08:35Z

torch/__init__.py

+    r"""Sets the default floating point dtype to :attr:`d`.
+    This dtype is:
+    1. The inferred dtype for python floats in :func:`torch.tensor`.
+    2. Used to infer dtype for python complex numbers. The default complex dtype is set to


You may want to make this a note for clarity:

"This setting also determines the inferred dtype of complex numbers. If the default floating point dtype is torch.float64 then complex numbers are inferred to have a dtype of torch.complex128, otherwise they are assumed to have a dtype of torch.complex64."

mruberry · 2020-07-11T07:11:38Z

torch/__init__.py

+       ``torch.complex128`` if default floating point dtype is ``torch.float64``,
+       otherwise it's set to ``torch.complex64``

    The default floating point dtype is initially ``torch.float32``.


This might also be noteworthy?

mruberry · 2020-07-11T07:12:33Z

torch/__init__.py

    Example::
-
-        >>> torch.tensor([1.2, 3]).dtype           # initial default for floating point is torch.float32
+        >>> # initial default for floating point is torch.float32


Following the above comment, maybe it'd be nice to have an example that isn't torch.tensor based?

facebook-github-bot · 2020-07-14T16:14:17Z

@anjali411 merged this pull request in e888c3b.

Summary: Pull Request resolved: #41263 Test Plan: Imported from OSS Differential Revision: D22482989 Pulled By: anjali411 fbshipit-source-id: 2aadfbb84bbab66f3111970734a37ba74d817ffd

Update torch.set_default_dtype doc

5917148

[ghstack-poisoned]

anjali411 requested review from gchanan and mruberry July 10, 2020 16:56

gchanan requested changes Jul 10, 2020

View reviewed changes

Update on "Update torch.set_default_dtype doc"

e002a25

[ghstack-poisoned]

anjali411 requested a review from gchanan July 10, 2020 17:56

gchanan reviewed Jul 10, 2020

View reviewed changes

gchanan approved these changes Jul 10, 2020

View reviewed changes

Update on "Update torch.set_default_dtype doc"

2030496

[ghstack-poisoned]

anjali411 added a commit that referenced this pull request Jul 10, 2020

Update torch.set_default_dtype doc

f710ef7

ghstack-source-id: 736011b Pull Request resolved: #41263

anjali411 mentioned this pull request Jul 10, 2020

[v1.6.0] Release Tracker #40472

Closed

mruberry reviewed Jul 11, 2020

View reviewed changes

facebook-github-bot closed this in e888c3b Jul 14, 2020

facebook-github-bot added the merged label Jul 14, 2020

facebook-github-bot deleted the gh/anjali411/43/head branch July 18, 2020 14:16

mruberry added the Merged label Oct 28, 2020

Update torch.set_default_dtype doc #41263

Update torch.set_default_dtype doc #41263

Uh oh!

Conversation

anjali411 commented Jul 10, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dr-ci bot commented Jul 11, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💊 CI failures summary and remediations

ci.pytorch.org: 2 failed

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mruberry Jul 11, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented Jul 14, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

anjali411 commented Jul 10, 2020 •

edited

Loading

dr-ci bot commented Jul 11, 2020 •

edited

Loading

mruberry Jul 11, 2020 •

edited

Loading