Implement Tensor.__cuda_array_interface__. #11984

asford · 2018-09-22T20:59:57Z

Implements #11914, cc: @ezyang

Implements __cuda_array_interface__ for non-sparse cuda tensors,
providing compatibility with numba (and other cuda projects...).

Adds numba installation to the xenial-cuda9 jenkins test environments via direct installation in .jenkins/pytorch/test.sh and numba-oriented test suite in test/test_numba_integration.py.

See interface reference at:
https://numba.pydata.org/numba-doc/latest/cuda/cuda_array_interface.html

Add baseline test of python-level __cuda_array_interface__ for torch.Tensor. See interface reference at: https://numba.pydata.org/numba-doc/latest/cuda/cuda_array_interface.html

torch/tensor.py

+                "Use var.detach().__cuda_array_interface__ instead."
+            )
+
+        typestr = {


test/test_torch.py

            self.assertFalse(isinstance(idx, int))
            self.assertEqual(x[idx], x[int(idx)])

+    @unittest.skipIf(not TEST_NUMPY, "Numpy not found")


ezyang

Lack of actual numba test is blocker in my mind. I have a few other doc nits which I'll mention below.

torch/tensor.py

+        # hasattr(cpu_tensor, "__cuda_array_interface__") is False.
+        if not self.device.type == "cuda":
+            raise AttributeError(
+                "Not on cuda device, use Tensor.cuda() first: %r" %


torch/tensor.py

+        if self.requires_grad:
+            raise RuntimeError(
+                "Can't get __cuda_array_interface__ on Variable that requires grad. "
+                "Use var.detach().__cuda_array_interface__ instead."


Add initial pytorch/numba integration tests. Expand test common/common_cuda configuration to support optional numba installation. Add test/test_numba_integration.py, with baseline integration test cases.

ezyang · 2018-09-25T02:36:33Z

The dir test failure is real; you probably should filter out __cuda_array_interface__ from it. (Calling hasattr on a property executes the code.)

Install numba on xenial-cuda9 test runs, allowing integration tests.

test/test_autograd.py

Fix numba __cuda_array_interface__ integration check to resolve difference in py2/py3 hasattr behavior. Tidy documentation.

Update `torch.Tensor.__dir__` to only include `__cuda_array_interface__` if property will be present (i.e. a dense, cuda tensor). Preserved RuntimeError behavior of cuda tensors that require grad.

asford · 2018-09-26T01:34:13Z

@ezyang

This is ready for another round of review when you get the chance. I've:

Expanded the test suite to cover baseline numba integration and added numba installation to xenial-cuda9 py2 & py3 test configurations.

This could be expanded into a full end-to-end test, covering dispatch of a jit-compiled numba function over input torch.Tensors and/or a full test of autograd integration via numba-jitted forward/backward passes. I'm not entirely sure about the project's philosopy of what should live in the test layer vs examples, and I'd be happy to hear your thoughts.
Tidied up the cuda_array_interface dir behavior.
Tidied up the cuda_array_interface error messages.

I'm a bit concerned about adding the numba installation to your test runner, rather than folding the numba package into your test image. I've pinned to installed version to attempt to mitigate the risk of a dependency on an external package, but this still adds ~400mb of external package access to the test run.

ezyang · 2018-09-26T03:49:16Z

I'm a bit concerned about adding the numba installation to your test runner, rather than folding the numba package into your test image. I've pinned to installed version to attempt to mitigate the risk of a dependency on an external package, but this still adds ~400mb of external package access to the test run.

Well, here's another crazy idea. We don't necessarily need to test against numba; we just need to test against something that supports cuda_array_interface. There's no reason we couldn't make PyTorch support cuda_array_interface. So what we could do is (1) add read-in support for cuda_array_interface to PyTorch, and then (2) test that going to a dummy object with cuda_array_interface and then back again works. Then as long as we don't have correlated errors between the decoder and encoder, you have a reasonably robust test.

ezyang · 2018-09-26T04:12:30Z

Other possibility: we have the numba test but don't run it in CI (only run it on a nightly build, or something.)

avmgithub · 2018-09-26T13:43:11Z

@asford Would you mind if I got a version of your changes to test on ppc64le before it gets merged ? Specially if its going to be a default test in run_test.py. I see you require numba=0.39 cudatoolkit=9.0 for x86 (EDITED) and looks like they are both available on ppc64le. I'll see if I can merge your changes locally and run the test. Appreciate it . CC: @hartb @sdmonov

avmgithub · 2018-09-26T14:21:09Z

@asford (EDITED) I had to install the torch/tensor.py into my environment and the test test_cuda_array_interface passes (it does not require numba). I installed numba and looks like the other test passes also.

asford · 2018-10-02T15:43:27Z

@avmgithub Great! Thinks for following up on that test. I was racking my brain trying to understand how it could be failing in that way.

asford · 2018-10-02T15:54:11Z

@ezyang

I'm comfortable with either solution you've proposed, mainly trying to avoid having a drive-by contribution that makes your testing system more fragile. I've been burned a few times by having dependencies on large external downloads in CI and think it's ideal to avoid it if possible.

Having this run as a nightly may be best if you already have the infrastructure in place for that. Are the pytorch-examples run on a periodic basis as part of your CI?

Having the ability to read data from numba seems generally useful. You can approximate this now by creating a numbabased view of the "to" tensor and then copy data via numba from the "from" data buffer, but executing this in torch is more natural. Given that it both improves the testing situation and improves the interface I'm in favor of this approach.

Adding read support for __cuda_array_interface__ components makes this feature very similar to the existing dlpack utilities. I'm going to quickly review that implementation to make sure I'm not adding a bunch of duplicate logic.

ezyang · 2018-10-12T16:13:19Z

Having this run as a nightly may be best if you already have the infrastructure in place for that. Are the pytorch-examples run on a periodic basis as part of your CI?

Yes. @pjh5, do you think we could install numba when we do nightly build and tests?

Adding read support for cuda_array_interface components makes this feature very similar to the existing dlpack utilities. I'm going to quickly review that implementation to make sure I'm not adding a bunch of duplicate logic.

SGTM. In the meantime, I'm going to merge this without the CI change. Existence of test LGTM.

facebook-github-bot

ezyang is landing this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

asford · 2018-10-12T17:58:17Z

+1 @ezyang Thanks for driving this. Should we open a follow-up issue for the nightly?

ezyang · 2018-10-12T19:05:43Z

Yes please.

…

On October 12, 2018 10:59:21 AM PDT, Alex Ford ***@***.***> wrote: +1 @ezyang Thanks for driving this. Should we open a follow-up issue for the nightly? -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #11984 (comment)

-- Sent from my Android device with K-9 Mail. Please excuse my brevity.

Alex Ford added 2 commits September 22, 2018 13:49

Add Tensor.__cuda_array_interface__ test.

00dff9c

Add baseline test of python-level __cuda_array_interface__ for torch.Tensor. See interface reference at: https://numba.pydata.org/numba-doc/latest/cuda/cuda_array_interface.html

Implement Tensor.__cuda_array_interface__

67f8984

asford requested review from apaszke, colesbury, ezyang, gchanan, soumith and zdevito as code owners September 22, 2018 20:59

Delint __cuda_array_interface__ implementation.

e7c7a45

ezyang reviewed Sep 23, 2018

View reviewed changes

torch/tensor.py

"Use var.detach().__cuda_array_interface__ instead."

)

typestr = {

This comment was marked as off-topic.

Sign in to view

This comment was marked as off-topic.

Sign in to view

This comment was marked as off-topic.

Sign in to view

ezyang reviewed Sep 23, 2018

View reviewed changes

ezyang requested changes Sep 23, 2018

View reviewed changes

ezyang reviewed Sep 23, 2018

View reviewed changes

Alex Ford added 4 commits September 23, 2018 11:13

Update cuda array interface exception messages.

7945bd0

Add byte order to cuda array typestr.

ad5d168

Add initial numba integration tests.

6562418

Add initial pytorch/numba integration tests. Expand test common/common_cuda configuration to support optional numba installation. Add test/test_numba_integration.py, with baseline integration test cases.

Tidy numba tests, add conversion error cases.

c773698

Fix test error in test_autograd.

0ab1171

asford force-pushed the asford/cuda_array_interface branch from 746a582 to 999b39f Compare September 25, 2018 05:59

Alex Ford added 2 commits September 24, 2018 23:08

Enable numba_integration test suite.

9cec76c

Add numba to xenial-cuda9 tests.

655a016

Install numba on xenial-cuda9 test runs, allowing integration tests.

asford force-pushed the asford/cuda_array_interface branch 3 times, most recently from 2a33110 to 3a2012c Compare September 25, 2018 18:53

apaszke reviewed Sep 25, 2018

View reviewed changes

test/test_autograd.py Outdated Show resolved Hide resolved

Alex Ford added 3 commits September 25, 2018 13:55

Fix py2/3 behavior delta in numba integration test.

cb87d7e

Fix numba __cuda_array_interface__ integration check to resolve difference in py2/py3 hasattr behavior. Tidy documentation.

Delint.

d49bfc1

Fix __dir__ implementation for __cuda_array_interface__.

fe24ad3

Update `torch.Tensor.__dir__` to only include `__cuda_array_interface__` if property will be present (i.e. a dense, cuda tensor). Preserved RuntimeError behavior of cuda tensors that require grad.

asford force-pushed the asford/cuda_array_interface branch from 39e968e to fe24ad3 Compare September 25, 2018 20:56

Pin numba version in test environment.

d7f0b8a

ezyang added 2 commits October 12, 2018 09:11

Don't install numba

8823c3b

clear one more newline

2f79b64

facebook-github-bot reviewed Oct 12, 2018

View reviewed changes

ezyang approved these changes Oct 12, 2018

View reviewed changes

asford mentioned this pull request Oct 12, 2018

Numba support should be included in nightly tests. #12614

Closed

facebook-github-bot closed this in 7a1b668 Oct 12, 2018

stuartarchibald mentioned this pull request Oct 19, 2018

Compatibility with CuPy numba/numba#3422

Closed

matthieuheitz mentioned this pull request Jan 8, 2019

Auto-convert GPU arrays that support the __cuda_array_interface__ protocol #15601

Closed

ezyang added open source merged labels Jun 24, 2019

casperdcl mentioned this pull request Feb 1, 2021

add __cuda_array_interface__ AMYPAD/CuVec#4

Closed

Implement Tensor.__cuda_array_interface__. #11984

Implement Tensor.__cuda_array_interface__. #11984

Uh oh!

Conversation

asford commented Sep 22, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

ezyang left a comment

Choose a reason for hiding this comment

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

ezyang commented Sep 25, 2018

Uh oh!

Uh oh!

asford commented Sep 26, 2018

Uh oh!

ezyang commented Sep 26, 2018

Uh oh!

ezyang commented Sep 26, 2018

Uh oh!

avmgithub commented Sep 26, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

avmgithub commented Sep 26, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

asford commented Oct 2, 2018

Uh oh!

asford commented Oct 2, 2018

Uh oh!

ezyang commented Oct 12, 2018

Uh oh!

facebook-github-bot left a comment

Choose a reason for hiding this comment

Uh oh!

asford commented Oct 12, 2018

Uh oh!

ezyang commented Oct 12, 2018 via email

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

asford commented Sep 22, 2018 •

edited

Loading

avmgithub commented Sep 26, 2018 •

edited

Loading

avmgithub commented Sep 26, 2018 •

edited

Loading