Repository navigation
Update _unpack_to_numpy function to convert JAX and PyTorch arrays to NumPy
#25887
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
8d6519c
7b992fa
d027cda
2cd9857
ec68363
3d294d1
a72a1ef
87ee115
e719e96
21cd9ce
cfe8edb
8ad4d1e
9acfb5b
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -2344,6 +2344,30 @@ def _picklable_class_constructor(mixin_class, fmt, attr_name, base_class): | |
| return cls.__new__(cls) | ||
|
|
||
|
|
||
| def _is_torch_array(x): | ||
| """Check if 'x' is a PyTorch Tensor.""" | ||
| try: | ||
| # we're intentionally not attempting to import torch. If somebody | ||
| # has created a torch array, torch should already be in sys.modules | ||
| return isinstance(x, sys.modules['torch'].Tensor) | ||
| except Exception: # TypeError, KeyError, AttributeError, maybe others? | ||
|
tacaswell marked this conversation as resolved.
Resolved
|
||
| # we're attempting to access attributes on imported modules which | ||
| # may have arbitrary user code, so we deliberately catch all exceptions | ||
| return False | ||
|
|
||
|
|
||
| def _is_jax_array(x): | ||
| """Check if 'x' is a JAX Array.""" | ||
| try: | ||
| # we're intentionally not attempting to import jax. If somebody | ||
| # has created a jax array, jax should already be in sys.modules | ||
| return isinstance(x, sys.modules['jax'].Array) | ||
| except Exception: # TypeError, KeyError, AttributeError, maybe others? | ||
| # we're attempting to access attributes on imported modules which | ||
| # may have arbitrary user code, so we deliberately catch all exceptions | ||
| return False | ||
|
|
||
|
|
||
| def _unpack_to_numpy(x): | ||
| """Internal helper to extract data from e.g. pandas and xarray objects.""" | ||
| if isinstance(x, np.ndarray): | ||
|
|
@@ -2358,6 +2382,12 @@ def _unpack_to_numpy(x): | |
| # so in this case we do not want to return a function | ||
| if isinstance(xtmp, np.ndarray): | ||
| return xtmp | ||
| if _is_torch_array(x) or _is_jax_array(x): | ||
| xtmp = x.__array__() | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is there any reason not to do
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think these are equivalent for the handled cases. While pandas claims you shouldn't call For me, either works.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @jakevdp @timhoffm What's your opinion about using if _is_torch_array(x):
xtmp = x.numpy(force=True) # or x.detach().cpu().numpy()
if _is_jax_array(x):
xtmp = x.__array__()
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Seems reasonable to use the
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @timhoffm To add more context, in JAX, cc @jakevdp.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. My hesitation about this is that memory overflows will get shunted to Matpolotlib, and then we will get the bug reports, whereas if people did There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
|
|
||
| # In case __array__() method does not return a numpy array in future | ||
| if isinstance(xtmp, np.ndarray): | ||
| return xtmp | ||
| return x | ||
|
|
||
|
|
||
|
|
||
Uh oh!
There was an error while loading. Please reload this page.