Remove getDeviceFromPtr and allocator from Type #21940
Conversation
pytorchbot
added
oncall: jit
Remove getDeviceFromPtr and allocator from Type gh-metadata: pytorch pytorch 21940 gh/li-roy/29/head
Remove getDeviceFromPtr and allocator from Type gh-metadata: pytorch pytorch 21940 gh/li-roy/29/head
Remove getDeviceFromPtr and allocator from Type gh-metadata: pytorch pytorch 21940 gh/li-roy/29/head
li-roy
requested review from
dzhulgakov,
ezyang,
gchanan and
smessmer
and removed request for
gchanan
tools/jit/gen_jit_dispatch.py
Outdated
|
|
||
|
|
||
| blacklisted_types = {'Storage', 'DimnameList?', 'ConstQuantizerPtr'} | ||
| blacklisted_types = {'Storage', 'DimnameList?', 'ConstQuantizerPtr', 'void*', 'std::function<void(void*)>'} |
There was a problem hiding this comment.
not really an issue with this diff, but doesn't the fact that you need to blacklist mean that the JIT is ignoring which functions aren't exposed to python? Which means if it's composed of all non-blacklisted-types I can use it in JIT but can't in frontend? That doesn't seem great.
| CPU: _frac_out_cpu | ||
| CUDA: _frac_out_cuda | ||
|
|
||
| - func: _from_blob(void* data, int[] sizes, int[] strides, std::function<void(void*)> deleter, *, ScalarType? dtype=None, Layout? layout=None, Device? device=None, bool? pin_memory=None) -> Tensor |
There was a problem hiding this comment.
This doesn't seem like the best solution, because native_functions is supposed to basically be python and now you need to teach codegen about these crazy things like std::function<void(void*)> deleter just to get type-dispatch.
Can we just hardcode this function in your registration table, i.e. just make it the equivalent of type? Because its' really just implementation detail stuff that doesn't need to be in native_functions (outside dispatch).
|
deferring to @gchanan for this one |
|
based on a chat with @li-roy I believe the current thinking is to put the minimal needed functionality onto pytorch/aten/src/ATen/Context.h Lines 61 to 72 in f8f583c |
|
The advantage being that we don't need to teach codegen about C++ specific things like |
Remove getDeviceFromPtr and allocator from Type gh-metadata: pytorch pytorch 21940 gh/li-roy/29/head
| } | ||
|
|
||
| virtual Device getDeviceFromPtr(void* data) const { | ||
| AT_ERROR("Cannot get default CUDA device without ATen_cuda library. ", CUDA_HELP); |
There was a problem hiding this comment.
this error message doesn't seem correct.
gchanan
left a comment
gchanan
left a comment
There was a problem hiding this comment.
just fix the error message, I think.
Remove getDeviceFromPtr and allocator from Type gh-metadata: pytorch pytorch 21940 gh/li-roy/29/head
Remove getDeviceFromPtr and allocator from Type gh-metadata: pytorch pytorch 21940 gh/li-roy/29/head
Summary: Pull Request resolved: pytorch/pytorch#21940 ghimport-source-id: 0a618878ae030f663b05662f83ac4b549a90ba29 Test Plan: Imported from OSS Differential Revision: D15893330 Pulled By: li-roy fbshipit-source-id: a3dfb6b4ed0c72f7f3efd00192fb63aabc9c5967
Summary: Pull Request resolved: pytorch#21940 ghimport-source-id: 0a61887 Test Plan: Imported from OSS Differential Revision: D15893330 Pulled By: li-roy fbshipit-source-id: a3dfb6b4ed0c72f7f3efd00192fb63aabc9c5967
Stack from ghstack:
In preparation of deleting Type. The getDeviceFromPtr logic is only used in from_blob, so I added _from_blob as a native function to dispatch the backend-based logic.
Differential Revision: D15893330