Add include_self flag to scatter_reduce #74607
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
[ghstack-poisoned]
Contributor
🔗 Helpful links
💊 CI failures summary and remediationsAs of commit dd1cbe1 (more details on the Dr. CI page): 💚 💚 Looks good so far! There are no failures yet. 💚 💚 This comment was automatically generated by Dr. CI (expand for details).Please report bugs/suggestions to the (internal) Dr. CI Users group. |
This was referenced Mar 23, 2022
`Tensor.scatter_reduce_(int64 dim, Tensor index, Tensor src, str reduce, *, bool include_input=True)`
- Add an argument `include_input` which indicates whether the value in the `self` Tensor at a given position is included in the reduction with the elements from `src` scattered to that position. For
`I_self = {all indices of self}`
`I_src= {all indices of src}`
`S = {indices of self modified by scatter}`
`self_indices_to_src_indices : I_self --> I_src` maps indices in `self` to a tuple of indices in `src` scattered to that index of `self`
Then for `s ∈ S` and `t ∈ I\S` when `include_input=False`
`self[s] = reduction_op(src[self_indices_to_src_indices[s]])`
`self[t] = self[t]`
and when `include_input=True` (regular scatter(reduce=op) behavior)
`self[s] = reduction_op(self[s], src[self_indices_to_src_indices[s]])`
`self[t] = self[t]`
The [`optional_out` case of pytorch_scatter.scatter](https://github.com/rusty1s/pytorch_scatter/blob/master/csrc/scatter.cpp#L32) can then be handled by
`torch.zeros(shape).scatter_reduce_(dim, index, src, reduce, include_input=False)`
[ghstack-poisoned]
`Tensor.scatter_reduce_(int64 dim, Tensor index, Tensor src, str reduce, *, bool include_input=True)`
- Add an argument `include_input` which indicates whether the value in the `self` Tensor at a given position is included in the reduction with the elements from `src` scattered to that position. For
`I_self = {all indices of self}`
`I_src= {all indices of src}`
`S = {indices of self modified by scatter}`
`self_indices_to_src_indices : I_self --> I_src` maps indices in `self` to a tuple of indices in `src` scattered to that index of `self`
Then for `s ∈ S` and `t ∈ I\S` when `include_input=False`
`self[s] = reduction_op(src[self_indices_to_src_indices[s]])`
`self[t] = self[t]`
and when `include_input=True` (regular scatter(reduce=op) behavior)
`self[s] = reduction_op(self[s], src[self_indices_to_src_indices[s]])`
`self[t] = self[t]`
The [`optional_out` case of pytorch_scatter.scatter](https://github.com/rusty1s/pytorch_scatter/blob/master/csrc/scatter.cpp#L32) can then be handled by
`torch.zeros(shape).scatter_reduce_(dim, index, src, reduce, include_input=False)`
[ghstack-poisoned]
`Tensor.scatter_reduce_(int64 dim, Tensor index, Tensor src, str reduce, *, bool include_input=True)`
- Add an argument `include_input` which indicates whether the value in the `self` Tensor at a given position is included in the reduction with the elements from `src` scattered to that position. For
`I_self = {all indices of self}`
`I_src= {all indices of src}`
`S = {indices of self modified by scatter}`
`self_indices_to_src_indices : I_self --> I_src` maps indices in `self` to a tuple of indices in `src` scattered to that index of `self`
Then for `s ∈ S` and `t ∈ I\S` when `include_input=False`
`self[s] = reduction_op(src[self_indices_to_src_indices[s]])`
`self[t] = self[t]`
and when `include_input=True` (regular scatter(reduce=op) behavior)
`self[s] = reduction_op(self[s], src[self_indices_to_src_indices[s]])`
`self[t] = self[t]`
The [`optional_out` case of pytorch_scatter.scatter](https://github.com/rusty1s/pytorch_scatter/blob/master/csrc/scatter.cpp#L32) can then be handled by
`torch.zeros(shape).scatter_reduce_(dim, index, src, reduce, include_input=False)`
[ghstack-poisoned]
`Tensor.scatter_reduce_(int64 dim, Tensor index, Tensor src, str reduce, *, bool include_input=True)`
- Add an argument `include_input` which indicates whether the value in the `self` Tensor at a given position is included in the reduction with the elements from `src` scattered to that position. For
`I_self = {all indices of self}`
`I_src= {all indices of src}`
`S = {indices of self modified by scatter}`
`self_indices_to_src_indices : I_self --> I_src` maps indices in `self` to a tuple of indices in `src` scattered to that index of `self`
Then for `s ∈ S` and `t ∈ I\S` when `include_input=False`
`self[s] = reduction_op(src[self_indices_to_src_indices[s]])`
`self[t] = self[t]`
and when `include_input=True` (regular scatter(reduce=op) behavior)
`self[s] = reduction_op(self[s], src[self_indices_to_src_indices[s]])`
`self[t] = self[t]`
The [`optional_out` case of pytorch_scatter.scatter](https://github.com/rusty1s/pytorch_scatter/blob/master/csrc/scatter.cpp#L32) can then be handled by
`torch.zeros(shape).scatter_reduce_(dim, index, src, reduce, include_input=False)`
[ghstack-poisoned]
`Tensor.scatter_reduce_(int64 dim, Tensor index, Tensor src, str reduce, *, bool include_input=True)`
- Add an argument `include_input` which indicates whether the value in the `self` Tensor at a given position is included in the reduction with the elements from `src` scattered to that position. For
`I_self = {all indices of self}`
`I_src= {all indices of src}`
`S = {indices of self modified by scatter}`
`self_indices_to_src_indices : I_self --> I_src` maps indices in `self` to a tuple of indices in `src` scattered to that index of `self`
Then for `s ∈ S` and `t ∈ I\S` when `include_input=False`
`self[s] = reduction_op(src[self_indices_to_src_indices[s]])`
`self[t] = self[t]`
and when `include_input=True` (regular scatter(reduce=op) behavior)
`self[s] = reduction_op(self[s], src[self_indices_to_src_indices[s]])`
`self[t] = self[t]`
The [`optional_out` case of pytorch_scatter.scatter](https://github.com/rusty1s/pytorch_scatter/blob/master/csrc/scatter.cpp#L32) can then be handled by
`torch.zeros(shape).scatter_reduce_(dim, index, src, reduce, include_input=False)`
[ghstack-poisoned]
`Tensor.scatter_reduce_(int64 dim, Tensor index, Tensor src, str reduce, *, bool include_input=True)`
- Add an argument `include_input` which indicates whether the value in the `self` Tensor at a given position is included in the reduction with the elements from `src` scattered to that position. For
`I_self = {all indices of self}`
`I_src= {all indices of src}`
`S = {indices of self modified by scatter}`
`self_indices_to_src_indices : I_self --> I_src` maps indices in `self` to a tuple of indices in `src` scattered to that index of `self`
Then for `s ∈ S` and `t ∈ I\S` when `include_input=False`
`self[s] = reduction_op(src[self_indices_to_src_indices[s]])`
`self[t] = self[t]`
and when `include_input=True` (regular scatter(reduce=op) behavior)
`self[s] = reduction_op(self[s], src[self_indices_to_src_indices[s]])`
`self[t] = self[t]`
The [`optional_out` case of pytorch_scatter.scatter](https://github.com/rusty1s/pytorch_scatter/blob/master/csrc/scatter.cpp#L32) can then be handled by
`torch.zeros(shape).scatter_reduce_(dim, index, src, reduce, include_input=False)`
[ghstack-poisoned]
`Tensor.scatter_reduce_(int64 dim, Tensor index, Tensor src, str reduce, *, bool include_self=True)`
- Add an argument `include_self` which indicates whether the value in the `self` Tensor at a given position is included in the reduction with the elements from `src` scattered to that position. For
`I_self = {all indices of self}`
`I_src= {all indices of src}`
`S = {indices of self modified by scatter}`
`self_indices_to_src_indices : I_self --> I_src` maps indices in `self` to a tuple of indices in `src` scattered to that index of `self`
Then for `s ∈ S` and `t ∈ I\S` when `include_input=False`
`self[s] = reduction_op(src[self_indices_to_src_indices[s]])`
`self[t] = self[t]`
and when `include_input=True` (regular scatter(reduce=op) behavior)
`self[s] = reduction_op(self[s], src[self_indices_to_src_indices[s]])`
`self[t] = self[t]`
The [`optional_out` case of pytorch_scatter.scatter](https://github.com/rusty1s/pytorch_scatter/blob/master/csrc/scatter.cpp#L32) can then be handled by
`torch.zeros(shape).scatter_reduce_(dim, index, src, reduce, include_input=False)`
[ghstack-poisoned]
`Tensor.scatter_reduce_(int64 dim, Tensor index, Tensor src, str reduce, *, bool include_self=True)`
- Add an argument `include_self` which indicates whether the value in the `self` Tensor at a given position is included in the reduction with the elements from `src` scattered to that position. For
`I_self = {all indices of self}`
`I_src= {all indices of src}`
`S = {indices of self modified by scatter}`
`self_indices_to_src_indices : I_self --> I_src` maps indices in `self` to a tuple of indices in `src` scattered to that index of `self`
Then for `s ∈ S` and `t ∈ I\S` when `include_input=False`
`self[s] = reduction_op(src[self_indices_to_src_indices[s]])`
`self[t] = self[t]`
and when `include_input=True` (regular scatter(reduce=op) behavior)
`self[s] = reduction_op(self[s], src[self_indices_to_src_indices[s]])`
`self[t] = self[t]`
The [`optional_out` case of pytorch_scatter.scatter](https://github.com/rusty1s/pytorch_scatter/blob/master/csrc/scatter.cpp#L32) can then be handled by
`torch.zeros(shape).scatter_reduce_(dim, index, src, reduce, include_input=False)`
[ghstack-poisoned]
`Tensor.scatter_reduce_(int64 dim, Tensor index, Tensor src, str reduce, *, bool include_self=True)`
- Add an argument `include_self` which indicates whether the value in the `self` Tensor at a given position is included in the reduction with the elements from `src` scattered to that position. For
`I_self = {all indices of self}`
`I_src= {all indices of src}`
`S = {indices of self modified by scatter}`
`self_indices_to_src_indices : I_self --> I_src` maps indices in `self` to a tuple of indices in `src` scattered to that index of `self`
Then for `s ∈ S` and `t ∈ I\S` when `include_input=False`
`self[s] = reduction_op(src[self_indices_to_src_indices[s]])`
`self[t] = self[t]`
and when `include_input=True` (regular scatter(reduce=op) behavior)
`self[s] = reduction_op(self[s], src[self_indices_to_src_indices[s]])`
`self[t] = self[t]`
The [`optional_out` case of pytorch_scatter.scatter](https://github.com/rusty1s/pytorch_scatter/blob/master/csrc/scatter.cpp#L32) can then be handled by
`torch.zeros(shape).scatter_reduce_(dim, index, src, reduce, include_input=False)`
Next step is to move this logic into the kernel
[ghstack-poisoned]
cpuhrsch
approved these changes
Apr 1, 2022
`Tensor.scatter_reduce_(int64 dim, Tensor index, Tensor src, str reduce, *, bool include_self=True)`
- Add an argument `include_self` which indicates whether the value in the `self` Tensor at a given position is included in the reduction with the elements from `src` scattered to that position. For
`I_self = {all indices of self}`
`I_src= {all indices of src}`
`S = {indices of self modified by scatter}`
`self_indices_to_src_indices : I_self --> I_src` maps indices in `self` to a tuple of indices in `src` scattered to that index of `self`
Then for `s ∈ S` and `t ∈ I\S` when `include_input=False`
`self[s] = reduction_op(src[self_indices_to_src_indices[s]])`
`self[t] = self[t]`
and when `include_input=True` (regular scatter(reduce=op) behavior)
`self[s] = reduction_op(self[s], src[self_indices_to_src_indices[s]])`
`self[t] = self[t]`
The [`optional_out` case of pytorch_scatter.scatter](https://github.com/rusty1s/pytorch_scatter/blob/master/csrc/scatter.cpp#L32) can then be handled by
`torch.zeros(shape).scatter_reduce_(dim, index, src, reduce, include_input=False)`
Next step is to move this logic into the kernel
[ghstack-poisoned]
Contributor
Author
|
@pytorchbot merge this please |
Collaborator
|
Merge failed due to Matched rule superuser, but it was not reviewed yet by any of:simonhollis,davidxili,xinyang0,Jack-Khuu,sidneyfletcher, ... |
Contributor
Author
|
@pytorchbot merge this please |
Collaborator
|
Merge failed due to Matched rule superuser, but it was not reviewed yet by any of:shz117,chaekit,frankseide,anirbanraywork,kavoor, ... |
Contributor
Author
|
@pytorchbot merge this |
Collaborator
|
Merge failed due to Matched rule superuser, but it was not reviewed yet by any of:lessw2020,bilalsal,sluks,brianjo,wenleix, ... |
Contributor
Author
|
@pytorchbot merge this |
Collaborator
|
Merge failed due to Matched rule superuser, but it was not reviewed yet by any of:z-a-f,kit1980,jnkwok1,tktrungna,janeyx99, ... |
Contributor
|
@pytorchbot merge this |
Collaborator
|
Merge failed due to Matched rule superuser, but it was not reviewed yet by any of:andrewconnors,aazzolini,larryliu0820,ziky90,bradleyhd, ... |
Contributor
|
@pytorchbot merge this |
Contributor
|
Hey @mikaylagawarecki. |
facebook-github-bot
pushed a commit
that referenced
this pull request
Apr 7, 2022
Summary: Pull Request resolved: #74607 Approved by: https://github.com/cpuhrsch Test Plan: contbuild & OSS CI, see https://hud.pytorch.org/commit/pytorch/pytorch/e9a8e6f74ac037ed3a16b99d0bd48bdaafc73825 Reviewed By: b0noI Differential Revision: D35404309 Pulled By: mikaylagawarecki fbshipit-source-id: 8cf9158b04344fc583dae80b6829019713f5366d
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Stack from ghstack:
Tensor.scatter_reduce_(int64 dim, Tensor index, Tensor src, str reduce, *, bool include_self=True)include_selfwhich indicates whether the value in theselfTensor at a given position is included in the reduction with the elements fromsrcscattered to that position. ForI_self = {all indices of self}I_src= {all indices of src}S = {indices of self modified by scatter}self_indices_to_src_indices : I_self --> I_srcmaps indices inselfto a tuple of indices insrcscattered to that index ofselfThen for
s ∈ Sandt ∈ I\Swheninclude_input=Falseself[s] = reduction_op(src[self_indices_to_src_indices[s]])self[t] = self[t]and when
include_input=True(regular scatter(reduce=op) behavior)self[s] = reduction_op(self[s], src[self_indices_to_src_indices[s]])self[t] = self[t]The
optional_outcase of pytorch_scatter.scatter can then be handled bytorch.zeros(shape).scatter_reduce_(dim, index, src, reduce, include_input=False)Next step is to move this logic into the kernel