implement double backwards for MaxPool3d #5328
Conversation
|
Could you run some benchmarks to see if the memory usage increase from changing MaxUnpool3d to not pack indices is significant? I think someone (@apaszke?) mentioned Resnet as a network that uses MaxUnpool3d; it would be nice to see if the memory usage increases significantly or not under resnet. edit: Ignore this comment, the memory usage is the same |
|
Oh, resnets use the 2d version, and I thought we’re modifying that. Will this make 3d consistent with the other pooling layers? |
|
Yeah this makes it more consistent. Also, I don't see any reason for this to use more memory than the old implementation. The way indices are being stored is changed, but we're still using the same number of bits in the tensor. |
zou3519
left a comment
zou3519
left a comment
There was a problem hiding this comment.
The math looks good to me. I had some minor comments.
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
zou3519
left a comment
zou3519
left a comment
There was a problem hiding this comment.
More nits about the naming that existed previously in the code (before your changes). Feel free to ignore
| /* loop over output */ | ||
| int64_t i, j, ti; | ||
| real *ip = input_p + k * itime * iwidth * iheight; | ||
| for (ti = 0; ti < otime; ti++) |
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
| for (y = start_h; y < end_h; y += dilationH) | ||
| { | ||
| for (x = 0; x < kernel_w; x++) | ||
| for (x = start_w; x < end_w; x += dilationW) |
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
| { | ||
| for (i = 0; i < oheight; i++) | ||
| { | ||
| for (j = 0; j < owidth; j++) |
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
| { | ||
| for (i = 0; i < iH; i++) | ||
| { | ||
| for (j = 0; j < iW; j++) |
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
| { | ||
| for (i = 0; i < iH; i++) | ||
| { | ||
| for (j = 0; j < iW; j++) |
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
4 participants
fixes #4497.
test plan: added double backwards check for maxpool3d tests