Support N-D tensors in Bilinear #5764

li-roy · 2018-03-14T02:15:41Z

Closes #5601.

goldsborough · 2018-03-14T04:17:56Z

@pytorchbot retest this please

li-roy · 2018-03-15T18:15:41Z

@pytorchbot retest this please

aten/src/ATen/native/Linear.cpp

+namespace at { namespace native {
+
+Tensor bilinear(const Tensor& input1, const Tensor& input2, const Tensor& weight, const Tensor& bias) {
+  auto b_input1 = input1.unsqueeze(-2).unsqueeze(-2);


torch/nn/modules/linear.py

    Shape:
        - Input: :math:`(N, \text{in1_features})`, :math:`(N, \text{in2_features})`
-        - Output: :math:`(N, \text{out_features})`
+          where :math:`*` means any number of additional dimensions. All but the last


aten/src/ATen/native/Linear.cpp

+  auto b_input2 = input2.unsqueeze(-2).unsqueeze(-1);
+
+  auto output = at::matmul(at::matmul(b_input1, weight), b_input2);
+  output = output.squeeze(-1).squeeze(-2).sum(-1);


aten/src/ATen/native/Linear.cpp

+
+Tensor bilinear(const Tensor& input1, const Tensor& input2, const Tensor& weight, const Tensor& bias) {
+  if (input1.dim() != input2.dim()) {
+    throw std::runtime_error("Inputs should have the same number of dimensions");


aten/src/ATen/native/Linear.cpp

+  }
+  for (int64_t i = 0; i < input1.dim() - 1; i++) {
+    if (input1.size(i) != input2.size(i)) {
+      throw std::runtime_error("Batch dimensions of inputs do not match");


aten/src/ATen/native/Linear.cpp

-  if (input1.dim() != input2.dim()) {
-    throw std::runtime_error("Inputs should have the same number of dimensions");
-  }
+  AT_ASSERT(input1.dim() == input2.dim(), "bilinear(): input dimensions do not match: got %d and %d",


aten/src/ATen/native/Linear.cpp

-    throw std::runtime_error("Bias sizes does not match weight size");
+    AT_ASSERT(input1.size(i) == input2.size(i),
+              "bilinear(): input batch dimensions do not match at dim %d: got %d and %d",
+              i, input1.size(i), input2.size(i));


li-roy · 2018-03-17T00:26:23Z

@pytorchbot retest this please

zou3519

🥇          🥇 🥇 🥇 🥇    🥇 🥇 🥇       🥇 🥇🥇 🥇 🥇 
🥇          🥇                 🥇          🥇    🥇   🥇 
🥇          🥇     🥇 🥇      🥇           🥇    🥇   🥇 
🥇          🥇        🥇       🥇          🥇    🥇   🥇 
🥇 🥇 🥇    🥇 🥇 🥇 🥇      🥇          🥇    🥇   🥇

soumith · 2018-03-17T15:57:54Z

I like richard's enthusiasm :D

li-roy force-pushed the bilinear branch from 0b24420 to afd9734 Compare March 14, 2018 22:30

Roy Li added 2 commits March 14, 2018 16:32

support n-d inputs in bilinear and move to aten

969dbfd

support n-d inputs in bilinear and move to aten

1e8917d

li-roy force-pushed the bilinear branch from afd9734 to 1e8917d Compare March 14, 2018 23:34

zou3519 reviewed Mar 15, 2018

View reviewed changes

aten/src/ATen/native/Linear.cpp

namespace at { namespace native {

Tensor bilinear(const Tensor& input1, const Tensor& input2, const Tensor& weight, const Tensor& bias) {

auto b_input1 = input1.unsqueeze(-2).unsqueeze(-2);

This comment was marked as off-topic.

Sign in to view

add asserts to bilinear inputs

e50d16b

zou3519 reviewed Mar 16, 2018

View reviewed changes

torch/nn/modules/linear.py

Shape:

- Input: :math:`(N, \text{in1_features})`, :math:`(N, \text{in2_features})`

- Output: :math:`(N, \text{out_features})`

where :math:`*` means any number of additional dimensions. All but the last

This comment was marked as off-topic.

Sign in to view

zou3519 reviewed Mar 16, 2018

View reviewed changes

aten/src/ATen/native/Linear.cpp Outdated

auto b_input2 = input2.unsqueeze(-2).unsqueeze(-1);

auto output = at::matmul(at::matmul(b_input1, weight), b_input2);

output = output.squeeze(-1).squeeze(-2).sum(-1);

This comment was marked as off-topic.

Sign in to view

zou3519 reviewed Mar 16, 2018

View reviewed changes

aten/src/ATen/native/Linear.cpp Outdated

}

for (int64_t i = 0; i < input1.dim() - 1; i++) {

if (input1.size(i) != input2.size(i)) {

throw std::runtime_error("Batch dimensions of inputs do not match");

This comment was marked as off-topic.

Sign in to view

address comments

9ede25a

zou3519 reviewed Mar 16, 2018

View reviewed changes

cast int64_t in asserts

263eb7c

zou3519 approved these changes Mar 17, 2018

View reviewed changes

soumith merged commit 1dcad08 into pytorch:master Mar 17, 2018

li-roy deleted the bilinear branch April 19, 2018 17:39

Support N-D tensors in Bilinear #5764

Support N-D tensors in Bilinear #5764

Uh oh!

Conversation

li-roy commented Mar 14, 2018

Uh oh!

goldsborough commented Mar 14, 2018

Uh oh!

li-roy commented Mar 15, 2018

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

li-roy commented Mar 17, 2018

Uh oh!

zou3519 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

soumith commented Mar 17, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

zou3519 left a comment •

edited

Loading