Gradient tests which were dependant only on cached_session#1173
Gradient tests which were dependant only on cached_session#1173Oceania2018 merged 3 commits intoSciSharp:masterfrom
Conversation
| var y = tf.identity(x); | ||
| var dy_indices = tf.placeholder(TF_DataType.TF_INT32); | ||
| var dy_values = tf.placeholder(TF_DataType.TF_FLOAT); | ||
| Tensor dy = new IndexedSlices(dy_values, dy_indices); |
There was a problem hiding this comment.
@Wanglongzhi2001 this one fails because IndexedSlices has null shape. Should I create an issue?
There was a problem hiding this comment.
After comparing the implementations of TensorFlow and TensorFlow.NET, it seems that TensorFlow.NET just simpley does not throw error when dense_shape is null.
https://github.com/tensorflow/tensorflow/blob/704c1918a05574215d0499a74905f9a43fab01d5/tensorflow/python/framework/indexed_slices.py#L435-L438
I'm sorry that I'm not very familiar with tf1, is it correct to use IndexSlice in your example? Or users need to manually pass in the dense_shape parameter? After all, in most case in tensorflow, dense_shape is manually passed in.
https://github.com/tensorflow/tensorflow/blob/704c1918a05574215d0499a74905f9a43fab01d5/tensorflow/python/client/session_test.py#L884-L920
There was a problem hiding this comment.
I just ported an existing test from TensorFlow. You may that this code was commented before.
The problem is in conversion from IndexedSlices to Tensor. Python TensorFlow doesn't have any restrictions on gradients interface and it looks like it converts IndexedSlices internally in a different way. I'm trying to run Python TensorFlow tests now to compare, but it is not easy to configure them for the first time.
There was a problem hiding this comment.
This is what's really going on here:
IndexedSlices actually is a CompositeTensor.
If we go to _GradientsHelper we'll find out that it shoud be able to convert CompositeTensor to Tensor by its own rules.
https://github.com/tensorflow/tensorflow/blob/8fcc1b465a354c2277dc84b7503443013c693541/tensorflow/python/ops/gradients_util.py#L535
In other words gradients conditionally should support the following signature:
public (Tensor | CompositeTensor)[] gradients((Tensor | CompositeTensor)[] ys,
(Tensor | CompositeTensor)[] xs,
(Tensor | CompositeTensor)[] grad_ys = null,
...);
Where (Tensor | CompositeTensor) is an abstract union type.
However, it is not implemented yet.
The code is just skipped.
So, I just commented and marked the test as "TODO".
I can also leave a comment in
gradients if necessary.
There was a problem hiding this comment.
I'm agree with you. And it seems that the implementation of CompositeTensor is a relatively big feature. So it may not be implemented in the near future. Anyway, really thank you for your contribution. ^_^
| + rng.random(shape).astype(np.float32); | ||
| } | ||
|
|
||
| var a = functionOf(Array.Empty<Tensor>(), 3); |
There was a problem hiding this comment.
@Wanglongzhi2001 this one fails because created Tensors have null in their op field. Should I create an issue?
| // tf.Graph().as_default(); | ||
| // var x = tf.constant(1.0, shape: new long[] { 2, 2 }); | ||
| // var y = tf.constant(3.0, shape: new long[] { 3, 1 }); | ||
| // var grads = tf.gradients(new[] { y }, new[] { x }, unconnected_gradients: "zero"); |
There was a problem hiding this comment.
@Wanglongzhi2001 there's no unconnected_gradients parameter here yet.
|
LGTM @Oceania2018 |
3 participants
testAggregationMethodAccumulateN
testAggregationMethodAddN
testAggregationMethodTree
testSingletonIndexedSlices - broken
testDependentYs
testPartialDerivatives
testStopGradients - broken