-
Notifications
You must be signed in to change notification settings - Fork 75.2k
Fix crash of tf.sparse.split when axis is not a scalar. #53695
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
copybara-service
merged 3 commits into
tensorflow:master
from
yongtang:53660-tf.sparse.split-crash
Jan 10, 2022
Merged
Fix crash of tf.sparse.split when axis is not a scalar. #53695
copybara-service
merged 3 commits into
tensorflow:master
from
yongtang:53660-tf.sparse.split-crash
Jan 10, 2022
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This PR tries to fix the crash of tf.sparse.split when axis is not a scalar, as was raised in 53660. This PR fixes 53660. Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
…t a scalar Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
mihaimaruseac
requested changes
Jan 7, 2022
Collaborator
mihaimaruseac
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: Can you move it a few lines down near the other OP_REQUIRES please?
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Member
Author
|
Thanks @mihaimaruseac for the review. The PR has been updated. Please take a look. |
mihaimaruseac
approved these changes
Jan 7, 2022
mihaimaruseac
pushed a commit
that referenced
this pull request
Jan 25, 2022
PiperOrigin-RevId: 420811652 Change-Id: I83742482770ba0bf7c3ccd57508c40fb9cdbe2f7
mihaimaruseac
pushed a commit
that referenced
this pull request
Jan 25, 2022
PiperOrigin-RevId: 420811652 Change-Id: I83742482770ba0bf7c3ccd57508c40fb9cdbe2f7
mihaimaruseac
pushed a commit
that referenced
this pull request
Jan 25, 2022
PiperOrigin-RevId: 420811652 Change-Id: I83742482770ba0bf7c3ccd57508c40fb9cdbe2f7
mihaimaruseac
pushed a commit
that referenced
this pull request
Jan 25, 2022
PiperOrigin-RevId: 420811652 Change-Id: I83742482770ba0bf7c3ccd57508c40fb9cdbe2f7
mihaimaruseac
added a commit
that referenced
this pull request
Jan 25, 2022
…1d56923600b40d9a95a04ad5-on-r2.8 Merge pull request #53695 from yongtang:53660-tf.sparse.split-crash
mihaimaruseac
added a commit
that referenced
this pull request
Jan 25, 2022
Merge pull request #53695 from yongtang:53660-tf.sparse.split-crash
mihaimaruseac
added a commit
that referenced
this pull request
Jan 25, 2022
…1d56923600b40d9a95a04ad5-on-r2.7 Merge pull request #53695 from yongtang:53660-tf.sparse.split-crash
mihaimaruseac
added a commit
that referenced
this pull request
Jan 25, 2022
Merge pull request #53695 from yongtang:53660-tf.sparse.split-crash
nluehr
added a commit
to NVIDIA/tensorflow
that referenced
this pull request
Mar 30, 2022
Backport PR tensorflow/tensorflow#53695
bensternlieb
added a commit
to ButterflyNetwork/tensorflow
that referenced
this pull request
Apr 12, 2022
* Add op-determinism info to version 2.7 release notes
* Move the change to the right TF release (2.7)
* tflite_runtime: Loosen numpy requirement
It's a common use case that using OpenCV with tflite_runtime. Since OpenCV requires minimum 1.19.3, we'd better loosen the numpy requirement of tflite_runtime.
PiperOrigin-RevId: 411936173
Change-Id: Idfda9382e5af534a2fc1beffcadc6428acc12201
* [lite] Add flag TFLITE_ENABLE_HEXAGON and update build rules to only define it if compiling for arm on non-apple devices.
PiperOrigin-RevId: 414330792
Change-Id: I88e534e506c03554c2cb6b079ba2bde07f1244e1
* [lite] Remove explicit #define for TFLITE_ENABLE_HEXAGON. This should be enabled by separate build target.
PiperOrigin-RevId: 414750624
Change-Id: I0650e96f81e840f4f885c7f459a59d75620e3743
* Temporary disable XNNPACK on armhf Linux Python wheel build
PiperOrigin-RevId: 411954096
Change-Id: I0464fe508d1a0d724799905c16acdf7a698c7f34
* tflite_runtime: Add a way to override wheel's platform name
PiperOrigin-RevId: 414570169
Change-Id: I853bd6725c54595e712977f12d6c14b3d610068d
* Sort signature inputs by input key instead of tensor name when loading a TF1 model in TF2.
Signatures are saved using nest.flatten, which sorts the keys alphabetically for determinism. This can cause a mismatch between the lifted signature function's arg_keywords and structured_input_signature, resulting in users not being able to re-save the signature.
Detailed example below.
Say there is a SavedModel signature with the inputs:
```
inputs['a'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 1)
name: input_2:0
inputs['b'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 2)
name: input_1:0
```
When this signature is loaded using the TF2 API, it becomes a concrete function with arg keywords = `['b', 'a']` (sorted by tensor name)
The `structured_input_signature` becomes `([], {'b': spec_for_b, 'a': spec_for_a})`
When this function is exported as a signature in a new SavedModel, the structured inputs are flattened and zipped with the arg keywords. However, the flattened `structured_input_signature` is `[spec_for_a, spec_for_b]`, since `nest.flatten` sorts the keys for determinism. This results in the wrong spec being mapped to the wrong arguments.
PiperOrigin-RevId: 417889747
Change-Id: Icdce72a43033027c18f5530068f2d6fd47e8e32d
* Internal change
PiperOrigin-RevId: 411896058
Change-Id: Ia031058247e3cf382957a6662d3f9e1cbb481ca2
* Fix out of bound error in ReverseSequence Op shape function
PiperOrigin-RevId: 411896080
Change-Id: I7e59a38e2f960886edf2b6c54ed5a84e86a9b193
* Fix out of bound access in DequantizeOp by adding check for axis < input dimension
PiperOrigin-RevId: 411214268
Change-Id: I3249d2a69ddc82f182c589a3a5bbfb71543f4b29
* Fix Integer overflow error in Dequantize op shape function, by adding a bound check on axis.
PiperOrigin-RevId: 412121389
Change-Id: I3088dbad9e90f9998d406b618c16694388a9dfb4
* Add negative bound check for row and column pooling_sequence in FractionalAvgPoolGrad op to avoid out of bound heap access
PiperOrigin-RevId: 413837346
Change-Id: I2b86034101df31bee161abcb781755e236c7bccd
* Fix integer overflow leading to divide by zero error in Unravel index kernel when dimensions product exceeds max int value.
PiperOrigin-RevId: 413250052
Change-Id: I9450b6e8acecd2e881a64b882e2b7c70e8e9289a
* Fix Segfault in Concat V2 shape function.
PiperOrigin-RevId: 412120654
Change-Id: I3ff915faea694f9ad8b00024e9af2de9909011be
* [tf.data] Set limit on number of threads used in threadpool_dataset.
PiperOrigin-RevId: 410922677
Change-Id: Ib25814a99043ab10805b5d2d7088ae0e0b7b04fd
* Add a check for pad width to be a positive value.
PiperOrigin-RevId: 413275853
Change-Id: I261a8db9dabf5ce48a806a9e58129080c9fac619
* Fix more boosted trees vulnerabilities.
Please no longer use Boosted trees inside TF. The authors of the code no
longer work on this after getting their promotion and moving to other
teams.
* Add missing validation to sparse dense cwise ops.
PiperOrigin-RevId: 415543133
Change-Id: I5baf3284e919338afb96178c468ad3d3cb0d956c
* Prevent overflow in sparse dense cwise ops.
PiperOrigin-RevId: 415543171
Change-Id: I22dab7c41be2121ab5efe5403ca0e2f9b7cb24b8
* Add missing validation to `AddManySparseToTensorsMap`.
Sparse tensors have a set of requirements for the 3 components and not all of them were checked.
PiperOrigin-RevId: 415358027
Change-Id: I96cbb672999cd1da772c22fabbd15507e32e12dc
* Replace faulty overflow check with a builder for `TensorShape`.
Prevents an integer overflow that was not caught before.
PiperOrigin-RevId: 415381595
Change-Id: I76585ddedc912bd9f4a390aeafa8e2ced1a28863
* Add a check for Key being scalar tensor for MapStage and OrderedMapStage ops.
According to documentation[1][2], key must be int64 value, but this wasn't enforced and the ops would fail with check failure for non-scalar key value.
[1]https://www.tensorflow.org/api_docs/cc/class/tensorflow/ops/ordered-map-stage
[2]https://www.tensorflow.org/api_docs/cc/class/tensorflow/ops/map-stage
PiperOrigin-RevId: 413822112
Change-Id: I9d118faf990e6361900aa32272eff486ad9f0e2e
* Fix potential divide by zero error when executing FractionalMaxPool, when pooling ratio is higher than input size for a particular dimension.
PiperOrigin-RevId: 412151722
Change-Id: I06e57cbb8eca43816eff79eac264fa7aae8f7163
* Fixed model server crash:
Instead of calling the constructor of TensorShape, we call the BuildTensorShapeBase factory function and check the status of the calls in sparse_slice_op.
PiperOrigin-RevId: 402647675
Change-Id: Id9cba19c18ab0f16ea800c2b69d859935a54c2c7
* Add integer overflow and negative value check for some of the op kernels to avoid Check Failure in the InitDim called from constructor, before instantiating TensorShape.
PiperOrigin-RevId: 414610546
Change-Id: Icc713b41ef8515d2977c4c193a7bf19dd6792ffc
* Fix integer overflow error in SparseSlice op
PiperOrigin-RevId: 415399219
Change-Id: Ifb3cffcb5cfacb6eab151bec6b31d7a2d7f264d7
* Add integer overflow fixes for StringNGram, SparseSliceDataset and SparseSlice ops for identified usages.
PiperOrigin-RevId: 415639495
Change-Id: Ie09094928d21e8449a372075a5e88a50c628fb04
* Use `SetDimWithStatus` instead of `set_dim`.
This would prevent a crash due to overflow.
PiperOrigin-RevId: 416083115
Change-Id: Id2a0c11f5940ec8e28e4e39a37f506c875b5aca5
* Use builders for tensor shapes to prevent overflows.
PiperOrigin-RevId: 416083152
Change-Id: I7476345ce34039bde8afa657412ff0b7b64b03d9
* Prevent overflow causing crash in `SerializeManySparse`
PiperOrigin-RevId: 416083201
Change-Id: Ib5244a92ea4e84a1af5e7f3339e6876a88a533a7
* Add a fix for Check Failure due to integer overflow
PiperOrigin-RevId: 416216408
Change-Id: I16d3e2ce35d88b91e8a4ca38b9ebbbf7a05ee792
* Properly validate sparse tensor in `SparseTensorSliceDataset`
Existing validation was incomplete.
PiperOrigin-RevId: 415375048
Change-Id: I14cd18f29ede73286f3ffac35171bd15828997e9
* Fix check-fail when bincount ops are passed invalid values.
PiperOrigin-RevId: 415063028
Change-Id: I20f8dc09933ddca1111c4efbf9a3a1e863215d02
* Fix nullptr exception in QuantizedMaxPool op when empty list is sent to min_input or max_input parameters.
PiperOrigin-RevId: 413960973
Change-Id: I9e3ded593f3c4eabf0d6d5dc356e6a19a3ad2682
* Fix Null-pointer dereference in BuildXlaCompilationCache
If ConfigProto is not used, then use the default settings which is to allow all devices.
PiperOrigin-RevId: 420391800
Change-Id: I88161ad7042990aef678e77b597a2fb2c8f815be
* Cleanup and remove duplicate validation in `SparseCount`.
We have valdiation that is duplicated, checking different conditions, in different formats and failing to capture all cases. This should fix all the previous bugs.
PiperOrigin-RevId: 414886981
Change-Id: Ibf0bba0beb057b76d505324bb9487565daf95f01
* Further validate sparse tensor for `SparseCount`: indices must be valid within dense shape.
PiperOrigin-RevId: 414888122
Change-Id: I4552bd74c135ecd4bcb5448acc0a3ce9402d8286
* Prevent crash due to integer overflow followed by allocating negative sized array.
PiperOrigin-RevId: 414891322
Change-Id: I5df390e0dc1d9f115209293708950cdf9306931c
* Fix crash when importing an invalid graph with an empty op type to TFG
Found by proto fuzzing.
PiperOrigin-RevId: 413847837
Change-Id: Icac24d1b389c5661800fb4d622dff0b31d846cca
* Fix crash when importing an invalid graph with functions with empty names
Found by proto fuzzing.
PiperOrigin-RevId: 413918956
Change-Id: I2cb898d6561070cfbcf448ea0da5e838438f3e92
* Gracefully catch errors when importing an invalid graphdef in MLIR TFG
When importing a "generic function" in TFG, we don't build a Graph in memory and
so we need to implement a bit more checking in the importer itself.
This particular case catches duplicated names between nodes or between nodes and
function arguments, and fixes a crash found by the fuzzer.
PiperOrigin-RevId: 409331027
Change-Id: Ibaf6290f67908c020c5103a7e009bdffd88690e2
* [lite] Add check for bias_size is zero to avoid division by zero. This shouldn't happen for properly converted models. Just safety check
PiperOrigin-RevId: 416383645
Change-Id: If8e508bf696ae8ecfb927e69c139a8ccf7fe60cb
* [lite] Add validation check for dilation height/width to be positive integers.
PiperOrigin-RevId: 416429178
Change-Id: If7cdcddca54486434d9b2f06e7e2b401d7c3ee25
* [lite] Update TfLiteIntArrayCreate to return size_t
PiperOrigin-RevId: 416439896
Change-Id: I847f69b68d1ddaff4b1e925a09b8b69c1756653b
* [lite] Move MultiplyAndCheckOverflow to util to be able to share it.
PiperOrigin-RevId: 416897229
Change-Id: I5feb44881bdcbb6ed911da4f17c55bb978754059
* [lite] Check for overflow when creating required bytes.
PiperOrigin-RevId: 417629001
Change-Id: Ia7feb3ea8e988f4fd4b3c98c1a1fed4557d99fd7
* Prevent segfault in `embedding_lookup_sparse.cc`
Previous fixes missed one additional case.
PiperOrigin-RevId: 417676944
Change-Id: I8ab412155cf9b1e897448a6611d209eaa7ca9e66
* [lite] add validation check for sparse fully connected
PiperOrigin-RevId: 417629354
Change-Id: If96171c4bd4f5fdb01d6368d6deab19d1c9beca7
* Remove a `DCHECK`-fail, log an error instead.
`DCHECK` in debug mode results in crashes. TensorFlow has had multiple vulnerabilities due to this.
Outside of debug mode, `DCHECK` is a no-op.
A better alternative is to report an error to the log buffer and continue. This should happen both in debug mode and in prod mode.
PiperOrigin-RevId: 408375925
Change-Id: Id5b3e19c73f3fbe0cc4bba26ca44ff9607bb6356
* [lite] Add some safety checks to avoid out of bound access for sparsity format
PiperOrigin-RevId: 416910386
Change-Id: Ic0b4dc048dc4b5a6309c572b8c4c9f776e4db60a
* Prevent a crash due to heap OOB write in grappler.
PiperOrigin-RevId: 408318417
Change-Id: If095feb8c001e3a8ac4a85b7387b81e8309df47d
* Use `tempfile.mkstemp` instead of `tempfile.mktemp`.
The `tempfile.mktemp` function is [deprecated](https://docs.python.org/3/library/tempfile.html#tempfile.mktemp) due to [security issues](https://cwe.mitre.org/data/definitions/377.html).
The switch is easy to do.
PiperOrigin-RevId: 420335872
Change-Id: I331ec2544a08d3cc3063a74af342cceae655b3dc
* Use `tempfile.mkstemp` instead of `tempfile.mktemp`.
The `tempfile.mktemp` function is [deprecated](https://docs.python.org/3/library/tempfile.html#tempfile.mktemp) due to [security issues](https://cwe.mitre.org/data/definitions/377.html).
The switch is easy to do.
PiperOrigin-RevId: 420355705
Change-Id: If437973d0cd7686a221679d4123cb12f79697fe0
* Use `tempfile.mkstemp` instead of `tempfile.mktemp`.
The `tempfile.mktemp` function is [deprecated](https://docs.python.org/3/library/tempfile.html#tempfile.mktemp) due to [security issues](https://cwe.mitre.org/data/definitions/377.html).
The switch is easy to do.
PiperOrigin-RevId: 420359120
Change-Id: Ifb43401b1fd3e023c685dc3a74b3b655090e1ce6
* Use `tempfile.mkstemp` instead of `tempfile.mktemp`.
The `tempfile.mktemp` function is [deprecated](https://docs.python.org/3/library/tempfile.html#tempfile.mktemp) due to [security issues](https://cwe.mitre.org/data/definitions/377.html).
The switch is easy to do.
PiperOrigin-RevId: 420359138
Change-Id: I8afc97448b1e730ac5883c2033f3b0e544b8fb58
* Use `tempfile.mkstemp` instead of `tempfile.mktemp`.
The `tempfile.mktemp` function is [deprecated](https://docs.python.org/3/library/tempfile.html#tempfile.mktemp) due to [security issues](https://cwe.mitre.org/data/definitions/377.html).
The switch is easy to do.
PiperOrigin-RevId: 420359156
Change-Id: I992e93ed8423eef87bfcfc84b0c877131d6f916d
* Use `tempfile.mkstemp` instead of `tempfile.mktemp`.
The `tempfile.mktemp` function is [deprecated](https://docs.python.org/3/library/tempfile.html#tempfile.mktemp) due to [security issues](https://cwe.mitre.org/data/definitions/377.html).
The switch is easy to do.
PiperOrigin-RevId: 420359212
Change-Id: I172811749d2e7b901399f63df4fd1523447c6682
* Use `tempfile.mkstemp` instead of `tempfile.mktemp`.
The `tempfile.mktemp` function is [deprecated](https://docs.python.org/3/library/tempfile.html#tempfile.mktemp) due to [security issues](https://cwe.mitre.org/data/definitions/377.html).
The switch is easy to do.
PiperOrigin-RevId: 420359224
Change-Id: I7bfc1df9cf931f45ec85d4878874ef41b9c55474
* Use `tempfile.mkstemp` instead of `tempfile.mktemp`.
The `tempfile.mktemp` function is [deprecated](https://docs.python.org/3/library/tempfile.html#tempfile.mktemp) due to [security issues](https://cwe.mitre.org/data/definitions/377.html).
The switch is easy to do.
PiperOrigin-RevId: 420359231
Change-Id: If2049dbeb46fb8ff6df7c8e077cee8be3872e5b4
* Use `tempfile.mkstemp` instead of `tempfile.mktemp`.
The `tempfile.mktemp` function is [deprecated](https://docs.python.org/3/library/tempfile.html#tempfile.mktemp) due to [security issues](https://cwe.mitre.org/data/definitions/377.html).
The switch is easy to do.
PiperOrigin-RevId: 420359237
Change-Id: I7fa45e888deff612ca53a4f8610cfad8f28e9671
* Use `tempfile.mkstemp` instead of `tempfile.mktemp`.
The `tempfile.mktemp` function is [deprecated](https://docs.python.org/3/library/tempfile.html#tempfile.mktemp) due to [security issues](https://cwe.mitre.org/data/definitions/377.html).
The switch is easy to do.
PiperOrigin-RevId: 420360028
Change-Id: Icd8a7ba3e47c2ff63a26a2fe007737ef01c0cb1d
* Use `tempfile.mkstemp` instead of `tempfile.mktemp`.
The `tempfile.mktemp` function is [deprecated](https://docs.python.org/3/library/tempfile.html#tempfile.mktemp) due to [security issues](https://cwe.mitre.org/data/definitions/377.html).
The switch is easy to do.
PiperOrigin-RevId: 420360036
Change-Id: I13eb94736af3397261cf0d46214ddb5a2af9d92b
* Use `tempfile.mkstemp` instead of `tempfile.mktemp`.
The `tempfile.mktemp` function is [deprecated](https://docs.python.org/3/library/tempfile.html#tempfile.mktemp) due to [security issues](https://cwe.mitre.org/data/definitions/377.html).
The switch is easy to do.
PiperOrigin-RevId: 420363556
Change-Id: I3225120cd6545462174641581a365ead0eb179c3
* Use `tempfile.mkdtemp` instead of `tempfile.mktemp`.
The `tempfile.mktemp` function is [deprecated](https://docs.python.org/3/library/tempfile.html#tempfile.mktemp) due to [security issues](https://cwe.mitre.org/data/definitions/377.html).
The switch is easy to do.
PiperOrigin-RevId: 420369603
Change-Id: I2cf40b13f41cc01000c2c21a483a2d680194dba2
* Use `tempfile.mkdtemp` instead of `tempfile.mktemp` to create directories.
The `tempfile.mktemp` function is [deprecated](https://docs.python.org/3/library/tempfile.html#tempfile.mktemp) due to [security issues](https://cwe.mitre.org/data/definitions/377.html).
The switch is easy to do: just a name change
PiperOrigin-RevId: 420370858
Change-Id: I44a0849d161132eacd4f3881fdb615e09c0f02a2
* Use `tempfile.mkstemp` instead of `tempfile.mktemp`.
The `tempfile.mktemp` function is [deprecated](https://docs.python.org/3/library/tempfile.html#tempfile.mktemp) due to [security issues](https://cwe.mitre.org/data/definitions/377.html).
The switch is easy to do.
PiperOrigin-RevId: 420384092
Change-Id: I8721c09ccc4de589b5a45d38e7ebc440160c72b8
* Fix `logging_ops_test` which fails on Windows
PiperOrigin-RevId: 420423302
Change-Id: I688a5e77831561c034c3e5bb09bdb0cedea63801
* Fix `debugger_cli_common_test` which breaks Windows
PiperOrigin-RevId: 420425157
Change-Id: I29596c2ef7d0dc13e3678c48394d31ab0bfcbd24
* Validate `proto.dtype()` before calling `set_dtype()`.
This prevents a `DCHECK`-fail when the proto contains an invalid dtype for a tensor shape with 0 elements or for an incomplete tensor shape.
PiperOrigin-RevId: 408369083
Change-Id: Ia21a3e3d62a90d642a4561f08f3b543e5ad00c46
* Prevent null dereference read in `SpecializeType()`
For some adversarial protos, the attribute for a key might not exist.
PiperOrigin-RevId: 408382090
Change-Id: Ie7eabe532c9ff280fce5dce1f6cdb93c76c2e040
* Properly handle the case where `SpecializeType()` returns an error `Status`.
If the error case in `SpecializeType()` is reached, then we would get a crash when trying to access the value of an errorenous `StatusOr` object
PiperOrigin-RevId: 408380069
Change-Id: If3c3fc876dcf9384d5ec7a4985adc68c23ea7318
* Fix heap OOB read/write due to incorrect indexing.
PiperOrigin-RevId: 408578046
Change-Id: Ifc9ffea49e5890f55fcb2c27568611052c3ddcfa
* Prevent copying uninitialized data in `AssignOp`.
This prevents harder to debug undefined behaviors that cannot be traced back to the original tensor after assignments occur earlier in the graph execution. Several of these undefined behaviors are just reference bindings to null pointers, which are caught when running under ubsan/asan.
PiperOrigin-RevId: 408654780
Change-Id: Iad2ec40d43f5fd7ea016c20283356c12d5ddeab1
* Prevent integer overflow in `OpLevelCostEstimator::CalculateTensorSize`.
In order to not change the API, we return a negative value in case of overflow. A better fix is to change the API to return a status instead.
PiperOrigin-RevId: 408713061
Change-Id: I3771475b0c72a2844a3854086966562fd33f2da5
* Prevent integer overflow in `OpLevelCostEstimator::CalculateOutputSize`.
In order to not change the API, we return a negative value in case of overflow. A better fix is to change the API to return a status instead.
PiperOrigin-RevId: 408701427
Change-Id: Idf31e7f0bf18ca824d084fdd355e1f653f145c20
* Prevent integer overflow in `CalculateTensorSize`.
In order to not change the API, we return a negative value in case of overflow. A better fix is to change the API to return a status instead.
PiperOrigin-RevId: 408714915
Change-Id: I110ec4e1c5bbf4d7ca7ef7c068dfd3e8bc7190cd
* Prevent null dereference read in `GetInitOp`.
We have a map of maps. We test that the key exists in the first map but then we don't have any validation that this also means the second map has the needed key. In the scenarios where this is not the case, we'll dereference a nullptr, if we don't have this check
PiperOrigin-RevId: 408739325
Change-Id: If9bb7ed759aba1f3b56a34913f209508dbaf65ce
* Fix memory leak when a graph node is invalid.
If a graph node is invalid but a kernel is created then we set the kernel back to `nullptr` but we forget to delete it. Hence, we get a memory leak.
PiperOrigin-RevId: 408968108
Change-Id: I1d8a9d0d8988ed5e08be8b9f2004ce1b4cd11b7c
* Fix abort caused by allocating a too large vector.
We need to make sure that the number of dimensions in a shape is within limits.
PiperOrigin-RevId: 408997911
Change-Id: If59e1c23f2ec9c2d4ff4d8632fd62b2a7773a4eb
* Eliminate `CHECK`-fails from `IsSimplifiableReshape` via `MakeShape(<invalid shape>)`
PiperOrigin-RevId: 409166738
Change-Id: I7f0a3590b8acae3f3e3e2fe636e1f5ef285693cf
* Remove `CHECK`-fails from `IsSimplifiableReshape`
PiperOrigin-RevId: 409164987
Change-Id: I58c7dd459ff348c3dbae95e00c4c5e63b30a4e65
* Make `IsSimplifiableReshape` return `Status` instead of `bool`.
This is to allow remove `CHECK`-fails in subsequent commits.
PiperOrigin-RevId: 409160987
Change-Id: I3f050218a3832271395c4372a0b8ea05f1c03d80
* Prevent a null-pointer dereference / `CHECK`-fail in grappler.
PiperOrigin-RevId: 409187354
Change-Id: I369c249cca32e6c56ec193f0ebbf2f2768fc7d43
* Use `PartialTensorShape` instead of `TensorShape`.
`TensorShape` constructor throws a CHECK-fail if shape is partial/overflows which the other doesn't. We are only determining the number of elements in the shape and partial shape should be used as it returns negative number when needed.
PiperOrigin-RevId: 409205384
Change-Id: Ia56542ff9ec758f2c9ffc7e4dcc9fa7eecd86e7b
* Validate real and expected type of arguments to cwise ops.
Without this validation, it is possible to trigger a `CHECK`-fail denial of service.
This is a rollforward of a previous commit which was rolled back as it was relying on RTTI. This time we don't use RTTI, we replace `typeid(Tin).name()` with a double function call, `DataTypeString(DataTypeToEnum<Tin>::v())`.
PiperOrigin-RevId: 409340416
Change-Id: I96080b2796729a3a9b65e7c68307ac276070f2f0
* Prevent use after free in `DecodePng` kernel.
We are cleaning up the memory in `decode` and then we are using an `OP_REQUIRES` to check an invariant on the `decode` data.
PiperOrigin-RevId: 409299145
Change-Id: I4eb93aaca52483eb202e89b78df07fbb2f6cb254
* In case of error in `DecodePng`, end kernel execution immediately.
There are scenarios where we detect an error in `DecodePng` kernel, yet the execution continues for a while longer before an error is thrown. This is not safe.
PiperOrigin-RevId: 409299935
Change-Id: Ife488b410148032ae777f59bc51864e172553fdf
* Prevent memory leak in decoding PNG images.
PiperOrigin-RevId: 409300653
Change-Id: I6182124c545989cef80cefd439b659095920763b
* Eliminate debug `CHECK`-fail from `function.cc`
PiperOrigin-RevId: 409416119
Change-Id: I8376ee464d434e9b970ff0ad49edfdaa2a273cfe
* Eliminate `CHECK`-fail from `function.cc`.
PiperOrigin-RevId: 409414744
Change-Id: Ic854e12ab2edb88b165d32e2d632c4ee654d71ad
* Prevent `CHECK`-fail when building reference tensor.
The tensor constructor does not allow reference dtypes, as these should not show up explicitly. However, when passed these invalid types instead of building an invalid object the constructor crashes via a `CHECK`-fail. We have a static builder that properly handles this case but is not applicable given current usage.
Instead, before calling the constructor, we can check that the dtype is not a reference type and return an error otherwise, given that the dtype is user controlled so malicious users can trigger denial of service.
PiperOrigin-RevId: 409662503
Change-Id: I5892f831fde7f276cd7ab34519cf6b8061c71a59
* Prevent overflow in `CalculateTensorElementCount`
Grappler cost estimation sometimes computes the number of elements in a tensor by multiplying all the dimensions in a shape. However, these tensors can also be controlled by users so a malicious attacker can trigger overflow that can be exploited.
PiperOrigin-RevId: 409575048
Change-Id: I7a958875ba6f3ad9cb5b9943fe5d459efcbe4557
* Prevent overflow in grappler cost estimation of crop&resize op.
The crop parameters are user controlled, so we should make sure a user can not trigger an overflow maliciously.
PiperOrigin-RevId: 409670234
Change-Id: I7994734a98b037c5642e051240329d16f959aae4
* Prevent null pointer dereference in `mutable_graph_view`
PiperOrigin-RevId: 409684472
Change-Id: I577eb9d9ac470fcec0501423171e739a4ec0cb5c
* Prevent null pointer dereference in constant folding.
Under certain conditions, an invalid protobuf saved model with invalid nodes would be loaded. During optimization phase, Grappler optimizer will then dereference a null pointer.
PiperOrigin-RevId: 409683530
Change-Id: I1f10340a7ec384bc9bc587300390f1078cf5caa0
* Fix `CHECK`-failure caused by constant folding code.
We're losing a `const` qualifier here, but unless we get to use more `StatusOr` objects, this is the best alternative.
PiperOrigin-RevId: 410072241
Change-Id: I69535c91490f0d23facb9587d2ff59db0782cda6
* Prevent stack overflow when FunctionLib in GraphDef has a self-recursive function.
It is likely that no recursivity is supported, but we should handle this separately.
PiperOrigin-RevId: 414860329
Change-Id: I02a2270e86282b37362ddd485eeef16fb986a9e0
* Merge pull request tensorflow#52707 from elfringham:init_ops_test_fix
PiperOrigin-RevId: 416941851
Change-Id: Iefa5a9b841b053b36f6b105cd82c9d32d5e47850
* Merge pull request tensorflow#53695 from yongtang:53660-tf.sparse.split-crash
PiperOrigin-RevId: 420811652
Change-Id: I83742482770ba0bf7c3ccd57508c40fb9cdbe2f7
* Add static factory builders for (partial) tensor shapes.
We have them in `TensorShapeBase` but calling them directly from there is cumbersome:
```cc
TensorShapeBase<TensorShape>::BuildTensorShape(...)
TensorShapeBase<PartialTensorShape>::BuildTensorShape(...)
```
With this change, above lines change to
```cc
TensorShape::BuildTensorShape(...)
PartialTensorShape::BuildTensorShape(...)
```
This should make it easier to remove `CHECK`-fails from code, preventing breakages similar to multiple CVEs we've had this year.
PiperOrigin-RevId: 407922426
Change-Id: I9e12f112457567a46528d170dc76137d553c9e81
* Prevent `CHECK`-fail when decoding resource handles from proto
In certain scenarios, the proto might contain tensors that have too many elements (overflow). This is a `CHECK`-fail in general, but we should prevent this, given how many CVEs caused by that we have received this year (a large fraction of 200).
PiperOrigin-RevId: 408049766
Change-Id: I2ac20b247aa8ed9110846fbdb7a0a9401f2c168c
* Update tensorflow/core/grappler/optimizers/constant_folding.cc
* Update tensorflow/core/grappler/optimizers/constant_folding.cc
* Update tensorflow/core/grappler/optimizers/constant_folding.cc
* Update tensorflow/core/grappler/optimizers/constant_folding.cc
* Check for type inference error on node construction.
PiperOrigin-RevId: 409415804
Change-Id: Ieb6e020906b96f522bf8e2fa103715ddbbdc434a
* Fix crash when an invalid graph (empty handle_data) is provided to the TFG importer
Found by the fuzzer.
PiperOrigin-RevId: 413036574
Change-Id: Ib90e542c04ccc46416bce3201e23f339e5bd1097
* Fix crash when importing invalid graph (attribute with empty name) to TFG
Found by the fuzzer
PiperOrigin-RevId: 413037608
Change-Id: I7220932e261bd71848d8bd6d0ee654e83c7fa2e3
* Fix crash when importing invalid graph (func attribute with empty name) to TFG
Found by the fuzzer
PiperOrigin-RevId: 413200989
Change-Id: Ib81abccaf33f05a69c780c937710699f761ede72
* Fix crash on invalid graph importing to TFG when there is a func_attr with an empty key.
Found by fuzzing.
PiperOrigin-RevId: 414089748
Change-Id: Ibdbb5aed29f36abd4bb66f7ef854868dc6b9d95c
* lite: Update Windows tensorflowlite_flex.dll build
Removed "windows_export_all_symbols" feature since Flex delegate only requires
to expose `TF_AcquireFlexDelegate` symbol.
This change is needed for Issue#43367.
PiperOrigin-RevId: 424060071
Change-Id: I8874ce6b107f6db9c5445b65e55073ea46266c76
* Fix `OP_REQUIRES` cherry-pick error
* Bump the maximum threshold before erroring
PiperOrigin-RevId: 424653571
Change-Id: Ic2d9f3a7db627d78cde80ad415105f3d53735b3b
* Disable broken tests
* Insert release notes place-fill
* Update version numbers to 2.7.1
* Bump ICU to 69.1 to handle CVE-2020-10531.
See [CVE-2020-10531](https://nvd.nist.gov/vuln/detail/CVE-2020-10531), it's an integer overflow.
PiperOrigin-RevId: 424908014
Change-Id: Ib8d6f8b527fb1fd9d887eaf487afc17d42333c8a
* Revert "Bump ICU to 69.1 to handle CVE-2020-10531."
* Revert "Revert "Bump ICU to 69.1 to handle CVE-2020-10531.""
* Update third_party/icu/workspace.bzl
* Update third_party/icu/workspace.bzl
* Update RELEASE.md
* Set Env Variable to override Setuptools new behavior
PiperOrigin-RevId: 423468055
Change-Id: I5b148103e1372a5eb73570bc77face27dbd5f914
* Reorder tags to fix buildifier linting
* Disable flaky test
Co-authored-by: Duncan Riach <duncan@nvidia.com>
Co-authored-by: Frederic Bastien <fbastien@nvidia.com>
Co-authored-by: Mihai Maruseac <mihaimaruseac@google.com>
Co-authored-by: Terry Heo <terryheo@google.com>
Co-authored-by: Karim Nosir <karimnosseir@google.com>
Co-authored-by: Katherine Wu <kathywu@google.com>
Co-authored-by: Isha Arkatkar <ishark@google.com>
Co-authored-by: Andrew Audibert <aaudibert@google.com>
Co-authored-by: A. Unique TensorFlower <gardener@tensorflow.org>
Co-authored-by: Edward Loper <edloper@google.com>
Co-authored-by: Smit Hinsu <hinsu@google.com>
Co-authored-by: Mehdi Amini <aminim@google.com>
Co-authored-by: Mihai Maruseac <mihai.maruseac@gmail.com>
Co-authored-by: Dan Moldovan <mdan@google.com>
Co-authored-by: TensorFlow Release Automation <jenkins@tensorflow.org>
liutongxuan
pushed a commit
to liutongxuan/DeepRec
that referenced
this pull request
Jul 28, 2022
Backport PR tensorflow/tensorflow#53695
liutongxuan
pushed a commit
to liutongxuan/DeepRec
that referenced
this pull request
Aug 1, 2022
Backport PR tensorflow/tensorflow#53695
liutongxuan
pushed a commit
to liutongxuan/DeepRec
that referenced
this pull request
Aug 3, 2022
Backport PR tensorflow/tensorflow#53695
This was referenced Nov 4, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
comp:core
issues related to core part of tensorflow
ready to pull
PR ready for merge process
size:S
CL Change Size: Small
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR tries to fix the crash of tf.sparse.split when axis is not a scalar, as was raised in #53660.
This PR fixes #53660.
Signed-off-by: Yong Tang yong.tang.github@outlook.com