ARROW-16099: [JS] RecordBatches that are compressed should throw an error

Isaac Brodsky · domoritz · domoritz · commit f5166fe21969 · 2022-04-02T21:44:15.000-04:00
While https://issues.apache.org/jira/browse/ARROW-8674 is not implemented, ArrowJS will silently provide corrupt (not uncompressed) data to applications when compressed files are passed in. This happens to be the default from Python, e.g. with `df.to_feather('my_file')`. Opening as a draft as I didn't find how to add a sample file for testing that this behaves as expected. I'd appreciate any pointers on which part of the test suite we can assert this behavior in. Closes apache#12718 from isaacbrodsky/js-recordbatch-compression-throw Lead-authored-by: Isaac Brodsky <isaac@unfolded.ai> Co-authored-by: Dominik Moritz <domoritz@gmail.com> Signed-off-by: Dominik Moritz <domoritz@gmail.com>
diff --git a/js/src/ipc/metadata/message.ts b/js/src/ipc/metadata/message.ts
@@ -297,6 +297,9 @@ function decodeSchema(_schema: _Schema, dictionaries: Map<number, DataType> = ne
 
 /** @ignore */
 function decodeRecordBatch(batch: _RecordBatch, version = MetadataVersion.V4) {
+    if (batch.compression() !== null) {
+        throw new Error('Record batch compression not implemented');
+    }
     return new RecordBatch(batch.length(), decodeFieldNodes(batch), decodeBuffers(batch, version));
 }
 

Original file line number	Diff line number	Diff line change
`@@ -297,6 +297,9 @@ function decodeSchema(_schema: _Schema, dictionaries: Map<number, DataType> = ne`
`297`	`297`
`298`	`298`	`/** @ignore */`
`299`	`299`	`function decodeRecordBatch(batch: _RecordBatch, version = MetadataVersion.V4) {`
	`300`	`+ if (batch.compression() !== null) {`
	`301`	`+ throw new Error('Record batch compression not implemented');`
	`302`	`+ }`
`300`	`303`	`return new RecordBatch(batch.length(), decodeFieldNodes(batch), decodeBuffers(batch, version));`
`301`	`304`	`}`
`302`	`305`