-
Notifications
You must be signed in to change notification settings - Fork 165
Description
I've hit a few issues trying to get Compressed to work, which makes me think it's not very well used or supported. It was called out in #355 but there aren't any examples in the gallery.
-
Compressedpretty much only makes sense when used withGreedyBytesfor its subcon. The documentation instead focuses on the input, saying it should be used withPrefixed, but not why.The reason for
GreedyBytesis that this:Compressed(Bytes(10), 'zlib')
will read an unlimited amount of compressed data, and then return the first 10 bytes. While this:
FixedSized(10, Compressed(GreedyBytes, 'zlib'))
will read 10 bytes, and decompress it into the right size. I think
FixedSizedshould be included in the documentation, along with an explanation thatPrefixedis suitable when creating your own structure. PerhapsGreedyBytesshould be the default subcon? -
While the above is the only way it could work when looking closely, I was misled by the signature for the preceding
Checksum, and assumed the subcon was the output type. I used the first example, and as the compressed output was always larger than the input, there was no error. I didn't realise I'd got it wrong until I started fully parsing the output. I feel like there's some footgun-avoiding that could be added here, perhaps warning if the output isn't consumed, or changing the interface to matchChecksum, or vice versa. -
For the format I'm looking at, I have a compressed field with a separate length field, and while I can now parse it, I want to be able to populate the compressed length when building. I basically want
Prefixed, but where the prefix is in a different location. If I useFixedSized(this.compressed_size, Compressed(GreedyBytes, 'zlib')), I need to somehow provide a value forcompressed_size. I can't work out how to do that without doing the compression a second time myself. I can pull it out of thectxif the field follows the data, butRebuilddoesn't seem to work if it's before.Am I trying to make
Compresseddo something it's just not suited for? Do I need to create an adapter instead? IsFixedSizedthe wrong thing to use?