Skip to content

Expose fixed-budget LZ4 block compression - #349

Open
ac1982 wants to merge 1 commit into
python-lz4:masterfrom
ac1982:feat/compress-dest-size
Open

ac1982 wants to merge 1 commit into
python-lz4:masterfrom
ac1982:feat/compress-dest-size

Conversation

@ac1982

@ac1982 ac1982 commented Oct 15, 2026

Copy link
Copy Markdown

Fixes #328.

Add lz4.block.compress_dest_size(source, target_size, return_bytearray=False), returning (compressed_block, bytes_consumed). It wraps LZ4_compress_destSize and produces a raw block no larger than the requested budget. The consumed count can be passed to decompress(uncompressed_size=...) and used to advance through the input.

The new function requires LZ4 1.9.2 or newer because earlier versions have a documented destination-size compression bug. Existing APIs keep their version requirements. Buffer allocation is bounded by both the requested target and LZ4_compressBound; the GIL is released during compression. Documentation explains raw-block framing and zero consumption with tiny targets.

Validation on Linux ARM64 / Python 3.12:

  • 19,857 existing/new block and frame tests pass with system LZ4 1.10.0; final dedicated suite has 55 passing tests.
  • All 55 dedicated tests pass with bundled LZ4 1.9.4, UBSan, and Python's debug allocator.
  • Actual LZ4 1.9.1 rejects the new API, releases the input buffer, and still round-trips through existing APIs.
  • Flake8, 77 documentation tests, and HTML documentation build pass.

Windows and free-threaded Python were not tested. AI disclosure: implemented and tested with OpenAI Codex.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Expose LZ4_compress_destSize

1 participant