Skip to content

gh-88300: Support .xz stream padding in lzma.decompress() and LZMAFile - #161057

Open
gpshead wants to merge 2 commits into
python:mainfrom
gpshead:lzma-stream-padding-88300
Open

gpshead wants to merge 2 commits into
python:mainfrom
gpshead:lzma-stream-padding-88300

Conversation

@gpshead

@gpshead gpshead commented Nov 1, 2026 •

Copy link
Copy Markdown
Member

lzma.decompress() and LZMAFile now skip .xz stream padding (a run of null bytes whose size is a multiple of four) after a stream that starts with the .xz magic, and go on to decode the stream that follows; a damaged stream after the padding raises like any other damaged later stream. The skipping is done in the shared Python stream loop and turned on only by lzma, rather than with LZMA_CONCATENATED in C, because that would need a new finish method on LZMADecompressor and would reject the mixed .xz/.lzma data that FORMAT_AUTO decodes today. A run of null bytes whose size is not a multiple of four is still handed to the decompressor as before, so nothing that decodes today starts to fail on that account, although the format specification says it is an error. FORMAT_ALONE, FORMAT_RAW, bz2 and compression.zstd are unchanged.

Fixes #88300

…ZMAFile

Null bytes in multiples of four after an .xz stream are now skipped, so
that a stream following them is decoded instead of ignored.
@gpshead gpshead added needs backport to 3.14 bugs and security fixes needs backport to 3.15 bugs and security fixes labels Nov 1, 2026
@gpshead gpshead self-assigned this Nov 1, 2026
@gpshead gpshead removed their assignment Nov 1, 2026
Rename the shared stream option to xz_stream_padding (and the helpers to
_skip_xz_padding and _xz_padding_size): no other format handled here has
such padding, and the option needs stream_magic to be a bytes object.

A run of null bytes after an .xz stream whose size is not a multiple of
four is left to the decompressor, as before.  DecompressReader no longer
builds the whole run in memory to do that: it keeps the number of null
bytes between the first and the last block it read, and hands them back
in the blocks they were read in.  _xz_padding_size() limits the size of
the chunks it copies.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@gpshead
gpshead marked this pull request as ready for review November 12, 2026 00:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

lzma: stream padding in xz files

1 participant