Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 4 additions & 6 deletions .github/workflows/build_dist.yml
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ jobs:
- macos-15-intel # x86
- macos-latest # arm
- windows-latest
cibw_build: [cp39-*, cp310-*, cp311-*, cp312-*, cp313-*, cp313t-*, cp314-*, cp314t-*]
cibw_build: [cp39-*, cp310-*, cp311-*, cp312-*, cp313-*, cp314-*, cp314t-*, cp315-*, cp315t-*]
steps:
- name: Check out repository
uses: actions/checkout@v6
Expand Down Expand Up @@ -78,13 +78,12 @@ jobs:
echo "CIBW_ENVIRONMENT=PYLZ4_USE_SYSTEM_LZ4=False" >> "$GITHUB_ENV"
echo "CIBW_TEST_COMMAND=tox -c {project}" >> "$GITHUB_ENV"
- name: Build wheels
uses: pypa/cibuildwheel@v3.3.0
uses: pypa/cibuildwheel@v4.2.0
env:
# CIBW_ARCHS_LINUX: "x86_64 i686 aarch64"
CIBW_ARCHS_LINUX: "x86_64 i686"
CIBW_ARCHS_MACOS: "auto64" # since we have both runner arches
CIBW_ARCHS_WINDOWS: "AMD64 x86 ARM64"
CIBW_ENABLE: cpython-freethreading
CIBW_BUILD: ${{ matrix.cibw_build }}
CIBW_SKIP: "cp*-musllinux*"
CIBW_TEST_SKIP: "*-macosx_arm64 *-macosx_universal2:arm64 *-*linux_{ppc64le,s390x} *-win_arm64"
Expand All @@ -106,7 +105,7 @@ jobs:
matrix:
os:
- ubuntu-24.04-arm
cibw_build: [cp39-*, cp310-*, cp311-*, cp312-*, cp313-*, cp313t-*, cp314-*, cp314t-*]
cibw_build: [cp39-*, cp310-*, cp311-*, cp312-*, cp313-*, cp314-*, cp314t-*, cp315-*, cp315t-*]
steps:
- name: Check out repository
uses: actions/checkout@v6
Expand All @@ -122,12 +121,11 @@ jobs:
echo "CIBW_ENVIRONMENT=PYLZ4_USE_SYSTEM_LZ4=False" >> "$GITHUB_ENV"
echo "CIBW_TEST_COMMAND=tox -c {project}" >> "$GITHUB_ENV"
- name: Build wheels
uses: pypa/cibuildwheel@v3.3.0
uses: pypa/cibuildwheel@v4.2.0
env:
CIBW_ARCHS_LINUX: "aarch64"
CIBW_BUILD: ${{ matrix.cibw_build }}
CIBW_SKIP: "cp*-musllinux*"
CIBW_ENABLE: cpython-freethreading
CIBW_BEFORE_BUILD: "python -m pip install -U pip && python -m pip install tox"
- name: Save wheels
uses: actions/upload-artifact@v5
Expand Down
8 changes: 5 additions & 3 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,11 @@ library. As such, these LZ4 bindings should provide a drop-in alternative to the
compression libraries shipped with Python. The package provides context managers
and file handler support.

The bindings drop the GIL when calling in to the underlying LZ4 library, and is
thread safe. An extensive test suite is included.
The bindings drop the GIL when calling in to the underlying LZ4 library.
Stateless operations and operations using separate compression or decompression
contexts can run concurrently. Incremental frame and stream contexts are
mutable; a context must be owned by one thread at a time, or access to it must
be synchronized by the caller. An extensive test suite is included.

Documentation
=============
Expand All @@ -67,4 +70,3 @@ Licensing
=========
Code specific to this project is covered by the `BSD 3-Clause License
<http://opensource.org/licenses/BSD-3-Clause>`_

1 change: 1 addition & 0 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -208,6 +208,7 @@ def pkgconfig_installed_check(lib, required_version, default):
'Programming Language :: Python :: 3.12',
'Programming Language :: Python :: 3.13',
'Programming Language :: Python :: 3.14',
'Programming Language :: Python :: 3.15',
'Programming Language :: Python :: Free Threading',
],
)
25 changes: 24 additions & 1 deletion tests/frame/test_frame_7.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
import os
import threading
from concurrent.futures import ThreadPoolExecutor

import lz4.frame as lz4frame
import pytest
import os

test_data = [
(os.urandom(32) * 256),
Expand Down Expand Up @@ -100,3 +103,23 @@ def test_roundtrip_multiframe_4(data):

assert len(decompressed) == nframes * len(data)
assert data * nframes == decompressed


def test_independent_frame_contexts_can_run_concurrently():
workers = 4
barrier = threading.Barrier(workers)

def roundtrip(worker):
data = os.urandom(128 * 1024) + bytes([worker])
compressor = lz4frame.LZ4FrameCompressor()

barrier.wait()
compressed = compressor.begin() + compressor.compress(data)
compressed += compressor.flush()

return lz4frame.decompress(compressed), data

with ThreadPoolExecutor(max_workers=workers) as executor:
results = list(executor.map(roundtrip, range(workers)))

assert all(decompressed == data for decompressed, data in results)