This is a correctness- and robustness-focused release. The compressed bitstream format is unchanged and remains backwards-compatible; all fixes below preserve round-trip compatibility with streams produced by earlier 2.x versions.

Bug Fixes

• C core (extended format & init): fix a dropped run byte when a length-1 RLE run ends across poll calls (now re-emitted as a literal), invalidate the lazy-match cache after extended/RLE handling mutates the window, and reject extended streams before marking the decompressor configured so a retry can't reinterpret them as classic format. Adds a compile-time guard against a too-small stream work buffer. (#328)
• Python bindings: validate custom dictionary sizes — the compressor requires exactly 2**window bytes, and the decompressor accepts any buffer of at least 2**window bytes, using the window-size prefix (a too-small buffer previously indexed out of bounds) — trim zero-padding from short one-shot reads, support read-only file objects in the decompressor (with an over-return guard), and stop raising on empty write(). (#329)
• MicroPython: bounds-check window references in the viper decompressor and hoist the window-edge check out of the viper compressor match loop (fixing OOB reads), and align the native module's initialize_dictionary signature with CPython — initialize_dictionary(source, seed=None, literal=8). Note: passing literal positionally as the 2nd argument now raises; pass it as the 3rd argument or by keyword. (#330)
• ESP32: fix out-of-bounds window reads in the SIMD/scalar match search near the window boundary and on unaligned tails. (#333)
• WASM: flush() now defaults to write_token=true, matching the Python API, so a mid-stream flush keeps the decompressor byte-aligned. Pass flush(false) at true end-of-stream. (#332)
• CLI: the CLI no longer crashes when built without the C extensions (dictionary building raises a clear error instead), and TAMP_SANITIZE=1 on Windows now fails with a clear message instead of a NameError. (#334)
• Viper Compressor: require an exact dictionary size, matching the other compressors. (#335)

Optimizations

• Embedded find_best_match: word-at-a-time (SWAR) skip over non-matching bytes — ~16% faster compression on Cortex-M0+. (#331)
• ESP32 extended-mode search: faster longest-match search with compile-time long-verify selection. (#333)
• ESP32 decompression: word-at-a-time window and output copies. (#336)

Other

• ESP32 gains host-side differential and fuzz verification against the reference implementation. (#333)
• Packaging: the Python build tooling migrated from Poetry to uv. Binary wheels no longer bundle C/Cython sources and the Linux extension is symbol-stripped, and the sdist has been pruned of dev-only files — both artifacts are significantly smaller. Installation is unchanged. (#339)
• README benchmark and binary-size numbers refreshed to current measurements.

Full Changelog: v2.2.4...v2.3.0

What's Changed

• Use PyPI trusted publishing in upload_to_pypi job by @BrianPugh in #320
• Add a double-FLUSH guard to prevent dictionary-reset desync by @BrianPugh in #319

Full Changelog: v2.2.3...v2.2.4

What's Changed

• Fix non-corrupting off-by-one in ESP32-S3 SIMD match finder by @BrianPugh in #317. Thanks @BitsForPeople

Full Changelog: v2.2.2...v2.2.3

Bug Fixes

• Fix type-limits warnings by @BrianPugh in #313

Full Changelog: v2.2.1...v2.2.2

ESP32-centric release.

Bug Fixes

• Fix type mismatch when TAMP_ESP32=1. by @BrianPugh in #308. Thanks @PierreMConsulting!
• Fix esp32 micropython native module by @BrianPugh in #309

Full Changelog: v2.2.0...v2.2.1

Features

Add the build-dictionary command to the CLI that builds a custom dictionary from a given corpus. Used to improve compression on similar, short messages.

No changes to compressor/decompressor code in this release.

Full Changelog: v2.1.0...v2.2.0

Motivation

Add capabilities that allows for easy appending to a compressed stream (e.g. file) without having to reconstruct the compressor's state.

Summary

• Add dictionary reset via double-FLUSH token sequence, enabling append-to-stream without retaining prior compressor state. Both sides re-initialize the window and continue with a fresh dictionary.
• Uses the previously reserved more_header header bit to signal dictionary-reset-capable streams (as well as an additional header byte to follow, currently 0x00). Old decompressors (<2.1.0) reject these streams at the header, preventing silent corruption.
• Add conf.append mode: writes a FLUSH instead of a header, allowing a new compressor to resume an existing stream.

Protocol

• more_header (header byte 1, bit 0) now additionally signals dictionary_reset
• Header byte 2 added (all bits reserved, must be zero)
• Two consecutive FLUSH tokens signal dictionary re-initialization
• FLUSH is always emitted (even when byte-aligned) in more_header streams to support append detection

API

• TampConf.dictionary_reset / TampConf.append config fields
• tamp_compressor_reset_dictionary() (C) / Compressor.reset_dictionary() (Python/JS)
• Decompressor automatically handles double-FLUSH when more_header is set

Full Changelog: v2.0.0...v2.1.0

Summary

Tamp v2 adds two new compression token types via the extended header flag (enabled by default):

• RLE: Run-length encoding for repeated bytes (up to 241 per token)
• Extended Match: Longer pattern matches beyond v1's min_pattern_size + 13 limit (up to min_pattern_size + 131)

Compression improvements range from 0.2% to 21% depending on dataset, with binary/structured data benefiting most.

Breaking Changes

• The enhanced format (extended=True) cannot be decompressed by tamp <2.0.0. Use extended=False for backwards compatibility.
  • Header bit [1]: Previously reserved (always 0), now indicates extended format. Old decompressors reject v2 streams safely.
• Callback semantics: tamp_callback_t now consistently reports input bytes consumed (was inconsistent in v1).
  • Previously it was inconsistent between APIs.
• CLI: Now an optional extra (pip install tamp[cli]). This allows for the normal tamp package to be installed without all the CLI dependencies (for use as a normal python library).
• Dictionary initialization: tamp_initialize_dictionary takes a new literal parameter; per-literal seed tables for literal=5,6.

Migration Guide

See Migrating to v2 for full details.

Compression Results

Improvements

• Javascript:
  • Use bulk memory transfers for WASM data copying. 1.8x faster decomprssion; 1.03x faster compression.
  • Fix memory-leak if an invalid custom dictionary is provided.
  • See #285 for details

Full Changelog: v1.11.0...v1.11.1

Breaking Changes (C Library)

tamp_decompressor_init now requires a window_bits parameter.

Before:

tamp_decompressor_init(&decompressor, NULL, window);

After:

// window_bits specifies the maximum window size the buffer can hold (8-15)
// The buffer must be at least (1 << window_bits) bytes
tamp_decompressor_init(&decompressor, NULL, window, 10);

This change enables runtime validation that the provided window buffer is large enough for the data being decompressed, preventing buffer overflows when decompressing data with a larger window size than expected.

Features

• C Library: New Stream API - High-level functions for compressing/decompressing data using I/O callbacks. Works with any backend (stdio, LittleFS, FatFs, memory buffers, UART, etc.).

  #include "tamp/compressor.h"
  #include "tamp/common.h"
  
  // Compress a file using stdio
  FILE *in = fopen("input.bin", "rb");
  FILE *out = fopen("output.tamp", "wb");
  unsigned char window[1024];
  TampCompressor compressor;
  
  tamp_compressor_init(&compressor, NULL, window);
  tamp_compress_stream(
      &compressor,
      tamp_stream_stdio_read, in,    // read callback and handle
      tamp_stream_stdio_write, out,  // write callback and handle
      NULL, NULL,                    // optional size outputs
      NULL, NULL                     // optional progress callback
  );
  fclose(in);
  fclose(out);

  Built-in I/O backends (enable via compiler flags):

  Backend	Flag	Use Case
  Memory	-DTAMP_STREAM_MEMORY=1	In-memory compression
  stdio	-DTAMP_STREAM_STDIO=1	Desktop/POSIX, ESP-IDF VFS
  LittleFS	-DTAMP_STREAM_LITTLEFS=1	Embedded flash filesystems
  FatFs	-DTAMP_STREAM_FATFS=1	SD cards, USB storage

• Performance improvements:

  • 2-3x faster compression on desktop - New platform-optimized find_best_match implementation for x86_64 and aarch64 using SIMD-friendly bit manipulation.
  • Faster compression on embedded targets - Optimized match-finding for Cortex-M and other 32-bit microcontrollers.
  • Faster decompression - Optimized hot path and reduced branch mispredictions.

Full Changelog: v1.10.4...v1.11.0