Skip to content

perf: add C-contiguous fast path to toBuffer() eliminating per-call allocation - #35

Merged
CodeWithKyrian merged 1 commit into
mainfrom
perf/tobuffer-contiguous-fastpath
Jun 16, 2026
Merged

CodeWithKyrian merged 1 commit into
mainfrom
perf/tobuffer-contiguous-fastpath

Conversation

@CodeWithKyrian

Copy link
Copy Markdown
Contributor

This PR adds a C-contiguous fast path to toBuffer() (and intoBuffer()) that copies data directly from the underlying Rust buffer via memcpy, skipping the intermediate full-array allocation and copy that extract_array_* incurred on every call.

Motivation and Context

fromBuffer() has always been fast: one bulk memcpy into internal storage. toBuffer() was significantly slower and degraded as array sizes grew, making it a bottleneck in ML inference pipelines (e.g. ONNX runtime output buffers, growing KV caches).

Root cause: every toBuffer() call ran extract_array_* which allocated a new Vec and copied the entire array — even when the data was already C-contiguous and could be read in-place. The subsequent copy_array_to_buffer also walked elements one at a time via stride iteration instead of using a bulk slice copy.

What's Changed

  • Added C-contiguous fast path: contiguous arrays copy directly from the internal buffer (0 allocations, 1 memcpy — same cost profile as fromBuffer())
  • Non-C-contiguous views (sliced, transposed, strided) fall through to the existing extract_array_* path
  • copy_array_to_buffer now uses as_slice() + copy_from_slice instead of element-by-element stride iteration

Breaking Changes

None

@CodeWithKyrian
CodeWithKyrian merged commit 62695a1 into main Jun 16, 2026
14 checks passed
@CodeWithKyrian
CodeWithKyrian deleted the perf/tobuffer-contiguous-fastpath branch June 16, 2026 15:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant