Invert layout writers to push API - #9423

Summary

• invert the file/layout writer tree to a stateful push API with write, finish, and close
• drive independent struct and list children concurrently through capacity-one mailboxes
• retain the existing hierarchical SequenceId protocol so ordered byte sinks remain deterministic while key/value sinks may ignore ordering
• account for arrays retained by actor mailboxes, buffering/repartition strategies, and spawned compression in buffered_bytes()
• update the TPC-H writer and CUDA layout strategy to use the push API

Motivation

The previous public push writer fed a SendableArrayStream into a background stream-based strategy tree. This made incremental writing indirect and made strategy composition and buffered-byte ownership harder to follow.

The push inversion keeps one independently driven child per structural field and restores the old sequence identifiers to preserve deterministic on-disk segment ordering. Structural fan-out and finish poll sibling children concurrently, so ordered segment output does not depend on a sibling driver that is not being advanced.

Backpressure matches the previous writer's memory model: the public/root input, every structural child edge, and the segment-buffer sink each have capacity one. A slow VortexWrite fills the segment sink first, then leaf and structural mailboxes, and ultimately makes public Writer::write await. Compression retains its separate, explicitly bounded CPU-concurrency window. Queued and in-flight arrays remain byte-accounted until the receiving writer has processed them.

The stream compatibility adapter also explicitly drops its unused sequence pointer before finishing, preventing it from holding an ordering predecessor indefinitely.

Behavior

• Writer::bytes_written() reports bytes passed to the underlying VortexWrite.
• Writer::buffered_bytes() reports logical bytes retained by layout strategies and asynchronous work; it excludes output-sink buffering and allocator/statistics overhead.
• The writer does not add a separate sink flush layer; custom VortexWrite implementations retain control over sink flushing.

Performance

Release in-memory fixture: 16 numeric columns, 8 chunks, 16K rows/chunk, 16 MiB logical input. Each run used two warmups and ten measured iterations against the latest merged develop; each value below is the median of its ten measured iterations.

Mean 1-to-8-worker scaling was 3.10x on develop and 3.65x with the push writer.

Validation

• cargo test -p vortex-layout -p vortex-file
• cargo check -p vortex-layout -p vortex-file -p vortex-cuda -p vortex-bench
• cargo clippy --all-targets --all-features
• cargo +nightly fmt --all
• git diff --check

The layout tests include mailbox saturation/backpressure, cross-field compression overlap with deterministic segment IDs, buffered-byte accounting for queued and spawned work, nested structural ordering, dictionary codes-before-values, and zone maps after data.