Vectorized eager CPU kernels for Kotlin/Native targets (native RTF 2.7–5.8x JVM on identical workloads) #910

Problem

Eager execution performance is a JVM story today: DefaultCpuOps on the JVM uses Panama vector kernels (jdk.incubator.vector), and the priority-100 FFM native kernel pack (skainet-backend-native-cpu) is JVM-only. On Kotlin/Native targets (linuxX64, linuxArm64, macosArm64) the eager op implementations are scalar.

For a framework whose pitch is native performance, the K/N eager tier is the gap: correctness is there (see Evidence — native transcripts are byte-identical to JVM), speed is not.

Evidence (measured 2026-08-08)

Downstream consumer: whisper-cli — a KMP CLI running Whisper base multilingual and Moonshine v2 tiny-streaming-en end-to-end in the SKaiNET DSL on DirectCpuExecutionContext, shipping both a JVM jar and K/N release binaries built from the same source.

Machine: Intel i7-9750H (6C/12T), 32 GB, Linux 6.8. JVM = OpenJDK 21.0.11 + Panama vector kernels. Native = Kotlin/Native 2.4.10 linuxX64 release binary (scalar eager). Same CLI surface, same inputs, single-run methodology (CLI-shaped usage, JIT warmup included). Native transcripts byte-identical to JVM on every run.

Per stage: the Whisper encoder (conv1d + 1500-position attention, d_model 512) is ~6.6× slower on native (30.4 s → 199.8 s); the Moonshine stack (dim 320) ~3.1× (135 s → 421 s). The Panama kernels win most on large conv/matmul shapes.

Root cause (diagnosis)

platformDefaultCpuOpsFactory() is expect/actual per target; the JVM actual dispatches to vector kernels, the native actuals to plain-Kotlin scalar loops. There is no SIMD story for K/N: no intrinsics, no cinterop kernel library for the hot ops.

Proposal

Any of (in increasing ambition):

1. Port the FFM kernel pack to K/N via cinterop — the CMake libskainet_kernels library already exists per-OS/arch for the JVM path; expose the same C ABI to K/N and register it as a high-priority KernelProvider for native targets.
2. Hand-vectorized C kernels for the top-4 hot ops only — matmul, conv1d, softmax, scaledDotProductAttention — measured on Whisper/Moonshine workloads (these dominate the profile; Accelerate on macosArm64 could cover matmul for free where available).
3. Multi-threading for the eager path on native (the JVM path also wins through memory layout + vector width; native is currently single-thread scalar).

The consumer-side seam (KernelProvider service registry, priority ordering) already exists — this is a kernels-and-packaging effort, not an API change.

Acceptance

• On a named linuxX64 and macosArm64 machine, Whisper eager RTF on the K/N binary is within 2× of the JVM Panama-path RTF (same model, same audio, documented single-run methodology).
• No regression for targets without the kernel pack (scalar fallback preserved).
• whisper-cli links the same binary with no source change and republishes its RTF table (it currently labels native binaries "preview" with RTF ≤ 2× JVM as the GA gate — this issue is the tracked enabler).