createRandomAccessSource returns null on Android: full-file heap load OOMs on a 138 MiB GGUF (working ~40-line fix included) #922

What happened

On a physical Android phone (minSdk 24 app, android:largeHeap="true"), loading
SmolLM2-135M-Instruct Q8_0 (138 MiB GGUF) via
LlamaNetworkLoader.fromGguf({ randomAccessSource() }, QuantPolicy.NATIVE_OPTIMIZED)
froze the app ("isn't responding") immediately after the model download finished:

java.lang.OutOfMemoryError: Failed to allocate a 16 byte allocation with
... free bytes and ...MB until OOM

The 16-byte allocation is just the unlucky one — the heap was already full.

Root cause

The Android actual of the factory unconditionally returns null:

// skainet-io/skainet-io-gguf/src/androidMain/kotlin/sk/ainet/io/gguf/RandomAccessSourceFactory.android.kt:13
public actual fun createRandomAccessSource(filePath: String): RandomAccessSource? = null

with the comment "Returns null on Android as file access patterns differ. Callers
should fall back to legacy GGUFReader which loads the full file."

That null selects the non-streaming branch of the loader, which materialises the
entire file first. On Android that means roughly 138 MiB of raw file bytes + about
the same again in built tensors + parsing garbage, all live at once — north of
300 MB on an ART heap capped at 256 MB (512 MB with largeHeap). Unlike ONNX
Runtime / llama.cpp / TFLite, every tensor here is a Kotlin array on the managed
heap, so the cap really bites (tracked separately in #921).

Nothing about Android actually prevents random file access. RandomAccessFile and
positional FileChannel.read exist since API 1, and positional reads don't touch
the shared file pointer, so they satisfy the thread-safety contract documented on
RandomAccessSource (skainet-io/skainet-io-core/src/commonMain/kotlin/sk/ainet/io/RandomAccessSource.kt).

Fix we're running in production

We implemented the source ourselves and passed it in; the OOM disappeared
completely and the same phone now loads the model reliably:

private class AndroidRandomAccessSource(
    private val file: RandomAccessFile,
) : RandomAccessSource {

    private val channel = file.channel
    override val size: Long = file.length()

    override fun readAt(position: Long, length: Int): ByteArray {
        if (length == 0) return ByteArray(0)
        val buffer = ByteArray(length)
        val read = readAt(position, buffer, 0, length)
        return if (read < length) buffer.copyOf(read) else buffer
    }

    override fun readAt(position: Long, buffer: ByteArray, offset: Int, length: Int): Int {
        if (length == 0) return 0
        val target = ByteBuffer.wrap(buffer, offset, length)
        var at = position
        var total = 0
        while (target.hasRemaining()) {
            val read = channel.read(target, at)
            if (read <= 0) break
            at += read
            total += read
        }
        return total
    }

    override fun close() = file.close()
}

This mirrors JvmRandomAccessSource
(skainet-io/skainet-io-core/src/jvmMain/kotlin/sk/ainet/io/JvmRandomAccessSource.kt)
almost exactly.

Suggested shape of the upstream fix

• skainet-io-core currently has no androidMain source set, so the actual can't
  reuse JvmRandomAccessSource directly. Options: add an androidMain with an
  AndroidRandomAccessSource, or a shared jvmAndroidMain intermediate source set
  (the custom native64Main intermediate in skainet-io-core/build.gradle.kts is
  precedent for hand-wired intermediates).
• The same = null actual exists in skainet-io-safetensors and skainet-io-onnx
  androidMain — same latent OOM there; one shared implementation fixes all three.
• Bonus fix: TokenizerFactory.fromGguf(fields) needs StreamingGGUFReader.fields,
  which needs a RandomAccessSource — so today there is no way on Android to build
  a tokenizer from the GGUF's own metadata. This fix restores that too.
• Tests: PosixPreadRandomAccessSourceTest (native64Test, 11 cases) is a ready
  template; the repo currently has no Android test source sets, so a host-side test
  (withHostTest {} in the AGP-KMP android {} block) may be the pragmatic start.

Related: #589 covered the same gap for Kotlin/Native and is closed; Android was
left behind.

Measured on: SmolLM2-135M-Instruct Q8_0, SKaiNET 0.38.0 /
skainet-transformers 0.38.0, Kotlin 2.4.10, minSdk 24 / compileSdk 36, physical
Android phone.

We're happy to open a PR with the implementation + tests if the approach above
(androidMain in skainet-io-core, shared by the three io modules) sounds right.