Skip to content

[gemma3n] Split into its own family module + conformance (postponed) #377

Description

@michalharakal

Postponed marker (like qwen's #352 lane) — descoped from the llama+gemma conformance release by decision.

Gemma 3n is effectively its own family living inside llm-inference/gemma: hand-rolled Gemma3nRuntime/Gemma3nAttentionBackend/Gemma3nKvCache (never touches gemmaNetwork()), own loaders, AltUp/ActivationSparsity ops — ~2000 LOC with exactly one external consumer (Gemma3nIngestion → kgemma CLI). Its engine delegation still has the defect fixed for gemma4: Gemma3nWeightLoader forces token_embd → GEMMA_DEQUANTIZE_ALL and Gemma3nLoadConfig dequantizes the whole model because the hand-rolled runtime can't consume packed tensors. Maturity gate: 0/5.

When picked up:

  • Move Gemma3n* + AltUp + ActivationSparsity + the gemma-module AttentionBackend into llm-inference/gemma3n (+ llm-runtime/kgemma3n facade), per the Common structure and naming convention for model family modules #346 template — mechanical, one consumer
  • Decide DSL migration vs documented Option-B hand-rolled runtime (AltUp/PLE/sparsity are the DSL-expressibility question)
  • Packed loading (kill the dense-FP32-everything default)
  • Maturity gate (golden parity, model-gated smoke, entry-point parity)

Refs: #346, #374 (the gemma sweep that leaves 3n untouched).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions