Skip to content

DAG DSL: reshape drops shape; RoPE/groupNorm/upsample have no op; and dag{} lacks builders for supported NN ops (layerNorm/rmsNorm/batchNorm/silu/embedding) #666

Description

@michalharakal

DAG DSL → StableHLO: reshape drops its shape; and dag{} lacks builders for many already-supported NN ops

Correction to the original report: I wrongly listed the norm/embedding ops as "missing".
They are supported by the StableHLO converter. Accurate picture below.

Already supported by the converter (lower to real StableHLO, with tests)

conv1d, conv2d, maxPool2d, avgPool2d, batchNorm, layerNorm, rmsNorm,
scaledDotProductAttention, silu/swish (ActivationOperationsConverter), and
embedding (GatherOperationsConverter). See NeuralNetOperationsConverter +
LayerNormConverterTest / RmsNormConverterTest. The full transformer models live in
SKaiNET-developers/SKaiNET-transformers.

1. reshape lowers to an empty function (real converter bug)

reshape(value, Shape) → empty func.func ("Reshape operation requires a target shape
specification"); the target shape is dropped. flatten/transpose/unsqueeze work.

dag { val x = input<FP32>("x", TensorSpec("x", listOf(1,12), "FP32")); output(reshape(x, Shape(1,3,4))) }
// EXPECTED stablehlo.reshape 1x12 -> 1x3x4 ; ACTUAL empty module

2. Genuinely no op (composition/feature gaps)

  • RoPE / rotary position embedding — no converter op (needed by Llama/Gemma/Qwen).
  • groupNormlayerNorm/rmsNorm/batchNorm are present, groupNorm is not.
  • upsample / interpolate — blocks FPN-style vision necks.

3. Ergonomics: dag{} has no builder for the supported NN ops

The converter supports layerNorm/rmsNorm/batchNorm/silu/embedding, but the
dag { … } DSL exposes no builder for them, so a DAG author must hand-build a GraphNode
(as the converter tests do) to reach the supported lowering. Builders
(rmsNorm(x, scale), layerNorm(x, scale, bias), silu(x), embedding(table, ids), …)
would let CNN/transformer graphs be written end-to-end in the DSL.

cc #663

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions