Skip to content

Fix LengthsCapable containers dropping lengths for wav_lens-named layers (fixes #2986) - #3068

Open
oliver0006 wants to merge 1 commit into
speechbrain:developfrom
oliver0006:fix/lengths-capable-wav-lens
Open

Fix LengthsCapable containers dropping lengths for wav_lens-named layers (fixes #2986)#3068
oliver0006 wants to merge 1 commit into
speechbrain:developfrom
oliver0006:fix/lengths-capable-wav-lens

Conversation

@oliver0006

@oliver0006 oliver0006 commented Jul 16, 2026

Copy link
Copy Markdown

Motivation

Reported in #2986: batched inference through EncoderASR produces different transcripts than per-utterance inference, with divergences appearing early in the shorter (padded) utterances — not just in the tail.

Root cause: LengthsCapableSequential (and the lighter LengthsCapableChain) only forward lengths to layers whose forward() declares a parameter literally named lengths (speechbrain/utils/callchains.py::lengths_arg_exists). The HuggingFace integration lobes all name it wav_lens (forward(self, wav, wav_lens=None)), so in the standard encoder: !new:speechbrain.nnet.containers.LengthsCapableSequential inference setup, the wav2vec2/HuBERT/WavLM module is silently called without lengths → make_padding_masks(src, wav_len=None) returns None → the HF model runs with attention_mask=None, and self-attention attends over the zero-padding, shifting every frame of the shorter items in the batch.

Note this is also a train/inference mismatch: training recipes typically call the lobe directly (self.modules.wav2vec2(wavs, wav_lens)), so the attention mask is applied during training but silently dropped at batched inference time.

Changes

  • New speechbrain.utils.callchains.lengths_arg_name(func): returns the name of the relative-lengths argument (lengths or wav_lens), or None if there is none.
  • LengthsCapableSequential.forward and LengthsCapableChain.__call__ now pass the lengths under the layer's actual parameter name.
  • lengths_arg_exists is untouched, so the augmenter and enhancement call sites keep their exact behavior. The takes_lengths attributes are kept in sync (and now also report wav_lens-style layers).

Backward compatibility

  • Layers with a lengths parameter: identical call as before.
  • Layers with neither name: called without lengths, as before.
  • Layers with a wav_lens parameter (the HF lobes): now actually receive the lengths — the intended fix. This deliberately changes batched-inference outputs for affected pipelines, aligning them with the mask-aware training-time behavior.

The issue reporter validated the equivalent fix end-to-end via a user-side wrapper: the WER delta between per-utterance and batched inference dropped to ~1e-3 over 300 samples (the residual is the convolutional front-end seeing the padding, inherent to padded batching). This PR makes that wrapper unnecessary.

Test Plan

  • pytest tests/unittests/test_callchains.py tests/unittests/test_containers.py — 5 passed, including the new test_lengths_arg_name, test_lengths_capable_chain_wav_lens, and test_lengths_capable_sequential_lengths_dispatch (a wav_lens-named module actually receives the lengths through the container).
  • pytest --doctest-modules speechbrain/nnet/containers.py speechbrain/utils/callchains.py tests/unittests/test_augment.py — 17 passed (augmenter unaffected).
  • ruff check and ruff format --check clean on all changed files.

Fixes #2986.


Co-written by a human (@oliver0006) and Claude AI (Fable 5) working together.

LengthsCapableSequential and LengthsCapableChain only forwarded the
lengths argument to layers whose forward() declares a parameter
literally named 'lengths'. The HuggingFace integration lobes (wav2vec2,
HuBERT, WavLM, ...) name it 'wav_lens', so in inference pipelines such
as EncoderASR the encoder silently received no lengths: no attention
mask was built and self-attention attended over the zero-padding,
degrading batched transcriptions relative to per-utterance inference
(speechbrain#2986).

The containers now detect the actual parameter name ('lengths' or
'wav_lens' via the new callchains.lengths_arg_name helper) and pass the
lengths under that name. The public lengths_arg_exists helper keeps its
exact previous semantics, so the augmenter and enhancement call sites
are unaffected; the takes_lengths attributes are kept in sync and now
also report wav_lens-style layers.

Fixes speechbrain#2986

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@oliver0006
oliver0006 force-pushed the fix/lengths-capable-wav-lens branch from 2e0e92a to 6dadab4 Compare July 16, 2026 16:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Running inference per batch alters the generated transcript

1 participant