Fix LengthsCapable containers dropping lengths for wav_lens-named layers (fixes #2986) - #3068
Open
oliver0006 wants to merge 1 commit into
Open
Fix LengthsCapable containers dropping lengths for wav_lens-named layers (fixes #2986)#3068oliver0006 wants to merge 1 commit into
oliver0006 wants to merge 1 commit into
Conversation
LengthsCapableSequential and LengthsCapableChain only forwarded the lengths argument to layers whose forward() declares a parameter literally named 'lengths'. The HuggingFace integration lobes (wav2vec2, HuBERT, WavLM, ...) name it 'wav_lens', so in inference pipelines such as EncoderASR the encoder silently received no lengths: no attention mask was built and self-attention attended over the zero-padding, degrading batched transcriptions relative to per-utterance inference (speechbrain#2986). The containers now detect the actual parameter name ('lengths' or 'wav_lens' via the new callchains.lengths_arg_name helper) and pass the lengths under that name. The public lengths_arg_exists helper keeps its exact previous semantics, so the augmenter and enhancement call sites are unaffected; the takes_lengths attributes are kept in sync and now also report wav_lens-style layers. Fixes speechbrain#2986 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
oliver0006
force-pushed
the
fix/lengths-capable-wav-lens
branch
from
July 16, 2026 16:10
2e0e92a to
6dadab4
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Reported in #2986: batched inference through
EncoderASRproduces different transcripts than per-utterance inference, with divergences appearing early in the shorter (padded) utterances — not just in the tail.Root cause:
LengthsCapableSequential(and the lighterLengthsCapableChain) only forwardlengthsto layers whoseforward()declares a parameter literally namedlengths(speechbrain/utils/callchains.py::lengths_arg_exists). The HuggingFace integration lobes all name itwav_lens(forward(self, wav, wav_lens=None)), so in the standardencoder: !new:speechbrain.nnet.containers.LengthsCapableSequentialinference setup, the wav2vec2/HuBERT/WavLM module is silently called without lengths →make_padding_masks(src, wav_len=None)returnsNone→ the HF model runs withattention_mask=None, and self-attention attends over the zero-padding, shifting every frame of the shorter items in the batch.Note this is also a train/inference mismatch: training recipes typically call the lobe directly (
self.modules.wav2vec2(wavs, wav_lens)), so the attention mask is applied during training but silently dropped at batched inference time.Changes
speechbrain.utils.callchains.lengths_arg_name(func): returns the name of the relative-lengths argument (lengthsorwav_lens), orNoneif there is none.LengthsCapableSequential.forwardandLengthsCapableChain.__call__now pass the lengths under the layer's actual parameter name.lengths_arg_existsis untouched, so the augmenter and enhancement call sites keep their exact behavior. Thetakes_lengthsattributes are kept in sync (and now also reportwav_lens-style layers).Backward compatibility
lengthsparameter: identical call as before.wav_lensparameter (the HF lobes): now actually receive the lengths — the intended fix. This deliberately changes batched-inference outputs for affected pipelines, aligning them with the mask-aware training-time behavior.The issue reporter validated the equivalent fix end-to-end via a user-side wrapper: the WER delta between per-utterance and batched inference dropped to ~1e-3 over 300 samples (the residual is the convolutional front-end seeing the padding, inherent to padded batching). This PR makes that wrapper unnecessary.
Test Plan
pytest tests/unittests/test_callchains.py tests/unittests/test_containers.py— 5 passed, including the newtest_lengths_arg_name,test_lengths_capable_chain_wav_lens, andtest_lengths_capable_sequential_lengths_dispatch(awav_lens-named module actually receives the lengths through the container).pytest --doctest-modules speechbrain/nnet/containers.py speechbrain/utils/callchains.py tests/unittests/test_augment.py— 17 passed (augmenter unaffected).ruff checkandruff format --checkclean on all changed files.Fixes #2986.
Co-written by a human (@oliver0006) and Claude AI (Fable 5) working together.