Skip to content

Port reusable audio preprocessing components from VoiceMemoApp #10

Description

@michalharakal

Port reusable audio preprocessing components from VoiceMemoApp

Context

[VoiceMemoApp](https://github.com/paulbauriegel/VoiceMemoApp) is an Android application for fully on-device voice recording and transcription.

It already contains several audio preprocessing components that overlap with functionality we want to provide as part of SKaiNET-audio.

The goal of this issue is to move the reusable parts into SKaiNET-audio so that applications such as VoiceMemoApp can use the library instead of maintaining their own DSP implementation.

Scope

Port / implement the following functionality based on the VoiceMemoApp implementation:

  • MixedRadixFft

    • Radix 2 and 5
    • allocation-free processing
    • support for FFT sizes required by Whisper and Parakeet
  • Polyphase sinc resampler with Blackman window

  • Streaming WavAudioReader

    • PCM16
    • PCM24
    • PCM32
    • Float32
    • indexed/windowed access without loading the complete file into memory
  • Whisper-compatible 128-Mel frontend

  • MelConfig.whisperV3() preset

Where useful, the existing VoiceMemoApp implementation can serve as the reference implementation.

Validation

Add shared golden/reference tests for the 128-Mel frontend, ideally generated from the corresponding Hugging Face / Python preprocessing pipeline.

Suggested test data:

  • 2–3 audio clips
  • at least one clip shorter than 30 seconds
  • comparison against the reference Mel output with an appropriate numerical tolerance

This should give us a reproducible reference that can also be used by VoiceMemoApp.

Integration goal

Once the functionality is available in SKaiNET-audio, create a follow-up PR for VoiceMemoApp that replaces the corresponding local DSP implementation with the SKaiNET-audio dependency.

This gives us a real-world integration test and should remove a significant amount of duplicated audio preprocessing code from the application.

Open points

  • Confirm licensing / attribution for code derived from VoiceMemoApp
  • Confirm FFT/window configuration for Parakeet
  • Implement MixedRadixFft
  • Replace direct DFT in MelSpectrogram
  • Add polyphase sinc resampler
  • Extend WAV reader
  • Add MelConfig.whisperV3()
  • Add 128-Mel golden tests
  • Validate against VoiceMemoApp
  • Follow-up PR to migrate VoiceMemoApp to SKaiNET-audio

Reference

VoiceMemoApp

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions