Skip to content

hifi-decode: Head Switching Filtering, IF to Audio resampling improvement#187

Merged
VideoMem merged 33 commits intooyvindln:vhs_decodefrom
eshaz:hifi-decode-head-switch-filter
Feb 7, 2025
Merged

hifi-decode: Head Switching Filtering, IF to Audio resampling improvement#187
VideoMem merged 33 commits intooyvindln:vhs_decodefrom
eshaz:hifi-decode-head-switch-filter

Conversation

@eshaz
Copy link

@eshaz eshaz commented Feb 2, 2025

  • De-noise head switching pulses
    • The audio is high pass filtered to remove audible sound and sent to scipy.signal.find_peaks to detect the peaks that fit closely to the video system field rate.
    • Peaks are then smoothed using linear interpolation.
  • IF to audio resampling now uses sinc-fastest, sinc-medium for resampling quality medium and high options respectfully. This also removes the IF to audio low-pass filter, which is not needed with sinc resampling.
    • The better resampling method removes artifacts and noise that was audible in some cases.
    • On my machine, the decode is actually faster since the low-pass filter was removed.
    • It also helps with the head switching detection to have a tighter impulse response on the pulses so they are more easily detected and interpolated
  • Overlap blocks in the beginning to avoid periodic clicks when each block is appended to the audio.
  • Improve multi-threading by offloading soundfile encoding to it's own process and various audio processing tasks to the process pool.
  • Fix file extension handling.

Head Switching Noise Reduction example:

  • Top: Without head switching noise reduction
  • Bottom: With head switching noise reduction
    image

Head Switching peak detection debugging graph

head switch detection

@eshaz eshaz marked this pull request as ready for review February 2, 2025 06:44
@eshaz eshaz marked this pull request as draft February 3, 2025 00:37
@eshaz eshaz marked this pull request as ready for review February 3, 2025 02:43
@eshaz eshaz marked this pull request as draft February 4, 2025 04:55
@eshaz
Copy link
Author

eshaz commented Feb 4, 2025

Running a few tests after this latest multi threading update. With this latest update, I am able to get .70x decode rate on a computer that could only do .35x before.

@eshaz eshaz force-pushed the hifi-decode-head-switch-filter branch from 9abd7db to ac9bc0d Compare February 4, 2025 05:34
@eshaz
Copy link
Author

eshaz commented Feb 5, 2025

Tested thoroughly and everything is working well.

@eshaz eshaz marked this pull request as ready for review February 5, 2025 07:08
@VideoMem
Copy link
Collaborator

VideoMem commented Feb 6, 2025

I had no luck running this PR first try.

There seems to be an error here, in main.py

  with as_soundfile(input_file) as f:
        progressB = TimeProgressBar(f.frames, f.frames)
        try:
            print(f"Starting decode...")
            for block, is_last_block in f.blocks(blocksize=block_size, overlap=read_overlap):
                if exit_requested:
                    break

f.blocks does not return two values, at least in my soundfile version API.

Got a workaround with this:

           for block in f.blocks(blocksize=block_size, overlap=read_overlap):
                is_last_block = f.tell() == f.frames
                if exit_requested:
                    break

In my system it runs at 0.09x, considerably slower than before.
It decodes nice sound.

The stop button in the GUI freezes the app, I don't know if the pause works yet.

There is some EOFError kind of exceptions written in the console at file ending and the application doesn't ends.
Maybe the EOFError is not being handled properly when the input is a file.

What soundfile version are you using?

@eshaz
Copy link
Author

eshaz commented Feb 6, 2025

I updated the f.blocks call in the latest commit to fix an issue where piped data wasn't saving the last input, but forgot to double check the soundfile output. I'll get that fixed.

I have version 0.12.1 of soundfile installed. I believe it is the default version that comes in Ubuntu 24.04. I have noticed that the soundfile decoding of flac is really slow, and this might be causing the extra slowness for you. Soundfile decoding was recently added in #186, and before it was using ffmpeg, which decodes much faster, but has bugs for some people. I have been using flac and ffmpeg to pipe the decoded pcm through stdin, and this is much faster and I have never seen an issue doing this. I have an almost 20 year old Core 2 Duo machine that decodes at around 0.04x, but this is with using piped in data from flac. On my laptop, I get around 0.90x, and on an older server with lots of cores I'm getting around 0.97x.

Here's the command I'm using to test with:
flac -d -c --force-raw-format --endian little --sign signed hifi_rf.flac | ~/git/vhs-decode/hifi-decode -t 16 -n -f 20MHz --overwrite - hifi_rf_decoded.flac

I might be able to put the soundfile code in a thread or have it work asynchronously to try to speed it up, but it probably won't ever be as fast as using flac or ffmpeg

Clicking stop on the gui should clear out the decode queue and then gracefully stop. There is a delay before it actually stops while the queue is being cleared. I can update it so stop just immediately ends decoding, if that's what we want it to do.

All of the machines I own run Linux. I used the built in Python threading libraries that should be OS agnostic, but it's not impossible something is different on Windows or Mac.

@eshaz
Copy link
Author

eshaz commented Feb 6, 2025

@VideoMem I fixed the blocks issue, the gui, and the EOF exceptions.

I put the block reading into an async task, which really helps keep the decoders working when piping in from stdin. It might help soundfile, but I suspect that it's CPU bound.

I also moved the sound player into a process so the preview mode has less skips now. I can almost get real-time playback on my laptop.

@harrypm harrypm added the Audio Related to HiFi-Decode or Audio Only Tapes label Feb 6, 2025
@VideoMem
Copy link
Collaborator

VideoMem commented Feb 6, 2025

@eshaz
I found a minor issue ( it might be my numba version (0.59.0) ) when piping data from flac as you described.
Something about np.size() not supported.

I simply changed it to len() on @njit decorated functions and it started working.

Yes, there is some performance issues with soundfile, but by piping it goes a lot faster.
It overheated my laptop!

        # offset array copying logic implemented without numpy seems a bit faster
        out_int16_len = np.size(out_int16)
        overlap_size = min(overlap_size, out_int16_len)
        new_overlap = np.empty(overlap_size, dtype=np.int16)
        result = np.empty(overlap_size + out_int16_len, dtype=np.float64)
        
        # copy the overlapping data into result
        overlap_offset = out_int16_len - overlap_size
        if np.size(overlap_data) == 0:
            for i in range(overlap_size):
                overlap_value = out_int16[i + overlap_offset]
                
                new_overlap[i] = overlap_value
                result[i] = overlap_value

In main.py, changed it to:

        # offset array copying logic implemented without numpy seems a bit faster
        out_int16_len = len(out_int16)
        overlap_size = min(overlap_size, out_int16_len)
        new_overlap = np.empty(overlap_size, dtype=np.int16)
        result = np.empty(overlap_size + out_int16_len, dtype=np.float64)
        
        # copy the overlapping data into result
        overlap_offset = out_int16_len - overlap_size
        if len(overlap_data) == 0:
            for i in range(overlap_size):
                overlap_value = out_int16[i + overlap_offset]
                
                new_overlap[i] = overlap_value
                result[i] = overlap_value

All the other things I tested seems to be working.
The sound quality is good.

With that minor change it's ready for prime time.

@eshaz
Copy link
Author

eshaz commented Feb 7, 2025

@VideoMem Fixed the np.size() issue and it works just fine using len().

I also noticed that some of the audio processing was unintentionally using double precision floating points. I updated these to use float32, and this sped things up considerably. I'm able to get 1.13x decode rate on my laptop now. There's also a constant defined that sets the precision in HiFiDecode.py if anyone wants to change it back to float64.

@VideoMem VideoMem merged commit 56ca756 into oyvindln:vhs_decode Feb 7, 2025
@eshaz eshaz deleted the hifi-decode-head-switch-filter branch March 8, 2025 16:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Audio Related to HiFi-Decode or Audio Only Tapes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants