feat: v1.7.0 part 1 by lil-jon-crunk · Pull Request #226 · AvionBlock/VoiceCraft

lil-jon-crunk · 2026-05-14T07:56:22Z

Summary

This PR is the first part of the VoiceCraft 1.7.0 update.

It brings the browser/WebRTC transport work into the current codebase, wires it into the server transport layer, updates browser client support, and includes a few cleanup/follow-up fixes found during the port.

SineVector241 · 2026-05-15T10:16:24Z

                        while (x.ConnectionState == VcConnectionState.Connected)
                        {
-                            sw.SpinOnce();
+                            await Task.Delay(100);


Long timeout but whatever IG.

Kept it at 100ms intentionally to replace the previous spin wait with a cheap async wait while still allowing the background service to exit shortly after disconnect. This is not a connection timeout, just the service lifetime polling interval.

SineVector241 · 2026-05-15T10:44:04Z

+
+    private void ConfigureDataChannel(WebRtcSession session, RTCDataChannel dataChannel)
+    {
+        session.DataChannel = dataChannel;


Not exactly sure if WebRTC data channels can handle audio data streaming/packets through data channels as efficiently as its official streaming method.

lil-jon-crunk · 2026-05-15T11:32:58Z

Commit 87ae5c6 is just tests from my local branch. It will be removed, pushed by mistake

Scope this change to the browser WebRTC transport path only. - send numeric signaling message types expected by the server serializer - keep the browser data channel reliable/ordered for login and control packets - surface WebRTC close reasons and deny responses to the browser client instead of timing out - fix WebRTC server capacity handling so the first client is not rejected and login failures return a deny response Audio data transport and broader performance work are intentionally left out for a follow-up.

lil-jon-crunk · 2026-05-17T19:25:45Z

Update pushed for review: 8eac66ed (fix: tighten browser WebRTC connection flow).

Scope is intentionally limited to the browser WebRTC transport path:

use numeric signaling message types expected by the server serializer;
keep the browser data channel reliable/ordered for login and control packets;
surface WebRTC close reasons and deny responses to the browser client instead of waiting for a timeout;
fix WebRTC server capacity handling so the first client is not rejected and login failures return a deny response.

Audio data transport and broader performance changes were left out and should be handled separately.

Validation:

dotnet build VoiceCraft.Server\VoiceCraft.Server.csproj --no-restore
dotnet build VoiceCraft.Client\VoiceCraft.Client.Browser\VoiceCraft.Client.Browser.csproj --no-restore
dotnet build VoiceCraft.Client\VoiceCraft.Client.Windows\VoiceCraft.Client.Windows.csproj --no-restore
dotnet test VoiceCraft.sln --no-restore

Remove encoded audio bytes from McApiOnEntityAudioReceivedPacket. The McApi audio-received event now carries only entity id, timestamp, and frame loudness again. The actual audio payload remains out of McApi event transport for this part and can be handled through a separate channel later.

lil-jon-crunk · 2026-05-17T19:45:52Z

Follow-up pushed: eec5fe06 (fix: keep McApi audio event metadata-only).

This removes encoded audio bytes from McApiOnEntityAudioReceivedPacket again. The McApi audio-received event now carries only entity id, timestamp, and frame loudness. Actual audio payload transport is intentionally left out of this part and can be handled separately later.

Validation:

dotnet build VoiceCraft.Server\VoiceCraft.Server.csproj --no-restore
dotnet build VoiceCraft.Client\VoiceCraft.Client.Browser\VoiceCraft.Client.Browser.csproj --no-restore
dotnet test VoiceCraft.sln --no-restore

Make server cards open explicitly instead of relying on ListBox selection state, which fixes browser clicks that were swallowed by item content and edit/delete buttons. Implement WebRTC ping over the existing data channel path and teach the server to answer info requests for WebRTC sessions, while preserving the UDP unconnected info response behavior for LiteNet. Serialize browser WebRTC connect and ping attempts so the periodic ping path cannot close the active JS peer connection while a user connect is starting. Normalize browser WebAudio ScriptProcessor frame sizes to browser-supported powers of two and run the capture/playback loops directly on the WASM async loop instead of Task.Run. Queue remote ICE candidates until the answer is applied, demote rejected candidates to warnings, and guard WebRTC server session closing against recursive close callbacks. Audio encoding is now created lazily so ping and navigation do not initialize Opus. Full browser audio work remains intentionally deferred until OpusSharp fixes its browser static native linkage; after that we can continue the audio pipeline work without app-side OpusSharp.Natives or custom wasm linking hacks.

lil-jon-crunk · 2026-05-17T21:20:05Z

Pushed a follow-up commit with browser WebRTC fixes: 258fede (fix: stabilize browser WebRTC flow).

What changed:

Server cards now open through an explicit command instead of ListBox selection state. This fixes browser clicks that were swallowed by item content while keeping edit/delete buttons isolated.
WebRTC ping now uses the existing data channel path, and the server can answer info requests for WebRTC sessions without changing the existing LiteNet unconnected info response behavior.
Browser ping/connect are serialized so the periodic ping path cannot close the JS peer connection while a user connection is starting.
Browser audio device frame sizes are normalized to WebAudio-supported ScriptProcessor buffer sizes, which fixes the createScriptProcessor(960) failure.
Remote ICE candidates are queued until the answer is applied, and rejected candidates are downgraded to warnings so they do not look like fatal client errors.
WebRTC server session closing now returns early after the first removal, preventing recursive close callbacks.
Audio encoder creation is lazy now, so ping/navigation do not initialize Opus before actual audio capture is needed.

Current boundary:

Full browser audio work is intentionally deferred until OpusSharp fixes browser static native linkage. No app-side OpusSharp.Natives reference or custom wasm linking workaround was added here.
After the OpusSharp package fix lands, the next step is to continue the actual audio pipeline work on top of this WebRTC client/server baseline.

…ing commands from a listbox.

lil-jon-crunk · 2026-05-18T06:13:12Z

Updated in 5e2b148f.

What changed:

Browser WebRTC no longer hardcodes ICE servers in JS. The browser transport now receives ICE server config from NetworkSettings.WebRtcIceServers, with Google STUN as the default value and room to override it later.
Server WebRTC now has configurable ICE servers, bind address, ICE gather timeout, and UDP port range in WebRtcConfig. This is needed for real deployments where the WebRTC UDP ports must be explicitly exposed/mapped.
Trickle ICE candidates are normalized on the browser side, because server-side candidate messages can arrive without the candidate: prefix while the SDP answer contains the prefixed form.
Ping/connect now has a connection timeout, so the browser does not sit forever while ICE remains in checking.
WebRTC diagnostic logs are opt-in via localStorage['voicecraft.webrtc.debug'] = 'true', so normal users do not get console spam.
Opus encoder/decoder runtime selection now keeps iOS on the statically linked runtime and lets browser use the default runtime path expected by the OpusSharp-side changes.

Deployment note: during testing, signaling on 22633/tcp worked, but ICE stayed in checking when the UDP candidate port was not reachable. For hosted servers we should configure WebRtcConfig.PortRangeStart / PortRangeEnd and expose that UDP range, or use TURN if direct UDP cannot be exposed.

The local test package source was removed before committing; this commit uses the normal OpusSharp package version again. The remaining audio feature work is intentionally left for the next step after the OpusSharp fixes are settled.

Checked locally: server build, browser build, Core tests, Network tests, and Client tests.

lil-jon-crunk · 2026-05-18T06:41:30Z

Updated in a2b58c5.

This fixes the MacOS/iOS publish failures from CI:

VoiceCraft.Client.MacOS.NativeBackgroundService now implements the current IBackgroundService.StartServiceAsync<T>(Action<T, Action<string>, Action<string>>) signature.
VoiceCraft.Client.iOS.NativeBackgroundService was updated the same way.
The internal background task wrappers were switched back from Func<Task> to Action, matching the Windows/Linux/Browser implementations and the current VoiceViewModel call site.

lil-jon-crunk added 2 commits May 14, 2026 10:43

feat: add browser WebRTC transport for 1.7.0

ef1c183

Fix build for Windows & iOS

bc5f302