Machines fail with "missing IP address" after heavy churn; survives app restart AND full orbctl stop && orbctl start; orphaned bridge/vmenet interfaces remain with zero machines running #2688

Summary

After heavy churn of ephemeral Linux machines (hundreds of orbctl clone/orbctl create + orbctl delete -f cycles from automation, e.g. a self-hosted CI runner autoscaler), every subsequent machine creation fails with:

[-32098] create '<name>': setup: machine didn't start in 30s (missing IP address)

This is not a one-off timing fluke: it fails 100% of the time, for every machine, regardless of distro/arch, once it starts happening. It does not self-heal via:

• Quitting and relaunching the OrbStack app
• A full orbctl stop (stops the entire engine, confirmed via orbctl status → Stopped and the OrbStack Helper vmgr process disappearing entirely, all bridge10x/vmenetN host interfaces removed) followed by orbctl start

I have not yet tested whether a full macOS reboot clears it (didn't want to reboot a machine with other work running without further confirmation), but this is at least the second time this exact failure has recurred for us after a period of heavy machine churn, which suggests even a host reboot may only be a temporary reprieve rather than a real fix.

I also found what looks like a related leak: host-side bridge10x/vmenetN interface pairs are left behind even when zero machines currently exist that would need them.

Environment

• OrbStack version: 2.2.3 (2020300), commit c83556b0ef8f1ba9a33abbb194622b6b7a1c0307
• macOS: 26.1 (build 25B78)
• Hardware: MacBook Pro, Apple M3 Pro, 36 GB RAM
• Kernel: Darwin 25.1.0 ... RELEASE_ARM64_T6030

Steps to reproduce

I don't have a guaranteed from-scratch repro recipe (it took extended heavy churn to first trigger it), but once triggered it reproduces on literally every attempt:

$ orbctl create -a amd64 -u test ubuntu:24.04 diag-test
[-32098] create 'diag-test': setup: machine didn't start in 30s (missing IP address)

The heavy-churn workload that (as far as I can tell) triggered it: a small autoscaler clones a golden base image per CI job (orbctl clone runzero-vm-base-amd64 <job-vm>), and deletes it (orbctl delete -f <job-vm>) once the job finishes (or, in this incident, once the per-VM setup script's own failure trap fired and powered it off). Over about 20 minutes this produced 355 clone/delete cycles (confirmed by counting our own orchestrator's log lines), because every single clone failed the same "missing IP address" way and got cleaned up, so our own retry loop kept making more.

What I checked before assuming it was our own bug

1. Reproduced with a bare orbctl create, zero of our own tooling involved -- rules out anything about our clone/registration script.
2. ~/.orbstack/log/vmgr.log shows the underlying error directly:

   📦 scon | time="09-09 15:00:32" level=error msg="machine is missing IP addresses" container=<name> error="machine didn't start in 30s (missing IP address)"
   

   repeated for every single one of >100 consecutive attempts across different machine names, starting from the very first machine creation attempt after an OrbStack app restart.
3. The previous vmgr session's log (vmgr.1.log) had zero occurrences of this error and shut down cleanly (shutting down → normal unmount/poweroff sequence, [VM] stopped). The new session -- a completely fresh OrbStack Helper vmgr process -- failed on its first attempt. Since vmgr is launched with a -handoff flag (OrbStack Helper vmgr -build-id ... -handoff), I initially assumed this meant the underlying guest VM survives an app restart via handoff, and that a real cold stop of the whole engine would be needed to clear whatever state was stuck.
4. I tested that theory and it was wrong: orbctl stop (no args) genuinely tears down the whole engine -- confirmed via orbctl status reporting Stopped, the OrbStack Helper vmgr process exiting completely, and every bridge10x/vmenetN interface disappearing from ifconfig. orbctl start then brings up a new set of interfaces (vmenet0-vmenet3/bridge100-bridge103 were freshly created). Immediately after this genuinely cold engine cycle, a diagnostic orbctl create still failed identically with "missing IP address." So whatever is stuck survives a full engine restart, not just an app-level relaunch.
5. Orphaned host interfaces even with zero machines: after step 4's single failed orbctl create, orbctl list shows zero running or even existing job machines (only two golden/stopped base images and one unrelated stopped machine) -- yet ifconfig shows 6 bridge10x/vmenetN pairs still present (bridge100-bridge105, vmenet0-vmenet5), each on its own /23-/24 private subnet. One single failed create left behind 6 host-level virtual interfaces with nothing using them.

What I suspect is happening (hypothesis, not confirmed -- OrbStack is closed-source so I can't verify past this point)

Based on docs.orbstack.dev/architecture and log lines like 📦 scon | ... conbr0: port N(vethXXXX) entered ... in vmgr.log, machines appear to run as lightweight containers inside one shared guest kernel, bridged internally via conbr0 with addresses from an internal pool, while bridge10x/vmenetN are the macOS-side vmnet interfaces bridging that shared guest out to the host. If some part of that address/interface bookkeeping doesn't get released on a forced/abnormal teardown (orbctl delete -f, or a machine that powers itself off after an in-guest script failure rather than a clean orbctl delete), it would explain:

• Why it takes sustained heavy churn to first trigger (slow accumulation)
• Why it affects 100% of subsequent attempts once triggered (the allocator itself is stuck/exhausted, not a per-attempt race)
• Why a full engine restart doesn't clear it (host-side interface leak, item 5 above, rather than purely in-guest kernel state)

Related existing issues

These report the same exact error text ("machine didn't start in 30s (missing IP address)") for different specific triggers:

• Cannot create Intel machine without Rosetta (ip missing) #1674 -- missing Rosetta (still open)
• x64 Machine Fails to Create #2179, Can't create openSUSE 15.6 arm64 linux machine #2213, Can't create openSUSE tumbleweed arm64 linux machine #1871 -- specific arch/distro combos, all closed as fixed in v2.2.0

I'm on 2.2.3 (newer than the v2.2.0 fix) and still hit the identical error text, with no Rosetta/distro angle -- plain ubuntu:24.04 on amd64 (Rosetta is installed and other amd64 machines have worked fine on this same host in the past). That, plus the "recurs after heavy churn, survives a full engine restart, leaves orphaned host interfaces" pattern (which none of the above describe), makes me think this is either a regression of the same underlying address-allocation path, or a distinct trigger that happens to surface through the same generic error message. Filing separately since I can't confirm which from the outside.

Impact

For anyone running OrbStack VMs as an automated CI/build backend (ephemeral machine per job, high creation/deletion rate), this eventually wedges the entire host into being unable to create any new machine at all, with no in-app recovery path we've found short of (possibly) a full macOS reboot -- which, per our prior experience with this exact issue, may not even be durable.

Happy to gather more diagnostics if there's something specific that would help narrow this down further (I don't have visibility past orbctl/ifconfig/the log files under ~/.orbstack/log/ since OrbStack itself is closed-source).