fix: support iOS soft keyboard input in VM consoles - #16029

No actionable comments were generated in the recent review. 🎉

sequenceDiagram
  participant IOSSoftKeyboard
  participant InputModel
  participant VmKeyboardUtils
  participant RustSession
  IOSSoftKeyboard->>InputModel: inputVmKeyboardText(text)
  InputModel->>VmKeyboardUtils: Convert text to HID strokes
  VmKeyboardUtils-->>InputModel: Return key events
  InputModel->>RustSession: Send flutter_key_map events
  RustSession->>RustSession: Process physical key events in map mode

• Fix all pre-merge checks with AI

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

_{Comment @coderabbitai help to get the list of available commands.}

Request changes

The overall approach looks reasonable: the workaround is opt-in and narrowly scoped to iOS → Windows sessions with VM keyboard mode enabled, which helps limit the blast radius. However, this PR changes keyboard/focus/input-dispatch behavior in a stateful path, and I think the regression risk inside this mode is still medium-to-high.

I see two merge-blocking regressions, one compatibility concern, and a few places where the implementation could be narrowed further to reduce regression risk.

1. Toolbar modifiers are lost in the new VM text path

inputVmKeyboardText() bypasses the existing inputKey() path, which is responsible for forwarding the current toolbar modifier state:

alt: alt,
ctrl: ctrl,
shift: shift,
command: command

The new ASCII → HID path only synthesizes Shift when the character itself requires it.

As a result, with VM keyboard mode enabled:

• Ctrl + c from the toolbar becomes plain c

• Toolbar Shift + a does not behave like Shift+A

• Alt-based shortcuts are lost

• Command/Win-based shortcuts are lost

This looks like a real behavioral regression rather than an edge case.

The VM HID path should preserve the current modifier state. Character-required Shift should also be combined with the existing Shift state rather than being emitted independently twice.

Please add tests for at least:

• Ctrl+C

• toolbar Shift + a

• one Alt shortcut

• one Command/Win shortcut

2. The focus guard is too broad and can suppress external hardware keyboard events

The new condition:

if (isIOS &&
    inputModel.vmKeyboardMode &&
    inputModel.peerPlatform == kPeerPlatformWindows &&
    !node.hasPrimaryFocus) {
  return KeyEventResult.skipRemainingHandlers;
}

does not distinguish duplicate soft-keyboard events from real hardware keyboard events.

When the hidden text field has primary focus, RawKeyFocusScope is still an ancestor in the focus tree. Physical keyboard events can bubble through it, and returning skipRemainingHandlers here can prevent them from reaching the normal RustDesk key handling path.

This can regress external iPad keyboard behavior, including:

• arrow/navigation keys

• function keys

• physical Ctrl/Alt/Shift/Command

• shortcuts that are not representable as text

• potentially mismatched key-down/key-up delivery during focus transitions

The last case is particularly concerning because keyboard handling is stateful. If a key-down reaches the remote side but the corresponding key-up is suppressed after focus changes, the remote guest can end up with a stuck modifier/key state.

The suppression logic should be narrowed to the actual duplicate IME/soft-keyboard case instead of suppressing every bubbled key event whenever the text field owns primary focus.

Please add a widget/integration test for:

• VM mode enabled

• mobile text field focused

• external hardware key-down + key-up

• modifier key-down + key-up

• focus transition while a key is active

3. ASCII → HID mapping assumes a US physical keyboard layout

vm_keyboard_utils.dart explicitly converts ASCII characters to physical US keyboard usages.

That is fine for a US-layout guest, but HID map-mode input represents physical key positions, not abstract characters. On AZERTY, German, or other layouts, the same HID usage may produce a different character.

For example, text input containing a is converted to the physical US A key position. A guest using another layout may interpret that position differently.

I would not necessarily block the PR on this alone if the intended scope is explicitly “VM workaround for US-compatible guest layouts”, but the assumption should be documented and ideally reflected in tests or UI expectations.

4. Please minimize changes to shared keyboard/focus behavior

Since this is a workaround for a narrow case, I would strongly prefer the implementation to avoid changing generic input behavior unless it is strictly necessary.

In particular, I would avoid introducing broad conditions into shared focus/key-event handlers when the desired behavior can be implemented inside the VM-specific text-input path.

A safer direction would be:

• keep the existing non-VM keyboard and focus paths unchanged;

• branch into VM-specific behavior as late as possible, ideally only when converting text to HID events;

• avoid changing generic RawKeyFocusScope behavior to solve soft-keyboard duplication;

• avoid adding new persistent input state unless required;

• reuse existing key/modifier dispatch helpers where possible instead of creating a second partially independent keyboard state machine.

This reduces the number of input paths that need to be reasoned about and makes it much easier to verify that users outside the target scenario are unaffected.

5. Prefer reusing existing modifier/key dispatch logic over duplicating it

The current implementation creates a new low-level HID emission path. That means modifier behavior, ordering, press/release balancing, and future keyboard fixes now risk being implemented differently in two places.

If possible, please factor the smallest reusable primitive out of the existing key dispatch path and use it from both normal input and VM text input.

For example, instead of making inputVmKeyboardText() independently responsible for:

• modifier state

• Shift synthesis

• key-down ordering

• key-up ordering

• source selection

it would be safer to reuse the existing modifier-aware key emission logic and only override the part that is actually VM-specific: the HID usage and "flutter_key_map" source.

This would reduce divergence and lower the chance of future regressions when modifier handling changes elsewhere.

6. Keep fallback behavior completely unchanged

For any text that cannot be safely represented by the VM ASCII/HID mapping, the existing Unicode/IME path should remain exactly as it was before this PR.

I would avoid adding new preprocessing, filtering, focus handling, or special cases to the fallback path.

A useful regression test would verify that unsupported/non-ASCII input takes the exact pre-existing path and does not leave any synthetic modifier or key state behind.

Regression risk

I would classify the regression risk as medium-high overall, with an important distinction:

The narrow feature gating reduces the global blast radius, but once this mode is active the PR touches core keyboard, focus, and event-dispatch behavior. Those paths are especially sensitive to modifier state and balanced key-down/key-up handling.

The safest implementation here is therefore not just one that fixes the reported VMware issue, but one that changes as little shared input behavior as possible.

What looks good

The Rust-side routing approach is clean: using a dedicated "flutter_key_map" source to force only these synthesized events through map mode avoids changing the user's persistent keyboard mode.

The ASCII conversion is also easy to reason about, and falling back to the existing Unicode/IME path for unsupported text is the right behavior.

Conclusion

I would Request Changes before merging.

The main concept is good and appropriately scoped, but I think the following should be addressed first:

1. Preserve toolbar modifier state in the VM HID text path.

2. Narrow or remove the generic focus-event suppression so external hardware keyboard events are not swallowed.

3. Keep VM-specific behavior isolated from the existing normal keyboard/focus paths.

4. Reuse existing modifier/key dispatch logic where possible instead of maintaining a second keyboard state machine.

5. Add regression coverage for modifiers, hardware key-down/key-up, focus transitions, and fallback behavior.

The US-layout limitation should also be documented or otherwise made explicit.

Given how narrow the original bug is, I would prefer a smaller patch with fewer changes to shared keyboard/focus behavior, even if that means the VM-specific implementation is slightly more explicit.

Thanks for the detailed review. I agree that the focus guard and the physical-layout assumption needed to be tightened.

Updated in 5910b02e:

• Removed the VM-specific RawKeyFocusScope guard entirely. The shared hardware-keyboard and focus behavior now matches master, so VM mode cannot swallow hardware key-up or modifier events during focus transitions.
• Kept VM conversion only in the iOS soft-keyboard text callback.
• Reused the existing newKeyboardMode physical-key dispatch entry point. The flutter_key_map sentinel only forces Rust map mode for an explicit VM event.
• Preserved balanced Ctrl, Shift, Alt, and Command/Win modifiers. Ctrl/Shift coverage was already present; this update adds Alt and Command/Win tests.
• Documented that ASCII-to-HID conversion is a workaround for US-compatible guest keyboard layouts.
• Unsupported or mixed Unicode input is fully validated before any VM event is sent. Conversion returns false and the caller takes the exact pre-existing Unicode/IME path, with no synthetic modifier/key state created.

I did not add new focus integration behavior or tests because the risky focus behavior was removed rather than replaced; that file now has no diff from master. git diff --check passes. Flutter/Dart and Rust toolchains are not installed in this local preparation environment, so the automated tests still need to run in PR CI after workflow approval.

Request changes

Most of the issues from my previous review are now addressed:

• the broad RawKeyFocusScope guard has been removed;
• Ctrl/Shift/Alt/Command toolbar modifiers are now included in balanced VM HID sequences;
• the VM path is isolated to the iOS soft-keyboard callback;
• unsupported text is validated before any synthetic event is emitted;
• the US-layout assumption is documented.

There is still one stateful modifier issue I think should be fixed before merging.

inputVmKeyboardText() treats ctrl, shift, alt, and command as modifiers that need to be synthesized around every VM stroke. However, these fields are not toolbar-only state: handleKeyDownEventModifiers() also sets them when an external hardware modifier is physically held.

For example:

1. Hold physical Ctrl on an external iPad keyboard. The normal hardware path sends Ctrl-down and sets ctrl = true.
2. While Ctrl is still physically held, enter a character through the software keyboard with VM mode enabled.
3. inputVmKeyboardText() sees ctrl == true and emits another Ctrl-down, the key stroke, and a synthetic Ctrl-up.
4. The remote side now sees Ctrl released even though the physical Ctrl key is still held locally.
5. A subsequent physical key pressed before the real Ctrl-up can therefore arrive without Ctrl active on the remote side.

The same issue applies to Shift, Alt, and Command.

Please avoid synthesizing/releasing a modifier that is already represented by a currently held physical modifier. Ideally toolbar modifier state and physical modifier state should be distinguished; alternatively, the existing tracked physical modifier state (toReleaseKeys) could be used so the VM text path only synthesizes toolbar-only modifiers.

I would also add a regression test covering:

physical modifier down → VM soft-key input → another physical key while the modifier remains held → physical modifier up

and verify that the VM soft-key sequence does not prematurely release the physical modifier.

The existing mapper-level Unicode fallback test is useful, although an InputModel/caller-level test proving that unsupported mixed text emits no VM HID events and takes the existing Unicode path would still be a good follow-up.