From b4f9cd71407303087b607e0ae9c1b28b71397cb8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Uma=C4=B5o?= <107099960+umajho@users.noreply.github.com> Date: Tue, 24 Mar 2026 18:58:58 +0800 Subject: [PATCH] Much improved IME (#7967) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Closes #7809 * Closes #7876 * Closes #7908 * Supersedes #7877 * Supersedes #7898 * The author of the PR above replaced it with #7914, which additionally fixes another IME issue. I believe that fix deserves a separate PR. * Reverts #4794 * [x] I have followed the instructions in the PR template This approach is better than #7898 (#7914) because it correctly handles all three major IME types (Chinese, Japanese, and Korean) without requiring a predefined “IME mode”. ## Environments I haved tested this PR in
macOS 15.7.3 (AArch64, Host of other virtual machines) Run command: `cargo run -p egui_demo_app --release` Tested IMEs: - builtin Chinese IME (Shuangpin - Simplified) - builtin Japanese IME (Romaji) - builtin Korean IME (2-Set)
Windows 11 25H2 (AArch64, Virtual Machine) Build command: `cargo build --release -p egui_demo_app --target=x86_64-pc-windows-gnu --features=glow --no-default-features` (I cannot use `wgpu` due to [this bug](https://github.com/emilk/egui/issues/4381), which prevents debugging inside the VM. Anyways, the rendering backend should be irrelevant here.) Tested IMEs: - builtin Chinese IME (Shuangpin) - Sogou IME (Chinese Shuangpin) - WeType IME (Chinese Shuangpin) - builtin Japanese IME (Hiragana) - builtin Korean IME (2 Beolsik)
Linux [Wayland + IBus] (AArch64, Virtual Machine) Fedora KDE Plasma Desktop 43 [Wayland + IBus 1.5.33-rc2] (Not working at the moment because of [another issue](https://github.com/emilk/egui/issues/7485) that will be fixed by #7983. It is [a complicated story](https://github.com/emilk/egui/pull/7973#issuecomment-4074627603). ) > [!NOTE] > > IBus is partially broken in this system. The Input Method Selector refuses to select IBus. As a workaround, I have to open System Settings -> Virtual Keyboard and select “IBus Wayland” to start an IBus instance that works in egui. > > The funny thing is: the Chinese Intelligent Pinyin IME is broken in native Apps like System Settings and KWrite, but works correctly in egui! > >
Screencast: What > > ![2026-03-13 3 10 11 AM](https://github.com/user-attachments/assets/4001cf12-8089-46f5-9cf4-e41d8f77ee24) >
Build command: `cross build --release -p egui_demo_app --target=aarch64-unknown-linux-gnu --features=wayland,wgpu --no-default-features` (The Linux toolchain on my mac is somehow broken, so I used `cross` instead.) Tested IMEs: - Chinese Intelligent Pinyin IME (Shuangpin) - Japanese Anthy IME (Hiragana) - Korean Hangul IME
Linux [X11 + Fcitx5] (AArch64, Virtual Machine) Debian 13 [Cinnamon 6.4.10 + X11 + Fcitx5 5.1.2] Build command: `cross build --release -p egui_demo_app --target=aarch64-unknown-linux-gnu --features=x11,wgpu --no-default-features` Tested IMEs: - Chinese Shuangpin IME - Chinese Rime IME with `luna-pinyin` - Japanese Mozc IME (Hiragana) - Korean Hangul IME Unlike macOS and Linux + Wayland, key-release events for keys processed by the IME are still forwarded to `egui`. These appear to be harmless in practice. Unlike on Windows, however, they cannot be filtered reliably because there are no corresponding key-press events marked as “processed by IME”.
--- There are too many possible combinations to test (Operating Systems × [Desktop Environment](https://en.wikipedia.org/wiki/Desktop_environment)s × [Windowing System](https://en.wikipedia.org/wiki/Windowing_system)s × [IMF](https://wiki.archlinux.org/title/Input_method#Input_method_framework)s × [IME](https://en.wikipedia.org/wiki/Input_method)s × …), and I only have access to a limited subset. For example, Google Japanese Input refused to install on my Windows VM, and some paid Japanese IMEs are not accessible to me. Therefore, I would appreciate feedback from people other than me using all kinds of environments. ## Details There are two possible approaches to removing keyboard events that have already been processed by an IME: * Approach 1: Filter out events inside `egui` that appear to have been received during IME composition. * Approach 2: Filter out such events in the platform backend (terminology [borrowed from imgui](https://github.com/ocornut/imgui/blob/master/docs/BACKENDS.md#using-standard-backends), e.g. the `egui-winit` crate or the code under `web/` in the `eframe` crate.). Both approaches already exist in `egui`: * #4794 uses the first approach, filtering these events in the `TextEdit`-related code. * `eframe` uses the second approach in its web integration. See: Compared to the first approach, the second has a clear advantage: when events are passed from the platform backends into `egui`, they are simplified and lose information. In contrast, events in the platform backends are the original events, which allows them to be handled more flexibly. This is also why #7898 (#7914), which attempts to address the issue from within the `egui` crate, struggles to make all IMEs work correctly at the same time and requires manually selecting an “IME mode”: the events received by `egui` have already been reduced and therefore lack necessary information. A more appropriate solution is to consistently follow the second approach, explicitly requiring platform backends not to forward events that have already been processed by the IME to `egui`. This is the method used in this PR. Specifically, this PR works within the `egui-winit` crate, where the original `KeyboardInput` events can be accessed. At least for key press events, these can be used directly to determine whether the event has already been processed by the IME on Windows (by checking whether `logical_key` equals `winit::keyboard::NamedKey::Process`). This makes it straightforward to ensure that all IMEs work correctly at the same time. This PR also reverts #4794, which took the first approach. It filters out some events that merely look like they were received during IME composition but actually are not. It also messes up the order of those events along the way. As a result, it caused several IME-related issues. One of the sections in the Demonstrations below will illustrate these problems. ## Demonstrations
Changes not included in this PR for displaying Unicode characters in demonstrations Download `unifont-17.0.03.otf` from , and place it at `crates/egui_demo_app/src/unifont-17.0.03.otf`. In `crates/egui_demo_app/src/wrap_app.rs`, add these lines at the beginning of `impl WrapApp`'s `pub fn new`: ```rust { const MAIN_FONT: &'static [u8] = include_bytes!("./unifont-17.0.03.otf"); let mut fonts = egui::FontDefinitions::default(); fonts.font_data.insert( "main-font".to_owned(), std::sync::Arc::new(egui::FontData::from_static(MAIN_FONT)), ); let proportional = fonts .families .entry(egui::FontFamily::Proportional) .or_default(); proportional.insert(0, "main-font".to_owned()); cc.egui_ctx.set_fonts(fonts); } ``` (I took this from somewhere, but I forgot where it is. Sorry…)
[GNU Unifont](https://unifoundry.com/unifont/index.html) is licensed under [OFL-1.1](https://unifoundry.com/OFL-1.1.txt). ### This PR Fixes: Focus on a single-line `TextEdit` is lost after completing candidate selection with Japanese IME on Windows (#7809)
Screencast: ✅ Japanese IME now behaves correctly while Korean IME behaves as before ![7809](https://github.com/user-attachments/assets/6e92f6e6-fed4-46f4-96e1-e0e12dadc360)
### This PR Fixes: Committing Japanese IME text with Enter inserts an unintended newline in multiline `TextEdit` on Windows (#7876)
Screencast: ✅ Japanese IME now behaves correctly while Korean IME behaves as before ![7876](https://github.com/user-attachments/assets/03d2cb22-fd0c-45fe-9132-e59fa39bfcf3)
### This PR Fixes: Backspacing deletes characters during composition in certain Chinese IMEs (e.g., Sogou) on Windows (#7908)
Screencast: ✅ Sogou IME now behaves correctly ![7908](https://github.com/user-attachments/assets/2c63de28-26f0-4387-9c50-dceabfdbe99d)
### This PR Obsoletes #4794, because `egui` receives only IME events during composition from now on On Windows, “incompatible” events are filtered in `egui-winit`, aligning the behavior with other systems.
Screencasts Some Chinese IMEs on Windows: ![2026-03-13 12 25 37 AM](https://github.com/user-attachments/assets/064fd1c7-244b-4053-bd24-c65d768cd943) The default Japanese IMEs on Windows: ![2026-03-13 12 28 33 AM](https://github.com/user-attachments/assets/f799b0b5-350b-4b05-a769-bcef16255bdb)
The 2-set Korean IMEs handle arrow keys differently. It will be discussed in the next section. ### This PR Reverts #4794, because it introduced several bugs Some of its bugs have already been worked around in the past, but those workarounds might also be problematic. For example, #4912 is a workaround for a bug (#4908) introduced by #4794, and that workaround is in fact the root cause of the macOS backspacing bug I have worked around with #7810. (The reversion of #4912 is out of the scope of this PR, I will do that in #7983.) #### It Caused: Arrow keys are incorrectly blocked during typical Korean IME composition When composing Korean text using 2-Set IMEs, users should still be able to move the cursor with arrow keys regardless if the composition is committed. ##### Correct behavior
Screencasts macOS TextEdit: ![2026-03-12 8 04 15 PM](https://github.com/user-attachments/assets/24383568-f51c-4a74-9251-adfd942cad8f) Windows Notepad: ![2026-03-12 8 05 08 PM](https://github.com/user-attachments/assets/5a29a5b5-69b8-407b-b1a4-84fdb8f8847d) With #4794 reverted, `egui` also behaves correctly (tested on Linux + Wayland, macOS, and Windows): ![2026-03-12 8 03 51 PM](https://github.com/user-attachments/assets/fcb7f25c-1329-4eb1-82f2-1cea33dcca73)
##### Incorrect behavior caused by #4794 `remove_ime_incompatible_events` removed arrow-key events in such cases. As a result, the first arrow key press only commits the composition, and users need to press the arrow key again to move the cursor:
Screencast ![2026-03-12 8 06 40 PM](https://github.com/user-attachments/assets/6760c6bd-b6ce-44ea-b192-6bd165191c01)
This is essentially the same issue described here: https://github.com/emilk/egui/pull/7877#issuecomment-3852719948 #### It Caused: Backspacing leaves the last character in Korean IME pre-edit text not removed on macOS
Screencasts Before this PR: ![2026-03-17 10 48 12 PM](https://github.com/user-attachments/assets/88021e7e-caf6-4aa9-8f73-ecffc63cda06) After this PR: ![2026-03-17 10 47 23 PM](https://github.com/user-attachments/assets/379cd0db-24e0-4c0e-a5b4-edb37d3e1df7)
### Korean IMEs also use Enter to confirm Hanja selections, and will not work properly in the Korean “IME mode” proposed by #7898 (#7914)
Screencast: Korean IME using Enter and Space for confirmation (IBus Korean Hangul IME) The screencast below demonstrates that some Korean IMEs handle Hanja selection in a way similar to Japanese IMEs: the Up/Down arrow keys are used to navigate candidates, and Enter confirms the selected candidate. ![2026-03-13 6 39 17 AM](https://github.com/user-attachments/assets/0b054cc6-2251-4689-95a4-d69a9be36371)
Screencasts: Another example Using the built-in Korean IME on Windows, I type two lines: the first line in Hangul, and the second line as the same word converted to Hanja. Correct behavior in Notepad (reference): ![7914-ref](https://github.com/user-attachments/assets/1e9f9315-eb71-497a-b6e2-2b11eb6bbf7f) Behavior after applying this PR, which matches the Notepad behavior: ![7914-7967](https://github.com/user-attachments/assets/26f12b9f-9354-45b8-b2a8-ede28c34c5b1) Behavior after applying #7914 with the “IME mode” set to Korean (which is also the behavior before this PR being applied): ![7914-7914](https://github.com/user-attachments/assets/0f82a019-c491-4b64-a92a-d88d62dfbd84) On the second line, each time a Hanja character is confirmed, an unintended newline is inserted. This mirrors the Japanese IME issues that are supposed to be fixed by setting the “IME mode” to Japanese. (These Japanese IME issues are fixed in this PR as mentioned before.)
--- crates/egui-winit/src/lib.rs | 138 +++++++++++++++++-- crates/egui/src/data/input.rs | 4 + crates/egui/src/widgets/text_edit/builder.rs | 29 +--- 3 files changed, 131 insertions(+), 40 deletions(-) diff --git a/crates/egui-winit/src/lib.rs b/crates/egui-winit/src/lib.rs index 234a9989b..90f0311d5 100644 --- a/crates/egui-winit/src/lib.rs +++ b/crates/egui-winit/src/lib.rs @@ -9,6 +9,9 @@ #![expect(clippy::manual_range_contains)] +#[cfg(target_os = "windows")] +use std::collections::HashSet; + #[cfg(feature = "accesskit")] pub use accesskit_winit; pub use egui; @@ -106,6 +109,12 @@ pub struct State { allow_ime: bool, ime_rect_px: Option, + + /// Used by [`State::try_on_ime_processed_keyboard_input`] to track key + /// release events that should be filtered out. See comments in that method + /// for details. + #[cfg(target_os = "windows")] + pressed_processed_physical_keys: HashSet, } impl State { @@ -148,6 +157,8 @@ impl State { allow_ime: false, ime_rect_px: None, + #[cfg(target_os = "windows")] + pressed_processed_physical_keys: HashSet::new(), }; slf.egui_input @@ -364,25 +375,33 @@ impl State { is_synthetic, .. } => { - // Winit generates fake "synthetic" KeyboardInput events when the focus - // is changed to the window, or away from it. Synthetic key presses - // represent no real key presses and should be ignored. - // See https://github.com/rust-windowing/winit/issues/3543 if *is_synthetic && event.state == ElementState::Pressed { + // Winit generates fake "synthetic" KeyboardInput events when the focus + // is changed to the window, or away from it. Synthetic key presses + // represent no real key presses and should be ignored. + // See https://github.com/rust-windowing/winit/issues/3543 EventResponse { repaint: true, consumed: false, } } else { - self.on_keyboard_input(event); + let egui_wants_keyboard_input = self.egui_ctx.egui_wants_keyboard_input(); - // When pressing the Tab key, egui focuses the first focusable element, hence Tab always consumes. - let consumed = self.egui_ctx.egui_wants_keyboard_input() - || event.logical_key - == winit::keyboard::Key::Named(winit::keyboard::NamedKey::Tab); - EventResponse { - repaint: true, - consumed, + if let Some(response) = + self.try_on_ime_processed_keyboard_input(event, egui_wants_keyboard_input) + { + response + } else { + self.on_keyboard_input(event); + + // When pressing the Tab key, egui focuses the first focusable element, hence Tab always consumes. + let consumed = egui_wants_keyboard_input + || event.logical_key + == winit::keyboard::Key::Named(winit::keyboard::NamedKey::Tab); + EventResponse { + repaint: true, + consumed, + } } } } @@ -526,6 +545,91 @@ impl State { } } + #[cfg(not(target_os = "windows"))] + #[expect(clippy::unused_self, clippy::needless_pass_by_ref_mut)] + #[inline(always)] + fn try_on_ime_processed_keyboard_input( + &mut self, + _event: &winit::event::KeyEvent, + _egui_wants_keyboard_input: bool, + ) -> Option { + // `KeyboardInput` events processed by the IME are not emitted by + // `winit` on non-Windows platforms, so we don't need to do anything + // here. + + None + } + + #[cfg(target_os = "windows")] + #[inline(always)] + fn try_on_ime_processed_keyboard_input( + &mut self, + event: &winit::event::KeyEvent, + egui_wants_keyboard_input: bool, + ) -> Option { + if !self.allow_ime { + None + } else if event.logical_key == winit::keyboard::NamedKey::Process { + // On Windows, the current version of `winit` (0.30.12) has a bug + // where `KeyboardInput` events processed by the IME are still + // emitted. [^1] + // + // As a workaround, we detect these events by checking whether their + // `logical_key` is `winit::keyboard::NamedKey::Process`, and filter + // them out to keep behavior consistent with other platforms. + // + // `winit::keyboard::NamedKey::Process` is not documented in + // `winit`. Reading through its source code, we find that it is + // mapped from `VK_PROCESSKEY` on Windows [^2]. (On an unrelated + // note, Web is the only other platform that also uses it [^3].) + // According to Microsoft, “the IME sets the virtual key value + // to `VK_PROCESSKEY` after processing a key input message” [^4]. + // See also [^5]. + // (I can't find a documentation page dedicated to this value.) + // + // TODO(umajho): Remove this workaround once the `winit` bug is fixed + // and we've updated to a version that includes the fix. NOTE: Don't + // forget to also remove the `pressed_processed_physical_keys` field + // and its related code. + // + // [^1]: https://github.com/rust-windowing/winit/issues/4508 + // [^2]: https://github.com/rust-windowing/winit/blob/e9809ef54b18499bb4f2cac945719ecc2a61061b/src/platform_impl/windows/keyboard_layout.rs#L946 + // [^3]: https://developer.mozilla.org/en-US/docs/Web/API/UI_Events/Keyboard_event_key_values + // [^4]: https://learn.microsoft.com/en-us/windows/win32/api/imm/nf-imm-immgetvirtualkey#remarks + // [^5]: https://learn.microsoft.com/en-us/windows/win32/learnwin32/keyboard-input#character-messages + + self.pressed_processed_physical_keys + .insert(event.physical_key); + + Some(EventResponse { + repaint: false, + consumed: egui_wants_keyboard_input, + }) + } else if event.state == ElementState::Released + && self + .pressed_processed_physical_keys + .remove(&event.physical_key) + { + // Unlike key-presses, we can not tell whether a key-release event + // is processed by the IME or not by looking at its `logical_key`, + // because their `logical_key` is the original value (e.g. + // `winit::keyboard::Key::Character(…)`) rather than + // `winit::keyboard::Key::Named(winit::keyboard::NamedKey::Process)`. + // (See the screencast for Windows in [^1].) + // So we track the physical keys of processed key-presses and + // filter out the corresponding key-releases. + // + // [^1]: https://github.com/rust-windowing/winit/issues/4508 + + Some(EventResponse { + repaint: false, + consumed: egui_wants_keyboard_input, + }) + } else { + None + } + } + /// ## NOTE /// /// on Mac even Cmd-C is pressed during ime, a `c` is pushed to Preedit. @@ -1001,6 +1105,16 @@ impl State { let allow_ime = ime.is_some(); if self.allow_ime != allow_ime { self.allow_ime = allow_ime; + #[cfg(target_os = "windows")] + if !self.allow_ime { + // Defensively clear the set to avoid unexpected behavior. + // + // We don't do the same in `ime_event_disable` because the key + // release events for IME confirmation keys arrive after + // `winit::event::Ime::Disabled`. + self.pressed_processed_physical_keys.clear(); + } + profiling::scope!("set_ime_allowed"); window.set_ime_allowed(allow_ime); } diff --git a/crates/egui/src/data/input.rs b/crates/egui/src/data/input.rs index a52d40233..5e1680334 100644 --- a/crates/egui/src/data/input.rs +++ b/crates/egui/src/data/input.rs @@ -441,6 +441,10 @@ pub enum Event { Text(String), /// A key was pressed or released. + /// + /// ## Note for integration authors + /// + /// Key events that has been processed by IMEs should not be sent to `egui`. Key { /// Most of the time, it's the logical key, heeding the active keymap -- for instance, if the user has Dvorak /// keyboard layout, it will be taken into account. diff --git a/crates/egui/src/widgets/text_edit/builder.rs b/crates/egui/src/widgets/text_edit/builder.rs index a6c41e71c..1f103d2f8 100644 --- a/crates/egui/src/widgets/text_edit/builder.rs +++ b/crates/egui/src/widgets/text_edit/builder.rs @@ -993,13 +993,7 @@ fn events( let mut any_change = false; - let mut events = ui.input(|i| i.filtered_events(&event_filter)); - - if state.ime_enabled { - remove_ime_incompatible_events(&mut events); - // Process IME events first: - events.sort_by_key(|e| !matches!(e, Event::Ime(_))); - } + let events = ui.input(|i| i.filtered_events(&event_filter)); for event in &events { let did_mutate_text = match event { @@ -1232,27 +1226,6 @@ fn events( // ---------------------------------------------------------------------------- -fn remove_ime_incompatible_events(events: &mut Vec) { - // Remove key events which cause problems while 'IME' is being used. - // See https://github.com/emilk/egui/pull/4509 - events.retain(|event| { - !matches!( - event, - Event::Key { repeat: true, .. } - | Event::Key { - key: Key::Backspace - | Key::ArrowUp - | Key::ArrowDown - | Key::ArrowLeft - | Key::ArrowRight, - .. - } - ) - }); -} - -// ---------------------------------------------------------------------------- - /// Returns `Some(new_cursor)` if we did mutate `text`. fn check_for_mutating_key_press( os: OperatingSystem,