mirror of
https://github.com/emilk/egui.git
synced 2026-06-26 14:49:06 -04:00
Much improved IME (#7967)
<!--
Please read the "Making a PR" section of
[`CONTRIBUTING.md`](https://github.com/emilk/egui/blob/main/CONTRIBUTING.md)
before opening a Pull Request!
* Keep your PR:s small and focused.
* The PR title is what ends up in the changelog, so make it descriptive!
* If applicable, add a screenshot or gif.
* If it is a non-trivial addition, consider adding a demo for it to
`egui_demo_lib`, or a new example.
* Do NOT open PR:s from your `master` branch, as that makes it hard for
maintainers to test and add commits to your PR.
* Remember to run `cargo fmt` and `cargo clippy`.
* Open the PR as a draft until you have self-reviewed it and run
`./scripts/check.sh`.
* When you have addressed a PR comment, mark it as resolved.
Please be patient! I will review your PR, but my time is limited!
-->
* Closes #7809
* Closes #7876
* Closes #7908
* Supersedes #7877
* Supersedes #7898
* The author of the PR above replaced it with #7914, which additionally
fixes another IME issue. I believe that fix deserves a separate PR.
* Reverts #4794
* [x] I have followed the instructions in the PR template
This approach is better than #7898 (#7914) because it correctly handles
all three major IME types (Chinese, Japanese, and Korean) without
requiring a predefined “IME mode”.
## Environments I haved tested this PR in
<details><summary>macOS 15.7.3 (AArch64, Host of other virtual
machines)</summary>
Run command: `cargo run -p egui_demo_app --release`
Tested IMEs:
- builtin Chinese IME (Shuangpin - Simplified)
- builtin Japanese IME (Romaji)
- builtin Korean IME (2-Set)
</details>
<details><summary>Windows 11 25H2 (AArch64, Virtual Machine)</summary>
Build command: `cargo build --release -p egui_demo_app
--target=x86_64-pc-windows-gnu --features=glow --no-default-features`
(I cannot use `wgpu` due to [this
bug](https://github.com/emilk/egui/issues/4381), which prevents
debugging inside the VM. Anyways, the rendering backend should be
irrelevant here.)
Tested IMEs:
- builtin Chinese IME (Shuangpin)
- Sogou IME (Chinese Shuangpin)
- WeType IME (Chinese Shuangpin)
- builtin Japanese IME (Hiragana)
- builtin Korean IME (2 Beolsik)
</details>
<details><summary>Linux [Wayland + IBus] (AArch64, Virtual
Machine)</summary>
Fedora KDE Plasma Desktop 43 [Wayland + IBus 1.5.33-rc2]
(Not working at the moment because of [another
issue](https://github.com/emilk/egui/issues/7485) that will be fixed by
#7983. It is [a complicated
story](https://github.com/emilk/egui/pull/7973#issuecomment-4074627603).
)
> [!NOTE]
>
> IBus is partially broken in this system. The Input Method Selector
refuses to select IBus. As a workaround, I have to open System Settings
-> Virtual Keyboard and select “IBus Wayland” to start an IBus instance
that works in egui.
>
> The funny thing is: the Chinese Intelligent Pinyin IME is broken in
native Apps like System Settings and KWrite, but works correctly in
egui!
>
> <details><summary>Screencast: What</summary>
>
> 
> </details>
Build command: `cross build --release -p egui_demo_app
--target=aarch64-unknown-linux-gnu --features=wayland,wgpu
--no-default-features`
(The Linux toolchain on my mac is somehow broken, so I used `cross`
instead.)
Tested IMEs:
- Chinese Intelligent Pinyin IME (Shuangpin)
- Japanese Anthy IME (Hiragana)
- Korean Hangul IME
</details>
<details><summary>Linux [X11 + Fcitx5] (AArch64, Virtual
Machine)</summary>
Debian 13 [Cinnamon 6.4.10 + X11 + Fcitx5 5.1.2]
Build command: `cross build --release -p egui_demo_app
--target=aarch64-unknown-linux-gnu --features=x11,wgpu
--no-default-features`
Tested IMEs:
- Chinese Shuangpin IME
- Chinese Rime IME with `luna-pinyin`
- Japanese Mozc IME (Hiragana)
- Korean Hangul IME
Unlike macOS and Linux + Wayland, key-release events for keys processed
by the IME are still forwarded to `egui`. These appear to be harmless in
practice.
Unlike on Windows, however, they cannot be filtered reliably because
there are no corresponding key-press events marked as “processed by
IME”.
</details>
---
There are too many possible combinations to test (Operating Systems ×
[Desktop
Environment](https://en.wikipedia.org/wiki/Desktop_environment)s ×
[Windowing System](https://en.wikipedia.org/wiki/Windowing_system)s ×
[IMF](https://wiki.archlinux.org/title/Input_method#Input_method_framework)s
× [IME](https://en.wikipedia.org/wiki/Input_method)s × …), and I only
have access to a limited subset. For example, Google Japanese Input
refused to install on my Windows VM, and some paid Japanese IMEs are not
accessible to me. Therefore, I would appreciate feedback from people
other than me using all kinds of environments.
## Details
There are two possible approaches to removing keyboard events that have
already been processed by an IME:
* Approach 1: Filter out events inside `egui` that appear to have been
received during IME composition.
* Approach 2: Filter out such events in the platform backend
(terminology [borrowed from
imgui](https://github.com/ocornut/imgui/blob/master/docs/BACKENDS.md#using-standard-backends),
e.g. the `egui-winit` crate or the code under `web/` in the `eframe`
crate.).
Both approaches already exist in `egui`:
* #4794 uses the first approach, filtering these events in the
`TextEdit`-related code.
* `eframe` uses the second approach in its web integration. See:
<14afefa252/crates/eframe/src/web/events.rs (L173-L176)>
Compared to the first approach, the second has a clear advantage: when
events are passed from the platform backends into `egui`, they are
simplified and lose information. In contrast, events in the platform
backends are the original events, which allows them to be handled more
flexibly. This is also why #7898 (#7914), which attempts to address the
issue from within the `egui` crate, struggles to make all IMEs work
correctly at the same time and requires manually selecting an “IME
mode”: the events received by `egui` have already been reduced and
therefore lack necessary information.
A more appropriate solution is to consistently follow the second
approach, explicitly requiring platform backends not to forward events
that have already been processed by the IME to `egui`. This is the
method used in this PR. Specifically, this PR works within the
`egui-winit` crate, where the original `KeyboardInput` events can be
accessed. At least for key press events, these can be used directly to
determine whether the event has already been processed by the IME on
Windows (by checking whether `logical_key` equals
`winit:⌨️:NamedKey::Process`). This makes it straightforward to
ensure that all IMEs work correctly at the same time.
This PR also reverts #4794, which took the first approach. It filters
out some events that merely look like they were received during IME
composition but actually are not. It also messes up the order of those
events along the way.
As a result, it caused several IME-related issues. One of the sections
in the Demonstrations below will illustrate these problems.
## Demonstrations
<details><summary>Changes not included in this PR for displaying Unicode
characters in demonstrations</summary>
Download `unifont-17.0.03.otf` from
<https://unifoundry.com/pub/unifont/unifont-17.0.03/font-builds/>, and
place it at `crates/egui_demo_app/src/unifont-17.0.03.otf`.
In `crates/egui_demo_app/src/wrap_app.rs`, add these lines at the
beginning of `impl WrapApp`'s `pub fn new`:
```rust
{
const MAIN_FONT: &'static [u8] = include_bytes!("./unifont-17.0.03.otf");
let mut fonts = egui::FontDefinitions::default();
fonts.font_data.insert(
"main-font".to_owned(),
std::sync::Arc::new(egui::FontData::from_static(MAIN_FONT)),
);
let proportional = fonts
.families
.entry(egui::FontFamily::Proportional)
.or_default();
proportional.insert(0, "main-font".to_owned());
cc.egui_ctx.set_fonts(fonts);
}
```
(I took this from somewhere, but I forgot where it is. Sorry…)
</details>
[GNU Unifont](https://unifoundry.com/unifont/index.html) is licensed
under [OFL-1.1](https://unifoundry.com/OFL-1.1.txt).
### This PR Fixes: Focus on a single-line `TextEdit` is lost after
completing candidate selection with Japanese IME on Windows (#7809)
<details><summary>Screencast: ✅ Japanese IME now behaves correctly while
Korean IME behaves as before</summary>
</details>
### This PR Fixes: Committing Japanese IME text with <kbd>Enter</kbd>
inserts an unintended newline in multiline `TextEdit` on Windows (#7876)
<details><summary>Screencast: ✅ Japanese IME now behaves correctly while
Korean IME behaves as before</summary>
</details>
### This PR Fixes: Backspacing deletes characters during composition in
certain Chinese IMEs (e.g., Sogou) on Windows (#7908)
<details><summary>Screencast: ✅ Sogou IME now behaves
correctly</summary>
</details>
### This PR Obsoletes #4794, because `egui` receives only IME events
during composition from now on
On Windows, “incompatible” events are filtered in `egui-winit`, aligning
the behavior with other systems.
<details><summary>Screencasts</summary>
Some Chinese IMEs on Windows:
The default Japanese IMEs on Windows:
</details>
The 2-set Korean IMEs handle arrow keys differently. It will be
discussed in the next section.
### This PR Reverts #4794, because it introduced several bugs
Some of its bugs have already been worked around in the past, but those
workarounds might also be problematic. For example, #4912 is a
workaround for a bug (#4908) introduced by #4794, and that workaround is
in fact the root cause of the macOS backspacing bug I have worked around
with #7810. (The reversion of #4912 is out of the scope of this PR, I
will do that in #7983.)
#### It Caused: Arrow keys are incorrectly blocked during typical Korean
IME composition
When composing Korean text using 2-Set IMEs, users should still be able
to move the cursor with arrow keys regardless if the composition is
committed.
##### Correct behavior
<details><summary>Screencasts</summary>
macOS TextEdit:
Windows Notepad:
With #4794 reverted, `egui` also behaves correctly (tested on Linux +
Wayland, macOS, and Windows):
</details>
##### Incorrect behavior caused by #4794
`remove_ime_incompatible_events` removed arrow-key events in such cases.
As a result, the first arrow key press only commits the composition, and
users need to press the arrow key again to move the cursor:
<details><summary>Screencast</summary>
</details>
This is essentially the same issue described here:
https://github.com/emilk/egui/pull/7877#issuecomment-3852719948
#### It Caused: Backspacing leaves the last character in Korean IME
pre-edit text not removed on macOS
<details><summary>Screencasts</summary>
Before this PR:
After this PR:
</details>
### Korean IMEs also use <kbd>Enter</kbd> to confirm Hanja selections,
and will not work properly in the Korean “IME mode” proposed by #7898
(#7914)
<details><summary>Screencast: Korean IME using <kbd>Enter</kbd> and
<kbd>Space</kbd> for confirmation (IBus Korean Hangul IME)</summary>
The screencast below demonstrates that some Korean IMEs handle Hanja
selection in a way similar to Japanese IMEs: the
<kbd>Up</kbd>/<kbd>Down</kbd> arrow keys are used to navigate
candidates, and <kbd>Enter</kbd> confirms the selected candidate.
</details>
<details><summary>Screencasts: Another example</summary>
Using the built-in Korean IME on Windows, I type two lines: the first
line in Hangul, and the second line as the same word converted to Hanja.
Correct behavior in Notepad (reference):
Behavior after applying this PR, which matches the Notepad behavior:
Behavior after applying #7914 with the “IME mode” set to Korean (which
is also the behavior before this PR being applied):
On the second line, each time a Hanja character is confirmed, an
unintended newline is inserted. This mirrors the Japanese IME issues
that are supposed to be fixed by setting the “IME mode” to Japanese.
(These Japanese IME issues are fixed in this PR as mentioned before.)
</details>
This commit is contained in:
Umaĵo
GitHub
@@ -9,6 +9,9 @@
|
||||
|
||||
#![expect(clippy::manual_range_contains)]
|
||||
|
||||
#[cfg(target_os = "windows")]
|
||||
use std::collections::HashSet;
|
||||
|
||||
#[cfg(feature = "accesskit")]
|
||||
pub use accesskit_winit;
|
||||
pub use egui;
|
||||
@@ -106,6 +109,12 @@ pub struct State {
|
||||
|
||||
allow_ime: bool,
|
||||
ime_rect_px: Option<egui::Rect>,
|
||||
|
||||
/// Used by [`State::try_on_ime_processed_keyboard_input`] to track key
|
||||
/// release events that should be filtered out. See comments in that method
|
||||
/// for details.
|
||||
#[cfg(target_os = "windows")]
|
||||
pressed_processed_physical_keys: HashSet<winit::keyboard::PhysicalKey>,
|
||||
}
|
||||
|
||||
impl State {
|
||||
@@ -148,6 +157,8 @@ impl State {
|
||||
|
||||
allow_ime: false,
|
||||
ime_rect_px: None,
|
||||
#[cfg(target_os = "windows")]
|
||||
pressed_processed_physical_keys: HashSet::new(),
|
||||
};
|
||||
|
||||
slf.egui_input
|
||||
@@ -364,25 +375,33 @@ impl State {
|
||||
is_synthetic,
|
||||
..
|
||||
} => {
|
||||
// Winit generates fake "synthetic" KeyboardInput events when the focus
|
||||
// is changed to the window, or away from it. Synthetic key presses
|
||||
// represent no real key presses and should be ignored.
|
||||
// See https://github.com/rust-windowing/winit/issues/3543
|
||||
if *is_synthetic && event.state == ElementState::Pressed {
|
||||
// Winit generates fake "synthetic" KeyboardInput events when the focus
|
||||
// is changed to the window, or away from it. Synthetic key presses
|
||||
// represent no real key presses and should be ignored.
|
||||
// See https://github.com/rust-windowing/winit/issues/3543
|
||||
EventResponse {
|
||||
repaint: true,
|
||||
consumed: false,
|
||||
}
|
||||
} else {
|
||||
self.on_keyboard_input(event);
|
||||
let egui_wants_keyboard_input = self.egui_ctx.egui_wants_keyboard_input();
|
||||
|
||||
// When pressing the Tab key, egui focuses the first focusable element, hence Tab always consumes.
|
||||
let consumed = self.egui_ctx.egui_wants_keyboard_input()
|
||||
|| event.logical_key
|
||||
== winit::keyboard::Key::Named(winit::keyboard::NamedKey::Tab);
|
||||
EventResponse {
|
||||
repaint: true,
|
||||
consumed,
|
||||
if let Some(response) =
|
||||
self.try_on_ime_processed_keyboard_input(event, egui_wants_keyboard_input)
|
||||
{
|
||||
response
|
||||
} else {
|
||||
self.on_keyboard_input(event);
|
||||
|
||||
// When pressing the Tab key, egui focuses the first focusable element, hence Tab always consumes.
|
||||
let consumed = egui_wants_keyboard_input
|
||||
|| event.logical_key
|
||||
== winit::keyboard::Key::Named(winit::keyboard::NamedKey::Tab);
|
||||
EventResponse {
|
||||
repaint: true,
|
||||
consumed,
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -526,6 +545,91 @@ impl State {
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(not(target_os = "windows"))]
|
||||
#[expect(clippy::unused_self, clippy::needless_pass_by_ref_mut)]
|
||||
#[inline(always)]
|
||||
fn try_on_ime_processed_keyboard_input(
|
||||
&mut self,
|
||||
_event: &winit::event::KeyEvent,
|
||||
_egui_wants_keyboard_input: bool,
|
||||
) -> Option<EventResponse> {
|
||||
// `KeyboardInput` events processed by the IME are not emitted by
|
||||
// `winit` on non-Windows platforms, so we don't need to do anything
|
||||
// here.
|
||||
|
||||
None
|
||||
}
|
||||
|
||||
#[cfg(target_os = "windows")]
|
||||
#[inline(always)]
|
||||
fn try_on_ime_processed_keyboard_input(
|
||||
&mut self,
|
||||
event: &winit::event::KeyEvent,
|
||||
egui_wants_keyboard_input: bool,
|
||||
) -> Option<EventResponse> {
|
||||
if !self.allow_ime {
|
||||
None
|
||||
} else if event.logical_key == winit::keyboard::NamedKey::Process {
|
||||
// On Windows, the current version of `winit` (0.30.12) has a bug
|
||||
// where `KeyboardInput` events processed by the IME are still
|
||||
// emitted. [^1]
|
||||
//
|
||||
// As a workaround, we detect these events by checking whether their
|
||||
// `logical_key` is `winit::keyboard::NamedKey::Process`, and filter
|
||||
// them out to keep behavior consistent with other platforms.
|
||||
//
|
||||
// `winit::keyboard::NamedKey::Process` is not documented in
|
||||
// `winit`. Reading through its source code, we find that it is
|
||||
// mapped from `VK_PROCESSKEY` on Windows [^2]. (On an unrelated
|
||||
// note, Web is the only other platform that also uses it [^3].)
|
||||
// According to Microsoft, “the IME sets the virtual key value
|
||||
// to `VK_PROCESSKEY` after processing a key input message” [^4].
|
||||
// See also [^5].
|
||||
// (I can't find a documentation page dedicated to this value.)
|
||||
//
|
||||
// TODO(umajho): Remove this workaround once the `winit` bug is fixed
|
||||
// and we've updated to a version that includes the fix. NOTE: Don't
|
||||
// forget to also remove the `pressed_processed_physical_keys` field
|
||||
// and its related code.
|
||||
//
|
||||
// [^1]: https://github.com/rust-windowing/winit/issues/4508
|
||||
// [^2]: https://github.com/rust-windowing/winit/blob/e9809ef54b18499bb4f2cac945719ecc2a61061b/src/platform_impl/windows/keyboard_layout.rs#L946
|
||||
// [^3]: https://developer.mozilla.org/en-US/docs/Web/API/UI_Events/Keyboard_event_key_values
|
||||
// [^4]: https://learn.microsoft.com/en-us/windows/win32/api/imm/nf-imm-immgetvirtualkey#remarks
|
||||
// [^5]: https://learn.microsoft.com/en-us/windows/win32/learnwin32/keyboard-input#character-messages
|
||||
|
||||
self.pressed_processed_physical_keys
|
||||
.insert(event.physical_key);
|
||||
|
||||
Some(EventResponse {
|
||||
repaint: false,
|
||||
consumed: egui_wants_keyboard_input,
|
||||
})
|
||||
} else if event.state == ElementState::Released
|
||||
&& self
|
||||
.pressed_processed_physical_keys
|
||||
.remove(&event.physical_key)
|
||||
{
|
||||
// Unlike key-presses, we can not tell whether a key-release event
|
||||
// is processed by the IME or not by looking at its `logical_key`,
|
||||
// because their `logical_key` is the original value (e.g.
|
||||
// `winit::keyboard::Key::Character(…)`) rather than
|
||||
// `winit::keyboard::Key::Named(winit::keyboard::NamedKey::Process)`.
|
||||
// (See the screencast for Windows in [^1].)
|
||||
// So we track the physical keys of processed key-presses and
|
||||
// filter out the corresponding key-releases.
|
||||
//
|
||||
// [^1]: https://github.com/rust-windowing/winit/issues/4508
|
||||
|
||||
Some(EventResponse {
|
||||
repaint: false,
|
||||
consumed: egui_wants_keyboard_input,
|
||||
})
|
||||
} else {
|
||||
None
|
||||
}
|
||||
}
|
||||
|
||||
/// ## NOTE
|
||||
///
|
||||
/// on Mac even Cmd-C is pressed during ime, a `c` is pushed to Preedit.
|
||||
@@ -1001,6 +1105,16 @@ impl State {
|
||||
let allow_ime = ime.is_some();
|
||||
if self.allow_ime != allow_ime {
|
||||
self.allow_ime = allow_ime;
|
||||
#[cfg(target_os = "windows")]
|
||||
if !self.allow_ime {
|
||||
// Defensively clear the set to avoid unexpected behavior.
|
||||
//
|
||||
// We don't do the same in `ime_event_disable` because the key
|
||||
// release events for IME confirmation keys arrive after
|
||||
// `winit::event::Ime::Disabled`.
|
||||
self.pressed_processed_physical_keys.clear();
|
||||
}
|
||||
|
||||
profiling::scope!("set_ime_allowed");
|
||||
window.set_ime_allowed(allow_ime);
|
||||
}
|
||||
|
||||
@@ -441,6 +441,10 @@ pub enum Event {
|
||||
Text(String),
|
||||
|
||||
/// A key was pressed or released.
|
||||
///
|
||||
/// ## Note for integration authors
|
||||
///
|
||||
/// Key events that has been processed by IMEs should not be sent to `egui`.
|
||||
Key {
|
||||
/// Most of the time, it's the logical key, heeding the active keymap -- for instance, if the user has Dvorak
|
||||
/// keyboard layout, it will be taken into account.
|
||||
|
||||
@@ -993,13 +993,7 @@ fn events(
|
||||
|
||||
let mut any_change = false;
|
||||
|
||||
let mut events = ui.input(|i| i.filtered_events(&event_filter));
|
||||
|
||||
if state.ime_enabled {
|
||||
remove_ime_incompatible_events(&mut events);
|
||||
// Process IME events first:
|
||||
events.sort_by_key(|e| !matches!(e, Event::Ime(_)));
|
||||
}
|
||||
let events = ui.input(|i| i.filtered_events(&event_filter));
|
||||
|
||||
for event in &events {
|
||||
let did_mutate_text = match event {
|
||||
@@ -1232,27 +1226,6 @@ fn events(
|
||||
|
||||
// ----------------------------------------------------------------------------
|
||||
|
||||
fn remove_ime_incompatible_events(events: &mut Vec<Event>) {
|
||||
// Remove key events which cause problems while 'IME' is being used.
|
||||
// See https://github.com/emilk/egui/pull/4509
|
||||
events.retain(|event| {
|
||||
!matches!(
|
||||
event,
|
||||
Event::Key { repeat: true, .. }
|
||||
| Event::Key {
|
||||
key: Key::Backspace
|
||||
| Key::ArrowUp
|
||||
| Key::ArrowDown
|
||||
| Key::ArrowLeft
|
||||
| Key::ArrowRight,
|
||||
..
|
||||
}
|
||||
)
|
||||
});
|
||||
}
|
||||
|
||||
// ----------------------------------------------------------------------------
|
||||
|
||||
/// Returns `Some(new_cursor)` if we did mutate `text`.
|
||||
fn check_for_mutating_key_press(
|
||||
os: OperatingSystem,
|
||||
|
||||
Reference in New Issue
Block a user