migrations/egui
1
0
Fork 0
mirror of https://github.com/emilk/egui.git synced 2026-09-01 22:30:03 -04:00
Code Activity

Implement proper visuals for IME composition (#8083)

Browse Source 
<!--
Please read the "Making a PR" section of
[`CONTRIBUTING.md`](https://github.com/emilk/egui/blob/main/CONTRIBUTING.md)
before opening a Pull Request!

* Keep your PR:s small and focused.
* The PR title is what ends up in the changelog, so make it descriptive!
* If applicable, add a screenshot or gif.
* If it is a non-trivial addition, consider adding a demo for it to
`egui_demo_lib`, or a new example.
* Do NOT open PR:s from your `master` branch, as that makes it hard for
maintainers to test and add commits to your PR.
* Remember to run `cargo fmt` and `cargo clippy`.
* Open the PR as a draft until you have self-reviewed it and run
`./scripts/check.sh`.
* When you have addressed a PR comment, mark it as resolved.

Please be patient! I will review your PR, but my time is limited!
-->

* Closes N/A
* [x] I have followed the instructions in the PR template

This PR adds visual support for IME composition, including the cursor
and conversion segment.
These visuals works (mostly) well on native platforms (`egui-winit`). On
the web (`eframe/web`), support is limited by browser capabilities:
Chromium works well, Firefox shows partial improvement, and Safari
remains subpar.

> [!NOTE]
>
> For `eframe` on Windows, this feature is currently gated behind the
`windows_new_ime_composition_visuals` feature flag.

## Details

We extend `egui::ImeEvent::Preedit(String)` to `egui::ImeEvent::Preedit
{ text: String, active_range_chars: Option<std::ops::Range<usize>> }`.
The new `active_range_chars` field enables rendering of:
- the cursor (when the range is empty), and
- the conversion segment (when the range is non-empty)

in IME composition.

In `egui-winit`, we now use the range provided by
`winit::event::Ime::Preedit` instead of ignoring it.

In `eframe/web`, we derive the range from `selectionStart` and
`selectionEnd` on the text agent. This mapping is fully accurate only in
Chromium, but represents the best available approach for now.

## Demonstrations

### Chinese IMEs (Shuangpin)

We can see where the cursor is now.

| What | With this PR | Without this PR |
|-|-|-|
| macOS builtin |<video
src=https://github.com/user-attachments/assets/487c7e7c-ef6d-4a86-8dbc-8c71871b4470
/>|<video
src=https://github.com/user-attachments/assets/49bd5a60-4b90-4e4a-99e0-cd01d3f7030c
/>|
| macOS builtin (light)|<video
src=https://github.com/user-attachments/assets/e84546f6-947b-4cea-a87e-fda903f49164
/>|——|
| Windows builtin |<video
src=https://github.com/user-attachments/assets/fd331884-1f0c-4822-a99e-8140aed54815
/>|——|
| Wayland iBus Intelligent Pinyin |<video
src=https://github.com/user-attachments/assets/b6796c75-1c4e-45e5-b43a-5d8dea320485
/>|——|
| Chromium (Chrome) macOS | Identical to `macOS builtin`. |——|
| Safari macOS | We can now differentiate between IME composition and
text selection, but we still can't tell where the cursor is. |——|
| Firefox (Zen) macOS | Identical to `macOS builtin`. |——|

### Japanese IMEs

We can see where the conversion segment is now.

| What | With this PR | Without this PR |
|-|-|-|
| macOS builtin |<video
src=https://github.com/user-attachments/assets/f2994cd4-a966-4ff0-9590-d263c202ec76
/>|<video
src=https://github.com/user-attachments/assets/7cf5ff35-003d-4f60-8fbf-15c725c3ecb9
/>|
| macOS builtin (light)|<video
src=https://github.com/user-attachments/assets/6f562bdd-12fc-4486-b37b-8fcf11643295
/>|——|
| Windows builtin |<video
src=https://github.com/user-attachments/assets/f0905659-5335-4034-abda-c25cf8f2fd57
/>|——|
| Wayland iBus Anthy |<video
src=https://github.com/user-attachments/assets/94cd3a24-3158-4d79-ae02-d9b30fdfa738
/>|——|
| Chromium (Chrome) macOS | Identical to `macOS builtin`. |——|
| Safari macOS | We can now differentiate between IME composition and
text selection, but we still can't tell where the conversion segment is.
|——|
| Firefox (Zen) macOS | (Limited improvement.) <video
src=https://github.com/user-attachments/assets/3daf9b63-6e75-467b-8515-31c2a44adf61
/> |——|

### Korean IMEs

We can clearly tell whether we are in composition (in contrast to
selection) now.

| What | With this PR | Without this PR |
|-|-|-|
|macOS builtin|<video
src=https://github.com/user-attachments/assets/73ca28c7-22a0-493f-8f4d-c6e59a2dec54
/>|<video
src=https://github.com/user-attachments/assets/f582de7d-7ec0-48fe-910f-0139ef1620d3
/>|
|macOS builtin (light)|<video
src=https://github.com/user-attachments/assets/269f03bd-6f95-498b-9fb1-1adcb043c738
/>|——|
| Windows builtin| (With a workaround for [this `winit`
bug](https://github.com/emilk/egui/pull/8083#issuecomment-4206742668)
applied.) <video
src=https://github.com/user-attachments/assets/1e82583d-0c41-4f1c-98cf-0606bee5af05
/>|——|
| Wayland iBus Hangul |<video
src=https://github.com/user-attachments/assets/8c9a0de1-9027-4b37-93a3-e9da0251d176
/>|——|
| Chromium (Chrome) macOS | Identical to `macOS builtin`. |——|
| Safari macOS | Identical to `Windows builtin`. |——|
| Firefox (Zen) macOS | Identical to `macOS builtin`. (ignoring the fact
that the composition breaks when typing the second Hangul. (This bug
predates this PR.)) |——|

---------

Co-authored-by: lucasmerlin <hi@lucasmerlin.me>
This commit is contained in:
Umaĵo
2026-06-26 00:21:19 +08:00
committed by GitHub
parent a8d09eb60d
commit 5bf62ca4b3
11 changed files with 480 additions and 37 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
crates/egui/src/data/input/ime_event.rs
View File

@@ -12,7 +12,10 @@ pub enum ImeEvent {
///
/// An empty preedit string indicates that the IME has been dismissed, while
/// a non-empty preedit string indicates that the IME is active.
Preedit(String),
Preedit {
text: String,
active_range_chars: Option<std::ops::Range<usize>>,
},
/// IME composition ended with this final result.
///

105
crates/egui/src/style.rs
View File

@@ -1026,6 +1026,7 @@ pub struct Visuals {
pub widgets: Widgets,
pub selection: Selection,
pub ime_composition: ImeComposition,
/// The color used for [`crate::Hyperlink`],
pub hyperlink_color: Color32,
@@ -1192,6 +1193,36 @@ pub struct Selection {
pub stroke: Stroke,
}
/// Visual style for IME composition.
#[derive(Clone, Copy, Debug, PartialEq)]
#[cfg_attr(feature = "serde", derive(serde::Deserialize, serde::Serialize))]
#[cfg_attr(feature = "serde", serde(default))]
pub struct ImeComposition {
/// Stroke used to underline the actively composed segment.
pub active_underline_stroke: Stroke,
/// Stroke used to underline those non-active segments.
pub inactive_underline_stroke: Stroke,
/// If `true`, IME (Input Method Editor) composition (preedit) text is rendered
/// the legacy way: visually indistinguishable from a text selection, with the
/// cursor always shown at the end of the composition.
///
/// If `false`, egui renders proper IME composition visuals: the cursor position
/// inside the composition is shown, and the active conversion segment is
/// highlighted (using the strokes configured above) distinctly from the rest of the
/// composition. This makes composing Chinese, Japanese and Korean text much
/// clearer.
///
/// The legacy visuals have known shortcomings, but the new visuals are not yet
/// fully reliable on every platform either (e.g. `winit` reports an incorrect
/// cursor position for Korean IMEs on Windows), so this remains configurable.
///
/// Defaults to `true` on Windows (because of the aforementioned `winit` bug) and
/// to `false` everywhere else.
pub legacy_visuals: bool,
}
/// Shape of the handle for sliders and similar widgets.
#[derive(Clone, Copy, Debug, PartialEq)]
#[cfg_attr(feature = "serde", derive(serde::Deserialize, serde::Serialize))]
@@ -1468,6 +1499,7 @@ impl Visuals {
weak_text_color: None,
widgets: Widgets::default(),
selection: Selection::default(),
ime_composition: ImeComposition::default(),
hyperlink_color: Color32::from_rgb(90, 170, 255),
faint_bg_color: Color32::from_additive_luminance(5), // visible, but barely so
extreme_bg_color: Color32::from_gray(10), // e.g. TextEdit background
@@ -1531,6 +1563,7 @@ impl Visuals {
},
widgets: Widgets::light(),
selection: Selection::light(),
ime_composition: ImeComposition::light(),
hyperlink_color: Color32::from_rgb(0, 155, 255),
faint_bg_color: Color32::from_additive_luminance(5), // visible, but barely so
extreme_bg_color: Color32::from_gray(255), // e.g. TextEdit background
@@ -1594,6 +1627,48 @@ impl Default for Selection {
}
}
impl ImeComposition {
fn dark() -> Self {
// Same as the default value of [`TextCursorStyle::stroke`] in dark mode.
let active_underline_stroke = Stroke::new(2.0, Color32::from_rgb(192, 222, 255));
let inactive_underline_stroke = Stroke {
width: active_underline_stroke.width,
color: active_underline_stroke.color.linear_multiply(0.5),
};
Self {
active_underline_stroke,
inactive_underline_stroke,
legacy_visuals: Self::default_legacy_visuals(),
}
}
fn light() -> Self {
// Same as the default value of [`TextCursorStyle::stroke`] in light mode.
let active_underline_stroke = Stroke::new(2.0, Color32::from_rgb(0, 83, 125));
let inactive_underline_stroke = Stroke {
width: active_underline_stroke.width,
color: active_underline_stroke.color.linear_multiply(0.5),
};
Self {
active_underline_stroke,
inactive_underline_stroke,
legacy_visuals: Self::default_legacy_visuals(),
}
}
/// The default of [`Self::legacy_visuals`]: `true` on Windows (where `winit`
/// reports an incorrect cursor position for Korean IMEs), `false` elsewhere.
const fn default_legacy_visuals() -> bool {
cfg!(windows)
}
}
impl Default for ImeComposition {
fn default() -> Self {
Self::dark()
}
}
impl Widgets {
pub fn dark() -> Self {
Self {
@@ -2113,6 +2188,34 @@ impl Selection {
}
}
impl ImeComposition {
pub fn ui(&mut self, ui: &mut crate::Ui) {
let Self {
active_underline_stroke,
inactive_underline_stroke,
legacy_visuals,
} = self;
ui.label("IME composition");
ui.checkbox(legacy_visuals, "Legacy visuals").on_hover_text(
"If enabled, IME composition (preedit) text looks like a text selection \
with the cursor at the end. If disabled, the cursor position and active \
conversion segment are shown.",
);
Grid::new("ime_composition").num_columns(2).show(ui, |ui| {
ui.label("Active underline stroke");
ui.add(active_underline_stroke);
ui.end_row();
ui.label("Inactive underline stroke");
ui.add(inactive_underline_stroke);
ui.end_row();
});
}
}
impl WidgetVisuals {
pub fn ui(&mut self, ui: &mut crate::Ui) {
let Self {
@@ -2169,6 +2272,7 @@ impl Visuals {
weak_text_color,
widgets,
selection,
ime_composition,
hyperlink_color,
faint_bg_color,
extreme_bg_color,
@@ -2376,6 +2480,7 @@ impl Visuals {
ui.collapsing("Widgets", |ui| widgets.ui(ui));
ui.collapsing("Selection", |ui| selection.ui(ui));
ui.collapsing("IME composition", |ui| ime_composition.ui(ui));
ui.collapsing("Misc", |ui| {
ui.add(Slider::new(resize_corner_size, 0.0..=20.0).text("resize_corner_size"));

144
crates/egui/src/text_selection/visuals.rs
View File

@@ -1,6 +1,17 @@
use std::sync::Arc;
use crate::{Galley, Painter, Rect, Ui, Visuals, pos2, vec2};
use emath::Pos2;
use epaint::{
Stroke,
text::{
CharIndex,
cursor::{CCursor, LayoutCursor},
},
};
use crate::{
Galley, Painter, Rect, Ui, Visuals, pos2, text_selection::text_cursor_state::cursor_rect, vec2,
};
use super::CCursorRange;
@@ -121,6 +132,135 @@ pub fn paint_text_selection(
}
}
#[expect(clippy::too_many_arguments)]
pub(crate) fn paint_ime_preedit_text_visuals(
pos: Pos2,
ui: &Ui,
painter: &Painter,
galley: &Arc<Galley>,
row_height: f32,
preedit_range: std::ops::Range<CCursor>,
mut relative_active_range: Option<std::ops::Range<CCursor>>,
time_since_last_interaction: f64,
) {
/// Instead of implementing [`PartialOrd`] and [`Ord`] for [`CCursor`] to
/// make [`std::ops::Range::is_empty`] available, we use this helper
/// function instead.
///
/// These traits are intentionally not implemented because
/// [`CCursor::prefer_next_row`] makes it difficult to define a clear
/// ordering between two [`CCursor`]s.
fn is_cursor_range_empty(range: &std::ops::Range<CCursor>) -> bool {
range.start.index == range.end.index
}
if is_cursor_range_empty(&preedit_range) {
return;
}
if let Some(relative_active_range) = &mut relative_active_range
&& relative_active_range.end.index > preedit_range.end.index - preedit_range.start.index
{
relative_active_range.end.index = preedit_range.end.index - preedit_range.start.index;
}
let visuals = ui.visuals();
let active_underline_stroke = visuals.ime_composition.active_underline_stroke;
let inactive_underline_stroke = visuals.ime_composition.inactive_underline_stroke;
if let Some(relative_active_range) = &relative_active_range
&& !is_cursor_range_empty(relative_active_range)
{
if relative_active_range.start.index > CharIndex::ZERO {
paint_underlines(
pos,
painter,
galley,
galley.layout_from_cursor(preedit_range.start),
galley.layout_from_cursor(preedit_range.start + relative_active_range.start.index),
inactive_underline_stroke,
);
}
paint_underlines(
pos,
painter,
galley,
galley.layout_from_cursor(preedit_range.start + relative_active_range.start.index),
galley.layout_from_cursor(preedit_range.start + relative_active_range.end.index),
active_underline_stroke,
);
if !is_cursor_range_empty(
&(relative_active_range.end..(preedit_range.end - preedit_range.start.index)),
) {
paint_underlines(
pos,
painter,
galley,
galley.layout_from_cursor(preedit_range.start + relative_active_range.end.index),
galley.layout_from_cursor(preedit_range.end),
inactive_underline_stroke,
);
}
} else {
paint_underlines(
pos,
painter,
galley,
galley.layout_from_cursor(preedit_range.start),
galley.layout_from_cursor(preedit_range.end),
inactive_underline_stroke,
);
}
if let Some(relative_active_range) = relative_active_range
&& is_cursor_range_empty(&relative_active_range)
{
let active_cursor = preedit_range.start + relative_active_range.start.index;
let cursor_rect = cursor_rect(galley, &active_cursor, row_height);
paint_text_cursor(
ui,
painter,
cursor_rect.translate(pos.to_vec2()),
time_since_last_interaction,
);
}
}
fn paint_underlines(
pos: Pos2,
painter: &Painter,
galley: &Arc<Galley>,
min: LayoutCursor,
max: LayoutCursor,
stroke: Stroke,
) {
for ri in min.row..=max.row {
let placed_row = &galley.rows[ri];
let row = &placed_row.row;
let left = if ri == min.row {
row.x_offset(min.column)
} else {
0.0
};
let right = if ri == max.row {
row.x_offset(max.column)
} else {
row.size.x
};
let offset_y = placed_row.pos.y + row.size.y;
painter.line_segment(
[pos + vec2(left, offset_y), pos + vec2(right, offset_y)],
stroke,
);
}
}
/// Paint one end of the selection, e.g. the primary cursor.
///
/// This will never blink.
@@ -130,7 +270,7 @@ pub fn paint_cursor_end(painter: &Painter, visuals: &Visuals, cursor_rect: Rect)
let top = cursor_rect.center_top();
let bottom = cursor_rect.center_bottom();
painter.line_segment([top, bottom], (stroke.width, stroke.color));
painter.line_segment([top, bottom], stroke);
if false {
// Roof/floor:

72
crates/egui/src/widgets/text_edit/builder.rs
View File

@@ -827,10 +827,15 @@ impl TextEdit<'_> {
false
};
let should_paint_ime_visuals_the_legacy_way = ui.visuals().ime_composition.legacy_visuals;
if ui.is_rect_visible(inner_rect) {
let has_focus = ui.memory(|mem| mem.has_focus(id));
if has_focus && let Some(cursor_range) = state.cursor.range(&galley) {
if has_focus
&& (state.cursor_purpose.is_selection() || should_paint_ime_visuals_the_legacy_way)
&& let Some(cursor_range) = state.cursor.range(&galley)
{
// Add text selection rectangles to the galley:
paint_text_selection(&mut galley, ui.visuals(), &cursor_range, None);
}
@@ -862,12 +867,37 @@ impl TextEdit<'_> {
// * Don't repaint the ui because of a blinking cursor in an app that is not in focus
let viewport_has_focus = ui.input(|i| i.focused);
if viewport_has_focus {
text_selection::visuals::paint_text_cursor(
ui,
&painter,
primary_cursor_rect,
now - state.last_interaction_time,
);
let time_since_last_interaction = now - state.last_interaction_time;
let cursor_purpose = if should_paint_ime_visuals_the_legacy_way {
&TextEditCursorPurpose::Selection
} else {
&state.cursor_purpose
};
match cursor_purpose {
TextEditCursorPurpose::Selection => {
text_selection::visuals::paint_text_cursor(
ui,
&painter,
primary_cursor_rect,
time_since_last_interaction,
);
}
TextEditCursorPurpose::ImeComposition { active_range } => {
text_selection::visuals::paint_ime_preedit_text_visuals(
galley_pos,
ui,
&painter,
&galley,
row_height,
{
let [start, end] = cursor_range.sorted_cursors();
start..end
},
active_range.clone(),
time_since_last_interaction,
);
}
}
}
if ui.memory(|mem| mem.owns_ime_events(id)) {
// Set IME output (in screen coords) when text is editable and visible
@@ -1182,25 +1212,37 @@ fn events(
// integration, but since this guard is harmless for well-
// behaved integrations and also fixes the issue described
// above, it is good enough for now.
ImeEvent::Preedit(composition_text) | ImeEvent::Commit(composition_text)
ImeEvent::Preedit {
text: composition_text,
..
}
| ImeEvent::Commit(composition_text)
if composition_text.is_empty()
&& !matches!(
state.cursor_purpose,
TextEditCursorPurpose::ImeComposition
) =>
&& !state.cursor_purpose.is_ime_composition() =>
{
None
}
ImeEvent::Preedit(composition_text) | ImeEvent::Commit(composition_text)
ImeEvent::Preedit {
text: composition_text,
..
}
| ImeEvent::Commit(composition_text)
if composition_text == "\n" || composition_text == "\r" =>
{
None
}
ImeEvent::Preedit(preedit_text) => {
ImeEvent::Preedit {
text: preedit_text,
active_range_chars,
} => {
state.cursor_purpose = if preedit_text.is_empty() {
TextEditCursorPurpose::Selection
} else {
TextEditCursorPurpose::ImeComposition
TextEditCursorPurpose::ImeComposition {
active_range: active_range_chars.clone().map(|range| {
CCursor::new(range.start)..CCursor::new(range.end)
}),
}
};
let mut ccursor = clear_preedit_text(text, &cursor_range);

24
crates/egui/src/widgets/text_edit/state.rs
View File

@@ -1,5 +1,7 @@
use std::sync::Arc;
use epaint::text::cursor::CCursor;
use crate::mutex::Mutex;
use crate::{
@@ -85,6 +87,24 @@ pub(crate) enum TextEditCursorPurpose {
#[default]
Selection,
/// The cursor is used for IME composition.
ImeComposition,
/// The cursor is used for IME composition. Its direction is irrelevant in
/// this case.
ImeComposition {
/// An optional cursor/segment within the composing text itself,
/// relative to the start of the composing region. Its direction is
/// irrelevant.
///
/// When `None`, no active range is displayed.
active_range: Option<std::ops::Range<CCursor>>,
},
}
impl TextEditCursorPurpose {
pub(crate) fn is_selection(&self) -> bool {
matches!(self, Self::Selection)
}
pub(crate) fn is_ime_composition(&self) -> bool {
matches!(self, Self::ImeComposition { .. })
}
}