5bf62ca4b3
<!-- Please read the "Making a PR" section of [`CONTRIBUTING.md`](https://github.com/emilk/egui/blob/main/CONTRIBUTING.md) before opening a Pull Request! * Keep your PR:s small and focused. * The PR title is what ends up in the changelog, so make it descriptive! * If applicable, add a screenshot or gif. * If it is a non-trivial addition, consider adding a demo for it to `egui_demo_lib`, or a new example. * Do NOT open PR:s from your `master` branch, as that makes it hard for maintainers to test and add commits to your PR. * Remember to run `cargo fmt` and `cargo clippy`. * Open the PR as a draft until you have self-reviewed it and run `./scripts/check.sh`. * When you have addressed a PR comment, mark it as resolved. Please be patient! I will review your PR, but my time is limited! --> * Closes N/A * [x] I have followed the instructions in the PR template This PR adds visual support for IME composition, including the cursor and conversion segment. These visuals works (mostly) well on native platforms (`egui-winit`). On the web (`eframe/web`), support is limited by browser capabilities: Chromium works well, Firefox shows partial improvement, and Safari remains subpar. > [!NOTE] > > For `eframe` on Windows, this feature is currently gated behind the `windows_new_ime_composition_visuals` feature flag. ## Details We extend `egui::ImeEvent::Preedit(String)` to `egui::ImeEvent::Preedit { text: String, active_range_chars: Option<std::ops::Range<usize>> }`. The new `active_range_chars` field enables rendering of: - the cursor (when the range is empty), and - the conversion segment (when the range is non-empty) in IME composition. In `egui-winit`, we now use the range provided by `winit::event::Ime::Preedit` instead of ignoring it. In `eframe/web`, we derive the range from `selectionStart` and `selectionEnd` on the text agent. This mapping is fully accurate only in Chromium, but represents the best available approach for now. ## Demonstrations ### Chinese IMEs (Shuangpin) We can see where the cursor is now. | What | With this PR | Without this PR | |-|-|-| | macOS builtin |<video src=https://github.com/user-attachments/assets/487c7e7c-ef6d-4a86-8dbc-8c71871b4470 />|<video src=https://github.com/user-attachments/assets/49bd5a60-4b90-4e4a-99e0-cd01d3f7030c />| | macOS builtin (light)|<video src=https://github.com/user-attachments/assets/e84546f6-947b-4cea-a87e-fda903f49164 />|——| | Windows builtin |<video src=https://github.com/user-attachments/assets/fd331884-1f0c-4822-a99e-8140aed54815 />|——| | Wayland iBus Intelligent Pinyin |<video src=https://github.com/user-attachments/assets/b6796c75-1c4e-45e5-b43a-5d8dea320485 />|——| | Chromium (Chrome) macOS | Identical to `macOS builtin`. |——| | Safari macOS | We can now differentiate between IME composition and text selection, but we still can't tell where the cursor is. |——| | Firefox (Zen) macOS | Identical to `macOS builtin`. |——| ### Japanese IMEs We can see where the conversion segment is now. | What | With this PR | Without this PR | |-|-|-| | macOS builtin |<video src=https://github.com/user-attachments/assets/f2994cd4-a966-4ff0-9590-d263c202ec76 />|<video src=https://github.com/user-attachments/assets/7cf5ff35-003d-4f60-8fbf-15c725c3ecb9 />| | macOS builtin (light)|<video src=https://github.com/user-attachments/assets/6f562bdd-12fc-4486-b37b-8fcf11643295 />|——| | Windows builtin |<video src=https://github.com/user-attachments/assets/f0905659-5335-4034-abda-c25cf8f2fd57 />|——| | Wayland iBus Anthy |<video src=https://github.com/user-attachments/assets/94cd3a24-3158-4d79-ae02-d9b30fdfa738 />|——| | Chromium (Chrome) macOS | Identical to `macOS builtin`. |——| | Safari macOS | We can now differentiate between IME composition and text selection, but we still can't tell where the conversion segment is. |——| | Firefox (Zen) macOS | (Limited improvement.) <video src=https://github.com/user-attachments/assets/3daf9b63-6e75-467b-8515-31c2a44adf61 /> |——| ### Korean IMEs We can clearly tell whether we are in composition (in contrast to selection) now. | What | With this PR | Without this PR | |-|-|-| |macOS builtin|<video src=https://github.com/user-attachments/assets/73ca28c7-22a0-493f-8f4d-c6e59a2dec54 />|<video src=https://github.com/user-attachments/assets/f582de7d-7ec0-48fe-910f-0139ef1620d3 />| |macOS builtin (light)|<video src=https://github.com/user-attachments/assets/269f03bd-6f95-498b-9fb1-1adcb043c738 />|——| | Windows builtin| (With a workaround for [this `winit` bug](https://github.com/emilk/egui/pull/8083#issuecomment-4206742668) applied.) <video src=https://github.com/user-attachments/assets/1e82583d-0c41-4f1c-98cf-0606bee5af05 />|——| | Wayland iBus Hangul |<video src=https://github.com/user-attachments/assets/8c9a0de1-9027-4b37-93a3-e9da0251d176 />|——| | Chromium (Chrome) macOS | Identical to `macOS builtin`. |——| | Safari macOS | Identical to `Windows builtin`. |——| | Firefox (Zen) macOS | Identical to `macOS builtin`. (ignoring the fact that the composition breaks when typing the second Hangul. (This bug predates this PR.)) |——| --------- Co-authored-by: lucasmerlin <hi@lucasmerlin.me>
eframe: the egui framework
eframe is the official framework library for writing apps using egui. The app can be compiled both to run natively (for Linux, Mac, Windows, and Android) or as a web app (using Wasm).
To get started, see the examples.
To learn how to set up eframe for web and native, go to https://github.com/emilk/eframe_template/ and follow the instructions there!
There is also a tutorial video at https://www.youtube.com/watch?v=NtUkr_z7l84.
For how to use egui, see the egui docs.
eframe defaults to using wgpu for rendering (with an option to change to glow), and on native it uses egui-winit.
To use on Linux, first run:
sudo apt-get install libxcb-render0-dev libxcb-shape0-dev libxcb-xfixes0-dev libxkbcommon-dev libssl-dev
You need to either use edition = "2024", or set resolver = "2" in the [workspace] section of your to-level Cargo.toml. See this link for more info.
You can opt-in to the using egui_glow for rendering by enabling the glow feature and setting NativeOptions::renderer to Renderer::Glow.
Alternatives
eframe is not the only way to write an app using egui! You can also try egui-miniquad, bevy_egui, egui_sdl2_gl, and others.
You can also use egui_glow and winit to build your own app as demonstrated in https://github.com/emilk/egui/blob/main/crates/egui_glow/examples/pure_glow.rs.
Limitations when running egui on the web
eframe and egui compiles to Wasm using either WebGPU (when available) or WebGL2 for rendering, and almost nothing else from the web tech stack. This has some benefits, but also produces some challenges and serious downsides.
- Rendering: Getting pixel-perfect rendering right on the web is very difficult.
- Search: you cannot search an egui web page like you would a normal web page.
- Bringing up an on-screen keyboard on mobile: there is no JS function to do this, so
eframefakes it by adding some invisible DOM elements. It doesn't always work. - Mobile text editing is not as good as for a normal web app.
- No integration with browser settings for colors and fonts.
- Accessibility: There is an experimental screen reader for
eframe, but it has to be enabled explicitly. There is no JS function to ask "Does the user want a screen reader?" (and there should probably not be such a function, due to user tracking/integrity concerns).eguisupports AccessKit, but as of early 2024, AccessKit lacks a Web backend.
In many ways, eframe is trying to make the browser do something it wasn't designed to do (though there are many things browser vendors could do to improve how well libraries like egui work).
The suggested use for eframe are for web apps where performance and responsiveness are more important than accessibility and mobile text editing.
Companion crates
Not all rust crates work when compiled to Wasm, but here are some useful crates have been designed to work well both natively and as Wasm:
Name
The frame in eframe stands both for the frame in which your egui app resides and also for "framework" (eframe is a framework, egui is a library).