When an agent screenshots the app using the mcp and then interacts with
the app by clicking at coords, they see the coords in the native image
coords. Since the mcp does everything else in logical coordinates, it
helps if the image they see is also in logical resoltution, so we always
downscale it to 1.0.
I've added this here to avioid having to decode and re-encode the image
in the mcp.
Unfortunately it only does downscaling for now, since adding some way to
upscale the image just for the screenshot would add a lot of complexity,
and might be invasive from a plugin.
I've also changed the submit call to take a closure, to make it easier
to use other transport channel (makes the implementation for reruns mcp
nicer).
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Lucas Meurer <lucas@rerun.io>
Introduces live inspection for running egui apps over a small TCP
request/response protocol, plus the `egui::Plugin` that serves it.
This is the minimal surface to get the egui mcp in, we may want to
extend this in the future to add support for the inspection gui.
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Related to #56 (Improve text — tracking issue)
## Summary
This PR integrates [harfrust](https://crates.io/crates/harfrust) (a
pure-Rust port of HarfBuzz) into epaint's text layout pipeline,
replacing the character-by-character glyph positioning with proper
OpenType text shaping.
### What this enables
- **GPOS kerning**: most modern fonts only ship kerning in GPOS tables
(not the legacy `kern` table). Pairs like "AV", "VA", "AT" are now
properly tightened.
- **GSUB substitutions**: ligatures (fi, fl), contextual alternates, and
other OpenType features.
- **Combining marks**: diacritics (e.g. ɔ̃) are positioned via anchor
tables instead of being rendered as standalone replacement glyphs.
### Before/After
#### Kerning, etc.
<img width="838" height="726" alt="before_main"
src="https://github.com/user-attachments/assets/f0f26d5f-b117-43a6-b39c-ea40d2e73836"
/>
<img width="838" height="726" alt="after_harfrust"
src="https://github.com/user-attachments/assets/d983e5da-486c-4f39-bd4f-5782a90c6b39"
/>
#### Ligatures
<img width="1117" height="698" alt="before_closeup"
src="https://github.com/user-attachments/assets/7a3b08b4-cf6f-45b7-98ba-07c473cd3b02"
/>
<img width="1117" height="698" alt="after_closeup"
src="https://github.com/user-attachments/assets/6cfc5f21-d32f-4f09-be0c-59c8c553d44f"
/>
### Architecture
The shaping integrates into the existing pipeline without changing the
public API:
1. **`Font::segment_into_runs`** — segments text into contiguous runs by
font face (grapheme-cluster aware, never splits combining sequences)
2. **`FontFace::shape_text`** — calls harfrust to shape each run,
returning glyph IDs + positioned advances/offsets
3. **`layout_shaped_run`** — emits `Glyph` structs from the shaping
output, with NOTDEF fallback to other font faces for missing glyphs
4. **Buffer recycling** — `FontsImpl` pools a `harfrust::UnicodeBuffer`
to avoid per-layout allocations
### Disclaimer
I'm far from being a good Rust programmer. Claude Code did most of the
heavy lifting here. I did my best and used my limited knowledge to avoid
making too many mistakes. If this PR isn't up to quality standards,
please don't hesitate to close it.
## Test plan
- [x] `cargo test -p epaint` — all 18 text tests pass, including 6 new
ones
- [x] `cargo clippy -p epaint --all-features` — clean
- [x] `cargo fmt` — clean
- [ ] Snapshot tests need regeneration (expected: shaping changes glyph
positions)
- New tests added:
- `test_gpos_kerning` — verifies GPOS kerning tightens "AV", "VA", "AT"
pairs
- `test_combining_diacritics` — combining tilde doesn't add extra width
- `test_shaping_basic_latin` — sanity check for Latin text
- `test_shaping_empty_string` — empty input doesn't panic
- `test_shaping_multiple_newlines` — newline splitting works correctly
- `test_shaping_mixed_font_fallback` — Latin + emoji in same string
---------
Co-authored-by: Emil Ernerfeldt <emil.ernerfeldt@gmail.com>
<!--
Please read the "Making a PR" section of
[`CONTRIBUTING.md`](https://github.com/emilk/egui/blob/main/CONTRIBUTING.md)
before opening a Pull Request!
* Keep your PR:s small and focused.
* The PR title is what ends up in the changelog, so make it descriptive!
* If applicable, add a screenshot or gif.
* If it is a non-trivial addition, consider adding a demo for it to
`egui_demo_lib`, or a new example.
* Do NOT open PR:s from your `master` branch, as that makes it hard for
maintainers to test and add commits to your PR.
* Remember to run `cargo fmt` and `cargo clippy`.
* Open the PR as a draft until you have self-reviewed it and run
`./scripts/check.sh`.
* When you have addressed a PR comment, mark it as resolved.
Please be patient! I will review your PR, but my time is limited!
-->
Update some dependencies to get rid of old ones (especially nice to get
rid of windows-sys 0.45...).
* Closes <https://github.com/emilk/egui/issues/THE_RELEVANT_ISSUE>
* [x] I have followed the instructions in the PR template
* Fix for https://github.com/emilk/eframe_template/issues/223
* Related: https://github.com/gfx-rs/wgpu/pull/9319
By default, we would only turn on the WebGPU backend on web, which means
browsers without WebGPU support would just crash.
You can still opt-out of all the default `wgpu` features by enabling
`eframe/wgpu_no_default_features` instead of `eframe/wgpu`
* [x] I have followed the instructions in the PR template
This updates wgpu to v29 across the egui crate stack.
There a a few API changes due to the requirement to provide a display
handle up front to properly support GLES on linux. I have done my best
to make the api changes as reasonable as possible, but I don't have all
the greater project context, so lmk if things should be done a bit
differently.
I've also updated glow to 0.17 to make cargo deny happy, there are no
source changes. I'm not sure how you want to land these.
---------
Co-authored-by: lucasmerlin <hi@lucasmerlin.me>
<!--
Please read the "Making a PR" section of
[`CONTRIBUTING.md`](https://github.com/emilk/egui/blob/main/CONTRIBUTING.md)
before opening a Pull Request!
* Keep your PR:s small and focused.
* The PR title is what ends up in the changelog, so make it descriptive!
* If applicable, add a screenshot or gif.
* If it is a non-trivial addition, consider adding a demo for it to
`egui_demo_lib`, or a new example.
* Do NOT open PR:s from your `master` branch, as that makes it hard for
maintainers to test and add commits to your PR.
* Remember to run `cargo fmt` and `cargo clippy`.
* Open the PR as a draft until you have self-reviewed it and run
`./scripts/check.sh`.
* When you have addressed a PR comment, mark it as resolved.
Please be patient! I will review your PR, but my time is limited!
-->
* Closes N/A
* [x] I have followed the instructions in the PR template
This was mostly from last month, but I never got around to submitting
it.
This PR adds font variation coordinates to the `TextFormat` struct, and
uses them when rendering text. The coordinates are stored in a
`SmallVec`; I've chosen to store up to 2 inline, which makes it take up
24 bytes (the minimum possible for a `SmallVec`). The variation axis
tags are stored as the `font_types::Tag` type, which I've chosen to
re-export from `epaint::text`.
The variation coordinates are resolved to a `skrifa::Location` during
font rendering/scaling, and are cached in the same way as all the other
scaled metrics. I've renamed the `ScaledMetrics` struct to
`StyledMetrics`, since it now also contains the resolved variation
coordinates. I haven't benchmarked the performance of text layout with
variation coordinates, but the existing text layout performance is
unchanged.
I've replaced the API for manually overriding a font's weight
(https://github.com/emilk/egui/pull/7790) with an API for manually
overriding any variation coordinates via `FontTweak`. This should
support the same use case as #7790 while being substantially more
flexible.
I have *not* yet added any higher-level API for mapping style attributes
(weight, width, slant, etc) to variation coordinates or to different
font faces within a single family. That's a pretty huge can of worms,
and it'd involve rethinking the split between `FontId` and `TextFormat`
(and whether `FontId` is so big that we should provide a way to reuse
it). This API is intentionally pretty low-level for now.
Likewise, I've intentionally not used variation coordinates when
computing a font's row height. I can't think of any fonts that change
their vertical metrics depending on variation axes, so this should be
fine for now.
---------
Co-authored-by: Emil Ernerfeldt <emil.ernerfeldt@gmail.com>
<!--
Please read the "Making a PR" section of
[`CONTRIBUTING.md`](https://github.com/emilk/egui/blob/main/CONTRIBUTING.md)
before opening a Pull Request!
* Keep your PR:s small and focused.
* The PR title is what ends up in the changelog, so make it descriptive!
* If applicable, add a screenshot or gif.
* If it is a non-trivial addition, consider adding a demo for it to
`egui_demo_lib`, or a new example.
* Do NOT open PR:s from your `master` branch, as that makes it hard for
maintainers to test and add commits to your PR.
* Remember to run `cargo fmt` and `cargo clippy`.
* Open the PR as a draft until you have self-reviewed it and run
`./scripts/check.sh`.
* When you have addressed a PR comment, mark it as resolved.
Please be patient! I will review your PR, but my time is limited!
-->
* [x] I have followed the instructions in the PR template
A number of separate commits to possibly easily revert some of them.
General idea: selectively update dependencies to remove transitive
dependencies and multiple versions etc. As well as updating "major" (the
one that `cargo update` doesn't update) version for some in Cargo.toml.
Rendering pipelines in `vello_cpu` wasn't obvious. Now both are used.
<!--
Please read the "Making a PR" section of
[`CONTRIBUTING.md`](https://github.com/emilk/egui/blob/main/CONTRIBUTING.md)
before opening a Pull Request!
* Keep your PR:s small and focused.
* The PR title is what ends up in the changelog, so make it descriptive!
* If applicable, add a screenshot or gif.
* If it is a non-trivial addition, consider adding a demo for it to
`egui_demo_lib`, or a new example.
* Do NOT open PR:s from your `master` branch, as that makes it hard for
maintainers to test and add commits to your PR.
* Remember to run `cargo fmt` and `cargo clippy`.
* Open the PR as a draft until you have self-reviewed it and run
`./scripts/check.sh`.
* When you have addressed a PR comment, mark it as resolved.
Please be patient! I will review your PR, but my time is limited!
-->
* Closes N/A
* [x] I have followed the instructions in the PR template
I'll probably come back to this and clean it up a bit. This PR
reimplements ab_glyph's functionality on top of Skrifa, a somewhat
lower-level font API that's being used in Chrome now.
Skrifa doesn't perform rasterization itself, so I'm using
[vello_cpu](https://github.com/linebender/vello) from the Linebender
project for rasterization. It's still in its early days, but I believe
it's already quite fast. It also supports color and gradient fills, so
color emoji support will be easier.
Skrifa also supports font hinting, which should make text look a bit
nicer / less blurry.
Here's the current ab_glyph rendering:
<img width="1592" height="1068" alt="image"
src="https://github.com/user-attachments/assets/2385b66e-23f8-4c6e-b8c2-ea90e0eea4e4"
/>
Here's Skrifa *without* hinting--it looks almost identical, but there
are some subpixel differences, probably due to rasterizer behavior:
<img width="1592" height="1068" alt="image"
src="https://github.com/user-attachments/assets/a815f3e9-65ac-4940-bc00-571177bef53d"
/>
Here's Skrifa *with* hinting:
<img width="1592" height="1068" alt="image"
src="https://github.com/user-attachments/assets/d6cc0669-3537-4377-bba9-ed5ef09664db"
/>
Hinting does make the horizontal strokes look a bit bolder, which makes
me wonder once again about increasing the font weight from "light" to
"regular".
---------
Co-authored-by: Emil Ernerfeldt <emil.ernerfeldt@gmail.com>
## Short bluesky announcement:
We just released egui 0.33.0!
Highlights:
- `egui::Plugin` a improved way to create and access egui plugins
- [kitdiff](https://github.com/rerun-io/kitdiff), a viewer for
egui_kittest image snapshots (and a general image diff tool)
- better kerning (check the diff on
[kitdiff](https://rerun-io.github.io/kitdiff/?url=https://github.com/emilk/egui/pull/7431))
https://github.com/user-attachments/assets/971f0493-6dae-42e5-8019-58b74cf5d203
## Relaese Changelog:
egui is an easy-to-use immediate mode GUI for Rust that runs on both web
and native.
Try it now: <https://www.egui.rs/>
egui development is sponsored by [Rerun](https://www.rerun.io/), a
startup building an SDK for visualizing streams of multimodal data.
# egui 0.33.0 changelog
Highlights from this release:
- `egui::Plugin` a improved way to create and access egui plugins
- [kitdiff](https://github.com/rerun-io/kitdiff), a viewer for
egui_kittest image snapshots (and a general image diff tool)
- better kerning
### Improved kerning
As a step towards using [parley](https://github.com/linebender/parley)
for font rendering, @valadaptive has refactored the font loading and
rendering code. A result of this (next to the font rendering code being
much nicer now) is improved kerning.
Notice how the c moved away from the k:
### `egui::Plugin` trait
We've added a new trait-based plugin api, meant to replace
`Context::on_begin_pass` and `Context::on_end_pass`.
This makes it a lot easier to handle state in your plugins. Instead of
having to write to egui memory it can live right on your plugin struct.
The trait based api also makes easier to add new hooks that plugins can
use. In addition to `on_begin_pass` and `on_end_pass`, the `Plugin`
trait now has a `input_hook` and `output_hook` which you can use to
inspect / modify the `RawInput` / `FullOutput`.
### kitdiff, a image diff viewer
At rerun we have a ton of snapshots. Some PRs will change most of them
(e.g. [the](https://github.com/rerun-io/rerun/pull/11253/files)
[one](https://rerun-io.github.io/kitdiff/?url=https://github.com/rerun-io/rerun/pull/11253/files)
that updated egui and introduced the kerning improvements, ~500
snapshots changed!).
If you really want to look at every changed snapshot it better be as
efficient as possible, and the experience on github, fiddeling with the
sliders, is kind of frustrating.
In order to fix this, we've made
[kitdiff](https://rerun-io.github.io/kitdiff/).
You can use it locally via
- `kitdiff files .` will search for .new.png and .diff.png files
- `kitdiff git` will compare the current files to the default branch
(main/master)
Or in the browser via
- going to https://rerun-io.github.io/kitdiff/ and pasting a PR or
github artifact url
- linking to kitdiff via e.g. a github workflow
`https://rerun-io.github.io/kitdiff/?url=<link_to_pr_or_artefact>`
To install kitdiff run `cargo install --git
https://github.com/rerun-io/kitdiff`
Here is a video showing the kerning changes in kitdiff ([try it
yourself](https://rerun-io.github.io/kitdiff/?url=https://github.com/rerun-io/rerun/pull/11253/files)):
https://github.com/user-attachments/assets/74640af1-09ba-435a-9d0c-2cbeee140c8f
### Migration guide
- `egui::Mutex` now has a timeout as a simple deadlock detection
- If you use a `egui::Mutex` in some place where it's held for longer
than a single frame, you should switch to the std mutex or parking_lot
instead (egui mutexes are wrappers around parking lot)
- `screen_rect` is deprecated
- In order to support safe areas, egui now has `viewport_rect` and
`content_rect`.
- Update all usages of `screen_rect` to `content_rect`, unless you are
sure that you want to draw outside the `safe area` (which would mean
your Ui may be covered by notches, system ui, etc.)
Adds an accessibility inspector plugin that shows the current AccessKit
tree:
https://github.com/user-attachments/assets/78f4f221-1bd2-4ce4-adf5-fc3b00f5c16c
Macos has a built in accessibility inspector, but it doesn't seem to
work with AccessKit / eframe so this provides some insight into the
accesskit state.
This also showed a couple issues that are easy to fix:
- [ ] Links show up as `Label` instead of links
- [ ] Not all supported actions are advertised (e.g. scrolling)
- [ ] The resize handles in windows shouldn't be focusable
- [ ] Checkbox has no value
- [ ] Menus should have the button as parent widget (not 100% sure on
this one)
Currently the plugin lives in the demo app, but I think it should be
moved somewhere else. Maybe egui_extras?
This could also be relevant for #4650
Adds a helper to quickly see whats going on in a kittest test.
Not all test have snapshots, but when debugging tests it might still be
useful to see whats actually going on, so this adds a helper fn that
renders a snapshot image to a temporary file and opens it with the
default image viewer:
https://github.com/user-attachments/assets/08785850-0a12-4572-b9b5-cea36951081c
This PR is a continuation of #4915 by @frederik-uni and @lucasmerlin
that introduces support for keeping egui content within the 'safe area'
on iOS (avoiding the notch / dynamic island / menu bar etc.), with the
following changes:
- `SafeArea` now wraps `MarginF32` and has been renamed to
`SafeAreaInsets` to clarify its purpose.
- `InputState::screen_rect` is now marked as deprecated in favour of
either `viewport_rect` (which contains the entire screen), or
`content_rect` (which is the viewport rect with the safe area insets
removed).
- I added some comments to the safe area insets logic pointing out the
[safe area API coming in winit
v0.31](https://github.com/rust-windowing/winit/issues/3910).
---------
Co-authored-by: frederik-uni <147479464+frederik-uni@users.noreply.github.com>
Co-authored-by: Lucas Meurer <hi@lucasmerlin.me>
Co-authored-by: Emil Ernerfeldt <emil.ernerfeldt@gmail.com>
