migrations/egui
1
0
Fork 0
mirror of https://github.com/emilk/egui.git synced 2026-09-02 23:00:04 -04:00
Code Activity

Allow downscaling image in GetScreenshot inspection request (#8248)

Browse Source 
When an agent screenshots the app using the mcp and then interacts with
the app by clicking at coords, they see the coords in the native image
coords. Since the mcp does everything else in logical coordinates, it
helps if the image they see is also in logical resoltution, so we always
downscale it to 1.0.

I've added this here to avioid having to decode and re-encode the image
in the mcp.
Unfortunately it only does downscaling for now, since adding some way to
upscale the image just for the screenshot would add a lot of complexity,
and might be invasive from a plugin.

I've also changed the submit call to take a closure, to make it easier
to use other transport channel (makes the implementation for reruns mcp
nicer).

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Lucas Meurer <lucas@rerun.io>
This commit is contained in:
Lucas Meurer
2026-06-23 10:51:42 +02:00
committed by GitHub
parent 7be2e9a2ae
commit 5ca09dc0b5
6 changed files with 158 additions and 58 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
crates/egui_inspection/README.md
View File

@@ -15,7 +15,7 @@ serves it. An external inspector — such as the
- resize the window (`Resize`).
The protocol is strictly request → response, which maps cleanly onto both a TCP socket and a
unary RPC (so the same machinery can be tunnelled over e.g. gRPC).
unary RPC (so the same machinery can be tunnelled over another transport).
> **Screenshots need a visible window.** Reading the tree and injecting input work even while
> the app is in the background, but capturing a screenshot requires a rendered frame — which
@@ -26,8 +26,8 @@ unary RPC (so the same machinery can be tunnelled over e.g. gRPC).
## What it's for
`egui_inspection` is the shared foundation for tools that observe or drive an egui app from
the outside. Anything that speaks the protocol (over TCP, or another transport like gRPC)
can be a consumer:
the outside. Anything that speaks the protocol (over TCP, or another transport) can be a
consumer:
- **[`egui_mcp`](https://crates.io/crates/egui_mcp)** — an MCP server that exposes the app to
AI agents and other tooling: query the widget tree, click / type / scroll, take screenshots.