fixed-width/glass

**Give your coding agent hands for the app it's building** — launch it, drive it, and verify the
result, without burning a screenshot on every step.
A Rust MCP server that gives an AI coding agent a closed **build →
see → interact → debug** loop over external native GUI applications.
glass lets an agent launch a GUI app, capture what is on screen, inject mouse and keyboard input, read
the app's logs, and detect visual changes — so a coding agent can build and debug UI applications
independently instead of asking the user "does this look right?".
glass drives apps as an external black box, so it works with any native GUI app regardless of toolkit
or language. It has two Linux backends (**X11** and **Wayland**), a **Windows** backend, an
**Android** backend (an AVD emulator, driven over adb from any host), an **iOS** backend (native
apps in the Simulator over xcrun simctl, with input and the accessibility tree via idb_companion,
including a two-finger pinch), and a **macOS** backend, behind a platform-agnostic core.
The same input tools work across backends. Default Linux displays and mobile emulators keep input
separate from your desktop; native Windows and macOS currently share it. glass_capabilities with
scope: "session" reports click, scroll and text support and whether each can affect your desktop.
See it
An agent building a GUI app runs it under glass, reproduces a bug **from the accessibility tree**
(no screenshots), fixes the code, and re-verifies — the loop it otherwise can't close on its own.
Try it yourself.
Try it in 60 seconds
1. Download glass for your platform from the Releases page
and connect it to your agent — glass works with any MCP host
(see which are verified).
2. Get the example app — clone this repo, or download
examples/tasks-demo/tasks_demo.py (on Linux it needs
sudo apt install python3-gi gir1.2-gtk-4.0).
3. Paste this to your agent:
> Use glass to run examples/tasks-demo/tasks_demo.py with accessibility on. There's a bug: clicking
> **Add** doesn't add the typed task. Reproduce it by driving the UI and checking the
> accessibility tree (don't just screenshot), then find and fix the bug in the code and verify a
> task actually appears.
Your agent launches the app, reproduces the bug from the accessibility tree, fixes the one-line
wiring bug, and confirms the task appears — the whole build → see → interact → debug loop, start to
finish. (glass-mcp doctor checks your environment if anything's off.)
The loop in practice
An agent can fill in a form, click Save, and check that the app reports success. When the app exposes
an accessibility tree, the agent works with named controls and verifies changes from text:
Use glass_find_elements to inspect candidates when the intended target is not unique or known well
enough to act on directly. Use glass_a11y_snapshot to explore the app's controls. See the
tool reference for click modes and platform behavior.
For a canvas or custom-rendered app with no accessibility tree, drive it by pixels instead —
glass_screenshot, glass_click {x,y}, and glass_diff, which returns changed_pct + a bbox as
text, so routine checks between screenshots cost no vision tokens. Why the loop is shaped this way:
the build → see → interact → debug loop.