AI agents

An AI agent working on your app (Claude Code, Cursor, Codex…) can use devui as you do: read and set the controls, look at the canvas, measure the frame and capture it. Its tools answer in text, with pictures where a picture says it, and every number is the steady one the panel shows.

Connecting an agent

devui's Vite plugin serves the tools over MCP, at /__devui/mcp on your dev server:

// vite.config.ts
import { devuiFiles } from '@haxiomic/devui/vite';
export default { plugins: [devuiFiles()] };

Then tell the agent where it is, once. For Claude Code:

claude mcp add --transport http devui http://localhost:5173/__devui/mcp

Other agents take the same address as an MCP server "over HTTP" (Streamable HTTP).

The app must be open in a browser, in a visible tab: the tools run in the page, and a hidden tab draws no frames. With several pages open, a call goes to the one in front; list_pages lists them, and every tool takes page to name one.
Only this machine can call it, even with vite --host. devuiFiles({ agent: { network: true } }) opens it to others on your network; agent: false turns it off.
A change an agent makes is yours: saved to the values file (or the browser's storage) and undone with ⌘Z, like a change made in the panel.

Without Vite, or from an agent that drives a browser itself (Playwright, Chrome DevTools), the same tools are in the page: await __devui.call('list_controls') answers { text, images }, the images as base64 PNG. __devui.gui is the GUI a call acts on unless it names one (the one with the most in it): from the console, __devui.gui.add(material) adds controls as code would.

The tools

Tool	Does
`list_pages`	The pages open on the dev server: address, title, visible, their GUIs
`list_controls`	Every folder, control (path = value, type, range or options, description), button and plot, then the commands. With `path`, one folder, or one control in full
`set_values`	Sets controls by path, as one undo step. Says what each became, and why any couldn't be set
`run_command`	Runs a command by id, or presses a button by path
`screenshot`	The app's canvas just after its next frame, as a PNG (`size`: its long side, 768 px by default)
`read_texture`	A texture row's numbers (each channel's min, max, mean, NaN and Inf) and picture, read now from the GPU or CPU
`read_plot`	A plot's series: latest, min, mean, p95 and max, and its values over time
`measure`	A few seconds of frames: frame interval, and with a frame scope each pass's GPU and CPU time, counts per frame and GPU memory. With `change`, an A/B/A of some values
`capture_frame`	Captures the next frame and lists it: passes (`p0`…) and draws (`d0`…) by name, with their targets, counts, times and inputs, then the findings
`capture_draw`	One draw in full: call, state, uniforms, inputs and where each came from, attributes, calls, and (`shaders: true`) its program's source
`capture_picture`	A pass's target after or before it, a draw's result drawn again at full size, or what a draw read, as a PNG with each channel's min, max, mean and NaN/Inf
`compare_captures`	Two captures matched pass by pass and draw by draw: what's new, gone, or differs
`open_capture`	Opens a `.devui-capture` file the dev server serves (`file`: a path under the project, an absolute path, or a URL) and lists it as `capture_frame` does. Its draws are drawn again on this page, each pass checked against the file's own picture of it; the other capture tools then work on it

open_capture fetches the file from the page, as the page would (cookies go only to its own origin): a path the dev server serves, an absolute path within Vite's server.fs.allow, or any URL the page can fetch. It opens what it's given, and opening runs the file's shaders on the page (a WebGL2 file's in the app's context, see Capture files in the frame scope guide): have it open your own captures, and open others' in the viewer.

Values are JSON, as list_controls shows them: 0.8, "#ffeedd", [0, 1, 2]. An option can be set by its label too ("ACESFilmic" for three's tone mapping). A colour with alpha is listed as color with alpha and takes "#ffeedd80" or "oklch(0.9 0.03 60 / 0.5)"; one without keeps its colour and leaves the alpha out. A wrong path answers with the nearest ones.

Measurements an agent can trust. Every time is a steady one over the span: a pass's GPU time is the low quarter of its readings (a reading counts the GPU's waits too, which only add), or their mean where the GPU's clock is coarse (Chrome rounds WebGPU's timestamps to steps of 65.5 µs, and a quick pass reads 0 or a step); its CPU time the mean (the browser's clock steps by 0.1 ms unless the page is cross-origin isolated). Each run's numbers come from its own frames. Where GPU times are worked out over a sweep of frames (Chrome on macOS: see GPU time in the frame scope's guide), a pass's time is from the readings taken in that run alone. The run goes on past seconds until every pass has been read twice (at 10 fps, 15 s or so), or three sweeps have gone by, at most a minute more; not at all where no more can come (the app at rest, or timing its own passes). The answer says how long it ran. An A/B (measure with change) runs A, then B, then A again, and marks a difference no bigger than A's own drift as possibly noise. The values are put back after, and nothing's saved. A pass is listed by its target's name, numbered where several share one, so the rows of an A/B whose B makes its targets anew still line up.

The capture's tools need a frame scope (frames(gui, renderer), see Frame scope), on WebGL2 or WebGPU, and measure needs one for GPU time and passes. A scope the panel offers (gui.perf() given a context, or three.js's extension, attaches it when its folder is first opened) is attached for the first tool that needs it, as opening the folder would. The folder stays folded. With more than one, capture_frame takes the one drawing the most, and the tools after it the one that took the capture; scope names one. They name passes and draws as the capture window does, by their targets and labels. On WebGPU a capture keeps no picture of each draw: a draw's picture is its pass drawn again up to it, as in the capture window. Pictures are read back at full size: the numbers are every texel's, and the PNG is scaled down. Floats are clipped to 0–1 and shown in sRGB, as the panel shows them; range: "fit" stretches the picture's own range instead, for HDR, data and depth (depth always is). A depth picture after a pass that cleared it is fitted to what the pass drew, as the capture window fits it: what's still the clear shows white, and the answer says what it was cleared to. NaN and Inf show magenta.

A typical session: list_controls; measure to see where the time goes; capture_frame and its findings (draws that change nothing, clears drawn over, passes nothing reads, overdraw); capture_draw and capture_picture on what looks wrong; a change (set_values, or the code); then measure or capture_frame again and compare_captures.