NexusWindows for Claude

Windows computer use for Claude, in the background. An MCP server (nexus-windows-claude) that lets Claude Desktop and Claude Code see and operate Windows desktop applications while the user keeps their mouse, keyboard and foreground window.

Version 3.0.2 · Windows 10/11 · Python 3.14 · MIT licence

Part of the Nexus family: NexusWindows (Claude · Codex · Antigravity), NexusBrowser (repo), NexusAndroid (repo). Project page: julienpiron.fr/nexuswindows.

Why

Anthropic's own computer use (in Cowork and Claude Code inside Claude Desktop) is a beta for Pro and Max subscribers: "Team and Enterprise plans don't have access to computer use at this time" (Claude Help Center). NexusWindows for Claude is a local MCP server, so it works with Claude Desktop and Claude Code on any plan that allows local MCP servers, Team and Enterprise included.

Screenshot-and-click computer use moves the real cursor and brings windows to the front, so the person at the PC has to stop working while the agent acts. Most Windows applications can be driven without that: native controls accept window messages, modern frameworks expose UI Automation, and Chromium can be reached through its accessibility tree. This server picks, per UI family, a technique that was measured not to steal focus, reads the effect back, and only falls back to the real mouse or keyboard when the user explicitly allows it.

NexusWindows family

Three MCP servers, one per agent, built separately because each model needs its tools shaped for it, with the same promise: the user keeps their mouse, keyboard and foreground window, and every action is read back.

Every step reports ✓ verified, = already done, ~ unknown (never replayed) or ✗ failed or refused. The three share the guardrail policy, the foreground token, the emergency stop (Ctrl+Alt+Shift+Esc), a common journal format and a common bench.

Installation

Windows 10 or 11. You do not need Python: uv downloads Python 3.14 by itself.

One step (installs uv if missing, downloads and checks the server, registers it with Claude Code and declares it in Claude Desktop's configuration, merged after a backup; an entry of your own is kept, and with the Claude Code plugin enabled no second server is added): open Windows PowerShell (Start menu, type powershell) and paste:

irm https://raw.githubusercontent.com/pironjulien/NexusWindows-Claude/main/setup.ps1 | iex

Then close Claude completely and open it again. setup.ps1 is short and can be read first; running it again only does what is missing.

Claude Code plugin (the same server, plus a skill; uv required):

/plugin marketplace add pironjulien/nexus-plugins
/plugin install nexus-windows@nexus-plugins

Claude Desktop extension (uv required): download nexus-windows-claude-<version>.mcpb from the latest release, then in Claude Desktop open Settings → Extensions and drop the file there (or Advanced settings → Install extension), and click Install. With the Microsoft Store version of Claude Desktop, double-clicking the file does not open it: Windows has no app for .mcpb.

By hand. Install uv (official package):

winget install --id=astral-sh.uv -e

then restart Claude so that it sees uv, and register the server from PyPI. Claude Code:

claude mcp add --scope user -e PYTHONUTF8=1 --transport stdio nexus-windows -- uvx --python 3.14 nexus-windows-claude

Claude Desktop: Settings → Developer → Edit Config, add this to claude_desktop_config.json (inside the existing mcpServers if there is one), save, then quit Claude completely and reopen it:

{
  "mcpServers": {
    "nexus-windows": {
      "command": "uvx",
      "args": ["--python", "3.14", "nexus-windows-claude"],
      "env": { "PYTHONUTF8": "1" }
    }
  }
}

The first start downloads about 100 MB (Python 3.14 and the server's dependencies); later starts take under a second. Until 3.0.0 the package was named nexus-windows; that name still works and installs nexus-windows-claude.

From a clone (development):

git clone https://github.com/pironjulien/NexusWindows-Claude.git
cd NexusWindows-Claude
$env:UV_PROJECT_ENVIRONMENT = "$env:LOCALAPPDATA\claude-windows-computer-use\venv"   # optional: venv outside the clone
uv sync --frozen

Register it with Claude Code (user scope):

claude mcp add --scope user `
  -e "UV_PROJECT_ENVIRONMENT=$env:LOCALAPPDATA\claude-windows-computer-use\venv" -e PYTHONUTF8=1 `
  --transport stdio nexus-windows '--' uv run --project "$PWD" --frozen python -m nexus_windows

For Claude Desktop from a clone:

{
  "mcpServers": {
    "nexus-windows": {
      "command": "uv",
      "args": ["run", "--project", "C:/path/to/NexusWindows-Claude", "--frozen", "python", "-m", "nexus_windows"],
      "env": { "PYTHONUTF8": "1" }
    }
  }
}

Local state (logs, routines) lives in %USERPROFILE%\.claude\nexus-claude\; set NEXUS_WINDOWS_HOME to use another folder.

Tools

Step actions: click, double_click, right_click, type, set, clear, key, select, check, uncheck, toggle, expand, collapse, scroll, focus, menu, wait, drag, hover. A target is an [N] from the last look, a visible name ("OK"), a role-qualified name ("button:OK", "edit:Name", "checkbox:Subscribe") or an x,y point in the last image.

Usage

Typical flow: look(window="Notepad"), then one act with all the steps, then read the per-step report.

act(window="Settings form", steps=[
  {"do": "set",   "target": "edit:Name", "text": "Ada",
   "check": {"target": "Name", "value": "Ada"}},
  {"do": "check", "target": "Subscribe"},
  {"do": "click", "target": "button:Save", "expect": "Saved"}
], remember="save-form")

Later: act(window="Settings form", routine="save-form").

What it does

Only what changed (then="changes", show="changes")

After an action, act(then="changes") (or look(show="changes")) returns only what changed since the last list shown for that window: + added, ~ changed (text, state or moved), - removed; every other element keeps its [N]. A common scroll of many elements is one line naming them as ranges ([2-21]), not a full list. Comparison is made in the window's own frame, so moving the window is not a change. A removed [N] is never given again, and an element whose change was cut from a long report loses its [N] until the next look. When almost everything changed (60 %), the full list is returned. Measured: 2 to 2.6 times shorter than the full list after a click.

Text read from the image (OCR fallback)

When a window publishes almost no UI Automation elements (fewer than 6: Tk, canvases) or a Chromium page stops publishing its content, the text of the window image is read with the Windows built-in OCR engine (Windows.Media.Ocr, local, offline) and listed as [N] Text "…" (OCR) with its position; such an element can be targeted by number or by name (click at its position, recomputed if the window moved). Measured on the lab apps: 10 to 21 ms per window; text correct 16/16 (WPF), 20/23 (Edge), 6/7 (Tk).

Four outcomes per step

Each step is reported as one of:

• ✓ the effect was read back (value, state, window, image);
• = the state requested by the step's check was already true, so nothing was sent;
• ~ the input was sent but its effect could not be read (never counted as a success);
• ✗ failure, refusal, or postcondition not met; the batch stops there.

A batch ends with a tally of the four counts.

Postconditions (check)

A step may carry check: one element and the state it must reach (value, contains, checked, selected, expanded, focused, enabled, exists). Already true before the step: the step is skipped (=). True after: ✓. False after: ✗ and the batch stops. A state the element does not publish stays unknown, never false.

Routines

act(remember="name") saves a batch only if every step ended ✓ or = and every target is a visible name (never an [N] or a point). {{x}} marks variable text, filled at replay with params. act(routine="name") finds each target again by name and re-verifies every step; an application update is reported and a failed replay marks the routine for review. Routines are stored per executable under %USERPROFILE%\.claude\nexus-claude\routines. An optional mirror folder (NEXUS_ROUTINES_MIRROR) can share definitions between PCs; typed text is never mirrored in clear (a routine whose text is not a {{x}} template stays local).

Verified opening of paths

launch(app=<folder or document>) opens through the file association (ShellExecuteEx, no activation requested), then checks what actually opened: the folder shown in File Explorer (read through Shell.Application) or a window titled with the file name. Scripts, shortcuts and installers are refused.

Per UI family

• Native Win32: direct messages to controls (BN_CLICKED, CB_SETCURSEL/LB_SETCURSEL with their notification, EM_REPLACESEL, menu WM_COMMAND, TCM_SETCURFOCUS), read back. UI Automation is avoided where it was measured to steal focus (button Invoke) or to fail silently (list Select).
• Win32 and WinForms drop-down lists: opened and closed with CB_SHOWDROPDOWN, read back with CB_GETDROPPEDSTATE (expand, collapse, or click on the list). Choosing in an open list closes it first (Windows treats that close as a cancel and would drop a choice made before it), then sends CB_SETCURSEL, CBN_SELCHANGE and CBN_SELENDOK to the parent and reads the application back (index and label); an empty or other value is ✗. While a list is open its UI Automation tree is not walked (it chains to every top-level window of the desktop): the options are read from the ComboLBox pop-up itself.
• WinForms, WPF, UWP, WinUI: UI Automation with AutoSetFocus disabled and bounded timeouts. A WPF drop-down list left open by a choice is closed and the choice read back.
• Chromium and Electron, without activation: for an app already open without DevTools, multiline fields, rich editors, <select>, keys and Ctrl/Alt shortcuts go through a simulated activation: WM_NCACTIVATE and WM_ACTIVATE are sent to that window only (Chromium takes its activation state from these messages; Windows does not change the foreground), DOM focus is set with MSAA accSelect(SELFLAG_TAKEFOCUS) and read back, characters go as WM_CHAR, and the deactivation is posted so it is processed after them. The page receives trusted keydown and input events. One caveat was measured: an app launched a moment ago may use its launch-time foreground right once (about 10 ms, handed back by the guard), so the first simulated activation per process is treated like a brief activation. Apps started with launch(devtools=true) are driven through the DevTools protocol instead.
• Steam (CEF in an SDL frame): recognised as Chromium; launch(app="Steam") adds Steam's own -cef-disable-occlusion -cef-force-accessibility switches so the UI stays readable while covered.
• Tk: no UI Automation; its text is found by the OCR fallback. Clicks are sent with the mouse button marked pressed for Tk's thread only (Tk reads it with GetKeyState), keys and shortcuts get a simulated focus (WM_SETFOCUS to that window only, never WM_ACTIVATE), lists scroll through their native scroll bar; all read back. A wheel on a covered Tk canvas is lost (Tk routes it to the window on top) and refused.
• Scrolling a covered Chromium page without DevTools: the wheel is dropped by Chromium when another window is on top, and scrolling areas expose no ScrollPattern; the next hidden line of the area is brought into view (UI Automation ScrollItem) and the line's offset inside its area is read back.

Capture

PrintWindow (10 to 26 ms, covered windows included), Windows.Graphics.Capture only as a fallback. Images follow the common Nexus image rule: at most 1568 px and 1.15 MP, so that Claude models do not downscale them (coordinates stay exact); lossless WebP up to 200 KB, then lossy until the image fits 500 KB, same size (a 1429×804 screen: 48 KB instead of 84 KB in PNG). To read small text, zoom with region rather than a larger image. An image identical to the last one sent for that window (or zoomed area) is not attached again; the reply says so, and show="image" always attaches it. Coordinates are bound to the window: followed if it moves, refused if it was resized.

Agent at work

While an action runs (act, launch, window; never the reads), a small logo spins on itself in the bottom-right corner of the driven window's screen, like a game loading screen, with a breathing blue halo. It goes away 1.6 s after the last action. The overlay is always on top, borderless, click-through, never activated, absent from the taskbar and excluded from screen captures (the model never sees it in look). NEXUS_WINDOWS_INDICATOR=0 turns it off.

Real mouse and keyboard

Only with act(foreground=true), which the user must approve. The window then stays in front for the whole batch; cursor and foreground are handed back at the end; any physical input from the user during the batch stops it.

Working alongside other agents

The server cooperates with other desktop agents on the same PC (it is used next to a Codex layer, NexusWindows for Codex, and an Antigravity layer) through two small file-based protocols. Any agent can implement them.

Foreground token (%USERPROFILE%\.nexus\foreground.lock): taken only while really holding the mouse or the foreground (real input, brief activation, window put in front), never for background actions. Holding it means an exclusive LockFileEx on 1 byte at offset 2^32, opened with read, write and delete sharing; Windows releases it when the handle closes, including when the process dies, so there is no stale token. The holder writes {"owner", "pid", "since", "purpose"} at the start of the file for diagnostics only. Waiters wait at most a few seconds (8 s here), then do nothing and say who holds it. A foreground guard that sees another agent holding the token follows that agent's window instead of taking the foreground back.

Emergency stop (Ctrl+Alt+Shift+Esc, files in %USERPROFILE%\.nexus\): every agent tries RegisterHotKey(MOD_NOREPEAT); one gets it, the others retry every 3 s. The holder writes a heartbeat to stop.owner every 3 s. On a press it rewrites stop with {"by", "pid", "at", "hold_s": 120, "nonce"}. Every agent watches stop at least every 0.2 s and before each action: a new version younger than 120 s blocks all desktop actions for the remaining time (reads stay allowed). The stop is latched in memory on a monotonic clock: deleting the file or changing the clock does not lift it, a new press re-arms it.

Architecture

flowchart LR
    Claude["Claude Code or Claude Desktop"] -->|"MCP stdio"| Server["server.py MCP tools"]
    Server --> Worker["COM worker thread with hard timeout"]
    Worker --> Snapshot["snapshot and capture"]
    Worker --> Actions["actions per UI family"]
    Actions --> Win32["Win32 messages"]
    Actions --> UIA["UI Automation"]
    Actions --> Chromium["Chromium simulated activation and DevTools"]
    Actions --> Real["real input only with foreground true"]
    Server --> Safety["policy, emergency stop, foreground guard"]
    Safety --> Shared["shared files in USERPROFILE nexus"]
    Real --> Shared

src/nexus_windows/
  reloader.py      stdio supervisor: restarts the server on new code and replays the client handshake
  server.py        MCP tools, dedicated COM thread, per-call log
  actions.py       executor facade, assembled from the act_*.py modules
  act_base.py      targets read back live, technique chains, outcomes, shared refusals
  act_win32.py     Win32 messages
  act_uia.py       UI Automation (WinForms, WPF, UWP/WinUI) and dispatch per family
  act_chromium.py  Chromium/Electron/CEF: verified click, DevTools, simulated activation, <select>
  act_pointer.py   points, message clicks, scrolling, drag, hover
  act_wait.py      text presence and waiting
  snapshot.py      window + popups, [N] elements, image, coordinates, changes since the last list
  ocr.py           Windows built-in OCR (Windows.Media.Ocr) for windows without UI Automation
  uia.py           UI Automation with a single cross-process cache request
  methods.py       Win32 message techniques
  cdp.py           minimal Chrome DevTools Protocol client
  capture.py       PrintWindow / Windows.Graphics.Capture
  foreground.py    foreground monitoring and restoration, brief activation
  agentlock.py     shared foreground token
  postcond.py      step postconditions (check)
  routines.py      verified routines per application
  physical.py      the user's physical input (Raw Input, no hook), told apart from injected input
  launch.py        background launch, DevTools, throwaway browser profiles
  steam.py         Steam switches, clean restart, DevTools port
  realinput.py     real mouse/keyboard (SendInput), only when authorised
  policy.py        where to act and what to launch (hard blocks of the shared policy, launch switches)
  act_confirm.py   confirmation of irreversible gestures (confirm: true), whatever key or click triggers them
  shared/          the guard-rail policy shared with NexusWindows for Antigravity (garde-fous.json)
  packages.py      resumes Store apps that Windows suspended in the background
  offwindows.py    outside Windows: the server starts and lists its tools, every desktop tool says Windows is required
  stop.py          shared emergency stop
  freshness.py     detects a server whose code changed on disk
  codeprint.py     code fingerprint and location of the shared policy (no dependencies)
  worker.py        COM thread with a hard timeout
  log.py           JSONL log, 14 days
lab/     test applications (Win32, WinForms, WPF, Tk, a web page for Electron/Edge)
bench/   technique benchmarks and raw results
tests/   unit tests (pytest) and end-to-end tests (pytest -m live)

Measurements

All figures come from bench/RESULTS.md and the raw files in bench/results/ (measured on one PC, October 2026; the notes in RESULTS.md are in French).

Cross-agent parity bench (October 26, 2026)

The three NexusWindows editions (this one, for Codex and for Antigravity) ran the same 33 tasks, 3 rounds each, on the Electron, WinForms and WPF lab windows, placed at the bottom of the z-order under an opaque cover, nobody at the PC (the bench stops the moment the user touches the mouse or keyboard). The effect is judged by the application (event received or value read back), never by the tool's report. Raw results: bench/results/2026-10-26_1045-parity.json (summary in French next to it).

Gap between the three: 0. Every Electron task 3/3 for all three. The 15 failures are the same for all three and are background limits refused honestly, not false successes: WPF raw click, Ctrl+K in WPF, WPF context menu, and double click in WinForms and WPF (the cover sits over the point; WinForms ignores a double click posted to a covered point). "Intrusion" means the foreground was taken, the cursor moved or the clipboard was written during the attempt.

Offline test suite: see the Tests badge (test.ps1 on Windows, plus a startup check on Linux).

Side by side with Cua Driver and Windows-MCP (October 26, 2026)

Cua Driver 0.38.0 is the closest other tool: it also works in the background by default. Windows-MCP 0.8.8 is the most used Windows MCP server; it has no background mode. All three were driven through their real MCP servers (bench/cua_vs_nexus.py) on the same lab windows, covered by an opaque window, nobody at the PC (the bench refuses to start before 2 minutes without physical input and stops at the first one); each was used the way its own guide tells an agent (Cua: background first, then clicks, then its foreground fallback; Windows-MCP: bring the window forward with App switch, then act on the screen). The effect is judged by the application (event received or value read back), never by the tool's report. Foreground changes are sampled every millisecond, the real cursor before and after each attempt. 51 tasks across Win32, WinForms, WPF, Tk, Electron and Edge (bench/tally_cua.py, raw results in bench/results/2026-10-26_1210-cua-vs-nexus.json). To reproduce: python bench/cua_vs_nexus.py on an idle Windows PC, with Cua Driver and Windows-MCP installed as described at the top of the script.

Windows-MCP reached the effect on all 9 Win32 tasks once its App switch brought the window forward. On the WinForms, WPF, Tk, Electron and Edge windows the switch left the window behind 30 times, and its click or typing then landed on the window on top (here the bench's cover; in real use, the user's own window) while it reported the click: 36 "done" answers with no effect, which says more about the absence of a background mode than about its reporting, so they are not counted as false successes in the table. 5 more attempts failed in the bench adapter (an item of an open list or menu was not found on screen).

Misses of this server in this run: WPF raw click, wheel and Ctrl+K, a wheel on a covered canvas (Tk, Electron, Edge), and Win32 Ctrl+K this time; each was reported as refused or as an unknown outcome (~), never as done.

History, from 3.0.1 (35 of 45 covered rows) to 3.0.2 (October 23, 44 of 51 against Cua's 28, details in bench/RESULTS.md): Tk clicks and shortcuts, Win32 Ctrl+K, and scrolling covered lists. Still refused, with the cause read in each toolkit's source and measured: WPF clicks, wheel and shortcuts, a wheel on a covered canvas, input at a point of a hidden Electron page. Taken from Cua: reading only what changed (then="changes"). Not taken: its injected WPF click (moves the user's real cursor), CDP clicks in an already open Chrome (turns on remote debugging in the user's real profile). Tried and dropped: simulated activation for WPF shortcuts (WPF takes the foreground), WM_POINTERWHEEL (Windows refuses to post it), WM_VSCROLL to the Chromium renderer (no effect), InjectSyntheticPointerInput (goes to the window on top).

Cross-agent bench (October 16, 2026)

The same task was run by the three Windows desktop agents used on this PC (this server, the Codex layer and the Antigravity layer) on WinForms and WPF forms created without activation at the bottom of the z-order, while the user kept working: fill three fields, tick a box, open the "Formule" drop-down list, choose "Bravo" in the open list, validate. Proof is the result label read back in the application. Physical input was forbidden and counted.

What still keeps this server from 20: clicking "Validate" stays ~ without an expect (the application did receive it), and a click on a closed WPF list is ~ where expand is ✓.

Security

• Emergency stop: Ctrl+Alt+Shift+Esc interrupts the running action at its next checkpoint (typing included) and refuses every new action for 2 minutes, across conversations and for the other agents (protocol above).
• Hard blocks (shared policy, src/nexus_windows/shared/garde-fous.json, same file as NexusWindows for Antigravity): the secure desktop (UAC, lock and sign-in screens) and password managers are never read nor driven. A missing or unreadable policy refuses every action (fail closed).
• Confirmation of irreversible gestures: paying, buying, ordering, transferring money, sending, publishing, posting, deleting permanently, emptying the bin, formatting, uninstalling, resetting, cancelling a subscription, deleting an account or making a repository public need confirm: true on the step, which the model sets only on the user's explicit order or yes. The check covers what the gesture triggers: click by [N], by name or by coordinates, double-click, Enter (target, default button, nearby send button), Ctrl+Enter, Space, Alt+letter, a typed line break DevTools pages and Win32 menus. A routine never records nor replays confirm.
• Approval bound to the exact action: an outward gesture (sending, calling, paying, publishing, transferring) also needs accord: "<token>". The first try is refused before any input with a summary of the exact action (trigger, process, window title, values of the window's editable fields) and a single-use token; only after the user's own yes is the same step sent again with confirm: true and that token. Any change in between voids it.
• Prompt injection signal: text read on screen that looks like instructions aimed at an AI gets a warning appended to the response (a signal only; it can never authorise an action).
• Launch: a browser never gets a profile chosen by the model, a DevTools port or a loaded extension (devtools=true opens a fresh throwaway profile). Consoles, File Explorer, the taskbar and other applications are not refused: the server is meant for a user who already lets Claude run commands; keep Claude's own permission prompts on if you want a gate there.
• Password fields: never typed into. Typed text is never logged (only its length).
• Accounts: a browser started with devtools=true gets a throwaway profile without account or sync, checked after launch and deleted afterwards.
• Screen content is data: every look reminds the model that on-screen text is not an instruction.
• Up-to-date code: the entry point is a small stdio supervisor (reloader.py). Before each tool call it compares the code on disk with the running server's; if it changed, it waits for calls in progress, starts a fresh server and replays the client's handshake, so an update needs no reconnection (/mcp). A server whose code changed between two checks refuses act, launch and window, and the call is replayed on the fresh one.
• Timeouts: an abandoned call is cancelled at its next checkpoint instead of finishing later.
• Clipboard: never used.

Privacy

NexusWindows for Claude runs entirely on your PC. It has no account, no telemetry and no analytics. Its only network request of its own is the update check: at most once a day, at startup and in the background, it reads the latest version number from PyPI (https://pypi.org/pypi/nexus-windows-claude/json, nothing sent but the request itself) and, when it was started by uvx, prepares that version in uv's cache for the next start of Claude. NEXUS_UPDATE_CHECK=off turns this off. DevTools connections stay on 127.0.0.1.

• What it reads: window titles, the UI Automation tree and window images of the applications Claude works in, only when a tool is called. Password managers and the secure desktop are never inspected (look) nor driven (act); windows still lists their window titles, like the taskbar does. The OCR fallback uses the Windows built-in engine on the PC; no image leaves it for OCR. look(window="screen") reads the whole screen as you see it, every visible window included. The notifications tool reads the Windows notification database read-only (app, time, texts; never modified), masking security codes.
• Where it goes: tool results are returned to Claude Desktop or Claude Code, which send them to Anthropic under Anthropic's own privacy policy; this server sends nothing anywhere else.
• What it stores, under %USERPROFILE%\.claude\nexus-claude\ (or NEXUS_WINDOWS_HOME): a JSONL debug log kept 14 days (tools, techniques, durations, errors; typed text is replaced by its length), the routines you chose to remember, and throwaway browser profiles created by devtools=true (deleted afterwards). Images are written to the log only if you set NEXUS_WINDOWS_DEBUG_IMAGES=1. Under %USERPROFILE%\.nexus\: the shared foreground token and emergency-stop state, and status\windows-claude.json (installed and latest version, update state) with the date of the last check in windows-claude\update.json. Under %LOCALAPPDATA%\NexusJournal\windows\claude\ (or NEXUS_JOURNAL_HOME): a short monthly JSONL journal, in the same format as the other NexusWindows tools (app executable name, framework, action, technique, outcome, duration, whether the foreground was used, and the first words of an error, cut before any quote, colon or bracket); never typed text, screen text, labels, titles or paths. The server never deletes it.
• Removing it: remove the server from Claude (claude mcp remove nexus-windows), then delete those folders (and %LOCALAPPDATA%\NexusJournal).

Questions or concerns: open an issue on this repository.

Limits

• Code comments are in French. Everything an MCP client receives (instructions, tool descriptions, step reports, refusal messages) is in English.
• Windows only. Tested on Windows 11 with the lab apps and a handful of real applications; not every application has been checked.
• Applications without UI Automation that ignore posted input (some games, canvases) need real input, so the user's approval and the foreground; the OCR fallback finds their text but does not make background input possible.
• WPF: raw clicks on a point, the mouse wheel and keyboard shortcuts are refused in the background (WPF drops a click while the real cursor is not over its window, and takes the foreground when given focus); controls with UI Automation patterns work.
• A mouse wheel on a covered Chromium or Tk canvas has no effect (both route it to the window on top); it is refused or reported as ~, never as a success. Lists and scrolling areas scroll without the wheel (see Per UI family).
• A covered Electron page that has gone hidden (it had the foreground once, then lost it) stops rendering: clicks at a point are reported as ~.
• Store apps that Windows suspends in the background are resumed (IPackageDebugSettings.Resume, no foreground); one that Windows still does not render can only be shown by taking the foreground; this is reported.
• In Chromium, a page that reacts very slowly to each keystroke can exceed the read-back time and is reported as ✗.
• An effect with nothing readable (a file written by a button, for example) stays ~; add a check that observes it.
• The end-to-end suite (pytest -m live) drives real windows and refuses to start if the PC was used less than 2 minutes ago.

Development

pwsh -NoProfile -File test.ps1         # unit tests, ruff check and format; venv and caches outside the clone
pwsh -NoProfile -File test.ps1 -Full   # also the end-to-end tests on the lab windows
uv run python bench/bench.py           # technique benchmark (when nobody is using the PC)

Licence

MIT, see LICENSE. Copyright (c) 2026 Julien Piron.