Skip to content

Latest commit

 

History

8 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

glance — the window, as text, as it changes

Point it at a window. glance reads what is on the screen the way a person does — from the application's own accessibility tree when there is one, from pixels when that is all there is — and turns every change into one line of text with a time on it. It never types, never clicks, never sends a byte anywhere. Its lines are what a coordinator's clock, an agent, or a resident reads next.

Status: 0.3.0 — built, --selftest green (132 checks), not yet on the tools page. Stages 0 to 2 work on this box: the UIA reader, the change stream, the strip, the pixel reader (DXGI, tile signatures, Windows OCR), the rules, the facts, the spool, the tape and --mcp. Stage 3 was measured on a loaded machine and on a real RDP session, and the one thing it found wrong — recognition running on the capture thread — is fixed: OCR runs on a worker with a queue depth of one per region, and a change's second look is taken from the window's own rendering rather than the next frame. Change-to-line is 18 ms through the tree and 44–183 ms through pixels. Stage 4 (the optional judge, and the second reader a bigger OCR model could be) is untouched. The design is in docs/BLUEPRINT.md, the rules in CLAUDE.md, the rules-file format in docs/RULES.md, the tools-page brief in docs/DESIGN-BRIEF-tools-page.md.

One job

It compiles a window into text. The tools page's six house rules are why it stops there: one job; free and MIT; runs beside the work on a strip; keyboard first; built on a loaded machine; the same answer for humans and agents. The vertical adds two more, kept by construction: nothing leaves the building (no network stack is linked) and no hands (no input injection is linked; the build fails if either appears, and glance --about prints the receipt read from its own import table).

Why it exists

On a transplant floor the donor record is a cloud web application the OPO does not control and cannot integrate with, often reached through Citrix or RDP where the workstation sees only pixels. Every tool that wants to help a coordinator has to read what the coordinator reads. On a developer's machine the same is true of every native window an agent cannot see: the build panel, the console, the dialog that is blocking. The screen is the one integration everyone already has. glance is the honest way to read it: the accessibility tree first, because it is text with names on it; pixels second, only where they changed, with the OS's own OCR.

Use

glance --window "iTransplant*"                     one line per change: HH:MM  source  path = value
glance --window W --rules rules/edr.yml --facts    caseclock's facts tape on stdout
glance --window W --json                           one JSON object per change, the fact inside when a rule fires
glance --window W --spool                          lane rows: t_mono_ns<TAB>glance<TAB>world<TAB>[win:X] path = value
glance --window W --read                           one walk of the tree (or one OCR of the pixels), then exit
glance --window W --no-uia                         pixels only (Citrix, RDP, a canvas)
glance --window W --strip                          the strip beside the lines; glancew.exe is the strip alone
glance --paths                                     the windows glance can see
glance --mcp                                       read-only MCP over stdio: glance_windows glance_read glance_changes glance_facts glance_status
glance --about                                     the organ's self-description, with the import-table receipt
glance --verify TAPE.jsonl                         walk a tape's chain (glance's, caseclock's — one format)
glance --selftest · --bench · --fixture edr        the oracle: a synthetic EDR window glance creates itself

W is a title (a substring, or a glob with */?), class:NAME, pid:N, hwnd:0x.., exe:NAME or region:x,y,w,h; the rules file's window: is the default. Keys on the strip: Ctrl+L target · Space pause · Ctrl+T pin · F facts · J json · W lines · C copy · Esc.

What it does

  • Reads the window through UI Automation: every control with a name and a value, walked once, then watched by property and structure events (a slow re-walk is the safety net). Changes arrive as lines: 21:30 uia Labs / Serology drawn = 21:30 (was 21:28) — label, path, value, time. A Label: value text splits at the colon; a field's label is its name.
  • Where the tree is empty (Citrix, RDP, a canvas) or with --no-uia, duplicates the desktop with DXGI, cuts the window into tiles (32 logical px), keeps a 9×8 difference hash and a 4×4 grid of means per tile, and runs Windows OCR only on the bands of tiles that moved, widened to whole lines. A value with no label on its own line takes the nearest text to its left as its label. The line says so: 21:31 pixels Serology drawn → "21:30" (0.90). A line whose tiles did not move cannot have changed, and a change read once is a candidate: it is read a second time before it becomes a line, so a window mid-repaint does not print half-drawn text. Recognition runs on its own worker, one job per region, so a slow read never blocks the next frame, and the second look comes from the window's own rendering rather than the next frame. Windows above the target in z-order mask the tiles under them; a mostly covered window, or one on another virtual desktop, is read from its own rendering (PrintWindow) once a second. Frames live in memory for one hash and one OCR call and are zeroed; nothing is ever written.
  • Writes every change to a tape (append-only, BLAKE2b-256 chained, REGISTRAR's core/tape.py format, text only — never a frame) and to a spool in the family's lane form, which fusord tails and caseclock reads.
  • Extracts facts by rules you own: rules/edr.yml maps a label glob to an event name and a time parser, and glance prints {"event":"serology_drawn","minutes":-150,"source":"uia:Labs/Serology drawn","confidence":1,"entered_at":"23:40 (-1d)","t0":"2026-09-03"} for caseclock's clock. The rules are data; the EDR's vocabulary is a file the site edits.
  • Shows a strip: glance · iTransplant — Chrome · UIA · 3 changes/min · 0 frames kept · 0 bytes sent.
  • Same answer for agents: --json, --mcp, --paths, all reading one change stream.

What it never does

No network stack. No SendInput, no PostMessage to the target, no clicks, no keys, no SetForegroundWindow: it has no hands and the linker proves it. No frames on disk, ever: the tape holds compiled text. No PHI leaves the machine because nothing leaves the machine; a redaction rule set keeps listed labels out of every output, and on the pixel path it pins the row beside such a label so nothing read there ever becomes a line; --phi-safe withholds values that look like a name, an MRN or a date of birth. No model inside: the optional judge that decides when a change deserves a sentence is a separate gear a later stage may add, off by default.

Where it sits

the EDR on screen ──▶ glance (UIA / pixels → lines) ──▶ caseclock (facts → the clock)
                                     └──▶ the spool (a lane the resident tails)
                                     └──▶ an agent (--json, --mcp)

facet finds files, vramtop shows the card, everywho shows the disk, caseclock keeps the clock, glance reads the screen. Five tools, five jobs, one tape shape.

Build

build.bat (VS 2022, /W4 /WX, static CRT, C++/WinRT from the SDK) produces glance.exe and glancew.exe, then fails the build if a network DLL is among the dependents or an input API is in the import table. glance --selftest before every commit.

MIT · Access Intellect LLC · built inside an OPO by someone who did the job.

About

The window, as text, as it changes: reads a window through its own accessibility tree, or through pixels and the OS's OCR where there is no tree (Citrix, RDP), and turns every change into one timestamped line, a fact for a clock, or JSON for an agent. Never types, never clicks, never sends a byte. One Windows exe, no network stack, no model. MIT.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages