From dd15bc5a62458a879ab9c236222ea6e03f76ee88 Mon Sep 17 00:00:00 2001 From: Timo Kramer Date: Mon, 31 Aug 2026 14:56:33 +0200 Subject: [PATCH 1/6] plan --- PLAN.md | 507 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 507 insertions(+) create mode 100644 PLAN.md diff --git a/PLAN.md b/PLAN.md new file mode 100644 index 0000000..544921f --- /dev/null +++ b/PLAN.md @@ -0,0 +1,507 @@ +# charm.clj — Improvement Plan + +Findings from a full review of `src/` (~5.5k lines), the examples, docs and CI, +plus a review of what changed in JLine since the project started (3.30.6 → +4.3.1, the current 4.x tip). Everything here was verified by running it: +benchmarks, escape-injection probes, the test suite (152 tests / 940 assertions, +all passing at time of review), and `javap` inspection of the local +`jline-terminal-4.3.1.jar`. + +Ordered by impact within each section. Each item lists the evidence, the +location, and a concrete suggestion. + +--- + +## Top priorities + +1. **Background color detection / adaptive colors (J1)** — first thing to solve. + JLine 4 exposes `getDefaultBackgroundColor()` (OSC 10/11 query); wire it up + to offer light/dark-adaptive styling, and wire in the dormant + `detect-color-profile` / `downgrade-color` code while in there. +2. **Sanitize untrusted content before it reaches the terminal**, and make + `strip-ansi` honest — OSC 52 injection writes attacker data into the user's + system clipboard. This is a real vulnerability, not a hardening nit. (S1, S2) +3. **Rewrite the event loop** to block on the channel, drain, and render once per + frame — fixes the 100 msg/s ceiling and makes `:fps` real in a single change. + (P1, P2, P3) +4. **Fix `:fg 240`, `:strikethrough` and `"pgup"`** — three small bugs that make + documented features, the library's own default styling, and Page Up/Down + silently do nothing. (U1, U2, U3) + +--- + +## 1. Performance + +### P1 — Event loop caps the library at ~100 messages/second + +`src/charm/program.clj:242` + +```clojure +(when-let [_ (a/ BEL` | **writes attacker data into the user's system clipboard** (iTerm2, kitty, foot, wezterm, tmux with `set-clipboard`) | +| `ESC ] 2 ; … BEL` | rewrites the terminal window/tab title | +| `ESC ] 8 ; ; BEL` | injects a hyperlink — visible text says one thing, the link goes elsewhere | +| `ESC P … ESC \` (DCS) | passes through untouched | +| `ESC [ 2K CR` | erases the line and rewrites it — content can hide what it just displayed | + +Highest-severity item. `AttributedString/fromAnsi` is an SGR parser, not a +sanitizer, and it is currently the only thing between untrusted input and the +terminal. + +**Suggestion:** add `charm.ansi/sanitize` that keeps SGR and drops +OSC/DCS/APC/PM/SOS, C1, and C0 other than `\n` / `\t`. Apply it in `render!` by +default, with an opt-out for content the app authored itself. + +### S2 — `strip-ansi` doesn't strip + +`src/charm/ansi/width.clj:13`. It is the function anyone displaying untrusted +data will reach for, and it leaves everything in S1 intact. It also *mangles* +private CSI rather than removing it: `ESC [ ?1049h` comes out as the visible +text `1049h`. + +**Suggestion:** make it exhaustive as part of S1. + +### S3 — Width measurement is wrong for unrecognised escapes + +`"file" ESC "]2;PWNED" BEL ".txt"` measures **14** but displays **8**. Every +downstream decision — truncate, pad, border, join, overlay — is then computed +from a wrong number, so injected content also breaks the frame. + +**Suggestion:** falls out of S1 once sanitisation happens before measurement. + +### S4 — `set-window-title` doesn't escape its argument + +`src/charm/render/screen.clj:52`. A BEL or ESC in the title terminates the OSC +early and lets the caller's string open a new one. Verified: title +`"hi" ESC "]52;c;ZXZpbA=="` emits two OSCs, the second a clipboard write. + +`copy-to-clipboard` (`src/charm/render/screen.clj:59`) is likewise unbounded — +terminals cap OSC 52 payloads and truncate silently. + +**Suggestion:** strip control characters from the title; document and enforce a +size cap on the clipboard payload. + +### S5 — Signal handlers are registered and never restored + +`src/charm/program.clj:228,233`. `run`'s `finally` restores terminal attributes +but not the WINCH/INT handlers. `Signals/register` returns the previous handler; +nothing captures it. + +In a REPL — which the README actively promotes — Ctrl+C stays hijacked after a +program exits, feeding a closure that puts into a closed channel, so SIGINT is +silently swallowed for the rest of the session. + +**Suggestion:** capture the return value of `Signals/register` and restore it in +the `finally`. + +### S6 — No shutdown hook + +`finally` covers exceptions but not `System/exit` or SIGTERM. The terminal is +left in raw mode, cursor hidden, alt-screen active. +`doc/examples/src/examples/timer.clj:198` calls `System/exit 1` from inside a +running program, so this is reachable from the repo's own code. + +**Suggestion:** register a shutdown hook that restores attributes, shows the +cursor and exits the alt screen. + +### S7 — CI supply chain + +`.github/workflows/ci.yml`: + +- `bash <(curl https://raw.githubusercontent.com/babashka/babashka/master/install)` + executes an unpinned script from a moving branch. +- Actions are pinned by tag, not commit SHA — including third-party + `DeLaGuardo/setup-clojure@13.4`. +- The `release` job holds `CLOJARS_PASSWORD` with `contents: write` and fires on + every push to `main`. +- `git config set user.email GIT_COMMITTER_EMAIL` is missing a `$` — it sets the + literal string, not the secret. + +**Suggestion:** pin actions by SHA, pin the babashka installer to a tag, and fix +the `$`. Consider gating release on a tag rather than every push to `main`. + +--- + +## 3. Usability + +### U1 — `:fg 240` silently does nothing + +`doc/api/styling.md:198` documents it as "ANSI 256 shorthand", but +`src/charm/style/color.clj:118` dispatches on `(:type color)`, which is `nil` for +an integer, and returns the style unchanged. Verified: +`(style/render (style/style :fg 240) "hi")` → `"hi"`, no escapes. + +Used in the library's own defaults — `src/charm/components/text_input.clj:97` +(placeholder) and `src/charm/components/list.clj:305` (item descriptions) — and +in ~15 places across docs and examples. So a documented feature, the library's +default styling, and most of the sample code all render as plain text. + +**Suggestion:** coerce integers and keywords to colour maps in `style` / +`with-fg` / `with-bg`, or throw on an unrecognised colour value. + +### U2 — `:strikethrough` silently does nothing + +Documented in the `doc/api/styling.md:32` options table, used in +`doc/examples/src/examples/todos.clj:20`, never applied by +`style->attributed-style` (`src/charm/style/core.clj:143`). + +**Suggestion:** add it to the `cond->`, and audit the docs table against that +function for anything else missing. + +### U3 — `"pgup"` / `"pgdown"` never match — Page Up/Down is dead in four components + +Key events carry `:key :page-up`; `key-match?` compares against +`(name :page-up)` = `"page-up"`. Verified: +`(msg/key-match? (msg/key-press :page-up) "pgup")` → `false`. + +Affects `src/charm/components/viewport.clj:29-30`, +`src/charm/components/list.clj:24-25`, +`src/charm/components/table.clj:28-29`, +`src/charm/components/paginator.clj:20-21`. + +**Suggestion:** rename the bindings to `"page-up"` / `"page-down"`, or accept +aliases in `key-match?`. Add a test that asserts every default binding string in +every component resolves against a real key event — this class of bug is +otherwise invisible. + +### U4 — `(style :padding 3)` throws + +`render` passes the raw value to `expand-box-values` → +`UnsupportedOperationException: count not supported on this type: Long`. +`with-padding` normalizes a bare number; the `style` constructor doesn't, so the +two entry points disagree. + +`hex` (`src/charm/style/color.clj:96`) likewise throws a raw +`NumberFormatException` on bad input rather than an `ex-info`. + +**Suggestion:** normalize in `style`, and wrap `hex` parse failures in `ex-info` +with the offending string. + +### U5 — Bracketed paste is built but never wired up + +`src/charm/render/core.clj:146-154` has the enable/disable calls and +`src/charm/input/keymap.clj:79-80` binds `:paste-start` / `:paste-end` — but +`run` has no option and never enables it. So pastes arrive as individual +keystrokes (throttled to 100/s by P1), and the markers are indistinguishable +from typed input. + +**Suggestion:** add a `:bracketed-paste` option, enable it in `start!`, and +coalesce everything between the markers into a single paste message. + +### U6 — Ctrl+C is intercepted and turned into a key message + +`src/charm/program.clj:233`. If the app's `update` doesn't handle `"ctrl+c"`, the +program cannot be killed from the keyboard. + +**Suggestion:** make it an option, default to quitting unless the app opts in, +and document it prominently either way. + +### U7 — Overflowing views are truncated from the top + +`src/charm/render/core.clj:184` keeps the *last* `height` lines. Sensible inline; +surprising full-screen, where a view one line too tall silently loses its title. + +**Suggestion:** document it, and make the direction configurable. + +### U8 — Component IDs are `(rand-int 1000000)` + +`src/charm/components/list.clj:88`, `src/charm/components/text_input.clj:79`, +`src/charm/components/viewport.clj:66`. ~1% collision chance at 150 components, +and it makes component state non-reproducible in tests. + +**Suggestion:** a counter or `gensym`. + +### U9 — Two overlapping key-matching APIs + +`msg/key-match?` and `keys/key-matches?`, with different pattern semantics +operating on different data shapes (message vs. event map). + +**Suggestion:** collapse to one; keep `msg/key-match?` as the public entry point. + +### U10 — `style` returns all 17 keys defaulted + +So `merge`ing a variant onto a base overwrites with `nil` / `false` instead of +inheriting. There is no `merge-style` / `inherit`, which is the first thing +anyone building a theme reaches for. + +**Suggestion:** add `merge-style` that ignores unset keys, or stop defaulting +every key in the constructor. + +### U11 — Nothing auto-sizes to the terminal + +Every component takes `:height 0` = unbounded, so each app hand-threads +window-size arithmetic. `doc/examples/src/examples/file_browser.clj:85-93` spends +real code on `chrome-height` bookkeeping. + +**Suggestion:** a "fill remaining space" affordance would remove a lot of +per-app boilerplate. + +--- + +## 4. JLine 4.x adoption + +The project started on JLine 3.30.6 (2026-01), moved to 4.0.10/4.0.12 (April — +the Mode 2027 grapheme work, ADR 007), and sits on 4.3.1 (July) — verified to be +the current tip of the 4.x line (3.30.16 is a maintenance backport on the old +branch). So the version is current; these items are 4.x capabilities the +library doesn't use yet, plus behavior changes to be aware of. + +Already absorbed: Mode 2027 grapheme clustering (ADR 007), the 4.3.1 ReDoS +guards, 4.1's `Display.update()` optimizations, and the shift+tab CSI Z binding +(done in-repo, `aa6a75e` — JLine still doesn't bind it). + +### J1 — Background color detection / adaptive colors ← do this first + +JLine 4 can query the terminal's actual default colors (OSC 10/11): + +```java +terminal.getDefaultBackgroundColor() ; -> int RGB, or -1 if unknown +terminal.getDefaultForegroundColor() +terminal.getPalette() ; -> ColorPalette +``` + +This is the primitive lipgloss uses for `HasDarkBackground` — light/dark +adaptive styling is one of the most-requested TUI features and charm.clj +currently has no way to do it. + +Related dormant code: `charm.style.color` already contains +`detect-color-profile` and `downgrade-color`, and **nothing calls either** +(verified by grep). JLine 4 also detects true-color from `COLORTERM` natively. + +**Suggestion:** +- Add `charm.terminal/dark-background?` (query `getDefaultBackgroundColor`, + compute luminance, sensible default when the query returns -1). +- Add an adaptive color type: `{:type :adaptive :light :dark }`, + resolved at render time against the detected background. +- Wire `detect-color-profile` + `downgrade-color` into the render path (or + delegate profile detection to JLine's `ColorPalette`) so ANSI-only terminals + degrade instead of getting raw true-color escapes. +- Expose the resolved profile/background on the program state (e.g. an + `:environment` msg at startup) so apps can branch on it. +- Note the interaction with U1: fixing integer-color coercion first avoids + building adaptive colors on top of a constructor that silently drops ints. + +### J2 — `KeyEvent` / `KeyParser`: revisit ADR 004 + +`org.jline.terminal.KeyParser/parse` (static, non-blocking — none of ADR 004's +BindingReader objections apply) returns a structured `KeyEvent`: type +(Character/Arrow/Function/Special/Unknown), `EnumSet` of Shift/Alt/Control, +raw sequence. This API did not exist when ADR 004 was written and overlaps +heavily with the ~500 hand-rolled lines in `charm.input.keys` + +`charm.input.keymap` (including the generated xterm modifier table). + +**Suggestion:** spike — run the corpus from `test/charm/input/keys_test.clj` +through `KeyParser.parse` and diff coverage. If it covers the table, delete +code; if not, document the gap in ADR 004 and keep the keymap. + +### J3 — `ScreenTerminal` for integration tests + +`org.jline.utils.ScreenTerminal` (consolidated 4.2, scrollback + cell decoding +exposed 4.3.0) is a full in-memory VT emulator: `write` the renderer's output, +then assert on the resulting screen grid via `dump` / `getHistory` and per-cell +accessors (`cellCodePoint`, `cellBold`, `cellFg`, …). + +The current integration-test pattern (ADR 003: dumb terminal + +`ByteArrayOutputStream`) asserts on raw escape bytes, which breaks whenever +`Display`'s diffing strategy changes even though the screen is identical. +Asserting on final screen cells tests what the user actually sees, and could +absorb some VHS visual tests without the ffmpeg/ttyd CI machinery (currently +commented out in `ci.yml`). + +**Suggestion:** add a test helper that renders a frame into a `ScreenTerminal` +and returns the grid; migrate the brittle byte-level assertions; update ADR 003. + +### J4 — `Terminal.trackMouse` now covers SGR + +`MouseSupport` in 4.3.1 emits `?1005/?1006/?1015` (verified in the jar), so +ADR 004's era gap is closed. `trackMouse(MouseTracking/{Off,Normal,Button,Any})` +maps 1:1 onto charm's `nil/:normal/:cell/:all` and could replace the raw escape +writes in `src/charm/render/core.clj:96-120`; `readMouseEvent(String)` accepts +the already-read prefix, addressing the "detect the prefix first" objection. +JLine then also cleans up mouse state on `close()`. + +**Suggestion:** replace enable/disable with `trackMouse`; keep the custom SGR +*parsing* (it works and is tested). + +### J5 — Terminal graphics: images (Kitty / iTerm2 / Sixel) + +4.0 added `TerminalGraphicsManager` with runtime protocol detection and +one-call display (`isGraphicsSupported`, `displayImage(terminal, file)`). +A genuine feature opportunity — image support Bubble Tea core doesn't have. + +Caveats: lives in the `impl` package (weaker stability guarantees) and depends +on `java.awt.BufferedImage` — expect unavailable on babashka and extra config +for native-image (`isJavaDesktopAvailable` guard exists for graceful +degradation). + +**Suggestion:** optional `charm.components.image` guarded by +`isGraphicsSupported`; document the platform matrix. + +### J6 — Signal handling: migrate to `Terminal.handle` + +JLine 4 modernized signals (4.1 FFM `sigaction`, 4.2 ISIG restoration, 4.3 +PosixSysTerminal interception). charm uses the static +`org.jline.utils.Signals/register` (`src/charm/program.clj:228,233`); +`Terminal.handle(Signal, SignalHandler)` is the supported route **and returns +the previous handler** — exactly what S5 needs to restore Ctrl+C after exit. + +**Suggestion:** fix S5 by migrating to `Terminal.handle`, one change for both. + +### J7 — Behavior changes to be aware of + +- **Terminals throw when used after close** (4.0 breaking change; 3.x was + silent). `run`'s cleanup ordering is fine, but `run-async`'s `:quit!` racing + a late `render!` against `term/close` would now throw. Audit alongside S5/S6. +- 4.1.2/4.1.3 fixed alt-screen cursor positioning and raw-mode ISIG behavior — + any local workarounds can be removed. +- New sibling modules `jline-prompt` / `jline-components` / `jline-shell` (4.0): + JLine growing its own UI story. Nothing to adopt, worth watching as + overlapping ecosystem. +- **Kitty keyboard protocol (CSI u): not in JLine** as of 4.3.1 — the xterm + modifier table in `keymap.clj` remains necessary; disambiguated keys + (Shift+Enter etc.) would need an in-repo implementation. + +--- + +## Suggested sequencing + +**Phase 0 — background color detection (J1)** +First thing to solve. Do U1 (integer-color coercion) as its opening step so +adaptive colors aren't built on a constructor that silently drops ints. + +**Phase 1 — correctness bugs, small and independent** +U2, U3, U4, P3, P7, P9, and the `$` fix in S7. + +**Phase 2 — security** +S1 + S2 + S3 together (one sanitiser), then S4, S5 via J6, S6. Pin CI actions +(S7). Audit the close-race from J7. + +**Phase 3 — event loop** +P1 + P2 as one change, then P6, P8. U5 and U6 land naturally on top. + +**Phase 4 — rendering architecture** +P4, then P5 (`AttributedString` through the layout stack). Largest change, +biggest headroom, best done once the loop above is stable. + +**Phase 5 — JLine adoption + API ergonomics** +J3 (ScreenTerminal tests — can also be pulled earlier, it's independent), +J2 spike, J4, J5. U7, U8, U9, U10, U11. From a80d88c6e8a433aec4721a5f8825a953584613eb Mon Sep 17 00:00:00 2001 From: Timo Kramer Date: Mon, 31 Aug 2026 20:16:20 +0200 Subject: [PATCH 2/6] feat: adaptive colors and terminal color environment detection MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Wires up light/dark-adaptive styling (J1) with integer-color coercion (U1) as its opening step, so adaptive colors aren't built on a constructor that silently drops ints. U1 — colors given as integers, keywords or hex strings were silently dropped, because apply-color-fg/bg dispatched on (:type color), which is nil for anything but a color map. This made a documented feature, the library's own default styling (help, list, text-input) and most sample code render as plain text. coerce-color now accepts integers as ANSI 256, keywords as ANSI 16 names and strings as hex, and throws an ex-info naming the offending value otherwise. Coercion lives in apply-color-fg/bg rather than in each constructor, so style, with-fg, with-bg and styled-str all pick it up at once. J1 — terminal/dark-background? queries JLine's getDefaultBackgroundColor (OSC 11) and computes luminance, defaulting to dark when the terminal doesn't answer. Adaptive colors are {:type :adaptive :light … :dark …}, built with style/adaptive and resolved at render time. run detects the profile and background once at startup, binds both dynamic vars around the event loop, and sends an :environment message so apps can branch themselves. The previously dormant detect-color-profile and downgrade-color are now live in the render path. Fixing downgrade-color was a prerequisite for that last part: its :ansi branch did (mod code 16) on a 256-cube index, which is colorimetrically meaningless — orange (rgb 255 128 0) came out cyan. Harmless while nothing called it, but wiring it in would have turned every RGB color on a TERM=xterm terminal into an effectively random one of 16. Replaced with a nearest-neighbour match against the existing ansi-hex table, plus a new ansi256->rgb covering the cube and grayscale ramp. The background probe is bounded by JLine's org.jline.terminal.probe.timeout (200ms default), so a terminal that ignores the query adds that much to startup once. Claude-Session: https://claude.ai/code/session_01CQeoheU7AxEmE5ZdATzg9q --- src/charm/message.clj | 17 +++ src/charm/program.clj | 188 +++++++++++++++++--------------- src/charm/style/color.clj | 154 +++++++++++++++++++------- src/charm/style/core.clj | 4 +- src/charm/terminal.clj | 32 ++++++ test/charm/message_test.clj | 11 ++ test/charm/style/color_test.clj | 102 ++++++++++++++++- test/charm/style/core_test.clj | 15 +++ test/charm/terminal_test.clj | 21 ++++ 9 files changed, 419 insertions(+), 125 deletions(-) diff --git a/src/charm/message.clj b/src/charm/message.clj index 88bb093..f6aee73 100644 --- a/src/charm/message.clj +++ b/src/charm/message.clj @@ -59,6 +59,18 @@ :ctrl ctrl :shift shift}) +(defn environment + "Create an environment message describing the terminal. + Sent once at program startup so apps can branch on the environment. + + Keys: + :color-profile - :ascii, :ansi, :ansi256, or :true-color + :dark-background? - whether the terminal background is dark" + [color-profile dark-background?] + {:type :environment + :color-profile color-profile + :dark-background? dark-background?}) + (defn focus "Create a focus gained message." [] @@ -103,6 +115,11 @@ [msg] (= :mouse (:type msg))) +(defn environment? + "Check if message is an environment message." + [msg] + (= :environment (:type msg))) + (defn focus? "Check if message is a focus event." [msg] diff --git a/src/charm/program.clj b/src/charm/program.clj index ef45909..65da86f 100644 --- a/src/charm/program.clj +++ b/src/charm/program.clj @@ -12,6 +12,7 @@ [charm.input.keymap :as km] [charm.message :as msg] [charm.render.core :as render] + [charm.style.color :as color] [charm.terminal :as term] [clojure.core.async :as a :refer [>! chan close! go]]) (:import @@ -183,7 +184,11 @@ The init function should return [initial-state cmd] or just initial-state. The update function receives (state msg) and returns [new-state cmd]. - Commands are optional and can be nil." + Commands are optional and can be nil. + + At startup an :environment message with the detected :color-profile and + :dark-background? is sent, and both are bound for the duration of the + program so colors resolve against the actual terminal." [{:keys [init update view running?] :as opts}] (let [opts (merge (default-opts) opts) {:keys [alt-screen mouse focus-reporting fps hide-cursor]} opts @@ -192,6 +197,12 @@ terminal (term/create-terminal) ^Attributes original-attrs (term/enter-raw-mode terminal) + ;; Detect the environment for adaptive styling. The background query + ;; reads the terminal's OSC response, so it must happen before the + ;; input loop starts consuming input. + color-profile (color/detect-color-profile) + dark-background? (term/dark-background? terminal) + ;; Create renderer renderer (render/create-renderer terminal :fps fps @@ -212,105 +223,110 @@ [init-result nil]) state (atom initial-state)] - (try - ;; Setup renderer - (render/start! renderer) - - ;; Setup mouse - (when mouse - (render/enable-mouse! renderer mouse)) - - ;; Setup focus reporting - (when focus-reporting - (render/enable-focus-reporting! renderer)) - - ;; Handle window resize signal - (Signals/register "WINCH" - (reify Runnable - (run [_] - (check-window-size! terminal msg-chan last-size)))) - - (Signals/register "INT" - (reify Runnable - (run [_] - (a/put! msg-chan (msg/key-press "c" :ctrl true))))) - - ;; Check initial window size - (check-window-size! terminal msg-chan last-size) - - ;; Start input loop (returns thread) - (let [^Thread input-thread (start-input-loop! terminal msg-chan running?)] - - ;; Execute init command - (execute-cmd! init-cmd msg-chan) - - ;; Render initial view - (render/render! renderer (view @state)) - - ;; Main event loop - (loop [] - (when @running? - (when-let [_ (a/ AttributedStyle/DEFAULT - (apply-color-fg fg) - (apply-color-bg bg))] - (.toAnsi (AttributedString. ^String text style))))) + Returns nil for nil; throws ex-info on unrecognised values." + [color] + (cond + (nil? color) nil + (and (map? color) (:type color)) color + (integer? color) (ansi256 color) + (and (keyword? color) (contains? ansi-colors color)) (ansi color) + (string? color) (hex color) + :else (throw (ex-info (str "Unrecognised color value: " (pr-str color)) + {:color color})))) ;; --------------------------------------------------------------------------- ;; Color Conversion @@ -173,6 +171,38 @@ ;; Map to 6x6x6 color cube (ansi256 (+ 16 (* 36 cube-r) (* 6 cube-g) cube-b))))) +(def ^:private cube-levels + "The six channel levels of the ANSI 256 color cube." + [0 95 135 175 215 255]) + +(defn ansi256->rgb + "Convert an ANSI 256 color to its RGB value." + [{:keys [code]}] + (cond + ;; 0-15: standard colors, from the hex table + (< code 16) (hex (nth ansi-hex code)) + + ;; 232-255: grayscale ramp + (>= code 232) (let [v (+ 8 (* 10 (- code 232)))] + (rgb v v v)) + + ;; 16-231: 6x6x6 color cube + :else (let [idx (- code 16)] + (rgb (nth cube-levels (quot idx 36)) + (nth cube-levels (mod (quot idx 6) 6)) + (nth cube-levels (mod idx 6)))))) + +(defn rgb->ansi16 + "Convert RGB to the nearest ANSI 16 color by distance in RGB space." + [{:keys [r g b]}] + (let [distance (fn [hex-str] + (let [{hr :r hg :g hb :b} (hex hex-str)] + (+ (* (- r hr) (- r hr)) + (* (- g hg) (- g hg)) + (* (- b hb) (- b hb)))))] + (ansi (first (apply min-key second + (map-indexed (fn [i h] [i (distance h)]) ansi-hex)))))) + (defn downgrade-color "Downgrade a color to fit a color profile." [color profile] @@ -180,8 +210,8 @@ :ascii (no-color) :ansi (case (:type color) :ansi color - :ansi256 (ansi (mod (:code color) 16)) - :rgb (ansi (mod (:code (rgb->ansi256 color)) 16)) + :ansi256 (rgb->ansi16 (ansi256->rgb color)) + :rgb (rgb->ansi16 color) color) :ansi256 (case (:type color) (:ansi :ansi256) color @@ -190,6 +220,56 @@ :true-color color color)) +(defn resolve-color + "Resolve a color value for rendering: coerce it to a color map, pick the + light or dark variant of adaptive colors against *dark-background?*, and + downgrade it to *color-profile*. Returns nil for nil." + [color] + (when-let [color (coerce-color color)] + (-> (if (= :adaptive (:type color)) + (coerce-color (if *dark-background?* (:dark color) (:light color))) + color) + (downgrade-color *color-profile*)))) + +;; --------------------------------------------------------------------------- +;; Color Application (via JLine AttributedStyle) +;; --------------------------------------------------------------------------- + +(defn apply-color-fg + "Resolve a color value and apply it as foreground to an AttributedStyle." + ^AttributedStyle [^AttributedStyle style color] + (let [color (resolve-color color)] + (if (or (nil? color) (= :none (:type color))) + style + (case (:type color) + :ansi (.foreground style (int (:code color))) + :ansi256 (.foreground style (int (:code color))) + :rgb (.foreground style (int (:r color)) (int (:g color)) (int (:b color))) + style)))) + +(defn apply-color-bg + "Resolve a color value and apply it as background to an AttributedStyle." + ^AttributedStyle [^AttributedStyle style color] + (let [color (resolve-color color)] + (if (or (nil? color) (= :none (:type color))) + style + (case (:type color) + :ansi (.background style (int (:code color))) + :ansi256 (.background style (int (:code color))) + :rgb (.background style (int (:r color)) (int (:g color)) (int (:b color))) + style)))) + +(defn styled-str + "Create a styled string with foreground and/or background color. + Returns the string with ANSI escape sequences applied." + [text & {:keys [fg bg]}] + (if (and (nil? fg) (nil? bg)) + text + (let [style (-> AttributedStyle/DEFAULT + (apply-color-fg fg) + (apply-color-bg bg))] + (.toAnsi (AttributedString. ^String text style))))) + ;; --------------------------------------------------------------------------- ;; Convenience Colors ;; --------------------------------------------------------------------------- diff --git a/src/charm/style/core.clj b/src/charm/style/core.clj index 2f78db6..e8e7aa9 100644 --- a/src/charm/style/core.clj +++ b/src/charm/style/core.clj @@ -21,7 +21,8 @@ "Create a style map. Options: - ;; Colors + ;; Colors (a color map, an ANSI 256 int like 240, an ANSI 16 + ;; name keyword like :red, or a hex string like \"#ff8000\") :fg - Foreground color :bg - Background color @@ -252,6 +253,7 @@ (def hex c/hex) (def ansi c/ansi) (def ansi256 c/ansi256) +(def adaptive c/adaptive) ;; Common colors (def black c/black) diff --git a/src/charm/terminal.clj b/src/charm/terminal.clj index 42bea63..5252b3c 100644 --- a/src/charm/terminal.clj +++ b/src/charm/terminal.clj @@ -29,6 +29,38 @@ [^Terminal terminal ^Attributes attrs] (.setAttributes terminal attrs)) +(defn default-foreground-color + "Get the terminal's default foreground color as a 0xRRGGBB int, + or nil if it cannot be determined." + [^Terminal terminal] + (let [color (.getDefaultForegroundColor terminal)] + (when-not (neg? color) color))) + +(defn default-background-color + "Get the terminal's default background color as a 0xRRGGBB int, + or nil if it cannot be determined. Queries the terminal (OSC 11) + where supported, so call this before reading terminal input." + [^Terminal terminal] + (let [color (.getDefaultBackgroundColor terminal)] + (when-not (neg? color) color))) + +(defn dark-color? + "True when a 0xRRGGBB color int is dark (luminance below 50%)." + [rgb] + (let [r (bit-and (unsigned-bit-shift-right rgb 16) 0xff) + g (bit-and (unsigned-bit-shift-right rgb 8) 0xff) + b (bit-and rgb 0xff)] + (< (+ (* 0.299 r) (* 0.587 g) (* 0.114 b)) 127.5))) + +(defn dark-background? + "True when the terminal's default background is dark. + Defaults to true when the background cannot be determined, + since dark terminal themes are the common case." + [^Terminal terminal] + (if-let [bg (default-background-color terminal)] + (dark-color? bg) + true)) + (defn get-size "Get terminal dimensions as {:width cols :height rows}." [^Terminal terminal] diff --git a/test/charm/message_test.clj b/test/charm/message_test.clj index 67ce2c6..c5d84a9 100644 --- a/test/charm/message_test.clj +++ b/test/charm/message_test.clj @@ -117,3 +117,14 @@ (is (msg/wheel? (msg/mouse :wheel-up :none 10 20))) (is (msg/wheel? (msg/mouse :wheel-down :none 10 20))) (is (not (msg/wheel? (msg/mouse :press :left 10 20)))))) + +(deftest environment-test + (testing "creates environment message" + (let [m (msg/environment :ansi256 true)] + (is (= :environment (:type m))) + (is (= :ansi256 (:color-profile m))) + (is (true? (:dark-background? m))))) + + (testing "environment? predicate" + (is (msg/environment? (msg/environment :true-color false))) + (is (not (msg/environment? (msg/quit)))))) diff --git a/test/charm/style/color_test.clj b/test/charm/style/color_test.clj index 20c6297..28da4c5 100644 --- a/test/charm/style/color_test.clj +++ b/test/charm/style/color_test.clj @@ -32,6 +32,52 @@ (is (= {:type :rgb :r 255 :g 255 :b 255} (c/hex "ffffff"))) (is (= {:type :rgb :r 0 :g 0 :b 0} (c/hex "000000"))))) +(deftest adaptive-color-test + (testing "creates adaptive colors" + (is (= {:type :adaptive :light {:type :ansi :code 0} :dark {:type :ansi :code 7}} + (c/adaptive c/black c/white))))) + +(deftest coerce-color-test + (testing "returns nil for nil" + (is (nil? (c/coerce-color nil)))) + + (testing "returns color maps unchanged" + (is (= c/red (c/coerce-color c/red))) + (is (= (c/rgb 1 2 3) (c/coerce-color (c/rgb 1 2 3))))) + + (testing "coerces integers to ANSI 256" + (is (= {:type :ansi256 :code 240} (c/coerce-color 240))) + (is (= {:type :ansi256 :code 0} (c/coerce-color 0)))) + + (testing "coerces ANSI 16 name keywords" + (is (= {:type :ansi :code 1} (c/coerce-color :red))) + (is (= {:type :ansi :code 12} (c/coerce-color :bright-blue)))) + + (testing "coerces hex strings" + (is (= {:type :rgb :r 255 :g 128 :b 0} (c/coerce-color "#ff8000")))) + + (testing "throws on unrecognised values" + (is (thrown? clojure.lang.ExceptionInfo (c/coerce-color :no-such-color))) + (is (thrown? clojure.lang.ExceptionInfo (c/coerce-color 1.5))))) + +(deftest resolve-color-test + (testing "resolves adaptive colors against *dark-background?*" + (let [color (c/adaptive c/black c/white)] + (binding [c/*dark-background?* true] + (is (= c/white (c/resolve-color color)))) + (binding [c/*dark-background?* false] + (is (= c/black (c/resolve-color color)))))) + + (testing "coerces the chosen adaptive variant" + (binding [c/*dark-background?* true] + (is (= {:type :ansi256 :code 254} (c/resolve-color (c/adaptive 236 254)))))) + + (testing "downgrades to *color-profile*" + (binding [c/*color-profile* :ansi256] + (is (= :ansi256 (:type (c/resolve-color (c/rgb 255 0 0)))))) + (binding [c/*color-profile* :ascii] + (is (= {:type :none} (c/resolve-color (c/rgb 255 0 0))))))) + (deftest styled-str-test (testing "applies foreground color" (is (= "\u001b[31mx\u001b[0m" (c/styled-str "x" :fg (c/ansi :red)))) @@ -55,7 +101,15 @@ (testing "returns unchanged for no colors" (is (= "x" (c/styled-str "x"))) - (is (= "x" (c/styled-str "x" :fg nil :bg nil))))) + (is (= "x" (c/styled-str "x" :fg nil :bg nil)))) + + (testing "coerces shorthand color values" + (is (= "\u001b[38;5;240mx\u001b[0m" (c/styled-str "x" :fg 240))) + (is (= "\u001b[31mx\u001b[0m" (c/styled-str "x" :fg :red)))) + + (testing "renders plain text under the :ascii profile" + (binding [c/*color-profile* :ascii] + (is (= "x" (c/styled-str "x" :fg (c/rgb 255 0 0))))))) (deftest rgb->ansi256-test (testing "converts RGB to ANSI 256" @@ -67,6 +121,52 @@ (is (= :ansi256 (:type result))) (is (>= (:code result) 232))))) ; Grayscale starts at 232 +(deftest ansi256->rgb-test + (testing "converts standard colors" + (is (= {:type :rgb :r 0 :g 0 :b 0} (c/ansi256->rgb (c/ansi256 0)))) + (is (= {:type :rgb :r 255 :g 0 :b 0} (c/ansi256->rgb (c/ansi256 9))))) + + (testing "converts the 6x6x6 cube" + ;; 196 is the cube's pure red corner + (is (= {:type :rgb :r 255 :g 0 :b 0} (c/ansi256->rgb (c/ansi256 196)))) + (is (= {:type :rgb :r 0 :g 0 :b 0} (c/ansi256->rgb (c/ansi256 16))))) + + (testing "converts the grayscale ramp" + (is (= {:type :rgb :r 8 :g 8 :b 8} (c/ansi256->rgb (c/ansi256 232)))) + (is (= {:type :rgb :r 238 :g 238 :b 238} (c/ansi256->rgb (c/ansi256 255)))))) + +(deftest rgb->ansi16-test + (testing "maps colors to the nearest ANSI 16 entry" + (is (= c/black (c/rgb->ansi16 (c/rgb 0 0 0)))) + (is (= c/bright-red (c/rgb->ansi16 (c/rgb 255 0 0)))) + (is (= c/bright-green (c/rgb->ansi16 (c/rgb 0 255 0)))) + (is (= c/bright-white (c/rgb->ansi16 (c/rgb 255 255 255))))) + + (testing "orange lands on a warm color, not an arbitrary one" + ;; Regression: (mod code 16) used to turn orange into cyan + (is (contains? #{c/yellow c/red c/bright-red c/bright-yellow} + (c/rgb->ansi16 (c/rgb 255 128 0)))))) + +(deftest downgrade-color-test + (testing ":ansi downgrade preserves hue" + (is (contains? #{c/yellow c/red c/bright-red c/bright-yellow} + (c/downgrade-color (c/rgb 255 128 0) :ansi))) + (is (contains? #{c/yellow c/red c/bright-red c/bright-yellow} + (c/downgrade-color (c/ansi256 214) :ansi)))) + + (testing ":ansi leaves ANSI 16 colors alone" + (is (= c/red (c/downgrade-color c/red :ansi)))) + + (testing ":ansi256 downgrades RGB only" + (is (= :ansi256 (:type (c/downgrade-color (c/rgb 255 128 0) :ansi256)))) + (is (= c/red (c/downgrade-color c/red :ansi256)))) + + (testing ":ascii drops all color" + (is (= {:type :none} (c/downgrade-color (c/rgb 255 128 0) :ascii)))) + + (testing ":true-color passes through" + (is (= (c/rgb 255 128 0) (c/downgrade-color (c/rgb 255 128 0) :true-color))))) + (deftest predefined-colors-test (testing "predefined colors are available" (is (= {:type :ansi :code 0} c/black)) diff --git a/test/charm/style/core_test.clj b/test/charm/style/core_test.clj index 6cf9068..bd12989 100644 --- a/test/charm/style/core_test.clj +++ b/test/charm/style/core_test.clj @@ -5,6 +5,21 @@ [charm.style.border :as b] [clojure.string :as str])) +(deftest color-shorthand-render-test + (testing "integer :fg renders as ANSI 256" + (is (str/includes? (s/render (s/style :fg 240) "hi") "\u001b[38;5;240m"))) + + (testing "keyword and hex string colors render" + (is (str/includes? (s/render (s/style :fg :red) "hi") "\u001b[31m")) + (is (str/includes? (s/render (s/style :fg "#ff0000") "hi") "\u001b["))) + + (testing "adaptive colors resolve against the background" + (let [st (s/style :fg (s/adaptive c/black c/white))] + (binding [c/*dark-background?* true] + (is (str/includes? (s/render st "hi") "\u001b[37m"))) + (binding [c/*dark-background?* false] + (is (str/includes? (s/render st "hi") "\u001b[30m")))))) + (deftest style-creation-test (testing "creates style with defaults" (let [st (s/style)] diff --git a/test/charm/terminal_test.clj b/test/charm/terminal_test.clj index 528727b..5e678ee 100644 --- a/test/charm/terminal_test.clj +++ b/test/charm/terminal_test.clj @@ -36,6 +36,27 @@ (finally (term/close t)))))) +(deftest dark-color?-test + (testing "dark colors are detected by luminance" + (is (term/dark-color? 0x000000)) + (is (term/dark-color? 0x1e1e2e)) + (is (not (term/dark-color? 0xffffff))) + (is (not (term/dark-color? 0xfafafa)))) + + (testing "luminance weights green heaviest" + (is (not (term/dark-color? 0x00ff00))) + (is (term/dark-color? 0x0000ff)))) + +(deftest dark-background?-test + (testing "returns a boolean, true when the background is unknown" + ;; In non-interactive environments JLine creates a dumb terminal, + ;; which cannot answer the OSC 11 background query. + (let [t (term/create-terminal)] + (try + (is (boolean? (term/dark-background? t))) + (finally + (term/close t)))))) + (deftest reader-writer-test (testing "get-reader returns non-nil" (let [t (term/create-terminal)] From 1af2dae2d6803243253b56fa0e660788ea399f27 Mon Sep 17 00:00:00 2001 From: Timo Kramer Date: Mon, 31 Aug 2026 20:16:35 +0200 Subject: [PATCH 3/6] docs: document adaptive colors, profiles and :environment Covers the API added alongside J1 and U1: style/adaptive and how it resolves against the detected background, the shorthand color forms (integer, keyword, hex string) that U1 made work, a table of the four color profiles and what each downgrades to, and the :environment message for apps that want to branch on the terminal themselves. Claude-Session: https://claude.ai/code/session_01CQeoheU7AxEmE5ZdATzg9q --- doc/api/messages.md | 36 ++++++++++++++++++++++++++++++++++++ doc/api/styling.md | 36 +++++++++++++++++++++++++++++++++++- 2 files changed, 71 insertions(+), 1 deletion(-) diff --git a/doc/api/messages.md b/doc/api/messages.md index d4f2737..6f18a2d 100644 --- a/doc/api/messages.md +++ b/doc/api/messages.md @@ -220,6 +220,42 @@ Requires `:focus-reporting true` in run options. [state nil])) ``` +## Environment Messages + +### environment + +```clojure +(msg/environment color-profile dark-background?) +``` + +Sent once at startup by `charm/run`, describing the terminal it detected: + +| key | value | +|---|---| +| `:color-profile` | `:ascii`, `:ansi`, `:ansi256`, or `:true-color` | +| `:dark-background?` | whether the terminal background is dark | + +Colors are already resolved against both for you — see +[Color Profiles](styling.md) and `charm/adaptive`. This message is for apps +that want to branch on the environment themselves, for example to pick an +ASCII fallback for box-drawing characters. + +### environment? + +```clojure +(msg/environment? msg) ; => boolean +``` + +```clojure +(defn update-fn [state msg] + (cond + (msg/environment? msg) + [(assoc state :dark? (:dark-background? msg)) nil] + + :else + [state nil])) +``` + ## Quit and Error Messages ### quit diff --git a/doc/api/styling.md b/doc/api/styling.md index ca6123b..a99db5e 100644 --- a/doc/api/styling.md +++ b/doc/api/styling.md @@ -166,6 +166,38 @@ Create an ANSI 256 color (0-255). - 16-231: 6x6x6 color cube - 232-255: Grayscale ramp +#### adaptive + +```clojure +(charm/adaptive light-color dark-color) +``` + +Create a color that resolves against the terminal's background: `light-color` +is used on light backgrounds, `dark-color` on dark ones. + +```clojure +(charm/style :fg (charm/adaptive (charm/hex "#333333") + (charm/hex "#dddddd"))) +``` + +The background is detected once at startup by `charm/run` (OSC 11), and +reported to your app as an [`:environment` message](messages.md). When the +terminal doesn't answer the query, charm assumes a dark background. + +### Color Profiles + +`charm/run` detects the terminal's color profile at startup and downgrades +colors to fit it, so the same styles work everywhere: + +| profile | detected when | colors become | +|---|---|---| +| `:true-color` | `COLORTERM` is `truecolor` or `24bit` | unchanged | +| `:ansi256` | `TERM` contains `256color` | nearest of the 256 palette | +| `:ansi` | any other `TERM` | nearest of the 16 basic colors | +| `:ascii` | `TERM` unset or `dumb` | dropped entirely | + +Outside of `charm/run` — in tests or scripts — colors are not downgraded. + ### Predefined Colors ```clojure @@ -195,7 +227,9 @@ charm/bright-white (charm/style :fg charm/red) (charm/style :fg (charm/rgb 255 128 0)) (charm/style :fg (charm/hex "#ff8000")) -(charm/style :fg 240) ; ANSI 256 shorthand +(charm/style :fg 240) ; ANSI 256 shorthand +(charm/style :fg :red) ; ANSI 16 name shorthand +(charm/style :fg "#ff8000") ; hex string shorthand ;; Background color (charm/style :bg charm/blue) From 574020c2cf09731af86a4c4ecd6e4451acffb920 Mon Sep 17 00:00:00 2001 From: Timo Kramer Date: Mon, 31 Aug 2026 20:16:44 +0200 Subject: [PATCH 4/6] docs: record phase 0 completion in plan MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Marks J1 and U1 done and records two premises the original review got wrong. First, charm never emitted raw true-color escapes for ANSI-only terminals to be spared from — AttributedString.toAnsi() without a terminal argument already collapses 24-bit color to the 256 palette, so making true color actually reach the terminal is separate Phase 4 work that will touch existing test assertions. Second, downgrade-color was not merely dormant but wrong, and wiring it in would have regressed TERM=xterm terminals had the nearest-color mapping not been fixed alongside. Also notes that the new dynamic vars are thread-local, which constrains the Phase 3/4 loop rewrite. Claude-Session: https://claude.ai/code/session_01CQeoheU7AxEmE5ZdATzg9q --- PLAN.md | 67 ++++++++++++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 57 insertions(+), 10 deletions(-) diff --git a/PLAN.md b/PLAN.md index 544921f..64d852a 100644 --- a/PLAN.md +++ b/PLAN.md @@ -14,10 +14,10 @@ location, and a concrete suggestion. ## Top priorities -1. **Background color detection / adaptive colors (J1)** — first thing to solve. - JLine 4 exposes `getDefaultBackgroundColor()` (OSC 10/11 query); wire it up - to offer light/dark-adaptive styling, and wire in the dormant - `detect-color-profile` / `downgrade-color` code while in there. +1. ~~**Background color detection / adaptive colors (J1)**~~ — **done**, together + with U1. JLine 4 exposes `getDefaultBackgroundColor()` (OSC 10/11 query); + wired up to offer light/dark-adaptive styling, with the dormant + `detect-color-profile` / `downgrade-color` code wired in alongside. 2. **Sanitize untrusted content before it reaches the terminal**, and make `strip-ansi` honest — OSC 52 injection writes attacker data into the user's system clipboard. This is a real vulnerability, not a hardening nit. (S1, S2) @@ -26,7 +26,7 @@ location, and a concrete suggestion. (P1, P2, P3) 4. **Fix `:fg 240`, `:strikethrough` and `"pgup"`** — three small bugs that make documented features, the library's own default styling, and Page Up/Down - silently do nothing. (U1, U2, U3) + silently do nothing. (U1 — **done**; U2, U3 outstanding) --- @@ -243,7 +243,7 @@ the `$`. Consider gating release on a tag rather than every push to `main`. ## 3. Usability -### U1 — `:fg 240` silently does nothing +### U1 — `:fg 240` silently does nothing — **done** `doc/api/styling.md:198` documents it as "ANSI 256 shorthand", but `src/charm/style/color.clj:118` dispatches on `(:type color)`, which is `nil` for @@ -258,6 +258,12 @@ default styling, and most of the sample code all render as plain text. **Suggestion:** coerce integers and keywords to colour maps in `style` / `with-fg` / `with-bg`, or throw on an unrecognised colour value. +**Resolved:** `charm.style.color/coerce-color` accepts integers (ANSI 256), +keywords (ANSI 16 names) and strings (hex), and throws an `ex-info` naming the +offending value otherwise. Coercion happens in `apply-color-fg` / `apply-color-bg` +rather than in each constructor, so every entry point — `style`, `with-fg`, +`with-bg`, `styled-str` — picks it up at once. + ### U2 — `:strikethrough` silently does nothing Documented in the `doc/api/styling.md:32` options table, used in @@ -369,7 +375,7 @@ Already absorbed: Mode 2027 grapheme clustering (ADR 007), the 4.3.1 ReDoS guards, 4.1's `Display.update()` optimizations, and the shift+tab CSI Z binding (done in-repo, `aa6a75e` — JLine still doesn't bind it). -### J1 — Background color detection / adaptive colors ← do this first +### J1 — Background color detection / adaptive colors — **done** JLine 4 can query the terminal's actual default colors (OSC 10/11): @@ -400,6 +406,45 @@ Related dormant code: `charm.style.color` already contains - Note the interaction with U1: fixing integer-color coercion first avoids building adaptive colors on top of a constructor that silently drops ints. +**Resolved:** + +- `charm.terminal/dark-background?` queries `getDefaultBackgroundColor`, computes + luminance via `dark-color?`, and defaults to dark when the query returns -1. +- Adaptive colors are `{:type :adaptive :light … :dark …}`, built with + `style/adaptive` and resolved at render time by `color/resolve-color`. +- `run` detects profile and background once at startup, binds + `*color-profile*` / `*dark-background?*` around the event loop, and sends an + `:environment` message so apps can branch themselves. +- Documented in `doc/api/styling.md` (adaptive colors, profile table) and + `doc/api/messages.md` (`:environment`). + +**Two premises above turned out to be wrong, and are worth recording:** + +1. *"ANSI-only terminals get raw true-color escapes."* They never did — and + neither does anything else. `AttributedString.toAnsi()` called without a + terminal argument collapses 24-bit colors to the 256 palette, so charm has + never emitted a `38;2;r;g;b` sequence: `(rgb 255 0 0)` renders as + `ESC[38;5;196m` even under the `:true-color` profile, and the existing tests + in `test/charm/style/color_test.clj` assert exactly that. Making true color + actually reach the terminal is a separate change — it needs the `toAnsi` + overload that takes a terminal — and it belongs with the Phase 4 render work, + since it changes those assertions. + +2. *`downgrade-color` was dormant but also wrong.* Its `:ansi` branch did + `(mod code 16)` on a 256-cube index, which is colorimetrically meaningless: + orange `(rgb 255 128 0)` came out **cyan**. Harmless while nothing called it, + but wiring it in would have turned every RGB color on a `TERM=xterm` terminal + into an effectively random one of 16 — worse than the previous + pass-everything-through behavior. Replaced with a nearest-neighbour match + against the existing `ansi-hex` table, plus a new `ansi256->rgb` covering the + cube and the grayscale ramp. Orange now lands on bright yellow, with a + regression test pinning it. + +**Caveat for the Phase 3/4 loop rewrite:** `*color-profile*` and +`*dark-background?*` are thread-local. The current single-threaded render path is +fine, but a `view` that renders parts on other threads would not see them — +`bound-fn` or explicit conveyance would be needed. + ### J2 — `KeyEvent` / `KeyParser`: revisit ADR 004 `org.jline.terminal.KeyParser/parse` (static, non-blocking — none of ADR 004's @@ -484,9 +529,11 @@ the previous handler** — exactly what S5 needs to restore Ctrl+C after exit. ## Suggested sequencing -**Phase 0 — background color detection (J1)** -First thing to solve. Do U1 (integer-color coercion) as its opening step so -adaptive colors aren't built on a constructor that silently drops ints. +**Phase 0 — background color detection (J1)** — **done** +U1 (integer-color coercion) landed as its opening step, so adaptive colors +aren't built on a constructor that silently drops ints. Also fixed +`downgrade-color`'s nearest-ANSI-16 mapping, which the wiring made reachable. +162 tests / 994 assertions passing, up from 152 / 940. **Phase 1 — correctness bugs, small and independent** U2, U3, U4, P3, P7, P9, and the `$` fix in S7. From 8604e8740026c99c8a1fe8dd6271842c29f29483 Mon Sep 17 00:00:00 2001 From: Timo Kramer Date: Fri, 11 Sep 2026 11:42:08 +0200 Subject: [PATCH 5/6] feat: pin the color environment via run options MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit :color-profile and :dark-background? override detection; nil, the default, keeps detecting. Useful for a terminal that misreports itself and for tests that need a fixed environment, and pinning :dark-background? also skips the OSC 11 query, so a terminal that never answers stops costing the probe timeout at every startup. An unknown profile throws an ex-info naming the value rather than falling through downgrade-color's case default and silently rendering undowngraded — the U1 lesson. The check lives in its own private fn so it is testable without a terminal. Also records in PLAN.md why the environment is ambient rather than part of the state: resolve-color needs it four frames below anything holding the state, because style/render returns a finished String with escapes already in it. The exit is styles-as-data resolved by the renderer, which is the same change real 38;2;r;g;b output needs, so the two should land together in Phase 4. Two things left alone until then, both invisible today: init runs before the binding opens (no component's init styles anything), and the vars are thread-local (core.async conveys the frame, so commands are fine). --- PLAN.md | 48 ++++++++++++++++++++++++++++++++----- doc/api/program.md | 8 +++++++ doc/api/styling.md | 9 +++++++ src/charm/program.clj | 40 ++++++++++++++++++++++++------- test/charm/program_test.clj | 16 ++++++++++++- 5 files changed, 105 insertions(+), 16 deletions(-) diff --git a/PLAN.md b/PLAN.md index 64d852a..e8eba14 100644 --- a/PLAN.md +++ b/PLAN.md @@ -415,8 +415,13 @@ Related dormant code: `charm.style.color` already contains - `run` detects profile and background once at startup, binds `*color-profile*` / `*dark-background?*` around the event loop, and sends an `:environment` message so apps can branch themselves. -- Documented in `doc/api/styling.md` (adaptive colors, profile table) and - `doc/api/messages.md` (`:environment`). +- `:color-profile` and `:dark-background?` are `run` options; `nil` (the + default) detects. Pinning `:dark-background?` also skips the OSC 11 query and + its probe timeout. An unknown profile throws rather than silently passing + through `downgrade-color`'s `case` default. +- Documented in `doc/api/styling.md` (adaptive colors, profile table, pinning), + `doc/api/program.md` (both options) and `doc/api/messages.md` + (`:environment`). **Two premises above turned out to be wrong, and are worth recording:** @@ -440,10 +445,41 @@ Related dormant code: `charm.style.color` already contains cube and the grayscale ramp. Orange now lands on bright yellow, with a regression test pinning it. -**Caveat for the Phase 3/4 loop rewrite:** `*color-profile*` and -`*dark-background?*` are thread-local. The current single-threaded render path is -fine, but a `view` that renders parts on other threads would not see them — -`bound-fn` or explicit conveyance would be needed. +**The dynamic vars are the interim mechanism, and Phase 4 retires them.** + +`resolve-color` needs the profile four frames below anything that was handed the +state: + +``` +(view state) → (list-view (:list state)) → (style/render item-style title) → resolve-color +``` + +`style/render` returns a finished `String` with escapes already in it, so the +"is this terminal true-color?" decision is made down there, in a function whose +signature is `(style & strings)`. Dynamic vars are the standard answer to that, +and the alternative — threading an environment argument through every +`xxx-view` — is viral across the whole component API. + +The exit is to stop resolving eagerly: `style/render` returns styled *data*, and +the renderer, which already holds the terminal, resolves colors when it emits. +No ambient value, no threading. That is the same change premise 1 above needs — +real `38;2;r;g;b` output requires the `toAnsi` overload that takes a `Terminal`, +which only the renderer has — so the two should land together. It breaks user +code that concatenates styled strings (`(str (style/render …) "more")`), which +is why it belongs with the Phase 4 render work and not here. + +Two smaller things deliberately left alone until then, both invisible today: + +1. `init` is called in `run`'s `let`, before the `binding` opens, so styling + done at init time resolves against the root values. No component's `init` + styles anything, so only user code that pre-renders styled strings in its + own `init` can reach it. +2. The vars are thread-local. core.async's `go` conveys the binding frame, so + commands are fine; a `view` that renders parts on app-owned threads or + futures is not. + +Both disappear with the renderer-owned environment, so fixing them now would be +wasted work. ### J2 — `KeyEvent` / `KeyParser`: revisit ADR 004 diff --git a/doc/api/program.md b/doc/api/program.md index 7b31350..8b732db 100644 --- a/doc/api/program.md +++ b/doc/api/program.md @@ -24,6 +24,14 @@ Run a TUI program with the Elm Architecture pattern. | `:focus-reporting` | boolean | `false` | Report focus in/out events | | `:fps` | int | `60` | Frames per second | | `:hide-cursor` | boolean | `true` | Hide terminal cursor | +| `:color-profile` | keyword | `nil` | `:ascii`, `:ansi`, `:ansi256` or `:true-color`; `nil` detects from `TERM`/`COLORTERM` | +| `:dark-background?` | boolean | `nil` | `nil` queries the terminal | + +The last two pin the [color environment](styling.md#color-profiles) instead of +detecting it, for a terminal that misreports itself or a test that needs a fixed +environment. Pinning `:dark-background?` also skips the OSC 11 background query, +and with it the probe timeout that a terminal which never answers costs at every +startup. **Example:** diff --git a/doc/api/styling.md b/doc/api/styling.md index a99db5e..0eabdb6 100644 --- a/doc/api/styling.md +++ b/doc/api/styling.md @@ -196,6 +196,15 @@ colors to fit it, so the same styles work everywhere: | `:ansi` | any other `TERM` | nearest of the 16 basic colors | | `:ascii` | `TERM` unset or `dumb` | dropped entirely | +Both halves of the environment can be pinned, for a terminal that misreports +itself or a test that needs a fixed environment: + +```clojure +(charm/run {:init ... :update ... :view ... + :color-profile :ansi256 + :dark-background? false}) +``` + Outside of `charm/run` — in tests or scripts — colors are not downgraded. ### Predefined Colors diff --git a/src/charm/program.clj b/src/charm/program.clj index 65da86f..ce395bb 100644 --- a/src/charm/program.clj +++ b/src/charm/program.clj @@ -63,7 +63,20 @@ :mouse nil ; nil, :normal, :cell, or :all :focus-reporting false :fps 60 - :hide-cursor true}) + :hide-cursor true + :color-profile nil ; nil - detect from the environment + :dark-background? nil}) ; nil - query the terminal + +(defn- resolve-color-profile + "Use a caller-supplied color profile, or detect one from the environment. + Throws an ex-info naming the value for an unknown profile." + [profile] + (cond + (nil? profile) (color/detect-color-profile) + (contains? color/color-profiles profile) profile + :else (throw (ex-info (str "Unknown color profile: " (pr-str profile)) + {:color-profile profile + :valid color/color-profiles})))) ;; --------------------------------------------------------------------------- ;; Event Loop @@ -181,14 +194,21 @@ :fps - Frames per second (default: 60) :hide-cursor - Hide cursor (default: true) :running? - Atom to control the event loop externally (default: internal atom) + :color-profile - :ascii, :ansi, :ansi256 or :true-color, overriding + detection (default: nil, detect from $TERM/$COLORTERM) + :dark-background? - Override the terminal background query (default: nil, + query the terminal) The init function should return [initial-state cmd] or just initial-state. The update function receives (state msg) and returns [new-state cmd]. Commands are optional and can be nil. - At startup an :environment message with the detected :color-profile and - :dark-background? is sent, and both are bound for the duration of the - program so colors resolve against the actual terminal." + At startup an :environment message with the :color-profile and + :dark-background? in effect is sent, and both are bound for the duration of + the program so colors resolve against the actual terminal. Pinning either + option skips its detection; pinning :dark-background? also skips the + terminal background query, and with it the probe timeout that terminals + which never answer would otherwise cost at every startup." [{:keys [init update view running?] :as opts}] (let [opts (merge (default-opts) opts) {:keys [alt-screen mouse focus-reporting fps hide-cursor]} opts @@ -197,11 +217,13 @@ terminal (term/create-terminal) ^Attributes original-attrs (term/enter-raw-mode terminal) - ;; Detect the environment for adaptive styling. The background query - ;; reads the terminal's OSC response, so it must happen before the - ;; input loop starts consuming input. - color-profile (color/detect-color-profile) - dark-background? (term/dark-background? terminal) + ;; Detect the environment for adaptive styling, unless the caller + ;; pinned it. The background query reads the terminal's OSC response, + ;; so it must happen before the input loop starts consuming input. + color-profile (resolve-color-profile (:color-profile opts)) + dark-background? (if (some? (:dark-background? opts)) + (:dark-background? opts) + (term/dark-background? terminal)) ;; Create renderer renderer (render/create-renderer terminal diff --git a/test/charm/program_test.clj b/test/charm/program_test.clj index 04a6a90..3e885c3 100644 --- a/test/charm/program_test.clj +++ b/test/charm/program_test.clj @@ -1,7 +1,8 @@ (ns charm.program-test (:require [clojure.test :refer [deftest is testing]] [charm.program :as p] - [charm.message :as msg])) + [charm.message :as msg] + [charm.style.color :as color])) ;; Note: Full program tests require terminal interaction. ;; These tests focus on command and message helper functions. @@ -48,3 +49,16 @@ (is (= :window-size (:type m))) (is (= 80 (:width m))) (is (= 24 (:height m)))))) + +(deftest resolve-color-profile-test + (let [resolve-profile #'p/resolve-color-profile] + (testing "a pinned profile is used as given" + (doseq [profile color/color-profiles] + (is (= profile (resolve-profile profile))))) + + (testing "nil detects from the environment" + (is (contains? color/color-profiles (resolve-profile nil)))) + + (testing "an unknown profile throws, naming the value" + (let [e (is (thrown? clojure.lang.ExceptionInfo (resolve-profile :256color)))] + (is (= :256color (:color-profile (ex-data e)))))))) From 7567f71aa6f6b1eaaddd8054b7529ced6f6dab68 Mon Sep 17 00:00:00 2001 From: Timo Kramer Date: Fri, 11 Sep 2026 13:25:15 +0200 Subject: [PATCH 6/6] fix: delegate nearest-color matching to JLine MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit org.jline.utils.Colors already did every conversion color.clj had hand-rolled, and did it better — matching in CIE Lab over the full 256-entry table via Colors$Distance, where ours measured Euclidean distance in RGB space over a partial one. rgb->ansi256 was the one that mattered: strictly worse than JLine in 73 of 125 probes, because it searched only the 16-231 cube and the 232-255 ramp — so an exact match among the first 16 entries was unreachable, and olive (128,128,0), which *is* entry 3, came back as 142 — and because it quantised as (v*5/255) as if the cube levels were evenly spaced when they are 0/95/135/175/215/255, rounding 128 up to 175 instead of down to 95. a80d88c made that reachable, and in doing so took the job away from an implementation that was already correct: before the wiring, apply-color-fg handed raw RGB to .foreground and JLine's toAnsi chose 208 for orange; after it, our matcher chose 214 and JLine only printed the index we had picked. downgrade-color now calls Colors/roundColor and Colors/roundRgbColor, and rgb->ansi256, rgb->ansi16, ansi256->rgb, ansi-hex and cube-levels are deleted — 22 insertions against 103 deletions. The :ansi profile keeps what the wiring genuinely gained: basic SGR (ESC[91m) rather than a 256-color code, which no-terminal toAnsi will not emit. coerce-color and adaptive stay ours; JLine has no equivalent for either. --- PLAN.md | 40 ++++++++++++++--- doc/api/styling.md | 3 ++ src/charm/style/color.clj | 79 +++++---------------------------- test/charm/style/color_test.clj | 46 +++++-------------- 4 files changed, 60 insertions(+), 108 deletions(-) diff --git a/PLAN.md b/PLAN.md index e8eba14..2c57002 100644 --- a/PLAN.md +++ b/PLAN.md @@ -423,7 +423,7 @@ Related dormant code: `charm.style.color` already contains `doc/api/program.md` (both options) and `doc/api/messages.md` (`:environment`). -**Two premises above turned out to be wrong, and are worth recording:** +**Three premises above turned out to be wrong, and are worth recording:** 1. *"ANSI-only terminals get raw true-color escapes."* They never did — and neither does anything else. `AttributedString.toAnsi()` called without a @@ -440,10 +440,40 @@ Related dormant code: `charm.style.color` already contains orange `(rgb 255 128 0)` came out **cyan**. Harmless while nothing called it, but wiring it in would have turned every RGB color on a `TERM=xterm` terminal into an effectively random one of 16 — worse than the previous - pass-everything-through behavior. Replaced with a nearest-neighbour match - against the existing `ansi-hex` table, plus a new `ansi256->rgb` covering the - cube and the grayscale ramp. Orange now lands on bright yellow, with a - regression test pinning it. + pass-everything-through behavior. + + It was first replaced with a hand-rolled nearest-neighbour match, which was + the wrong instinct — see premise 3. + +3. *All of the color conversion was JLine's already, and our copies were worse.* + `org.jline.utils.Colors` offers `rgbColor(int)` (palette index → RGB), + `roundColor(idx, max)` and `roundRgbColor(r, g, b, max)`, matching in CIE Lab + via `Colors$Distance`, over `DEFAULT_COLORS_256`. Measured against it: + + - `ansi256->rgb` was identical to `Colors/rgbColor` on every probe — pure + duplication. + - `rgb->ansi16` agreed on 5 of 6 probes; ours measured distance in RGB space. + - `rgb->ansi256` was **strictly worse in 73 of 125 probes**, for two reasons: + it only searched the 16–231 cube and the 232–255 ramp, so exact matches in + the first 16 entries were missed (olive `(128,128,0)` *is* entry 3; we + returned 142), and it quantised as `(v*5/255)` as if the cube levels were + evenly spaced when they are 0/95/135/175/215/255, so 128 became 175 rather + than 95. + + Worse, wiring `downgrade-color` into `apply-color-fg` *took this job away + from JLine*, which had been doing it correctly all along: before the wiring, + `apply-color-fg` passed raw RGB to `.foreground` and JLine's `toAnsi` chose + 208 for orange; after it, our matcher chose 214 and JLine merely printed it. + + `downgrade-color` now delegates to `Colors`, and the three conversion + functions plus `ansi-hex` and `cube-levels` are deleted. The `:ansi` profile + keeps the one thing the wiring genuinely gained — basic SGR codes + (`ESC[91m`) instead of a 256-color code, which no-terminal `toAnsi` will not + emit. + + Still ours, because JLine has no equivalent: `coerce-color` (ints, keywords, + hex) and `adaptive`. Also still ours, but duplicating terminfo `max_colors`: + `detect-color-profile`. **The dynamic vars are the interim mechanism, and Phase 4 retires them.** diff --git a/doc/api/styling.md b/doc/api/styling.md index 0eabdb6..2535fe6 100644 --- a/doc/api/styling.md +++ b/doc/api/styling.md @@ -194,6 +194,9 @@ colors to fit it, so the same styles work everywhere: | `:true-color` | `COLORTERM` is `truecolor` or `24bit` | unchanged | | `:ansi256` | `TERM` contains `256color` | nearest of the 256 palette | | `:ansi` | any other `TERM` | nearest of the 16 basic colors | + +Nearest is JLine's `org.jline.utils.Colors`, which measures distance in CIE Lab +rather than RGB space. | `:ascii` | `TERM` unset or `dumb` | dropped entirely | Both halves of the environment can be pinned, for a terminal that misreports diff --git a/src/charm/style/color.clj b/src/charm/style/color.clj index 57ab409..aad904b 100644 --- a/src/charm/style/color.clj +++ b/src/charm/style/color.clj @@ -9,7 +9,7 @@ (:require [clojure.string :as str]) (:import - [org.jline.utils AttributedString AttributedStyle])) + [org.jline.utils AttributedString AttributedStyle Colors])) ;; --------------------------------------------------------------------------- ;; Color Profile Detection @@ -75,11 +75,6 @@ :bright-cyan 14 :bright-white 15}) -(def ansi-hex - "ANSI 16 colors as hex values for conversion." - ["#000000" "#800000" "#008000" "#808000" "#000080" "#800080" "#008080" "#c0c0c0" - "#808080" "#ff0000" "#00ff00" "#ffff00" "#0000ff" "#ff00ff" "#00ffff" "#ffffff"]) - ;; --------------------------------------------------------------------------- ;; Color Construction ;; --------------------------------------------------------------------------- @@ -149,73 +144,23 @@ ;; Color Conversion ;; --------------------------------------------------------------------------- -(defn rgb->ansi256 - "Convert RGB to closest ANSI 256 color." - [{:keys [r g b]}] - (let [;; Check grayscale first (232-255) - gray-start 232 - gray-levels 24 - gray? (and (< (Math/abs (long (- r g))) 10) - (< (Math/abs (long (- g b))) 10) - (< (Math/abs (long (- r b))) 10)) - - ;; Convert to 6x6x6 color cube (16-231) - cube-value (fn [v] (int (Math/round (/ (* v 5.0) 255.0)))) - cube-r (cube-value r) - cube-g (cube-value g) - cube-b (cube-value b)] - (if gray? - ;; Map to grayscale ramp - (let [gray-idx (int (Math/round (* (/ (+ r g b) 3.0 255.0) (dec gray-levels))))] - (ansi256 (+ gray-start gray-idx))) - ;; Map to 6x6x6 color cube - (ansi256 (+ 16 (* 36 cube-r) (* 6 cube-g) cube-b))))) - -(def ^:private cube-levels - "The six channel levels of the ANSI 256 color cube." - [0 95 135 175 215 255]) - -(defn ansi256->rgb - "Convert an ANSI 256 color to its RGB value." - [{:keys [code]}] - (cond - ;; 0-15: standard colors, from the hex table - (< code 16) (hex (nth ansi-hex code)) - - ;; 232-255: grayscale ramp - (>= code 232) (let [v (+ 8 (* 10 (- code 232)))] - (rgb v v v)) - - ;; 16-231: 6x6x6 color cube - :else (let [idx (- code 16)] - (rgb (nth cube-levels (quot idx 36)) - (nth cube-levels (mod (quot idx 6) 6)) - (nth cube-levels (mod idx 6)))))) - -(defn rgb->ansi16 - "Convert RGB to the nearest ANSI 16 color by distance in RGB space." - [{:keys [r g b]}] - (let [distance (fn [hex-str] - (let [{hr :r hg :g hb :b} (hex hex-str)] - (+ (* (- r hr) (- r hr)) - (* (- g hg) (- g hg)) - (* (- b hb) (- b hb)))))] - (ansi (first (apply min-key second - (map-indexed (fn [i h] [i (distance h)]) ansi-hex)))))) - (defn downgrade-color - "Downgrade a color to fit a color profile." - [color profile] + "Downgrade a color to fit a color profile. + + Nearest-color matching is JLine's (`org.jline.utils.Colors`), which searches + the whole 256-color table — the 16 basic entries included — and measures + distance in CIE Lab rather than RGB space." + [{:keys [type code r g b] :as color} profile] (case profile :ascii (no-color) - :ansi (case (:type color) + :ansi (case type :ansi color - :ansi256 (rgb->ansi16 (ansi256->rgb color)) - :rgb (rgb->ansi16 color) + :ansi256 (ansi (Colors/roundColor (int code) 16)) + :rgb (ansi (Colors/roundRgbColor (int r) (int g) (int b) 16)) color) - :ansi256 (case (:type color) + :ansi256 (case type (:ansi :ansi256) color - :rgb (rgb->ansi256 color) + :rgb (ansi256 (Colors/roundRgbColor (int r) (int g) (int b) 256)) color) :true-color color color)) diff --git a/test/charm/style/color_test.clj b/test/charm/style/color_test.clj index 28da4c5..792263b 100644 --- a/test/charm/style/color_test.clj +++ b/test/charm/style/color_test.clj @@ -111,42 +111,6 @@ (binding [c/*color-profile* :ascii] (is (= "x" (c/styled-str "x" :fg (c/rgb 255 0 0))))))) -(deftest rgb->ansi256-test - (testing "converts RGB to ANSI 256" - ;; Pure red should map to color cube - (let [result (c/rgb->ansi256 {:r 255 :g 0 :b 0})] - (is (= :ansi256 (:type result)))) - ;; Gray should map to grayscale ramp - (let [result (c/rgb->ansi256 {:r 128 :g 128 :b 128})] - (is (= :ansi256 (:type result))) - (is (>= (:code result) 232))))) ; Grayscale starts at 232 - -(deftest ansi256->rgb-test - (testing "converts standard colors" - (is (= {:type :rgb :r 0 :g 0 :b 0} (c/ansi256->rgb (c/ansi256 0)))) - (is (= {:type :rgb :r 255 :g 0 :b 0} (c/ansi256->rgb (c/ansi256 9))))) - - (testing "converts the 6x6x6 cube" - ;; 196 is the cube's pure red corner - (is (= {:type :rgb :r 255 :g 0 :b 0} (c/ansi256->rgb (c/ansi256 196)))) - (is (= {:type :rgb :r 0 :g 0 :b 0} (c/ansi256->rgb (c/ansi256 16))))) - - (testing "converts the grayscale ramp" - (is (= {:type :rgb :r 8 :g 8 :b 8} (c/ansi256->rgb (c/ansi256 232)))) - (is (= {:type :rgb :r 238 :g 238 :b 238} (c/ansi256->rgb (c/ansi256 255)))))) - -(deftest rgb->ansi16-test - (testing "maps colors to the nearest ANSI 16 entry" - (is (= c/black (c/rgb->ansi16 (c/rgb 0 0 0)))) - (is (= c/bright-red (c/rgb->ansi16 (c/rgb 255 0 0)))) - (is (= c/bright-green (c/rgb->ansi16 (c/rgb 0 255 0)))) - (is (= c/bright-white (c/rgb->ansi16 (c/rgb 255 255 255))))) - - (testing "orange lands on a warm color, not an arbitrary one" - ;; Regression: (mod code 16) used to turn orange into cyan - (is (contains? #{c/yellow c/red c/bright-red c/bright-yellow} - (c/rgb->ansi16 (c/rgb 255 128 0)))))) - (deftest downgrade-color-test (testing ":ansi downgrade preserves hue" (is (contains? #{c/yellow c/red c/bright-red c/bright-yellow} @@ -161,6 +125,16 @@ (is (= :ansi256 (:type (c/downgrade-color (c/rgb 255 128 0) :ansi256)))) (is (= c/red (c/downgrade-color c/red :ansi256)))) + (testing ":ansi256 searches the basic 16 entries too" + ;; Olive is exactly entry 3. The old cube-only search returned 142. + (is (= (c/ansi256 3) (c/downgrade-color (c/rgb 128 128 0) :ansi256))) + (is (= (c/ansi256 5) (c/downgrade-color (c/rgb 128 0 128) :ansi256)))) + + (testing ":ansi256 picks the nearest cube entry, not an evenly-spaced guess" + ;; The cube levels are 0/95/135/175/215/255, not multiples of 51: the old + ;; (v*5/255) quantisation rounded 128 up to 175 and returned 214. + (is (= (c/ansi256 208) (c/downgrade-color (c/rgb 255 128 0) :ansi256)))) + (testing ":ascii drops all color" (is (= {:type :none} (c/downgrade-color (c/rgb 255 128 0) :ascii))))