Commit 9996984
authored
feat(tui): TUI enhancements, security-audit remediation, and multi-instance hardening (#103)
* feat(config): disable automatic turn recaps by default
Turn recaps now default off; the agent only recaps when asked. Add a
direct toggle — /config recaps on|off (also /settings recaps ...) —
that persists to the config file and reloads the shell. The interactive
/settings panel keeps its existing turn-recaps item.
Part of the Codex TUI adoption backlog (item 0.1).
* feat(tui): adaptive theme foundation — bg probe, color depth, blending
Port the Codex terminal-adaptation layer (codex-rs/tui) to Python:
- ui/color_utils.py: hex parse/format, linear RGB blend, BT.601 luma.
- ui/terminal_background.py: OSC 11 background probe (100ms timeout,
per-process cache, PYTHINKER_NO_BG_PROBE opt-out) and theme = "auto"
resolution with a dark fallback. /theme and the settings selector
accept the new value.
- terminal_capabilities.color_depth(): three usable color tiers
(truecolor/256/16 + none) honoring FORCE_COLOR levels and the
Windows Terminal WT_SESSION truecolor promotion.
- get_diff_colors(): 16-color terminals now get plain green/red
foreground diff styles instead of quantized hex background tints.
/theme now compares against the persisted setting rather than the
resolved active theme so users can pin dark/light while on auto.
Backlog items 1.1-1.4.
* feat(tui): renderer guards + md-fence table unwrapping
Adopt the Codex renderer-safety behaviors:
- Syntax-highlight size guard: code blocks beyond 512 KiB or 10k lines
render as plain text with a 'highlighting skipped (N lines)' notice
instead of paying an unbounded Pygments cost.
- Large diff guard: expanded diffs are capped at 400 rendered lines
(head + tail with an explicit omitted-line count) so one huge edit
cannot freeze or flood the terminal.
- Generic tool output switches from head-only to head-tail truncation,
keeping the start (identifies the result) and the end (the actionable
part) with an omitted-line notice.
- Fence unwrapping for tables: ```md/```markdown fences whose body
contains a header+delimiter table pair now render as markdown instead
of opaque code. Conservative Codex heuristics: other languages,
untagged fences, md fences without tables, and unclosed fences pass
through unchanged.
Backlog items 2.6, 3.1, 6.2, 7.5.
* docs(tasks): Codex TUI adoption gap analysis + Phase 1 plan
* feat(tui): reference-CLI design polish + probe hardening
User-directed design wave on top of the Codex adoption Phase 1:
- Transcript marker is now ⏺ (U+23FA) on macOS/Linux — Windows keeps the
text circle, ASCII mode keeps the star. Tool/assistant rows blink the
marker while running (reduced-motion pins it static) and settle to the
solid green marker when finished; thinking rows carry the marker too.
- Activity coral muted to a clay ramp (#C68D7E/#D8AC9E/#E9CDC2 dark,
#B26A52/#9E563E/#82412D light). Shimmer simplified from wave+splash to a
calm bidirectional sweep with settle beats; truecolor terminals get a
continuous cosine-blended sheen via ui.color_utils.blend, lower tiers
keep the discrete ramp.
- Activity/todo headers share one metadata design:
'Verb… (12s, ↓ 2.4k tokens, 45 t/s)' — parenthesized, comma-separated,
with a live tokens/sec readout on the working indicator and the pinned
todo header (sliding-window rate over the turn's context tokens).
- Thinking-effort frame colors form a cold→hot gradient ending on dark red
for xhigh (slate→blue→teal→amber→orange→red).
- Pinned todos: the active task title+box are coral; concurrent in-progress
rows read light grey so the running task stays unmistakable.
- Diff word-level highlights drop reverse-video for the theme's add/del
highlight backgrounds (GitHub-style emphasis, no glare).
- Turn recap is padded to the card inset instead of spanning edge-to-edge.
Hardening (findings from the in-app review validated and fixed):
- Terminal probe: catch select's ValueError (fd >= FD_SETSIZE), suppress
tcsetattr restore failures, cap the OSC reply buffer at 4 KiB, and
serialize the probe cache behind a lock.
- Generic tool-output head/tail truncation halves the char budget per side
so combined output can never exceed the limit.
- /settings arg parsing splits the mode string once.
- Added ~~~md tilde-fence unwrap coverage.
* feat(tui): reference-CLI layout, palette, and chrome refinements
Second design wave driven by side-by-side comparison with the reference
transcripts:
- Tool headers use the parenthesized single-line form — ⏺ Bash(cmd…),
⏺ Update(path) — ellipsizing at the terminal edge instead of wrapping.
- Result gutters no longer pad rows with trailing spaces to the terminal
edge (copy-clean, ragged-right like the reference).
- Todo list matches the reference: coral ■ box with bold default-color
(white) title for in-progress rows, green ✓ with struck muted titles
for done rows; coral stays on the top activity line.
- Diff palette set to the specified values: row tints #052e05/#3a0808,
sign/line-number accents #81C784/#E57373, content in the terminal's
default (white) over the tint. Also fixes a span-layering bug where
the row restyle buried word-level highlights.
- Dark text hierarchy: primary output #D4D4D4; UI chrome (gutters, line
numbers, expand hints, toolbar metadata) #6F6F6F.
- Thinking-effort input bars dim their gradient color 30% toward the
background pole so the frame hints without shouting.
- The 'N background agents' line is gone everywhere — the bottom toolbar
owns that count; the verb spinner + todos remain. Background subagent
status rows hang-indent under their label.
- Welcome banner gains a bold /init tip when the repo has no AGENTS.md
or CLAUDE.md.
* fix(tui): unify the two todo-list renderers into one design
The pinned todo list is drawn by two code paths — the live view during a
foreground turn and the prompt-side background status between turns —
and they had drifted apart (◼/◻/✔ vs ■/□/✓ glyphs, hex off-white vs
terminal-default bold titles, bright pending rows, no strikethrough).
That made running tasks appear to change style mid-session.
Both paths now render identically: coral ■ with a bold default-color
(white) title for running rows, muted □ pending, green ✓ with struck
muted titles when done.
* fix(tui): white running-task titles, bg-status metadata, diff palette consistency
Three fixes from live-session screenshots:
- Pinned todo rows no longer carry a muted base style: the bold
running-task title sets no color of its own, so the muted base bled
through and rendered it grey instead of the terminal-default white.
The base style is gone; prefix/icon/title each carry their own style.
- The background working status line ('Ebbing…') now shows the same
'(elapsed, ↓ Nk tokens, N t/s)' metadata as the live working
indicator: elapsed since background work appeared and a 1.5s
sliding-window token rate over the status snapshot, dropped first on
narrow terminals and reset when background work drains.
- Word-level diff highlights are gated on line similarity (ratio >= 0.5):
heavy single-line rewrites previously flooded the whole row with the
brighter highlight tint, reading as a different palette from plain
added/removed rows. Such rewrites now render as plain rows.
* feat(tui): elapsed/tokens/t-s metadata on the background status line
Completes the c8c0f05 commit: the prompt.py change was clobbered from
the index by a concurrent session's git operation before that commit
landed, leaving the renderer half-committed.
The background working status line ('Ebbing…') now carries the same
'(elapsed, ↓ Nk tokens, N t/s)' metadata as the live working indicator:
elapsed since background work first appeared and a 1.5s sliding-window
token rate over the status snapshot's context tokens. The metadata is
dropped first on narrow terminals and both trackers reset when
background work drains.
* fix(tui): transcript-row bullets use the record marker, not the list dot
Notification, progress-note, question-answered, and suggestion rows in
the transcript still rendered with BulletColumns' default list bullet.
They now carry the record marker in the block's accent color (severity
for notifications, green for progress/answers, accent for suggestions).
Genuine list rows — /help sections and nested subagent detail lines —
deliberately keep the list dot.
* feat(agents): structured prompt overhaul + subagent/background hardening
Refactor all default agent YAML prompts with explicit Mission / Hard
Constraints / Workflow / Output Contract sections for consistency and
clarity. Update system.md overlay accordingly.
Harden background manager, subagent runner, and agent tool to align
with the Codex TUI adoption (Phase 1) gap map: stale-record
reconciliation, resume contract enforcement, and interactive-visualizer
fixes. Update all affected unit and e2e tests.
* docs(changelog): add Unreleased entry for TUI enhancements
* fix: remediate 65 security and correctness audit findings
Implements the remediation plan for the validated audit findings
(1 Critical, 15 High, 29 Medium, 20 Low) across five phases, each fix
behind a TDD test and gated by `make check` plus a security review.
Phase 0 — shell permission classifier: close the read-only / auto-mode
bypass cluster (interior &/|& separators, casefolded base commands,
wrapper value-options, find/xargs/awk payloads, glued output
redirection, unsafe `git -c`, uv sub-namespaces and `uv run`
option-prefix bypass).
Phase 1 — confinement, egress, telemetry: symlink-resolve file
read/write/edit and grep before workspace/sensitive checks; fail-closed
SSRF with a connection-pinned resolver; bounded shell wait; Sentry
path/home redaction; invisible-char and case-folded sensitive-file
handling; sensitive-import gate; untrusted-output wrapping; subagent-id
path validation.
Phase 2 — tool dispatch, lifecycle, context integrity: MCP-vs-builtin
tool collisions; run_soul task-leak cleanup; restore-id/path traversal
guards; compaction rollback; mid-tool-cancel turn balance; cyclic-extend
detection.
Phase 3 — wire server, auth, web-server: wire read-loop hardening; OAuth
refresh / device-id / 403 handling; provider base_url validation; replay
watermark; session-leak cleanup; ZIP-import validation.
Phase 4 — UI/usage/CLI: ANSI sanitization at render boundaries (incl.
generic tool arg-key names); usage-meter consumed-vs-remaining;
reset-window loop guard; RunAgents approval fingerprint over child
prompts; owner-only MCP config; live Typer help; /restore traversal and
error handling; bounded approval-request store.
Review-found gaps were fixed with regression tests: uv-run option bypass,
grep symlink escape, SSH-key import gate, ANSI arg-key injection, and the
MCP-config / share-dir permission race. Also hardens auth JSON parsing.
* fix(tui): address CodeRabbit review findings
- app.py: detect AGENTS.md/CLAUDE.md in the session work_dir (awaiting the
async HostPath.exists) instead of process cwd, so the /init tip is correct
when cwd differs from the session path.
- color_utils.py: replace EN DASH with ASCII hyphen in the luma docstring.
- slash.py: clarify the recaps config-flag guidance (--config vs --config-file).
- terminal_background.py: debug-log swallowed OSC11 probe failures instead of
silently degrading, per the exception-handling rule.
- tests: annotate the autouse fixture return type; tighten the thinking-status
metadata assertion to validate the token-count block, not just any paren.
* feat(llm): controllable reasoning effort for Qwen on OpenCode Go
Qwen3.x/3.7 are hybrid thinking models: reasoning is toggleable per request
via the standard Anthropic thinking block, which the OpenCode Go
@ai-sdk/anthropic route (and Alibaba Model Studio's Anthropic-compatible
endpoint) accepts as {"type": "enabled", "budget_tokens": N} /
{"type": "disabled"}.
Previously OpenCode Go Qwen models got no thinking capability, so their
effort was uncontrollable — while the Alibaba plan already exposed it. Give
the Qwen family the controllable 'thinking' capability so create_llm routes
the selected effort through with_thinking, which (for these non-Claude
models) emits the budget-based payload and clamps xhigh/max -> high,
minimal -> low. GLM/MiniMax remain always_thinking (effort can't be turned
off); Qwen can.
Adds a clamp regression case pinning the budget-safe mapping for qwen3.7-max
so the budgets[...] lookup can never KeyError.
* feat(tui): static-grey input border with a top-right effort label
Stop recoloring the whole input border by thinking effort. The border is now
one static frame grey at every level; the effort signal moves to a small
label flushed right on the input's top border — a level-colored dot
(off->max: slate->blue->teal->amber->orange->red) plus the muted level word.
The dot carries the cold->hot color at full strength (it's a single glyph),
while the word uses a muted class so it never competes with the typed text.
The label is hidden for native-thinking models (always_thinking with no user
dial) and non-thinking models, and the rule auto-shortens by the measured
label width so the top line never wraps.
_prompt_separator_style no longer borrows thinking_frame_style, so all input
separators (top border and footer) render the same static frame grey.
* feat(tui): single top-right effort label; Qwen treated as native-thinking
Two related thinking-effort changes:
1. Remove the duplicated effort indicator from the footer (the
'agent <model> • <effort>' / 'native reasoning' text under the input).
Effort now shows in exactly one place — the top-border label added
earlier. The footer mode line is just 'agent <model>', degrading to the
bare mode on narrow terminals. Drops the now-orphaned
_thinking_footer_label helper.
2. Mark all Qwen models (qwen3.7-max/plus, qwen3.6-plus/flash, qwen3 coder
plus/flash on Alibaba; qwen* on OpenCode Go) as always_thinking instead
of the controllable 'thinking' capability. Qwen now matches GLM/MiniMax:
reasoning is native and always on, with no user effort dial and no
top-border effort label. Reasoning still flows over the Anthropic
thinking block both Anthropic-compatible routes accept.
* fix: address CodeRabbit review findings on the security-remediation diff
Critical permission-classifier bypasses:
- sudo long value-options (`sudo --user alice rm -rf /`) were not consumed, so
the wrapped destructive payload classified as the option's value. Add the long
forms to _SUDO_VALUE_OPTS.
- uv global options before `run` (`uv --directory repo run rm -rf /`) hid the
subcommand; the global flag's value was mistaken for it. Add
_uv_strip_global_opts and apply it in both mutation and destructive paths.
Major:
- file_restore: treat missing (None) or malformed-base64 content as a corrupt
restore point instead of silently writing an empty/garbage file.
- web/fetch: _ip_is_blocked now fails closed (blocks) on an unparseable address.
- scratchpad: stop unlinking the advisory lock file (split-inode race); keep it
persistent and add *.scratchpad.lock to the written .gitignore patterns.
- soul shutdown: don't re-await an already-finished task in the cleanup loop —
retrieve its exception without re-raising so the rest of shutdown still runs.
- pythinkersoul: on mid-tool interruption, keep the real results of calls that
already completed (captured via on_tool_result) and only synthesize the
interruption marker for still-pending calls.
Minor / nitpick:
- /import arg parsing now uses shlex via a shared parse_import_args helper
(soul/slash + ui/shell/export_import), preserving quoted paths.
- web/runner: offload the blocking wire-file stat with asyncio.to_thread;
document the intentional broad except at the per-message dispatch boundary.
- cli/vis: rename unused callback param to _ctx.
- tests: regression cases for both bypasses; strengthened import token-count
assertions; lock-file persistence test; minor annotations.
* fix: deep-scan remediation + multi-instance robustness hardening
Security/correctness (multi-agent review of the branch; every finding
verified against the live code before fixing):
- permission gate: classify awk shell-outs (print | "cmd", getline) as
mutating AND destructive; recognize xargs -L payloads
- Glob: resolve symlinks before the workspace boundary check
- transcript: sanitize ANSI in progress-note titles
- Grep: control-char field separators end path/lineno misparsing
(utf-8-codec.py) and make sensitive-file attribution exact
- StrReplaceFile: CRLF-translate LF-joined multi-line old strings
- /import: parse --force from the raw string; paths byte-preserved
- compaction restore: keep --add-dir files in reminders
- soul: settle shielded context writes across repeated cancellations
- web replay: stat failure replays full history instead of none
- Agent resume: malformed ids return a clean 'Agent not found'
- oauth: fail loud on missing refresh_token at login; carry expires_in
forward on refresh; never read an empty device id
- /theme auto: failed background probe can be re-probed via /theme
- markdown: fence walker honors CommonMark close rules (no info string)
- /restore: cheap id-format guard replaces loading every snapshot
Multi-instance robustness:
- per-session writer lock (.owner.lock) in the CLI and web worker
- pythinker.json mutations go through a locked read-modify-write
- JSONL appenders repair torn final lines; forks materialize atomically
- session/wire scanners skip non-object lines instead of crashing
- project memory: strict reads on mutation (no wipe on transient EIO),
journal capped at 100 recaps, atomic inbox claim, mtime-based recall
re-arm, flock acquisition off the event loop
Subagent orchestration:
- summary-continuation failure keeps the completed result
- hallucinated subagent types fail fast with the valid-type list
(Agent and RunAgents, before any child launches)
- background failures carry an Agent ID + resume hint; finalize is
guarded; runner crashes are surfaced via the done callback
- copy_for_role shares the live-task registry (no false orphans)
Cleanup: shared blink_visible() (12 copies), used_from_remaining()
(6 copies), is_local_host reuse, single realpath pass in write/replace,
cached thinking-frame blend, MCP cross-server shadow warning.
* refactor(auth): split openai.py into an auth/openai package
Mechanical, behavior-preserving split by responsibility: constants,
catalog, oauth_client, browser_flow, models, config_apply, login, plus
a package __init__ re-exporting the existing import surface. Applies
the accepted clean-code review items: JsonObject boundary typing with
narrow casts, _parse_chatgpt_model_item extraction,
_default_config_error / _handled_error_event helpers, and
private-helper renames (_first_present_field /
_first_non_empty_string_field). Test monkeypatch targets follow the
moved definitions. Includes the audit's 401/403 login message split,
which now lives in login.py.
* style(tests): apply ruff format to retargeted blink_visible patches
* fix(tui): prevent input top-border label from wrapping on narrow terminals
The flushed-right effort label was appended unconditionally; when the rule
was shorter than the label plus its gap, the line overflowed and wrapped,
contradicting the method's stated no-wrap invariant. Fall back to the plain
full-width rule when the label cannot fit, and add a regression test.1 parent 00e7b67 commit 9996984
222 files changed
Lines changed: 9751 additions & 2889 deletions
File tree
- packages
- pythinker-core/tests
- pythinker-host/src/pythinker_host
- pythinker-review/tests/unit
- src/pythinker_code
- acp
- agents/default
- approval_runtime
- auth
- openai
- background
- cli
- hooks
- memory
- plugin
- soul
- subagents
- telemetry
- tools
- agent
- background
- file
- shell
- web
- ui
- shell
- components
- selectors
- tool_renderers
- usage_adapters
- visualize
- utils
- rich
- vis/api
- web
- api
- runner
- wire
- tasks
- tests_e2e
- tests
- auth
- background
- core
- e2e
- hooks
- telemetry
- tools
- ui_and_conv
- ui
- usage_adapters
- utils
- vis
- web
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
77 | 77 | | |
78 | 78 | | |
79 | 79 | | |
| 80 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
15 | 15 | | |
16 | 16 | | |
17 | 17 | | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
18 | 24 | | |
19 | 25 | | |
20 | 26 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
114 | 114 | | |
115 | 115 | | |
116 | 116 | | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
117 | 129 | | |
118 | 130 | | |
119 | 131 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
156 | 156 | | |
157 | 157 | | |
158 | 158 | | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
159 | 163 | | |
160 | 164 | | |
161 | 165 | | |
| |||
282 | 286 | | |
283 | 287 | | |
284 | 288 | | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
285 | 293 | | |
286 | 294 | | |
287 | 295 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
99 | 99 | | |
100 | 100 | | |
101 | 101 | | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
102 | 108 | | |
103 | 109 | | |
104 | 110 | | |
| |||
143 | 149 | | |
144 | 150 | | |
145 | 151 | | |
146 | | - | |
| 152 | + | |
147 | 153 | | |
148 | 154 | | |
149 | 155 | | |
| |||
154 | 160 | | |
155 | 161 | | |
156 | 162 | | |
157 | | - | |
| 163 | + | |
158 | 164 | | |
159 | 165 | | |
160 | 166 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
118 | 118 | | |
119 | 119 | | |
120 | 120 | | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
121 | 125 | | |
122 | 126 | | |
123 | 127 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
179 | 179 | | |
180 | 180 | | |
181 | 181 | | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
182 | 205 | | |
183 | 206 | | |
184 | 207 | | |
| |||
Lines changed: 2 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
48 | 48 | | |
49 | 49 | | |
50 | 50 | | |
51 | | - | |
| 51 | + | |
| 52 | + | |
52 | 53 | | |
53 | 54 | | |
54 | 55 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
8 | 8 | | |
9 | 9 | | |
10 | 10 | | |
11 | | - | |
12 | | - | |
13 | | - | |
14 | | - | |
15 | | - | |
16 | | - | |
17 | | - | |
18 | | - | |
19 | | - | |
20 | | - | |
21 | | - | |
22 | | - | |
23 | | - | |
24 | | - | |
25 | | - | |
26 | | - | |
27 | | - | |
28 | | - | |
29 | | - | |
30 | | - | |
31 | | - | |
32 | | - | |
33 | | - | |
34 | | - | |
35 | | - | |
36 | | - | |
37 | | - | |
38 | | - | |
39 | | - | |
40 | | - | |
41 | | - | |
42 | | - | |
43 | | - | |
44 | | - | |
45 | | - | |
46 | | - | |
47 | | - | |
48 | | - | |
49 | | - | |
50 | | - | |
51 | | - | |
52 | | - | |
53 | | - | |
54 | | - | |
55 | | - | |
56 | | - | |
57 | | - | |
58 | | - | |
59 | | - | |
60 | | - | |
61 | | - | |
62 | | - | |
63 | | - | |
64 | | - | |
65 | 11 | | |
66 | 12 | | |
67 | 13 | | |
| |||
126 | 72 | | |
127 | 73 | | |
128 | 74 | | |
129 | | - | |
130 | | - | |
131 | | - | |
132 | | - | |
133 | 75 | | |
134 | 76 | | |
135 | 77 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
212 | 212 | | |
213 | 213 | | |
214 | 214 | | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
215 | 218 | | |
216 | 219 | | |
217 | 220 | | |
| |||
0 commit comments