Skip to content

Strip all ANSI control sequences in bounded terminal output buffer - #1266

Open
nordicnode wants to merge 2 commits into
CodebuffAI:mainfrom
nordicnode:fix-strip-all-ansi-control-sequences
Open

Strip all ANSI control sequences in bounded terminal output buffer#1266
nordicnode wants to merge 2 commits into
CodebuffAI:mainfrom
nordicnode:fix-strip-all-ansi-control-sequences

Conversation

@nordicnode

@nordicnode nordicnode commented Sep 3, 2026

Copy link
Copy Markdown

Summary

  • Switch BoundedOutputBuffer.append from stripColors to stripAnsi in sdk/src/tools/run-terminal-command.ts.
  • Update INCOMPLETE_ESCAPE_SEQUENCE_REGEX to match split ECMA-48 CSI (\x1b[...) and OSC (\x1b]...) sequences across chunk boundaries, tightened to require that incomplete OSC sequences do not already contain BEL (\x07) or ST (\x1B\\), preventing fully-terminated OSC codes from buffering and dropping trailing text.
  • Reorder ansiRegex in common/src/util/string.ts so CSI and OSC sequences take precedence over generic 2-character Fe escapes, and support both BEL (\x07) and ST (\x1B\\) terminators.
  • Rename internal tracker pendingColorSequence to pendingEscapeSequence.
  • Add unit tests in sdk/src/__tests__/run-terminal-command.test.ts verifying stripping of line clears (\x1b[2K), cursor movement (\x1b[1A), cursor visibility (\x1b[?25h), fully-terminated OSC sequences (BEL and ST), preservation of trailing text in the same chunk and final chunk, and chunk-split sequences.
  • Add unit tests in common/src/util/__tests__/string.test.ts verifying stripAnsi across colors, CSI control codes, OSC with BEL, OSC with ST, and Fe escapes.

Test plan

  • bun test --config=/dev/null --preload test/setup-env.ts src/__tests__/run-terminal-command.test.ts in sdk/ (17/17 pass)
  • bun test src/util/__tests__/string.test.ts in common/ (26/26 pass)
  • bun run build:sdk
  • bun run --cwd cli typecheck (0 errors)
  • bun freebuff/cli/build.ts 0.0.0-ci
  • bun cli/scripts/smoke-binary.ts cli/bin/freebuff (attempt 1/3 OK)
  • bun x prettier --check sdk/src/tools/run-terminal-command.ts sdk/src/__tests__/run-terminal-command.test.ts common/src/util/string.ts common/src/util/__tests__/string.test.ts
  • PR hygiene check passed

- Switch BoundedOutputBuffer.append from stripColors to stripAnsi in sdk/src/tools/run-terminal-command.ts.
- Update INCOMPLETE_ESCAPE_SEQUENCE_REGEX to correctly match split ECMA-48 CSI and OSC sequences across chunk boundaries.
- Rename internal tracker pendingColorSequence to pendingEscapeSequence.
- Add unit tests in sdk/src/__tests__/run-terminal-command.test.ts verifying stripping of line erases, cursor movement, cursor visibility, and chunk-split sequences.
@codebuff-team

Copy link
Copy Markdown
Contributor

Good catch on the underlying bug: stripColors was letting non-color CSI/OSC codes (\x1b[2K, \x1b[?25h, etc.) leak into the buffer, and switching to stripAnsi is the right fix. The rename (pendingColorSequencependingEscapeSequence) is a clean improvement, and the added tests for split CSI sequences across chunk boundaries are a solid addition.

One correctness concern with the new INCOMPLETE_ESCAPE_SEQUENCE_REGEX:

/\x1B(?:\[[0-?]*[ -/]*|\][^\x1B]*)?$/

The OSC branch \]\[^\x1B]*$ matches any run of non-ESC characters after ESC], with no check for the actual OSC terminator (\x07 BEL or ESC\ ST). That means a fully-terminated OSC sequence like \x1b]0;title\x07 at the tail of a chunk is still flagged as "incomplete" and gets buffered into pendingEscapeSequence rather than stripped and flushed. It only gets released once the pending buffer accidentally exceeds 32 chars from unrelated trailing text — otherwise, if that's the last append in the stream (process exits, no further append() calls), that trailing text is silently dropped from format()'s output since there doesn't appear to be a flush-on-finalize path shown in this diff.

Worth either tightening the OSC alternative to require it not already contain a BEL/ST (i.e. only match genuinely truncated sequences), or adding a test that appends an OSC sequence terminated by BEL as the final chunk and asserting it doesn't vanish from format().

Otherwise this is a well-scoped, testable change in sdk/ and worth porting once the OSC edge case is addressed.

@codebuff-team codebuff-team added bot:triaged Classified by the community triage bot pr:needs-work Right idea, not mergeable as written labels Sep 4, 2026
- Tighten INCOMPLETE_ESCAPE_SEQUENCE_REGEX in sdk/src/tools/run-terminal-command.ts
  to require incomplete OSC sequences not contain BEL (\x07) or ST (\x1B\), preventing
  fully-terminated OSC codes from being buffered as incomplete and dropping trailing text.
- Reorder ansiRegex in common/src/util/string.ts so CSI and OSC take precedence over
  generic 2-character Fe escapes, and support both BEL and ST terminators.
- Add unit tests in sdk/src/__tests__/run-terminal-command.test.ts verifying fully-terminated
  OSC stripping, preservation of trailing text in the same chunk and final chunk, and chunk-split OSC.
- Add unit tests in common/src/util/__tests__/string.test.ts verifying stripAnsi on colors,
  CSI controls, OSC BEL, OSC ST, Fe escapes, and plain text.
@nordicnode

Copy link
Copy Markdown
Author

Thanks for catching this edge case! We've updated the regex, corrected stripAnsi's OSC handling, and added comprehensive unit tests for both chunk-split and trailing OSC sequences:

  1. Tightened INCOMPLETE_ESCAPE_SEQUENCE_REGEX:
    Updated sdk/src/tools/run-terminal-command.ts to:

    const INCOMPLETE_ESCAPE_SEQUENCE_REGEX =
      /\x1B(?:\[[0-?]*[ -/]*|\][^\x07\x1B]*(?:\x1B)?)?$/

    The OSC branch \][^\x07\x1B]*(?:\x1B)? now excludes BEL (\x07) and ST (\x1B\\), ensuring that fully-terminated OSC sequences (with or without trailing text) are never flagged as incomplete or buffered into pendingEscapeSequence.

  2. Fixed stripAnsi in common/src/util/string.ts:
    Reordered ansiRegex so CSI (\[...) and OSC (\]...) sequences take precedence over the generic 2-character Fe escape class [@-Z\\-_] (which previously matched \x1B] as a 2-char escape and left 0;title\x07 unstripped), and added support for both BEL (\x07) and ST (\x1B\\) terminators:

    const ansiRegex =
      /\x1B(?:\[[0-?]*[ -/]*[@-~]|\][^\x07\x1B]*(?:\x07|\x1B\\)|[@-Z\\-_])/g
  3. Added Unit Tests:

    • In sdk/src/__tests__/run-terminal-command.test.ts:
      • Verified that fully-terminated OSC sequences with BEL (\x1b]0;title\x07) with trailing text in the same chunk (building...\x1b]0;title\x07done) are stripped without dropping trailing text.
      • Verified that OSC sequences as the final chunk (finished\x1b]0;title\x07) do not drop output in format().
      • Verified OSC sequences terminated with String Terminator (\x1b\\).
      • Verified split OSC sequences across chunk boundaries (both BEL-terminated and ST-terminated right at the \x1B byte).
    • In common/src/util/__tests__/string.test.ts:
      • Added unit tests for stripAnsi covering ANSI colors, CSI controls (\x1b[2K, \x1b[?25h, \x1b[1A), OSC BEL, OSC ST, 2-char Fe escapes, and plain text.

All tests, typechecks, SDK build, and binary smoke tests pass cleanly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bot:triaged Classified by the community triage bot pr:needs-work Right idea, not mergeable as written

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants