Skip to content

quiet: trim the two recurring kprintf sources, keep every verdict - #121

Merged
douglasmun merged 1 commit into
mainfrom
quiet/kprintf-runtime-noise
Aug 23, 2026
Merged

quiet: trim the two recurring kprintf sources, keep every verdict#121
douglasmun merged 1 commit into
mainfrom
quiet/kprintf-runtime-noise

Conversation

@douglasmun

Copy link
Copy Markdown
Owner

What

TinyOS grew ~2,150 kprintf sites as a debugging tool. Rather than sweep all of them, I measured two boots — idle 200s, and a ring-3 login + exec idling ~6 min — to find what actually reaches the screen.

Only ~187 of the 2,150 ever fire, and only two things recur:

Source Rate Action
EDR 60-second status report 10 lines/min, forever trimmed
Per-exec [ELF]/[PAE] load trace ~35 lines per exec behind a flag
Boot banner (191 lines) once untouched

The one-time boot banner is deliberately left alone — it's the machine stating its posture, and it costs nothing after boot.

EDR report: inverted from loud-when-idle to quiet-when-idle

The full block now prints only when a finding counter moves (threats, responses, TI matches); otherwise one line; nothing at all if even the scan counter is unchanged.

[EDR DAEMON] ========== STATUS REPORT ==========    <- first report, full baseline
... 8 lines ...
[EDR DAEMON] ===================================
[EDR DAEMON] 120s: 24 scans, 166 procs, 0 threats   <- then one line/minute
[EDR DAEMON] 180s: 36 scans, 250 procs, 0 threats

Two properties kept deliberately:

  • The first report always prints in full, so an EDR that never started stays distinguishable from one running clean.
  • Liveness stays observable, so a wedged daemon goes silent rather than continuing to print a reassuring block of zeroes. The old behaviour was exactly inverted — loudest when nothing was happening, which made the report that finally carried a nonzero threat count look identical to the 60 before it.

Per-exec trace: new kdbg(), off by default

17 pure-trace sites converted (12 elf.c, 5 pae.c) — load addresses, page-table plumbing, per-segment copies.

Every ERROR / SECURITY / WARNING / PASS / FAILED line is deliberately left on kprintf: a verbosity flag that can hide a signature refusal turns "absent" and "denied" into the same output, which is the status-surface failure this tree has hit four times.

New loglevel [normal|debug] restores the trace without a rebuild. Root-gated for the same reason pae/wxaudit are: the trace prints kernel physical addresses and CR3 values, which is ASLR-defeating material regardless of how mundane the individual lines look.

Measured

  • Post-boot output 185 → 104 lines
  • Recurring output 10 lines/min → 1
  • Boot banner unchanged at 191 lines

Verified

  • make clean + rebuild: 0 warnings under -Werror
  • auto-verify-exec.sh PASS (signature enforcement intact, hello.elf ran)
  • edr-rejoin-test.sh 10/10 PASS including both negative controls
  • Checked all 60 harnesses: none greps for a converted line
  • Round-trip proven on a live guest: 0 trace lines by default, 8 restored after loglevel debug

Not done, deliberately

I initially planned to remove the grep -v Suspicious filters in 29 harnesses as stale. They are not stale, and I dropped that change. edr-rejoin-test.sh:61 has a dedicated unit test asserting that marker is repaired, edr-rejoin.sh:14 records it as the high-frequency tear, and advanced detection is enabled by default (edr_advanced.c:528). "Did not appear in two boots" is not "cannot appear" — removing them would make 29 harnesses fail intermittently on a correct kernel.

🤖 Generated with Claude Code

TinyOS grew ~2,150 kprintf sites as a debugging tool. Measuring two boots
(idle 200s, and a ring-3 login + exec idling ~6min) showed only ~187 of them
ever fire, and only TWO things actually RECUR:

  1. the EDR 60-second status report -- 10 lines/minute, forever
  2. the per-exec [ELF]/[PAE] load trace -- ~35 lines every exec

Everything else in the post-boot tail is one-time. The one-time boot banner is
deliberately untouched: it is the machine stating its posture, and it costs
nothing after boot.

EDR status report: inverted from loud-when-idle to quiet-when-idle.

The full block now prints only when a FINDING counter moves (threats,
responses, TI matches); otherwise one line; and nothing at all if even the scan
counter is unchanged. Scans/processes are progress, not findings, so they do
not force the block.

Two properties kept deliberately. The FIRST report always prints in full, so an
EDR that never started stays distinguishable from one running clean. And
liveness stays observable, so a wedged daemon goes SILENT rather than
continuing to print a reassuring block of zeroes -- the old behaviour was
exactly inverted, loudest when nothing was happening, which made the report
that finally carried a nonzero threat count look identical to the 60 before it.

Per-exec trace: new kdbg(), off by default.

17 pure-trace sites converted (12 in elf.c, 5 in pae.c) -- load addresses,
page-table plumbing, per-segment copies. Every ERROR / SECURITY / WARNING /
PASS / FAILED line is deliberately left on kprintf: a verbosity flag that can
hide a signature refusal turns "absent" and "denied" into the same output,
which is the status-surface failure this tree has hit four times.

New `loglevel [normal|debug]` restores the trace without a rebuild. Root-gated
for the same reason pae/wxaudit are: the trace prints kernel physical addresses
and CR3 values, which is ASLR-defeating material regardless of how mundane the
individual lines look.

MEASURED: post-boot output 185 -> 104 lines; recurring output 10 lines/min -> 1.
Boot banner unchanged at 191 lines.

Verified:
  - make clean + rebuild: 0 warnings under -Werror
  - auto-verify-exec.sh PASS (signature enforcement intact, hello.elf ran)
  - edr-rejoin-test.sh 10/10 PASS including both negative controls
  - checked all 60 harnesses: none greps for a converted line
  - round-trip proven on a live guest: 0 trace lines by default,
    8 restored after `loglevel debug`

NOT done, deliberately: the `grep -v Suspicious` filters in 29 harnesses were
initially assumed stale and are NOT. edr-rejoin-test.sh:61 has a dedicated unit
test asserting that marker is repaired, edr-rejoin.sh:14 records it as the
high-frequency tear, and advanced detection is enabled by default
(edr_advanced.c:528). "Did not appear in two boots" is not "cannot appear";
removing them would make 29 harnesses fail intermittently on a correct kernel.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CEkhAhgTxbE5TgifyYf8v4
@douglasmun
douglasmun merged commit 00f0e6a into main Aug 23, 2026
2 checks passed
@douglasmun
douglasmun deleted the quiet/kprintf-runtime-noise branch August 23, 2026 14:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant