Core quality pass: sentence-cut clips, exact captions, two-pass loudness, lazy reframe (0.8.0)

[JeremySNR]

What does this change?

A quality pass over the five things the product lives on: where clips start and stop, captions matching the preview, audio mastering, speaker reframing throughput, and cleaner transcripts. Bumps the version to 0.8.0 with a dated CHANGELOG section so the release workflow can publish notes.

Clip boundaries

• The model reads the transcript as one sentence per line (derived from word punctuation, unpunctuated rambles split at pauses) instead of Whisper segments, which break mid-sentence, and is told to start and end every clip on a line. The ending and opening reviews use the same sentences.
• A start inside a sentence opens that sentence rather than jumping to the nearest boundary; an end inside a sentence completes it.
• Tightened clips keep 0.7 s after the last word so the export fade no longer ducks the final syllable.
• New scripts/eval-clips.ts measures boundary quality on saved projects, and with --rerun compares against fresh detection so prompt changes can be checked.

Captions

• Groups are laid out into explicit lines from a per-style em budget (measured glyph widths, aspect-aware), capped at two lines. The ASS export carries the breaks as \N with \q2 on caption events; the preview draws one non-wrapping block per line. Preview and export cannot wrap differently.
• Caption events are anchored middle-centre on the style line, matching the preview. Two-line captions used to sit half a block higher in the export.
• Finished groups hold for up to 1.5 s instead of blanking the frame on every pause.

Audio

• Two-pass loudness: measure, then one linear gain. Through loudnorm linear mode when peaks allow (range target raised to the measured range, which that mode requires), otherwise a plain gain into a true-peak limiter. Single-pass dynamic only when the source cannot be measured.

Reframe throughput

• Face tracking plus active speaker detection runs for the top tier only (eight clips, twelve for scores of 80+). The rest are analysed on demand when opened or exported, with refcounted cancellation and an editor status line with retry. Layout choices made while pending are kept.

Transcripts

• Whisper segments it flags as hallucinations are dropped with their words and never primed into the next chunk. Word timings are made monotonic with a minimum on-screen duration.

Also

• The "fit under N MB" field can be cleared and retyped; it commits on blur/Enter.

Why?

Making a long video into clips is judged in the first and last second of each clip and by whether the captions look like the editor promised. Each of these was leaking quality in a way you could see or hear: mid-sentence cuts, captions wrapping differently in the file, audio pumping on speech, an hour of face tracking before the first clip appeared.

How did you test it?

• npm test (324 tests, up from 231; new coverage for sentence derivation, boundary snapping, caption layout and anchor, loudness parsing and both paths, tier selection and merge rules, hallucination filtering, word timing repair)
• npm run typecheck
• npm run lint
• Checked the change in the running app (Xvfb smoke test with screenshots)
• Offline pipeline and quality scripts (real renders through the new audio chain)
• A scripted-model test drives the whole highlight pass end to end through a stubbed network layer: sentence-per-line prompt, mid-sentence model boundaries landing on sentence boundaries, the ending review extending a clip, the opening review advancing one, dedupe and clamping.
• An integration test runs the real on-demand reframe analysis on a synthetic video under a stubbed Electron, covering persistence, shared runs, cancellation refcounting and merging with concurrent edits.
• Loudness verified on the bundled ffmpeg: a wide-range source lands at -14.0 LUFS in linear mode; a peak-limited source lands at -14.5 LUFS with the true peak at -1.9 dBTP through the limiter.
• Rendered 9:16 frames for three caption styles and a long hook title and checked them visually: blocks centred on their anchors, explicit two-line breaks, the hook wrapping inside its margins.

Anything to watch out for?

Preview and export still share the planning code in src/shared/; the caption layout now lives there too (captionLayout.ts), so both sides get identical lines by construction. The width estimate is deliberately conservative for unknown uploaded fonts, so lines break a little early rather than risk overflow.

Older projects load unchanged: clips without a reframe status are treated as already analysed.

🤖 Generated with Claude Code

https://claude.ai/code/session_01XZCKJpD6vn9eFsymhmhHhg

---

Generated by Claude Code

---

Note

Medium Risk
Touches highlight detection, render audio, export IPC, and on-demand reframe on the critical path; behavior is heavily tested but regressions in framing or loudness would affect every export.

Overview
0.8.0 is a quality pass on clip boundaries, captions, export audio, reframe throughput, and transcript hygiene.

Clip picking now feeds the model one sentence per line (from word punctuation, with long unpunctuated runs split at pauses) instead of Whisper segments, and snaps starts/ends with snapClipStart / snapClipEnd so clips open and close on thoughts. Opening/ending reviews use the same sentence model. Tighten keeps more head/tail room so the export fade does not duck the last syllable. scripts/eval-clips.ts scores saved projects and can --rerun detection for prompt A/B checks.

Speaker reframing runs eagerly only for the top 8–12 clips; others stay reframeStatus: pending until the editor opens them or export runs ensureClipReframe, with merge rules so stale renderer saves do not wipe analysis and concurrent callers share one run.

Captions share a width-based captionLayoutBudget in captionLayout.ts for preview and ASS (\N + \q2), middle-centre pos to match preview, and a brief hold after each group.

Exports measure then normalize loudness (linear loudnorm or gain+limiter) instead of single-pass dynamic loudnorm.

Transcription drops hallucinated Whisper segments and words, primes chunks with trusted text only, and normalizes word timings for karaoke.

Also fixes the fit under N MB settings field (draft until blur/Enter) and bumps version/docs.

^{Reviewed by Cursor Bugbot for commit 8b89e0a. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Bugbot Autofix prepared a fix for the issue found in the latest run.

• ✅ Fixed: Start snap skips enclosing sentence
  • When the model lands deep inside a long sentence, snapClipStart now word-snaps immediately instead of falling through to the next-sentence snap path.

_{You can send follow-ups to the cloud agent here.}

^{Reviewed by Cursor Bugbot for commit 231653a. Configure here.}