Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions .github/workflows/ui-smoke.yml
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,7 @@ jobs:
node --check site/preview-v2/assets/mixer-native-compressor.js
node --check site/preview-v2/assets/mixer-native-harmony.js
python tests/native_harmony_assembly_test.py
node --check site/preview-v2/assets/mixer-native-vocal-throw.js
python tests/native_peq_assembly_test.py
python tests/approved_operator_ux_smoke.py
python scripts/build_ui_css.py --check
Expand All @@ -128,6 +129,32 @@ jobs:
- name: Block incomplete product release only when preparing product review
if: ${{ github.event_name == 'pull_request' && !github.event.pull_request.draft && startsWith(github.head_ref, 'integration/product-') }}
run: node scripts/check_console_puzzle.mjs --release
vocal-throw-preview:
name: Vocal Throw native editor desktop/mobile Chromium FAT
needs: quick
if: ${{ github.event_name == 'pull_request' && (startsWith(github.head_ref, 'feature/arson-vocal-throw-') || startsWith(github.head_ref, 'integration/arson-vocal-throw-')) }}
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install Chromium for targeted Vocal Throw UI
run: |
python -m pip install playwright
python -m playwright install --with-deps chromium
- name: Check exact mixer picker, native ACK and editor in desktop/mobile
run: python tests/approved_native_vocal_throw_smoke.py
- name: Keep compact editor screenshot evidence
if: ${{ always() }}
uses: actions/upload-artifact@v4
with:
name: ArSonMix-Vocal-Throw-Desktop-Mobile-${{ github.run_id }}
path: arsonmix-vocal-throw-*.png
if-no-files-found: ignore
retention-days: 7

operator-ux-fat:
name: Operator UX real Chromium / native-assembled GUI (5 viewports)
needs: quick
Expand Down
12 changes: 12 additions & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -405,6 +405,18 @@ if(BUILD_TESTING)
COMMAND ARMixNativeCompressorLifecycleTests)
add_dependencies(ARMixMixerRoutingTests ARMixNativeCompressorLifecycleTests)

add_executable(ARMixNativeVocalThrowTests tests/builtin_vocal_throw_test.cpp
engine/builtin_fx/VocalThrow.h)
target_compile_features(ARMixNativeVocalThrowTests PRIVATE cxx_std_20)
add_test(NAME NativeVocalThrowPhraseCaptureManualAutoAndNoHeap COMMAND ARMixNativeVocalThrowTests)
add_dependencies(ARMixMixerRoutingTests ARMixNativeVocalThrowTests)
add_executable(ARMixNativeMusicTempoTests tests/native_music_tempo_test.cpp
engine/core/NativeMusicTempoMonitor.h)
target_compile_features(ARMixNativeMusicTempoTests PRIVATE cxx_std_20)
add_test(NAME NativeMusicTempoSourceOnsetsAndFallback COMMAND ARMixNativeMusicTempoTests)
add_dependencies(ARMixMixerRoutingTests ARMixNativeMusicTempoTests)


add_executable(ARMixNativeMasterLoudnessTests tests/native_master_loudness_test.cpp engine/dsp/NativeMasterLoudness.h)
target_compile_features(ARMixNativeMasterLoudnessTests PRIVATE cxx_std_20)
add_test(NAME P1KMeasuredNativeBS1770StereoAnd4xTruePeak COMMAND ARMixNativeMasterLoudnessTests)
Expand Down
45 changes: 45 additions & 0 deletions docs/ARSON_VOCAL_THROW_BEAT_GRID_M3_20261109.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# ArSon Vocal Throw M3 — native music beat-phase quantized echo

**Branch:** `feature/arson-vocal-throw-phase-grid-m3-20261109`; builds on draft M2 PR #474 (which builds on M1 PR #469). Product issue #473. **Not a hardware-certified stage release.**

## What changed

A stable BPM alone cannot locate the *beat boundary*. M3 shares the existing native ChannelId audio renderer's monotonically increasing callback frame counter with its built-in FX processing, then quantizes **only the newly scheduled echo** when the selected MUSIC ChannelId has a credible beat-phase estimate.

```
ONE native ASIO/JUCE ChannelId render (epoch + graph revision)
beginAudioBlock(frames) -> monotonic native sample-frame start
NativeBeatFrameScope (thread-local, callback-only and restored on exit)
pre-FX MUSIC ChannelId -> 10ms nonmutating energy frames
each summary carries its exact observed audio end-frame (within 10ms)
SPSC -> existing control timer -> BPM, onset anchor, phase confidence
CONTROL publishes BPM + sample-frame anchor + epoch/revision to FX
-> atomic version-fenced snapshot (one read attempt per callback)
VOICE FX on SAME callback/thread
-> read versioned tempo + scoped native frame origin
-> if source/epoch/revision invalid: accepted Manual/TAP
-> if BPM stable, beat phase weak: BPM-derived interval, NO grid
-> if BPM and phase credible: snapshot anchor & interval at THROW
-> freeze voice with existing two-tail bounded PCM
-> schedule next echo at FIRST native music grid point >=
intended (phrase end + musical subdivision), with correction
bounded to ONE subdivision and zero change to dry PCM
-> already sounding echo tails retain frozen timing across BPM/graph edits
```

### Realtime invariant
No new audio callback, dynamic allocation, model inference, FFT, lock, waiting, JS audio, logging or file I/O from the callback. Channel PCM enters one mixer graph and is never processed twice. The timing scope is a trivially copied thread-local struct, no global audio clock thread. Tempo/phase inference (up to 121 BPM candidates) is CONTROL-only. Packed atomic scalar fields are protected by an even/odd version; the audio callback makes **one** bounded read attempt and immediately falls back on inconsistency.

### Confidence and fallback
- BPM confidence and beat-phase confidence are independent. A musical onset beat grid is used only if a majority of percussive onset strength agrees on a single pulse lattice.
- Ambiguous rhythmic patterns remain **AUTO BPM · BPM only**, not a speculative grid. A missing source or stale graph becomes **Manual / TAP** without losing the accepted preset.
- A 10ms analysis window means onset time has approximately up to one window of inherent beat-position uncertainty, even though *subsequent scheduling* uses exact callback sample indices. This is not sample-perfect transient detection.
- Phase quantization is measured against **pre-FX** selected music PCM, not against a fully latency-compensated external performance clock. Vendor VST3 latency, off-beat percussion, compressed music, drift, sample-rate changes, ASIO callback reentrancy and sparse backing must still pass field tests.

### Automated acceptance
- Native DSP unit test verifies an actual frozen phrase's first echo is scheduled on the music frame lattice and remains there after transport disappears, compared with Manual/TAP quarter-note fallback; repeats at 64/128-frame PCM chunks with callback heap forbidden.
- Native graph PCM test stamps the selected music ChannelId from one authoritative callback at variable 64+36 sample chunks, verifies anchored BPM/phase confidence at 72/95/120/160 BPM, one block-clock scope, frame counter continuity after a graph revision, and restoration of callback-only context.
- Desktop/mobile Chromium smoke explicitly distinguishes a native ACKed source's **BPM only** from **BEAT GRID** status. No user-facing opt-in or extra monitor/arm step.

### Deferred release qualification
Physical ASIO 64/128 frame loops at 44.1/48/96 kHz; on-stage singer/backing leakage, syncopated 4/4 and half/double-time, large FX PDC, dropout/replug, fade/no-click audition, real CPU/xrun vs bypass, and sustained stress still require field evidence. Do not merge while the PR chain and these safety gates remain unresolved.
45 changes: 45 additions & 0 deletions docs/ARSON_VOCAL_THROW_MUSIC_TEMPO_M2_20261108.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# ArSon Vocal Throw M2 — Native Music Channel Tempo Detection

**Status:** implementation and qualification draft, stacked on Vocal Throw M1 PR #469. M1 green head `4db384851e6f6378cca4fd037b744c98bd1f0ba3`; M2 is PR #474 and tracks issue #473.

## Signal authority

```
Authoritative ChannelId native PCM graph, single JUCE / ASIO callback
MUSIC ChannelId selected explicitly by singer/operator
-> PRE-FX non-mutating read-only music tap
-> 10ms stereo absolute-energy summary, bounded SPSC<Frame,2048>
-> existing 12Hz JUCE control timer: onset search + BPM confidence
-> accepted tempo precomputed into sample intervals, CONTROL only
-> per-VocalThrow atomic pending interval
-> next dry/vocal PCM callback copies interval BEFORE any THROW
-> new frozen vocal echo gets music interval, old tails keep prior schedule
VOICE ChannelId -> VocalThrow (no added dry PCM delay) -> WIRE/BUS/MAIN
```

**AUDIO callback contract:** no memory allocations, FFT, file I/O, logging, locks, networking, JS DSP, or new audio threads. Only the explicitly selected source channel's input stereo samples are summarized; every other channel skips the per-sample music tap. Existing JUCE callback remains the only audio authority. Core audio buffers and retained FX leases remain with the native graph.

**CONTROL contract:** only an accepted `ChannelId` with input / GROUP / FX_RETURN role and an actually scheduled PCM lane is a permissible tempo source. The native FX editor validates both its own vocal insert instance and source identity, device epoch and graph revision. Monitor/MAIN and the vocal channel itself are not accepted. On a valid structural graph recompile, an existing source ID is rebound with a new analysis generation; if absent or stale, the music clock is disabled, and Vocal Throw falls back to its saved BPM or TAP tempo.

## Detector

- A 10ms energy onset, 80ms transient refractory and adaptive floor reject a simple repeated rise from a sustained note.
- Bounded pairwise onset-period candidate scoring (60–180 BPM), with penalty for octave half/double candidates, three consistent estimates and confidence threshold.
- Native readback distinguishes **AUTO BPM locked**, **Listening**, and **Manual/TAP fallback**. A silence period exceeding 3.2s resets lock.
- UI conveys confidence as measured by estimator; this number is not field-certified reliability.

**Explicit limitations:** Music/onset BPM estimation does not provide sample-perfect beat-phase grid alignment. A recent transient anchor is retained for diagnostics only; it is not used as a globally compensated sample clock. Syncopated and heavily compressed music, live mic bleed, time-signature changes, half/double-time ambiguity and device-specific ASIO timing remain hardware field gates. Tempo selection is ephemeral song-wide, not yet persisted per Show. The fallback path always remains audible on the singer's dry stream.

## Qualification evidence

- Prior M1 Windows MSVC/JUCE: 63/63 CTest on exact SHA `4db384851e6f6378cca4fd037b744c98bd1f0ba3`.
- M2 Windows MSVC/JUCE: 64/64 CTest, including `NativeMusicTempoSourceOnsetsAndFallback`, on exact SHA `cce63231da52f69e6789873ecc406adba69aa1cb`. See [Actions run 42498186748](https://github.com/masarray/arsonmix/actions/runs/42498186748).
- Chromium FAT M2 native Vocal Throw picker, music source ACK, BPM readback and stale instance handling passed on same SHA.
- Later hardening: inbound tempo sampled before same-callback THROW and reset on plugin audio reprepare; corresponding regression checks for manual fallback and active tail interval stability. Qualification on the later SHA must be checked independently.
- Deterministic PCM: 72, 95, 120, 160 BPM; non-vocal source, stale revision, source change, dropout, default fallback. The CTest calls the *actual* native publisher to verify a music signal traverses the accepted graph, not only the isolated estimator.

**Not qualified:** physical ASIO 64/128 frame load at 44.1/48/96 kHz; prolonged simultaneous singer/backing test; live beat-grid sync / stereo echo hearing test; WASAPI secondary stream callback load; long-run xrun/restart and voice-music bleed reliability. Green CI is necessary but not sufficient for release.

## Next DSP/UX milestone

Prove stable beat phase with a source-specific latency-compensated anchor; handle half/double-time ambiguity with confidence floor and long-window hysteresis; permit song/Show fallback preference; add footswitch mapping and live capture metrics without callback work. Until then label UI as estimated **AUTO BPM** rather than **sample-accurate beat quantization**.
54 changes: 54 additions & 0 deletions docs/ARSON_VOCAL_THROW_NATIVE_20261108.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# ArSon Vocal Throw — native phrase-aware live delay

**Status: functional prototype on a separate feature branch; not a certified live-performance release.**

This is a first-class native ChannelId insert in ArSonMix's single JUCE PCM render graph, not a second audio callback, browser-generated sound or independent DAW.

## Implemented

- **Manual:** rolling ring buffer holds up to 1.5 seconds of the most recent vocal. THROW captures a word or phrase and schedules up to six decaying repeats. Two independent tail voices allow a subsequent phrase to overlap an existing echo instead of cutting it.
- **Auto:** energy envelope, minimum voice duration, silence threshold and cooldown detect the end of a sung phrase without letting the wet echo re-trigger itself.
- **Hybrid (default):** phrase-end automation and manual cue together, with manual-cue suppression of a subsequent auto event for the same phrase.
- **Tempo:** BPM entry (60-200), TAP BPM, quarter/eighth/dotted-eighth/half note subdivisions. Interval timing is in native samples at the actual device rate.
- **Mix:** direct dry input with zero added samples, stereo echo, decay, short fades, smoothed wet edits, and ducking while the singer is active. A bounded active-tap gain normalizer prevents dense repeats/overlapping phrases from amplifying the wet bus simply because taps stack; dry is never attenuated.
- **Real-time:** all rolling/frozen buffers are allocated off-callback; a fixed maximum 24-stereo-frame-per-sample capture copy budget per active tail avoids a single large callback memcpy. Contiguous copy chunks eliminate per-frame modulo, and the ring write cursor rolls over without division. A lock-free SPSC transfers host settings and manual cue to the audio callback. Callback never allocates/logs/locks.
- **Identity & persistence:** stable native catalog ID; ChannelId, device epoch, graph revision, and insert instance fencing. DSP completed ACK and later readback precede accepted display. Schema-versioned 48-byte AVTH settings blob validated during Show restore; audio capture and tails are not persisted.
- **UX:** a compact FX editor inside the exact approved /preview/ mixer; presets in header, uniform power/close, BPM/note/repeat, THROW, TAP and collapsible advanced controls. One shared FX stylesheet.

## Architecture

Hardware input → ChannelId mixer strip → upstream EQ/Compressor/VST3 → VocalThrowProcessor → downstream inserts → fader/pan → WIRE/BUS/MAIN → hardware output.

Core files: engine/builtin_fx/VocalThrow.h (DSP), VocalThrowProcessor.h (JUCE), src/LiveAudioEngine.h (graph install), src/Main.cpp (validated command/readback), src/NativeMixerSessionFxRestore.h (versioned state), scripts/compose_native_peq.py (one picker), web/approved-native-bridge.js (real insert), site/preview-v2/assets/mixer-native-vocal-throw.js (controls only).

## Tests / qualification

The ARMixNativeVocalThrowTests CTest covers zero-added-sample dry, tempo divisions, captured stereo echoes after manual cue, auto-end detection and no self-trigger, bad parameter rejection, 8/44.1/48 kHz sample rates and a global allocator guard around realtime processing. Additional deterministic tests cover 8/48/96 kHz two-tail preservation during a second cue, a full two-tail pool reporting cue refusal rather than overwriting a running phrase, suppression of manual/auto double-cues, and peak containment of dense six-tap echoes. Python native picker/compiled-gui tests enforce no extra mixer screen and real-ACK event names.

Before shipping: exact-SHA Windows MSVC + CTest, actual approved /preview/ Chromium screenshots, ASIO 64/128-frame stress at 44.1/48/96 kHz, live singer audition, device restart, bypass/tail behavior, and Show restore.

## Explicit limitations

1. **Automatic BPM analysis from the music/backing bus is not yet implemented.** Auto currently means automatic vocal phrase-end trigger, not auto-estimated music tempo. BPM is entered or TAP-derived. Full beat tracking requires an independent music-bus feed, beat confidence and phase tracking.
2. Voice end detector uses an adjustable energy-envelope heuristic, not linguistic speech recognition or a trained singing model. Breaths, falsetto and microphone bleed can cause false cues.
3. No MIDI footswitch binding, more than two simultaneous phrase layers, stretchable echo phrases, global music BPM transport integration, or direct song-snapshot binding in this milestone. When both frozen-tail slots are busy, further THROW requests are rejected (native ACK reports not applied) rather than cutting an existing musical phrase.
4. Green quick CI is not equivalent to real Windows native compilation or ASIO device certification. Zero reported PDC applies only to the FX DSP's dry path; end-to-end hardware/driver latency and callback CPU load still require ASIO measurements.
5. Existing channel routing and independent secondary audio clock remain authoritative; the FX insert makes no new claims about unresolved WIRE/SOURCE ONLY issues.

## Field test

Insert ID armix.builtin.vocal-throw.v1 into real ChannelId vocal strip; confirm graph ACK; sing “atau tidak” then press THROW at 72/95/120/160 BPM; try Auto with varying held notes and short breaths; toggle bypass while dry vocal remains direct; change device/quantum and reject stale instance requests. Compare physical ASIO callback xruns, stereo/feedback, click-free fades and singer subjective quality.

## 2026-11-08 — performance and musicality hardening audit

Audit of the first native revision identified these concrete risks:
- Triggering a second phrase reset `playing_` and overwrote the frozen buffer of the first tail; audible repeat discontinuity was possible.
- The original capture copied 24 frames via two per-frame ring modulo operations; this was an avoidable callback hot-path cost.
- Hybrid manual cues could be followed by auto-trigger of the same sung phrase, producing unintended double repeats.
- Wet level changed immediately on edits; close-spaced six-tap feedback-like echoes could produce unnecessarily high wet peaks.

Hardening: two independent frozen voices, bounded cue admission (explicit rejection if full), bulk contiguous ring capture, write cursor without modulo, continuous smoothing of wet/duck gain, current-phrase manual suppression of auto replay, and peak-aware wet overlap normalization. Silence counters are now saturating for multi-hour sessions.

**Realtime caveats:** No heap, locks, logging or file operations in `processStereo`, `throwNow`, or parameter application. Worst-case processing still includes two simultaneous 24-frame copy budgets, and an up-to-twelve-tap mix; on low-buffer ASIO that requires measured xrun and callback-duration evidence, not just source inspection. Extra frozen buffer memory is allocated only on channels with this effect. A full two-voice pool intentionally rejects a new cue, so polyphony and cue policies should be evaluated with vocalist/operator tests.

**Next release gates:** exact-SHA MSVC CTest and native JUCE compile, 64/128-frame ASIO at 44.1/48/96 kHz, overnight stability and lifecycle/device-rate restart, phrase-ending listening sessions including breath/bleed/falsetto, bypass/re-enable and FX graph routing, and latency/xrun baseline comparisons versus bypass. Beat tracking from an isolated backing/music bus (confidence, phase and half/double-time hysteresis) remains a separate architecture milestone, not an already delivered feature.
1 change: 1 addition & 0 deletions engine/builtin_fx/BuiltInFxCatalog.h
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
namespace armix::builtin_fx {
inline constexpr std::string_view channelEqId = "armix.builtin.channel-eq.v1";
inline constexpr std::string_view channelCompressorId = "armix.builtin.channel-compressor.v1";
inline constexpr std::string_view vocalThrowId = "armix.builtin.vocal-throw.v1";

enum class FilterKind : std::uint8_t { highPass, lowShelf, bell, highShelf, lowPass };
enum class Slope : std::uint8_t { db12 = 12, db24 = 24 };
Expand Down
Loading
Loading