Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
ed11632
feat(harmony): P0 time-sliced pitch correlation, sample-rate stable r…
masarray Nov 9, 2026
db33b3c
fix(harmony): phase-resilient vocal and cross-channel Music Guide DSP…
masarray Nov 9, 2026
8dcf69f
perf(harmony): P1 sample-rate-stable Color and P2 prepared note inter…
masarray Nov 9, 2026
b6e7968
test(harmony): tighten physical-time Color check and antiphase dry PC…
masarray Nov 9, 2026
b83597b
perf(harmony): P0 eight-lag realtime slices, complete-window evidence…
masarray Nov 9, 2026
bade068
fix(harmony): auto-rebind accepted FX and replay held user edits acro…
masarray Nov 9, 2026
1637a8b
test(harmony): exercise live graph revision, recreated DSP, slot move…
masarray Nov 9, 2026
44ff9e2
test(harmony): use Playwright keyword-only wait_for_function arg
masarray Nov 9, 2026
b9fd483
test(harmony): enforce accepted-graph auto-rebind without outdated st…
masarray Nov 9, 2026
770f390
fix(harmony): recover initial unadmitted FX and preserve slot label a…
masarray Nov 9, 2026
67a04bd
fix(harmony): confirm native key edits and real vocal pitch activity …
masarray Nov 9, 2026
4fd5d63
docs(harmony): source-backed natural ensemble DSP architecture and re…
masarray Nov 9, 2026
71e7c49
test(harmony): emulate continuous 12 Hz native console ticks for Manu…
masarray Nov 9, 2026
cc57c1f
test(harmony): drain outstanding native readback before asserting Man…
masarray Nov 9, 2026
bdfa32c
feat(harmony): P1 deterministic singer ensemble space with independen…
masarray Nov 9, 2026
f40d419
docs(harmony): PRD and ARD for backing-only perceived depth and lead …
masarray Nov 9, 2026
8c75946
feat(harmony): preserve lead and implement backing-only depth, damped…
masarray Nov 9, 2026
738ab4f
feat(harmony): causal pitch-synchronous vocal candidate with safe gra…
masarray Nov 9, 2026
55168b2
ci(harmony): targeted no-JUCE C++ DSP regression on draft PR changes
masarray Nov 9, 2026
0e9609e
fix(harmony): correct fractional window interpolation in native C++ c…
masarray Nov 9, 2026
94f768d
feat(harmony): native WAV A/B renderer for lead, legacy, synchronous …
masarray Nov 9, 2026
800942f
feat(harmony): psychoacoustic backing-only stage with twin reflection…
masarray Nov 9, 2026
604b822
test(harmony): compare previous and new backing depth with identical …
masarray Nov 9, 2026
94a4039
fix(harmony): heap-own full native FX restore fixtures to prevent Win…
masarray Nov 9, 2026
2230cd9
feat(harmony): frequency-shaped far-ear stage cue with rate-stable DS…
masarray Nov 9, 2026
c0cd439
feat(harmony): recess phantom-center backing direct energy without re…
masarray Nov 9, 2026
d9e7b79
feat(harmony): program-dependent backing room bloom and sub-bass clea…
masarray Nov 9, 2026
649409a
merge(harmony): reconcile natural D3 depth with Player M1 and Compres…
masarray Nov 9, 2026
afa6f2f
fix(harmony): restore musical vocal backing at full WET; space captio…
masarray Nov 9, 2026
f02f35e
fix(harmony-ui): preserve caption and inter-row halo clearance withou…
masarray Nov 9, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions .github/workflows/harmony-fast-dsp.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
name: Harmony native DSP fast verification
on:
pull_request:
types: [opened, reopened, synchronize]
paths:
- 'engine/builtin_fx/Harmony*.h'
- 'tests/harmony*.cpp'
- 'CMakeLists.txt'
- '.github/workflows/harmony-fast-dsp.yml'
permissions:
contents: read
concurrency:
group: harmony-fast-${{ github.event.pull_request.number }}
cancel-in-progress: true
jobs:
harmony-cpp:
name: Native DSP compile and PCM regression (Linux, no JUCE)
runs-on: ubuntu-latest
timeout-minutes: 7
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha }}
- name: Build and run synchronous vocal fixture
run: |
g++ -std=c++20 -O2 -Wall -Wextra tests/harmony_pitch_synchronous_test.cpp -o /tmp/arson-harmony-psola
/tmp/arson-harmony-psola
- name: Compile and run complete engine dry, pitch, stereo and graph-independent PCM fixtures
run: |
g++ -std=c++20 -O2 -Wall -Wextra tests/harmony_dsp_test.cpp -o /tmp/arson-harmony-engine
/tmp/arson-harmony-engine
- name: Verify stage separation, mono and exact PCM
run: |
g++ -std=c++20 -O2 -Wall -Wextra tests/harmony_stage_spatial_test.cpp -o /tmp/arson-harmony-spatial
/tmp/arson-harmony-spatial
- name: Compile and exercise native offline listening A/B renderer
run: |
g++ -std=c++20 -O2 -Wall -Wextra tools/harmony_offline_ab.cpp -o /tmp/arson-harmony-ab
/tmp/arson-harmony-ab --self-test
15 changes: 14 additions & 1 deletion CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -115,9 +115,22 @@ if(BUILD_TESTING)
add_executable(ARMixFxStructuralHandoffTests tests/live_fx_structural_handoff_test.cpp)
target_compile_features(ARMixFxStructuralHandoffTests PRIVATE cxx_std_20)
add_test(NAME R0BStructuralFxAckAndControlThreadRetirement COMMAND ARMixFxStructuralHandoffTests)
add_executable(ARMixHarmonyOfflineAb tools/harmony_offline_ab.cpp)
target_compile_features(ARMixHarmonyOfflineAb PRIVATE cxx_std_20)
add_test(NAME NativeHarmonyOfflineDryAndABRenderHarness COMMAND ARMixHarmonyOfflineAb --self-test)
add_executable(ARMixHarmonyPitchSyncTests tests/harmony_pitch_synchronous_test.cpp)
target_compile_features(ARMixHarmonyPitchSyncTests PRIVATE cxx_std_20)
add_test(NAME NativeHarmonyPitchSynchronousFormantCandidateAndBlockParity COMMAND ARMixHarmonyPitchSyncTests)
add_executable(ARMixHarmonySpatialTests tests/harmony_stage_spatial_test.cpp)
target_compile_features(ARMixHarmonySpatialTests PRIVATE cxx_std_20)
add_test(NAME NativeHarmonyBackingStageSpatialDepthAndMonoPDC COMMAND ARMixHarmonySpatialTests)
add_executable(ARMixHarmonyDspTests tests/harmony_dsp_test.cpp)
target_compile_features(ARMixHarmonyDspTests PRIVATE cxx_std_20)
add_test(NAME NativeHarmonyMultivoicePitchAndRealtimeFinitePCM COMMAND ARMixHarmonyDspTests)
# Non-gating synthetic engine-only CPU profile (not ASIO certification).
add_executable(ARMixHarmonyCpuProfileTests tests/harmony_cpu_profile_test.cpp)
target_compile_features(ARMixHarmonyCpuProfileTests PRIVATE cxx_std_20)
add_test(NAME NativeHarmonySyntheticCallbackCpuProfile COMMAND ARMixHarmonyCpuProfileTests)
add_executable(ARMixHarmonyMusicGuideTests tests/harmony_music_key_test.cpp)
target_compile_features(ARMixHarmonyMusicGuideTests PRIVATE cxx_std_20)
add_test(NAME NativeHarmonyCrossChannelMusicKeyAndManualFallback COMMAND ARMixHarmonyMusicGuideTests)
Expand All @@ -129,7 +142,7 @@ if(BUILD_TESTING)
add_test(NAME ApprovedNativeHarmonyEditorAckAndSingleGui COMMAND "${Python3_EXECUTABLE}" "${CMAKE_CURRENT_SOURCE_DIR}/tests/native_harmony_assembly_test.py")
add_executable(ARMixMixerRoutingTests tests/mixer_routing_plan_test.cpp)
target_compile_features(ARMixMixerRoutingTests PRIVATE cxx_std_20)
add_dependencies(ARMixMixerRoutingTests ARMixHarmonyDspTests ARMixHarmonyMusicGuideTests ARMixHarmonyChannelPcmTests)
add_dependencies(ARMixMixerRoutingTests ARMixHarmonyDspTests ARMixHarmonyMusicGuideTests ARMixHarmonyChannelPcmTests ARMixHarmonyCpuProfileTests)
add_test(NAME TenStripStaticRoutingAndMaster COMMAND ARMixMixerRoutingTests)
# M3D atomic native Session: the accepted Console roster cannot advance
# without a successful epoch/revision-gated PCM publication.
Expand Down
133 changes: 133 additions & 0 deletions docs/ARD_ARSON_HARMONY_DEPTH_ENGINE_20261109.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
# ARD — ArSon Harmony FX Backing-Only Spatial Depth Engine (v1.0)
**Date:** 2026-11-09 · **Status:** Proposed architecture with staged implementation checks
**Scope:** Native built-in Harmony on PR #487; companion [PRD](PRD_ARSON_HARMONY_NATURAL_DEPTH_20261109.md) and [Natural Ensemble](ARSON_HARMONY_NATURAL_ENSEMBLE_ARCHITECTURE_20261109.md).

## Evidence-led acoustic model
Human auditory distance relies on **direct/reverberant energy ratio**, overall source level, spectral balance and interaural cues. Increasing reverberant ratio commonly signals greater distance; absolute physical distance cannot be inferred from a simple Space knob. First-arriving direct sound determines much of localization (precedence); a small number of obvious repeats can produce slap/comb effects rather than a coherent room. More high-frequency absorption is a mixing cue, but **air absorption over just a few metres does not justify extreme LPF**. For a live singer, masking and consonant smearing are unacceptable.

- Bronkhorst & Houtgast, *Nature* 1999, distance versus DRR: https://www.nature.com/articles/17374
- Zahorik review, distance, intensity, DRR, spectral and interaural cues: https://pmc.ncbi.nlm.nih.gov/articles/PMC4744263/
- Litovsky et al. review of the precedence effect: https://pmc.ncbi.nlm.nih.gov/articles/PMC4310855/
- JASA 2025 frontal distance study (DRR or spectral centroid best single cue): https://pubmed.ncbi.nlm.nih.gov/40928551/
- Sound on Sound depth mixing (level/EQ/diffuse reverb versus clarity): https://www.soundonsound.com/techniques/creating-sense-depth-your-mix
- iZotope mix depth: https://www.izotope.com/community/blog/what-is-mix-depth-how-to-create-front-back-space
- FDN theory / realtime late room: https://artificial-audio.github.io/portfolio/fdn/

## Before: audited exact C++ issue
`engine/builtin_fx/HarmonyDsp.h::process` mixes lead with
`(1.f - wet_*0.65f*voicedGain_)` despite the lead being dry; at Wet=.45 / voicedGain=1 this is about -3 dB lead. This is **gain duck**, not proof of phase cancellation. Current `HarmonyEnsembleSpace.h` adds one early opposite-side tap/voice but has **no diffuse late field**. At most it generates scattered echoes and pan; it does not control DRR as an acoustic scene.

## One-source, one-room block graph
```text
Mono mic PCM + accepted ChannelId / native graph
|
+--> phase-safe analysis / voice targets --> pitch shifters (2-5)
| |
| +--> independent singer onset / timbre / pitch modulation
| |
| +--> short direct backing L/R (Space-dependent attenuation)
| +--> per-singer fixed-tap filtered early reflections
| +--> ONE shared small stereo diffuse late field [FDN]
| (backing send ONLY; fixed delay, stable feedback)
|
+--> exact delayed native L/R lead --> optional explicitly enabled Pitch Tune
| |
+-----------------------------> sum w backing L/R
-> explicit Gain -> existing PCM bus
```
Only existing `Settings::space`, wet, color and voices. No new scene schema, plugin routing, hidden monitor step, GUI Lab, or second ownership. `Space=0` disables audible room, not the valid PCM path. Diffuse field is never fed with lead.

## Time/frequency strategy
- **Front:** lead unchanged by wet, stereo and sample latency untouched. Pitch Tune explicitly replaces/corrects when user enables it; never implied by Wet.
- **Direct backup:** keep existing individually phase-offset 5–17 ms arrival, deterministic cent modulation and prepared pan L/C/R. At increasing Space, reduce direct backing gain mildly (candidate 0dB to -4dB) so backing remains intelligible but moves behind.
- **Backstage spectral contour:** one sample-rate-normalized low-pass in each voice with a smooth dry/filtered crossfade, evaluated on backing only. Candidate corner around 3–6 kHz and partial dry retention; do not physically claim air propagation at 3 m.
- **Early field:** use existing per-voice 12–20 ms fixed taps, with mild high-frequency damping and side distribution; no modulation of the delay tap by Space (avoids Doppler/comb chirp).
- **Late field:** one shared bounded 4-branch stereo feedback delay network (FDN), fixed integer delay lines with different non-harmonic lengths and a normalized Hadamard feedback transform. Feedback damping and per-line gain computed in non-RT `prepare(rate)` to target about 0.45–0.7 s decay. Conservative energy normalization, filtered output, no runaway or metallic tonal build-up. First prototype is room **ambience**, not cathedral/hall reverb.
- **Control:** `space` interpolates smoothly across width, direct gain, backing brightness, early/late send and return. `wet` scales backing energy only; disabled or Wet=0 must not leak old tail. All coefficients finite and clamped. No audio-owner reallocation or blocking.

## Proposed bounded C++ data ownership
- `HarmonyEnsembleSpace.h`: per-voice onset, identity modulation and 4096-sample early reflection rings; optional per-voice damping state. Precompute sample-rate coefficients on CONTROL `prepare(rate)`; callback only adds/multiplies.
- `HarmonyDepthRoom.h`: 4 shared, 8192-sample float delay rings (128 KiB); 4 damping states; 4 feedback gains; 1 monotonic write index. Delays up to 41 ms at 192k remain in bounds. A 4x4 orthonormal Hadamard transformation (scale 0.5) ensures controlled feedback energy. No `new`, FFT, `pow`, `sin`, `log`, device calls or synchronization in `processSample`.
- `HarmonyDsp.h`: keep lead path independent of `wet`; accumulate backing direct and early/late sends by voice, shared stereo room returns only in wet branch. Use existing 5-ms control smoothing and fixed instance state.
- Single publisher and callback: preserve bridge ACK/readback/graph lease, `ChannelHarmonyProcessor`, SPSC packets and `latencySamples()`. Declared PDC remains 1536 samples @48k until evidence warrants an independent P0 latency redesign.

## Risks, failure behavior and test design
| Risk | Defensive behavior / evidence |
|---|---|
| Room self-oscillation / metallic flutter | Precomputed feedback magnitude <1 and damping, orthogonal matrix; impulse decay < target floor by ~2 seconds; 32/44.1/48/96/192k CTest and extended silence |
| Mono cancellation / comb | Positive-polarity forward dry, no artificial anti-phase width; L+R output energy with 2/3/5 voices >0 and lead parity; evaluate ER frequency response with real voice |
| Space automation chirping | Interpolate gains/filter coefficients, **not** delay lengths; test 0->1->0 at arbitrary 64-sample boundaries, output finite without transient spike |
| Lead duck | Unit check dry equals original delayed PCM at wet=0,45,100 and multiple Space; actual lead equation cannot reference Wet |
| Reverb masks lyrics | Limit room gain, high-pass/LP spectrally soften only wet room; compare consonant intelligibility, DRR as measured against current impl |
| Real singer still robot | Keep depth independent; this architecture doesn't solve formant/grain synthesis; P0 resynthesis follows separate proof |
| Device restart / session | `prepare(rate)` resets room/filters and restarts deterministic state, no stale room tail from previous device; maintain GUI native ACK identity |

## No shortcut quality declaration
Do not equate "FDN compiled" with “feels like three people metres away.” First require baseline/updated dry and backing stems rendered at level parity, mono/stereo blind A/B for perceived front-back ranking, listening through nearfields/headphones/mono laptop speaker, and actual ASIO 48/96 kHz 64/128/256 buffers with full mixer FX rack and CPU p95/p99/XRUN data. The desired response is **distant but distinct, not washed out**. Exact room dimensions and HRTF positioning are not implemented and must not be shown as facts in GUI.

## 2026-11-09 implementation checkpoint D0–D1 (unqualified acoustically)

D0 implemented: `HarmonyDsp::leadPath` no longer references Wet; Pitch Tune still explicitly blends corrected lead. Direct backing attenuation is separate from dry lead. D1 implemented: `EnsembleSpace::shadeBacking` adds a rate-correct gentle LP shading to **backing only**; `BackingDepthRoom` adds a 4-branch fixed-delay, precomputed 0.55 s nominal FDN with frequency damping and non-anti-phase stereo outputs. Space smooths original pan, backing attenuation, early-field gain, late send and return. No new controls/schema or device calls; no extra plugin PDC.

Added CTest gates for 32/44.1/48/96/192 kHz impulse/tail feedback stability, reset/silence/mono compatibility, backing HF-shading boundedness and explicit pitch-tune lead preservation. This is an acoustically *unqualified* room candidate until real listening A/B and physical ASIO load.

## 2026-11-09 D2 — backing-stage acoustic depth candidate

Field feedback: the pitch-synchronous singer is now acceptably natural but backing appears too close to the original singer. Keep original lead and pitch resynthesis unchanged. New `HarmonyStageSpatial.h` introduces a fixed 0.22ms far-ear precedence cue, a **second per-voice early reflection** at 27–38ms (first remains 12–20ms), and stronger direct/reverberant separation. Existing ONE shared backing-only 4x4 FDN remains unchanged.

With Space `s` in [0,1], direct gain is `1-0.62s`, early send `0.49s`, late return `1.08s`. Positive-polarity stereo image; no exact HRTF or claimed physical metres, no extra audio latency or device, no delay-time modulation, no lead processing changes. An offline-only switch retains old spatial routing to compare against the current pitch engine. The one-knob GUI remains unchanged.

Test `NativeHarmonyBackingStageSpatialDepthAndMonoPDC` validates 32–192k, monotonic early/direct energy, three/five singer first-arrival and mono energy, 64/257 callback parity, declared 1536-sample PDC and Wet0 lead invariance. Aural user validation and physical ASIO xrun still required.

## 2026-11-09 P1 spatial refinement: frequency-dependent far-ear cue

After the stage placement milestone, the fixed 0.22ms far-ear precedence cue used the *same full-band vocal* for near and far ears, so lateral placement could still resemble ordinary pan/doubling rather than a believable backing position. The stage now precomputes one gentle 4.1kHz one-pole for the **contralateral-only** short-delay ring. The near-ear direct path, lead PCM, source pitch-synchronous resynthesis, fixed sample PDC, per-singer reflection timing, shared room, and existing Space control are unchanged. No speculative physical 3D/HRTF distance claims.

At Space 0 the far-ear blend is zero; at higher Space, far ear combines a proportion of the original with a short delayed, spectrally softened replica. Fixed delay and preprepared coefficient prevent audio-control chirps. This is a small independent cue, not new unbounded spectral processing. Synthetic tests compare 350Hz / 8kHz far-to-near energy at 48/96/192k, preserve full-band Space0 parity, plus existing per-singer reflection/DRR, dry polarity, mono and callback-block parity. Real singer/headphones/mono A/B still gate acceptance.

### 2026-11-09 center backing separation

The fixed short far-ear shadow cue now combines with a modest, **center-only backing direct** recess. With three singers the middle backing singer can otherwise overlap the dry center-lead image. At Space 1 its center direct level is 0.83 of its original while its reflected and one-room-send components remain unchanged. The effect tapers with physical stage direction and Space (fully neutral at Space 0), preserves side singer direct energy and never changes dry lead, pitch/formant engine or published PDC. Impulse regressions validate actual center direct/room ratio, no full muting or polarity tricks. Subjective quality is still an ear-test.


## 2026-11-09 D3 — conservatively focused musical backing room

`HarmonyRoomFocus.h` is a shared 170Hz high-pass on the BACKING ROOM SEND
only, and a 12ms attack / 240ms release lead-envelope follower that
modulates only the late room return (max 22% attenuation at Space=1).
Its input is the original latency-aligned dry lead, not a second channel.
During sustained lead, late diffusion is gently eased back; between
phrases the late field blooms again. Direct singers, all first reflections,
lead dry PCM and pitch/formant candidate are unaffected. No FFT, lookahead,
extra delay/PDC, GUI knob, saved schema change or realtime allocation.

Native regression measures the 75/600Hz frequency discrimination at
32/44.1/48/96/192k, bounded attack/release, Space0 unity and restart
clearing. Offline A/B adds `10_room_focus_off_space_060.wav` and
`11_room_focus_off_backing_only.wav` with the SAME stage and pitch
to isolate the room change. Synthetic tests are not a real singer or
physical ASIO qualification.


## 2026-11-09 D4 field fix — harmony wet presence and knob optical spacing

A recorded owner listening session found pitch quality noticeably human,
but backing was too thin even with 4 voices, 100% Wet and Space 60%.
`HarmonyDsp` previously applied `0.65*Wet/sqrt(voices)`, followed by
the existing Space direct attenuation. The new **control-thread** voice
law `Wet*(0.77+0.38*Wet)/sqrt(voices)` raises wet backing roughly
+3.2 dB at Wet45% and +4.95 dB at Wet100%, without ducking the original
dry lead or moving spatial placements. Wet0 is exactly zero; 2–5 voices
retain sqrt-count normalization. This is not a master limiter, and no
physical hardware headroom or singing quality is implied by the formula.
An offline-only before-process() switch produces the prior presence law
for matched-A/B WAV `12_presence_old_space_060.wav` and
`13_presence_old_backing_only.wav`. Synthetic self-test asserts backing
stem RMS increases in the expected bounded range and dry lead parity.

The screenshot showed COLOR/SPACE/WET/GAIN overlapping the glowing ring:
the pseudo ring extends 11px below the 68/59px knob but the old label
was separated by only a 1px gap. Add a compact 12px top margin and 11px
line height to the caption; preserve all editor dimensions, dial size,
pointer and parameter meanings. Chromium geometry checks real caption
clearance and value noncollision across existing test viewports.
Loading
Loading