[BUG] Dictation model (~3.4 GB) stays pinned in wired memory and never unloads, freezing 8 GB Macs

[cianhammo]

Describe the bug

On an 8 GB Mac, dictating loads the model (fluid-1-q4_k_m.gguf, 3.43 GB) into wired, non-pageable memory and keeps it there for the entire time the app is open. It is not released between dictations or when idle, only when the app is quit.

This pins roughly 43% of total RAM as unswappable memory. The system is then forced into heavy swap, causing severe lag, beachballing, and hard freezes that require a force-restart.

Because the model is wired for GPU/Neural-Engine inference, this memory does NOT appear against the FluidVoice process in Activity Monitor's normal Memory (RSS) column. It only shows as system-wide "wired" memory. That makes it very hard for users to attribute the slowdown to FluidVoice. In my case it presented as "5-6 GB wired with no visible process using it," which cost two days of misdiagnosis. Measuring wired memory (top / vm_stat) before and after quitting the app is the reliable way to observe it.

Reproduction steps

1. Use a Mac with 8 GB RAM. Note baseline wired memory: top -l 1 | grep PhysMem (mine: ~1.3 GB wired, ~69% free with the app closed).
2. Launch FluidVoice. Wired rises to ~1.9 GB (model not yet loaded).
3. Dictate once.
4. Observe wired memory jump to ~5.8 GB in a single step, at the moment of dictation (the 3.43 GB model + inference overhead being wired).
5. Stop dictating and leave the app idle. Wired stays at ~5.6-5.8 GB indefinitely. It does not drop between dictations.
6. Quit FluidVoice. Wired immediately drops back to ~1.3 GB and free RAM returns to ~69%.

The swing tracks the app in both directions and matches the model size, confirming the model is held in wired memory for the app's lifetime.

Expected behavior

The model should be released from memory after a period of idle (e.g. N minutes after the last dictation), or there should be a smaller-model option and/or a low-RAM warning. Wired memory should not remain elevated for the entire app session.

Actual behavior

Wired (non-pageable) memory jumps by ~3.9 GB the instant dictation starts and stays elevated for the entire app session. It is not released when idle or between dictations, only when FluidVoice is quit.

On an 8 GB Mac this leaves too little RAM for macOS and other apps, so the system swaps constantly, becomes severely laggy, beachballs, and eventually freezes hard enough to require a force-restart. FluidVoice itself does not crash — the whole system is starved of memory. Quitting FluidVoice immediately restores normal memory and performance.

App Version

1.6.1 (build 12)

macOS Version

15.5 (24F74)

Architecture

Apple Silicon

Logs or crash report

Evidence — wired (non-pageable) memory measured with `top -l 1 | grep PhysMem`:

FluidVoice closed:            wired ~1.3 GB   (free 69%)
FluidVoice open, idle:        wired ~1.9 GB   (model not yet loaded)
After ONE dictation:          wired ~5.8 GB   (free ~9%)  <-- +3.9 GB, single step
Idle for 2 min after:         wired held 5.5–5.8 GB, no decrease
FluidVoice quit:              wired dropped to ~1.3 GB   (free back to 69%)

Raw 10s sampling around the dictation (wired MB):
19:58:22  1869   (idle)
19:58:37  5810   <-- dictated; model loaded
19:58:50  5796
19:59:15  5599
20:00:12  5548   (still elevated, idle)

Config: FluidVoice 1.6.1 (build 12), macOS 15.5 (24F74), MacBook Air M2, 8 GB RAM.
Model: fluid-1-q4_k_m.gguf (3.43 GB).



=== vmmap: the model mapped into the FluidVoice process ===
mapped file  300f14000-3cc518000  [ 3.2G  44.6M  0K  0K]  r--/r-x SM=SHM
    /Users/cian/Library/Application Support/FluidIntelligence/Models/fluid-1-q4_k_m.gguf
mapped file  3f4d04000-3f9b48000  [ 78.3M 77.0M  0K  0K]  r--/r-x SM=SHM
    .../Models/MTP/gemma-4-E2B-it-Q8_0-MTP.gguf

=== vmmap -summary (FluidVoice), key line ===
mapped file                 3.5G   ...     (total mapped model/file memory)
TOTAL                       6.8G   ...

=== vm_stat at load (16 KB pages) ===
Pages wired down: 230474   -> ~3.6 GB wired, climbing to ~5.7 GB total system wired

=== system wired memory, measured with `top -l1 | grep PhysMem` ===
FluidVoice quit:   wired 1.36 GB, free 72%
FluidVoice + one dictation: wired 5.74 GB, free ~9%
Quit again:        wired 1.36 GB, free 72%

Config: FluidVoice 1.6.1 (build 12), macOS 15.5 (24F74), MacBook Air M2, 8 GB RAM.

Screenshots or screen recording

No response

[altic-dev]

Thanks for the report. How long did you wait for? It does offload after a certain time imo. Let me double check!

Building a smaller one for 8gb or so macs! Should fix this issues nevertheless

[cianhammo]

Thanks for the report. How long did you wait for? It does offload after a certain time imo. Let me double check!

It was open overnight, so this was a long idle test already. After a reboot the model loaded on my first dictation (wired memory jumped ~1.9 → ~5.8 GB), and it then stayed at ~5.6-5.8 GB continuously for 10+ hours idle overnight, with no dictation in that window. It never offloaded. Wired only returned to ~1.3 GB when I quit the app (confirmed repeatedly: quit → 1.3 GB, dictate → 5.8 GB). It happens every time I use the app now (even after the latest update)

[devzahirul]

I dug into this — the offload never happens because it isn't implemented: ASRService caches every provider instance and only releases them on app quit or model switch. There is no time-based unload anywhere for speech models (only the Private AI LLM runtime has one), and the audio-engine standby timer only retires the capture engine, not the model. Combined with the startup auto-load, the model stays wired from launch until quit, which matches your measurements exactly.

Opened #619 with a fix: the model unloads after a configurable idle window (default 30 min, Never available in Settings → Voice Engine) and reloads transparently through the existing ensureAsrReady() path on the next dictation. Verified on a Release build — the unload fires on schedule and the memory is returned while the app keeps running.