Repository navigation
Make PersonaPlex CPU speech generation realtime - #113
Conversation
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 3424c40. Configure here.
There was a problem hiding this comment.
Pull request overview
This PR refactors PersonaPlex CPU speech generation toward realtime by moving parallel matmul and attention execution into kernel-owned Stateforward.SML actors, adding new CPU-native kernel actor surfaces (matmul, temporal attention, f32 matvec), and introducing a bounded speech wavefront/frame orchestration that joins all child work inside the owning RTC boundary. It also hardens “threads” into a strict CPU budget across owner/stage/matmul workers and standardizes Zig/macOS toolchain targeting across maintained scripts.
Changes:
- Move parallel matmul ownership from
emel::text::generator::matmulintoemel::kernel::matmul, updating tools/tests to use the new lane pool + execution policy API. - Add kernel actor surfaces for Moshi temporal attention and f32 matvec (including new benches/tests) and extend Mimi initialization with a pre-dispatch bind contract + multi-route capacity/bind decisions.
- Update build/bench/parity/determinism/docs scripts to source shared Zig/macOS target configuration and to clamp build parallelism via
EMEL_BUILD_JOBS.
Reviewed changes
Copilot reviewed 128 out of 131 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tools/paritychecker/parity_engines.cpp | Switch paritychecker generator wiring to emel::kernel::matmul lane pool/policy. |
| tools/embedded_size/emel_probe/main.cpp | Switch embedded-size probe generator wiring to emel::kernel::matmul. |
| tools/docsgen/docsgen.cpp | Ensure rendered templates end with a newline. |
| tools/determinism_check/main.cpp | Switch determinism tool generator wiring to emel::kernel::matmul. |
| tools/decode_wavefront_eval/main.cpp | Switch decode wavefront eval wiring to emel::kernel::matmul. |
| tools/bench/speech/mimi_emel_parity_runner.cpp | Update Mimi initialize to include precomputed bind contract. |
| tools/bench/speech/codec_mimi_bench.cpp | Update Mimi bench fixture init to include bind contract; formatting tweaks. |
| tools/bench/model/tensor/window_streaming_bench.cpp | Switch bench generator wiring to emel::kernel::matmul. |
| tools/bench/kernel/f32_matvec_bench.cpp | New bench for f32 matvec reference vs exact-x4 (AArch64). |
| tools/bench/generation_bench.cpp | Switch generation bench to emel::kernel::matmul. |
| tools/bench/dependency_manifest.txt | Update manifest dependency from generator matmul detail to kernel matmul actor. |
| tools/bench/CMakeLists.txt | Force GGML_METAL OFF for all CPU-only benchmark reference lanes. |
| tools/bench/bench_dependency_manifest.cpp | Update compiled dependency record to kernel matmul actor path. |
| tests/text/generator/stream_window_tests.cpp | Switch generator test rigs to emel::kernel::matmul lane pool/policy. |
| tests/text/generator/determinism_tests.cpp | Switch row-slice utilities and matmul actor to kernel-owned matmul; update invariance test wiring. |
| tests/text/generator/detail_tests.cpp | Switch matmul actor fixture to kernel matmul; update diagnostics assertions path. |
| tests/text/generator/action_guard_tests.cpp | Switch include/types to kernel matmul; set backend lane mode explicitly. |
| tests/speech/predictor/moshi_executor_streaming_tests.cpp | Inject kernel matmul dependency into moshi executor tests. |
| tests/sm/sm_policy_tests.cpp | Add fork/join lane pool batch submission and worker-budget tests. |
| src/emel/text/generator/sm.hpp | Bind emel::kernel::matmul::sm and propagate lane mode into generator context. |
| src/emel/text/generator/prefill/guards.hpp | Gate parallel prefill on explicit kernel matmul lane mode. |
| src/emel/text/generator/matmul/sm.hpp | Remove old text-generator-owned matmul actor. |
| src/emel/text/generator/matmul/guards.hpp | Remove old text-generator-owned matmul guards. |
| src/emel/text/generator/matmul/events.hpp | Remove old text-generator-owned matmul events. |
| src/emel/text/generator/matmul/detail.hpp | Remove old text-generator-owned matmul lane pool/policy/detail helpers. |
| src/emel/text/generator/matmul/context.hpp | Remove old text-generator-owned matmul context + lane storage. |
| src/emel/text/generator/matmul/actions.hpp | Remove old text-generator-owned matmul actions. |
| src/emel/text/generator/layer/sm.hpp | Switch matmul lane-mode template parameter to kernel matmul lane mode. |
| src/emel/text/generator/initializer/actions.hpp | Pass matmul lane mode into backend prepare path. |
| src/emel/text/generator/guards.hpp | Gate decode parallel lanes on explicit kernel matmul lane mode. |
| src/emel/text/generator/context.hpp | Update generator context to kernel matmul actor pointer + lane mode field. |
| src/emel/text/generator/actions.hpp | Capture kernel matmul diagnostics and fold them into generator diagnostics totals. |
| src/emel/speech/predictor/moshi/executor/guards.hpp | Add explicit matmul-route guards and attention-lane route selection guards. |
| src/emel/speech/predictor/moshi/executor/context.hpp | Add kernel matmul + attention lane pool dependencies; allocate attention actor storage once. |
| src/emel/speech/generator/frame/guards.hpp | New frame-level generator guards (validation, child outcomes, callbacks). |
| src/emel/speech/generator/frame/events.hpp | New frame-level generator events and done/error outcome types. |
| src/emel/speech/generator/frame/errors.hpp | New frame-level generator error enum. |
| src/emel/speech/generator/frame/context.hpp | New frame-level generator context/dependencies concept and runtime ctx structs. |
| src/emel/speech/generator/frame/any.hpp | Add top-level SpeechGeneratorFrame alias. |
| src/emel/speech/generator/events.hpp | Add wavefront events, attribution structs, and wavefront runtime ctx types. |
| src/emel/speech/generator/errors.hpp | Extend speech generator errors for wavefront submit/attribution/drain/middle paths. |
| src/emel/speech/generator/context.hpp | Add wavefront generator mode, dependencies concept, stage pool types, and lane storage. |
| src/emel/speech/generator/any.hpp | Expose wavefront mode alias. |
| src/emel/speech/codec/mimi/sm.hpp | Split bind capacity/bind actions per route; add diagnostics capture event handling. |
| src/emel/speech/codec/mimi/events.hpp | Introduce opaque bind_contract + capture_diagnostics; require contract in initialize. |
| src/emel/speech/codec/mimi/detail.hpp | Add f32_matvec actor in runtime; add typed contract validation + typed bind entrypoints. |
| src/emel/speech/codec/mimi/any.hpp | Add public make_bind_contract() helper; formatting tweak. |
| src/emel/speech/codec/mimi/actions.hpp | Add typed bind actions (f32/native/q8) and codec diagnostics capture effect. |
| src/emel/sm.hpp | Extend fork/join lane pool with runtime worker budget + batch submission API. |
| src/emel/machines.hpp | Export new kernel machines (KernelAttention, F32Matvec, KernelMatmul). |
| src/emel/kernel/matmul/events.hpp | New kernel matmul actor event surface (serial/parallel, diagnostics, callbacks). |
| src/emel/kernel/matmul/detail.hpp | New kernel matmul lane_pool alias, lane_mode, execution_policy, and slice helper. |
| src/emel/kernel/matmul/context.hpp | New kernel matmul context with construction-time lane storage allocation. |
| src/emel/kernel/f32_matvec/events.hpp | New f32 matvec actor event surface + diagnostics. |
| src/emel/kernel/f32_matvec/context.hpp | New f32 matvec actor context counters. |
| src/emel/kernel/f32_matvec/any.hpp | New f32 matvec any header include. |
| src/emel/kernel/events.hpp | Move kernel_kind/detect_host_kind here; add generic kernel diagnostics events. |
| src/emel/kernel/attention/sm.hpp | New kernel attention actor state machine. |
| src/emel/kernel/attention/guards.hpp | New attention request validation (shape/alignment/disjoint ranges). |
| src/emel/kernel/attention/events.hpp | New attention execute request/result events. |
| src/emel/kernel/attention/context.hpp | New attention reusable workspace context. |
| src/emel/kernel/attention/any.hpp | New attention any header include. |
| src/emel/kernel/attention/actions.hpp | New head-range attention execution implementation + callback emission. |
| src/emel/kernel/any.hpp | Add configure_kind and capture_diagnostics handling; remove duplicated kernel_kind definition. |
| src/emel/kernel/aarch64/actions.hpp | Tighten NEON compile-time gating for q4_k_x8 scale decode helper. |
| snapshots/quality_gates/timing.txt | Update recorded gate timing snapshot. |
| snapshots/lint/clang_format.txt | Update clang-format snapshot list (added/removed files). |
| scripts/zig_toolchain.sh | New shared Zig/macOS CMake args for explicit deployment target/SDK/framework linkage. |
| scripts/test_with_coverage.sh | Make coverage compilers configurable; update tool checks; improve comments. |
| scripts/setup_whisper_cpp_reference.sh | Source zig_toolchain and propagate platform args for Zig builds. |
| scripts/setup_personaplex_mlx_reference.sh | Add probe/retry logic around Python import; avoid reinstall unless needed. |
| scripts/paritychecker.sh | Source build_jobs + zig_toolchain; clamp build parallelism; propagate platform args. |
| scripts/generate_docs.sh | Source build_jobs + zig_toolchain; clamp build parallelism; propagate platform args. |
| scripts/embedded_size.sh | Source zig_toolchain; pass platform args when using Zig. |
| scripts/check_macos_binary_target.sh | New macOS min-OS verification helper via otool. |
| scripts/check_determinism.sh | Source zig_toolchain; pass platform args. |
| scripts/build_with_zig.sh | Source zig_toolchain; pass platform args; verify macOS binary min-OS and launch. |
| scripts/bench.sh | Source zig_toolchain; pass platform args for Zig bench builds/compare/run-only flows. |
| scripts/bench_whisper_single_thread.sh | Source zig_toolchain; pass platform args for Zig builds. |
| scripts/bench_whisper_compare.sh | Source zig_toolchain; pass platform args for Zig builds. |
| scripts/bench_personaplex_compare.sh | Treat threads as 1/2/4/8 budget; improve Python selection; pass platform args; refactor invocation. |
| scripts/bench_moshi_lm_compare.sh | Source zig_toolchain; pass platform args for Zig builds. |
| scripts/bench_mimi_compare.sh | Switch compare build dir; source zig_toolchain; require zig; configure Zig toolchain + platform args. |
| scripts/bench_generation_reference_llama_cpp.sh | Source zig_toolchain; pass platform args for Zig builds. |
| scripts/bench_generation_compare.sh | Source zig_toolchain; pass platform args for Zig builds. |
| scripts/bench_embedding_reference_liquid.sh | Source zig_toolchain; pass platform args for Zig builds. |
| scripts/bench_embedding_compare.sh | Source zig_toolchain; pass platform args for Zig builds. |
| scripts/bench_diarization_compare.sh | Source zig_toolchain; pass platform args for Zig builds. |
| README.md | Update architecture index: add kernel actor docs; remove generator matmul doc. |
| goal.md | Update matmul actor naming to kernel-owned matmul. |
| docs/templates/README.md.j2 | Mark C++ implementation as discontinued in favor of emel.rs. |
| docs/determinism.md | Update determinism documentation to include new parallel surfaces and kernel matmul ownership. |
| docs/benchmarking.md | Document PersonaPlex Mimi selective Q8_0 artifact and current same-WAV input-code match. |
| coroutine-plan.md | Update test reference to new kernel matmul test path. |
| CMakeLists.txt | Update test list: remove old parallel matmul tests; add new kernel/attention/f32_matvec/matmul and speech wavefront/frame tests. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3424c40f33
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 24537c8f36
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d56cfc5f6e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fb4ab721d6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Codex Review: Something went wrong. Try again later by commenting “@codex review”. ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
Codex Review: Something went wrong. Try again later by commenting “@codex review”. ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e4ce5d73c0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Codex Review: Didn't find any major issues. Swish! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Summary
CPU performance
On the maintained 250-frame workload, the final requested-total-eight run used exactly:
It generated 20 seconds of speech in 14.7401 steady-state seconds: RTF 0.7370, or about 1.36× realtime. The output WAV remained bit-identical to the prior accepted output (
e030b8ba…d9).Verification
Known residual
The pinned moshi.cpp Mimi token-exact comparison remains 0.953125. The same frame/token pattern was reproduced with a preserved pre-optimization EMEL binary, so this is recorded as a pre-existing semantic-exactness gap rather than a regression from this PR. The bounded MLX comparison passes. A later scoped wrapper run also could not perform its redundant reference fetch because sandbox DNS blocked GitHub; the already-built maintained runner and hard-budget benchmark completed successfully.
This PR does not claim startup/prefill realtime, hard per-frame deadlines, full semantic parity, GPU/Metal performance, release, or deployment.
Note
Low Risk
Changes are planning/architecture documentation and diagram sync only; no executable code in this diff.
Overview
Updates
.planning/architecture(and matching.mermaidsources) so generated state diagrams match the refactored runtime actors described in the PR.New kernel documentation covers shared
kernel_matmul,kernel_f32_matvec, andkernel_attentionmachines (execute/prepare paths, parallel lane guards, done/error callbacks, diagnostics). The oldtext_generator_matmulmermaid diagram is removed in favor of the kernel-owned matmul actor.Logits sampler diagrams now branch temperature–top‑k completion on
temperature_top_k_selected_token_valid_vs invalid (invalid goes toerroredviamark_invalid_request_) instead of always succeeding after select.Speech codec Mimi init splits contract binding into f32 / native / q8 capacity decisions with matching bind effects, and adds
capture_diagnosticsself-loops from uninitialized and session-ready.Moshi predictor executor docs reflect serial vs parallel projection and logits matmul guards, multi-route temporal attention (1/2/4/8), an explicit invalid forced-text-token error path, and parallel depformer logits sampling.
Moshi tokenizer tail-phase detokenize now commits through
state_commit_generated_*(not full-frame commits), and invalid detokenize guards are renamed toguard_detokenize_request_invalid_.Reviewed by Cursor Bugbot for commit 88757eb. Bugbot is set up for automated code reviews on this repo. Configure here.