Performance: reduce GC and CPU overhead in AudioSegmentProcessor

[ysdede]

What changed:
Replaced the speechEnergies and silenceEnergies arrays in AudioSegmentProcessor.ts with O(1) running scalar sum and count variables (speechEnergySum, speechEnergyCount).

Why it was needed (bottleneck evidence):
During high-frequency audio chunk processing loops, dynamically pushing to arrays (this.state.speechEnergies.push(energy)) and using .reduce (this.state.speechEnergies.reduce((a, b) => a + b, 0) / this.state.speechEnergies.length) introduces unnecessary CPU and Garbage Collection (GC) overhead. silenceEnergies was also being appended to but never actually read or used for any calculations.

Impact:
Eliminates intermediate array allocations and .reduce() iteration per segment, providing a small but measurable reduction in memory allocations and CPU overhead in the VAD audio loop.

How to verify:
Run the test suite using npm run test and observe that AudioSegmentProcessor behaviors remain unchanged while no regressions occur. Build using bun run build.

---

PR created automatically by Jules for task 16316793501974944230 started by @ysdede

Summary by Sourcery

Optimize AudioSegmentProcessor speech energy tracking to reduce CPU and GC overhead during VAD processing loops.

Enhancements:

• Replace speech energy array accumulation with running sum and count to compute average speech energy in O(1) per chunk.
• Remove unused silence energy tracking and related state to simplify processor state and avoid unnecessary allocations.

Summary by CodeRabbit

• Refactor
  • Optimized internal audio segment processor's energy aggregation calculations.

[google-labs-jules]

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.

---

For security, I will only act on instructions from the user who triggered this task.

[qodo-code-review]

Review Summary by Qodo

Reduce GC and CPU overhead in AudioSegmentProcessor

✨ Enhancement

Walkthroughs

Description

• Replace array-based energy tracking with O(1) scalar sum/count variables
• Eliminate unnecessary array allocations and .reduce() iterations
• Remove unused silenceEnergies array that was never read
• Reduce GC and CPU overhead in high-frequency audio processing loop

Diagram

flowchart LR
  A["Array-based tracking<br/>speechEnergies[]<br/>silenceEnergies[]"] -->|"Replace with"| B["Scalar variables<br/>speechEnergySum<br/>speechEnergyCount"]
  B -->|"Benefits"| C["O(1) operations<br/>No allocations<br/>Lower GC pressure"]

Loading

File Changes

1. src/lib/audio/AudioSegmentProcessor.ts ✨ Enhancement +12/-15

Replace array-based energy tracking with scalar variables

• Replaced speechEnergies and silenceEnergies arrays with speechEnergySum and
 speechEnergyCount scalar variables in ProcessorState interface
• Updated energy tracking logic to use addition and increment instead of array push operations
• Changed average energy calculation from .reduce() to simple division of sum by count
• Removed unused silenceEnergies array initialization and accumulation
• Updated startSpeech() and startSilence() methods to initialize scalar variables instead of
 arrays
• Updated reset() method to initialize scalar variables to 0

src/lib/audio/AudioSegmentProcessor.ts

---

[coderabbitai]

📝 Walkthrough

Walkthrough

The AudioSegmentProcessor's speech energy tracking during silence-in-speech handling was refactored from storing arrays of individual energy values to maintaining running aggregates (sum and count). Average speech energy is now computed from these aggregates instead of array reduction, with corresponding updates to state initialization and reset logic.

Changes

Cohort / File(s)	Summary
Audio Energy Accumulation Refactoring src/lib/audio/AudioSegmentProcessor.ts	Replaced array-based energy storage (speechEnergies, silenceEnergies) with running aggregates (speechEnergySum, speechEnergyCount). Updated silence-in-speech handling to accumulate energy as sums instead of arrays, compute average energy via sum/count division, and adjusted state initialization/reset accordingly.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

• refactor(audio): harden speech stats cache safety #173: Modifies speech statistics aggregation and summary computation in AudioSegmentProcessor.ts, directly related to how energy data is processed and accumulated.

Poem

🐰 Arrays of energy fade away,
Running sums now lead the day,
Sum and count in harmony dance,
Silence-speech averages prance,
Cleaner code, a lighter way! ✨

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'Performance: reduce GC and CPU overhead in AudioSegmentProcessor' clearly and specifically describes the main change: replacing arrays with scalar variables to reduce garbage collection and CPU overhead in the AudioSegmentProcessor component.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch perf/audio-processor-gc-overhead-16316793501974944230

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[qodo-code-review]

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0)

Great, no issues found!

Qodo reviewed your code and found no material issues that require review

ⓘ The new review experience is currently in Beta. Learn more

[sourcery-ai]

Hey - I've left some high level feedback:

• Since speechEnergySum and speechEnergyCount are logically tied to a single speech segment, consider explicitly resetting them when ending speech (e.g., in startSilence or right after computing avgEnergy) to avoid stale state being accidentally reused if future logic reads these fields while inSpeech is false.

Prompt for AI Agents

Please address the comments from this code review:

## Overall Comments
- Since `speechEnergySum` and `speechEnergyCount` are logically tied to a single speech segment, consider explicitly resetting them when ending speech (e.g., in `startSilence` or right after computing `avgEnergy`) to avoid stale state being accidentally reused if future logic reads these fields while `inSpeech` is false.

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[coderabbitai]

🧹 Nitpick comments (1)

src/lib/audio/AudioSegmentProcessor.ts (1)

57-58: Consider: floating-point precision for extremely long segments.

Accumulating many small floating-point values into speechEnergySum can theoretically degrade precision over very long durations. Given the maxSegmentDuration limit and typical chunk rates, this is unlikely to be a practical concern—but worth noting if the processor is ever used without duration limits.

Also applies to: 252-253, 281-282

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@src/lib/audio/AudioSegmentProcessor.ts` around lines 57 - 58, The fields
speechEnergySum and speechEnergyCount can accumulate floating-point error over
extremely long segments; implement compensated summation (e.g., Kahan or
pairwise) for updating speechEnergySum by adding a companion variable (e.g.,
speechEnergyCompensation) and replace direct `speechEnergySum += value` sites
with the compensated-add routine wherever energy is accumulated (look for
methods that update speechEnergySum such as the frame/segment accumulation
functions referenced around lines 252-253 and 281-282), and keep
speechEnergyCount as-is; ensure final energy reads combine sum and compensation
when computing outputs.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@src/lib/audio/AudioSegmentProcessor.ts`:
- Around line 57-58: The fields speechEnergySum and speechEnergyCount can
accumulate floating-point error over extremely long segments; implement
compensated summation (e.g., Kahan or pairwise) for updating speechEnergySum by
adding a companion variable (e.g., speechEnergyCompensation) and replace direct
`speechEnergySum += value` sites with the compensated-add routine wherever
energy is accumulated (look for methods that update speechEnergySum such as the
frame/segment accumulation functions referenced around lines 252-253 and
281-282), and keep speechEnergyCount as-is; ensure final energy reads combine
sum and compensation when computing outputs.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: e8ea2b43-5917-482d-bf73-552958360fa6

📥 Commits

Reviewing files that changed from the base of the PR and between 474dbe6 and 40931cb.

📒 Files selected for processing (1)

• src/lib/audio/AudioSegmentProcessor.ts

[gemini-code-assist]

Code Review

This pull request refactors the AudioSegmentProcessor to optimize energy tracking by replacing the speechEnergies and silenceEnergies arrays with speechEnergySum and speechEnergyCount variables. This change improves memory efficiency and performance by calculating the average energy using a running sum instead of storing and reducing an array of values. I have no feedback to provide.