Large finished turns can OOM bb-server during timeline projection and cause a restart loop

[vaayne]

Agent-generated report. I asked an agent to investigate and file this from production logs. I reviewed the facts below; private hostnames and thread IDs have been removed.

Summary

bb-server 0.35.1 can exhaust the V8 heap while building the latest timeline for a large thread. One timeline request can materialize and decode tens of megabytes of event JSON before returning a small projected response. In my case this killed the process repeatedly and put the service into a systemd restart loop.

The service recovered once the timeline requests stopped, but the same workload can trigger it again.

Environment

• bb: 0.35.1, npm release
• Node.js: 22.23.2
• OS: Linux arm64
• Storage: SQLite with better-sqlite3
• Service manager: systemd with Restart=on-failure
• BB_FF_TIMELINE_WINDOW_EVENT_BUDGET: unset, so the default applies
• cgroup limits: MemoryHigh=640M, MemoryMax=768M
• Host: 1.8 GiB RAM, 1 GiB swap

What happened

Before the first crash, the server logged repeated event loop stalls during timeline projection. One request had this shape:

{
  "selectionStrategy": "standard-window",
  "pageKind": "latest",
  "segmentLimit": 20,
  "eventRowCount": 1306,
  "eventDataBytes": 29135518,
  "decodedEventCount": 1306,
  "projectedRowCount": 45,
  "responseRowCount": 45,
  "stageTimings": [
    { "stage": "event-query", "durationMs": 713.4 },
    { "stage": "event-json-decode", "durationMs": 597.7 },
    { "stage": "thread-view-projection", "durationMs": 76.6 }
  ]
}

Another request read 5,594 event rows totaling 6.2 MB and returned 15 projected rows:

{
  "selectionStrategy": "standard-window",
  "pageKind": "latest",
  "segmentLimit": 20,
  "eventRowCount": 5594,
  "eventDataBytes": 6228463,
  "decodedEventCount": 5594,
  "projectedRowCount": 15,
  "responseRowCount": 15
}

The process then failed around a 300 MB V8 heap:

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory

The native stacks varied between failures. Some ended in better_sqlite3.node while converting rows from Statement::JS_all; others ended in V8 JsonParser during JSON.parse.

systemd restarted the service after five seconds. Connected clients reconnected and requested timelines again, and the new process often crashed within 15 to 45 seconds. This repeated 11 times. Observed process memory peaks ranged from roughly 426 MB to 520 MB.

Reproduction shape

I do not have a small deterministic fixture yet, but the production trigger is consistent:

1. Create a thread with a large finished turn containing many tool events or large event payloads.
2. Open that thread so the client requests the latest timeline.
3. Restart the server, or let it restart while clients remain connected.
4. The server rebuilds the timeline, materializes a large event set, stalls the event loop, and can exhaust the V8 heap.
5. Reconnecting clients can turn the failure into a restart loop.

The logs showing 5,594 selected events are also surprising with the default timeline event budget. If a finished turn intentionally bypasses that budget, a single large turn still needs a byte or memory ceiling.

Expected behavior

A single thread or timeline request should not terminate the server. Oversized turns should be paginated, truncated with an explicit continuation, or rejected with a bounded per-request error that leaves the process healthy.

Suggested investigation

• Apply event count and byte limits before Statement.all() materializes all selected rows.
• Bound JSON decoding and projection by bytes, not only event count.
• Make the limit cover a single oversized finished turn.
• Avoid re-running the same fatal timeline projection immediately after clients reconnect.
• Add a regression fixture with a finished turn containing large tool payloads and assert that the latest timeline request stays within a fixed memory envelope.

I can provide more anonymized journal output or inspect the SQLite event distribution if that would help narrow down the query path.