fix(agent,memory): refuse session JSON truncation that wipes history

[cursor]

Summary

When serialized session JSON exceeded SESSION_JSON_MAX (128 KiB), append_exchange_to_session_json / compact_session_via_llm silently clipped mid-JSON and still returned success, so session_save persisted a corrupt payload. The next agent_run failed to parse the session and treated history as empty (permanent wipe on the following save). session_load had the same clip-into-invalid-JSON path.

Rewritten on current origin/development (includes #59 and #63). Do not cherry-pick the original Bot commit.

Trigger

Near-cap session (~124 KiB of valid JSON) plus a fat assistant reply (session cap is 128 KiB). Reproduced with a regression test; pre-fix truncation yields unparseable JSON.

Fix

• append_exchange_to_session_json / compact_session_via_llm: return -1 instead of truncating (skip session_save)
• session_load: reject blobs that do not fit the caller buffer instead of clipping

Test plan

• make test_agent (includes test_session_overflow_does_not_corrupt_history)
• make test_memory (includes test_session_load_rejects_oversized)
• CI=true make test
• make static

---

Note

Medium Risk
Changes session load/save semantics at the 128 KiB boundary—turns may not persist when over cap, but that avoids corrupt JSON and accidental history wipes in SQLite.

Overview
Fixes a data-loss bug where session history could be permanently wiped when serialized JSON hit the 128 KiB cap.

Load path: session_load no longer clips oversized blobs into invalid JSON. It returns SESSION_LOAD_TOO_LARGE and leaves the output buffer empty so callers can tell “too big” apart from “missing.”

Save path: append_exchange_to_session_json and LLM compaction now use copy_printed_session_json, which refuses mid-JSON truncation instead of saving corrupt payloads. If load failed with SESSION_LOAD_TOO_LARGE, agent_persist_session skips session_save so a later small turn cannot overwrite the oversized row with a fresh empty/history-less JSON.

Regression coverage in test_agent (overflow + oversize-load) and test_memory (test_session_load_rejects_oversized); CHANGELOG updated.

^{Reviewed by Cursor Bugbot for commit 1b54efd. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Stale comment

Signals are clean: Cursor Bugbot and Cursor Security Agent both passed with no findings that need human review. GitHub will not accept an approval from this automation on a PR it authored, so this is a non-blocking comment only. adriannoes is already requested as reviewer; no additional reviewers were assigned.

  

_{Sent by Cursor Approval Agent: Pull Request Router and Approver}

[cursor]

Summary

This is a correct fail-closed fix for a real history wipe. When serialized session JSON went over SESSION_JSON_MAX (128 KiB), append_exchange_to_session_json and compact_session_via_llm clipped with len = size - 1, session_save stored invalid JSON, the next cJSON_Parse treated history as empty, and the following persist wrote a fresh array. session_load had the same clip. The PR refuses the copy instead, skips save, and adds a regression that reproduces the near-cap + fat-reply trigger. Matches the #61 pattern. No merge blockers.

Must Fix

None. Size checks use n >= cap then memcpy(..., n + 1), so the NUL fits. On refuse, printed is cJSON_freed and the caller buffer is left untouched. agent_persist_session already skips session_save when append returns non-zero. test_session_overflow_does_not_corrupt_history seeds ~124 KiB of valid JSON, forces a 16 KiB reply, then asserts the stored blob still parses and the next agent_run still sees the marker. test_session_load_rejects_oversized covers the small-buffer load path. I ran test_memory locally (all passed). test_agent could not be linked in this image (router.c needs libcurl headers). GitHub static, test, release was still pending at review time.

Should Fix

1. Silent persist skip. agent_run still returns 0 when append/compact refuse, with no log. Channel dispatch uses a 32 KiB response buffer; ASAP task.request uses 256 KiB. A single ASAP reply cannot fit in the 128 KiB session blob, even on an empty session, so that turn is never written. Log session id and len vs cap when refusing.
2. session_load overflow is indistinguishable from a missing row. It zeros the out buffer, then returns -1. agent_prepare_context (around line 431) ignores the rc. session_save still has no cap. A valid oversized row (not produced by current agent writers) would look empty and be overwritten by the next small persist. Distinguish overflow from not-found, or skip persist when load failed for a present row.
3. Compaction is message-count only (msg_count > max_ctx). The new overflow test sets max_context_messages = 40 with two huge messages, so compact never runs. Near-cap sessions with few large messages will freeze: later turns keep skipping save. Size-based drop-oldest (or compact-until-it-fits) is the real product follow-up; out of this PR's integrity fix.

Nice to Have

• A compact-path overflow test (the plen >= session_buf_size branch has no direct coverage).
• Assert the fat reply (RRR...) is absent from the saved blob, so a "raise the cap and save anyway" change cannot sneak through. Today LOAD_CAP (130 KiB) already fails session_load if a ~140 KiB blob were stored, which is a decent implicit check.
• Reuse spy_provider for the second turn instead of cloning it as keep_history_provider.
• session_load in memory.h still talks about the implementation allocating messages_out; it does not mention oversized-buffer rejection.

Positive Highlights

• Comments state why truncation is fatal (next parse wipes history), not just what the if does.
• Both serialize sites were fixed, not only append. Load refuse matches.
• The agent regression is a real trigger (near-cap JSON + large reply), then a second turn that spies provider messages. That would have failed on the old clip-and-save path.
• CHANGELOG Unreleased records the behavior change.

  

_{Sent by Cursor Automation: Adrianno’s personal code review}

[cursor]

Signals are clean: Cursor Bugbot and Cursor Security Agent both passed with no findings that need human review. GitHub will not accept an approval from this automation on a PR it authored, so this is a non-blocking comment only. adriannoes is already requested as reviewer; no additional reviewers were assigned.

  

_{Sent by Cursor Approval Agent: Pull Request Router and Approver}