Skip to content

test(pod): paginable long-history seed scenario (Refs #4100) - #9415

Open
pepmach wants to merge 1 commit into
mainfrom
feat/seam-history-fixture
Open

test(pod): paginable long-history seed scenario (Refs #4100)#9415
pepmach wants to merge 1 commit into
mainfrom
feat/seam-history-fixture

Conversation

@pepmach

@pepmach pepmach commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Problem

No shipped seed scenario can paginate, so a history-paging trace cannot be
performed through the real routes at all.

The slot-detail fast path (api_chat_slot_detail, the no-limit branch) hardcodes
has_more = False and only flips it when read_rotated_messages_chained returns
rows — i.e. only when the session has size-rotated archive/ segments. Rotation
fires only once a transcript outgrows a 10 MiB byte budget while holding more
than 200 lines, so a scenario made of a handful of messages can never reach that
state and "load earlier" stays structurally unreachable.

This is the missing capability named in
#4100's stand-down
("a seed that can actually paginate (a long-history fixture)"). Refs #4100
this PR does not fix that issue, it unblocks verifying it.

Red

Unmodified worktree, pod seeded with the existing sessions-a-few, initial
slot load through the real authenticated route:

GET /api/chat/slots/pinned-work        -> HTTP 200
{"has_more": false, "next_before": 0, "total": 2}
messages returned: 2

has_more=false with next_before=0, so there is no cursor to page with and
the "load earlier" affordance has nothing to reach. Reproduced across the whole
registry: every session in all 13 shipped scenarios returns 0 rotate-archived
rows.

Why the fixture has to be generated, and why it is still checked in

A real rotation splits a >10 MiB transcript into a kept tail and an archived
head, so archive bytes + live bytes always exceed 10 MiB — a static fixture
that simply contains a post-rotation session cannot be small. Fixtures ship as
package data in the wheel and the sdist, and test_pod_seed_scenarios.py caps
each one at 64 KB precisely because every install pays for it.

The way out is to keep only the halves the paging seam needs:

  1. append the 60 rows destined to be rotated away (small, realistic)
  2. append 199 oversized filler rows, staying under the byte budget
  3. append one more, sized so the file crosses 10 MiB while the surviving 200
    lines still fit it — that rotation archives exactly the 60 head rows
  4. rewrite the transcript to empty through the real compaction path, which moves
    the 10 MiB of filler out into a compact segment
  5. append the 8 rows the fixture keeps live, title and pin the session

The fixture ships the rotate segment and the live transcript, both real
ConversationLog output at the production budgets. The compact segment is
discarded — a home reaches that same state once archive retention reclaims it,
and read_rotated_messages ignores compact segments by design. Result: ~20 KB
of package data for a session that genuinely paginates.

No product code changed, and the rotation budgets were not touched or hooked.

Green

Pod booted on this branch, seeded sessions-long-history, real authenticated
route:

GET /api/chat/slots
[{"key": "long-history", "title": "Long history paging", "pinned": true}]

GET /api/chat/slots/long-history                      (initial load, no cursor)
{"has_more": true, "next_before": 60, "total": 68}
messages returned: 8
first: "Follow-up 1: does the live tail still line up with the archived head..."

GET /api/chat/slots/long-history?before=60&limit=20   (follow-up cursor fetch)
{"has_more": true, "next_before": 40, "total": 68}
messages returned: 20
first: "Turn 21: walking the paging seam again - next_before is a collapsed..."
last:  "Turn 30: it reads the legacy pagination path is the only way back..."

GET /api/chat/slots/long-history?before=20&limit=20   (last page)
{"has_more": false, "next_before": 0, "total": 68}
returned: 20 | first: "Turn 1: walking the paging seam again - ..."

The second page is real archived content, and the walk terminates at the true
first message of the corpus instead of retiring at the rotation boundary.

Contract test

test/test_fixture_sessions_long_history.py, two tests, 0.7s:

  • the design point, through the real readers — 8 live rows, 60 rotated rows,
    the full chained corpus equals rotated + live (so the archived rows are a
    contiguous prefix and a single cursor is exact), chain_mid_rotation false,
    the collapsed rotated count that becomes next_before is 60, and the session
    is pinned so it restores as a slot whatever the mtime restore window is.
  • writer provenance — drives a rotation in-test and asserts the shipped
    segment's header key set, reason=rotate, count, and row key set match what
    _maybe_rotate writes today. A change to the rotation format turns this red
    rather than leaving the fixture describing a shape the product no longer
    produces.

The scenario also picks up the existing per-fixture invariants (manifest parses,
seeds into a fresh home, every JSON/JSONL line parses, no credential-shaped
text, under the 64 KB cap).

Scope

In: the scenario, its contract test, and the seed-scenario guide. Out:
chatSlice/ChatPage and anything else on the paging code path — open
#9130 is live in those files
and the traces above ran against this branch with no frontend change.

Gates

pytest -n0 on the new test plus the fixture registry, seed, helper, pod
scenario, sibling sessions-a-few and archive-pagination suites: 202 passed.
flake8 and mypy clean on the touched file. scripts/docs-lint.sh passes.

Pattern harvest

  • A fixture that must be produced by a writer is not automatically a static
    fixture.
    Check the arithmetic of the writer's own trigger before designing
    the tree: here the 10 MiB rotation gate makes "the whole post-rotation session"
    unshippable, while "the halves the seam reads" is 20 KB. The 64 KB cap in
    test_pod_seed_scenarios.py is the load-bearing constraint, and it is a test,
    not a guideline.
  • Discarding a real artifact can be more honest than authoring a small one.
    Running the real compaction path and then dropping its 10 MiB compact
    segment lands a state the product reaches on its own, whereas hand-writing a
    small live tail next to a real archive segment would have paired files no
    single writer run ever produced.
  • Pin generated fixture bytes against the generator, not just against the
    reader.
    Asserting the shipped segment's shape against a rotation driven
    in-test is what keeps "produced by the real writer" true a year from now;
    without it the claim decays into a commit-message assertion.
  • A scenario is only useful if the state survives seeding. The session had to
    be pinned: an unpinned fixture session is admitted to the slot list only while
    it is inside the mtime restore window, so the scenario would have worked when
    freshly seeded and silently produced no slot later.

No shipped seed scenario could paginate. The slot-detail fast path
answers has_more=false unless a session has size-rotated archive/
segments, and rotation only fires once a transcript outgrows a 10 MiB
budget, so every scenario built from a handful of messages leaves "load
earlier" structurally unreachable and a paging trace cannot be performed
through the real routes.

sessions-long-history seeds that state: one pinned session whose oldest
60 rows sit below a real rotation boundary. The initial slot load returns
the 8 live rows with has_more=true and next_before=60, and paging with
that cursor returns rows out of the archive.

Both shipped files are real ConversationLog output. The segment is
_maybe_rotate output at the production budgets, and the live transcript
is compaction output holding rows append wrote, so the fixture cannot
encode a rotation shape the writer does not produce. Only the rotate
segment and the live transcript are kept; the ~10 MiB compact segment the
recipe produces is discarded, which is the state a home reaches once
archive retention reclaims it. That keeps the fixture at ~20 KB of
package data, inside the 64 KB per-fixture cap.

The contract test pins the design point through the real readers and
pins the shipped segment's header and row shape against a rotation
driven in-test, so a change to the rotation format turns red instead of
leaving the fixture describing a shape the product no longer writes.
@pepmach
pepmach requested a review from a team as a code owner September 8, 2026 08:36
@pepmach
pepmach requested a review from bolichen97 September 8, 2026 08:36
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Sep 8, 2026
@github-actions

github-actions Bot commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Design Review (Fable 5) — ✅ PASS

Design-level review of c7d8191bbe23798958ca29646ca6a6d15df53261 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

A real, documented capability gap (#4100 stand-down) closed with real writer output, size-arithmetic-aware, and provenance-pinned against the generator — sound and proportionate.

Suggestions

  • The regeneration procedure lives as prose in fixture.yaml ("append 199 oversized filler rows… append one more sized so the file crosses 10 MiB"); when the provenance test goes red on a rotation-format change, whoever regenerates must re-derive that arithmetic by hand — checking in the generator script alongside the fixture makes regeneration mechanical instead of archaeological.

[DESIGN-REVIEWED] c7d8191

@github-actions

github-actions Bot commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Opus 4.8 Review — ✅ no blocking findings

Reviewed c7d8191bbe23798958ca29646ca6a6d15df53261 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] c7d8191

Verdict parsed from the review's SHA-scoped output markers for commit c7d8191bbe23798958ca29646ca6a6d15df53261.

False positive or not applicable? A repository writer can comment:
/ai-review override fable c7d8191bbe23798958ca29646ca6a6d15df53261: <one-sentence reason>

@github-actions

github-actions Bot commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5) — ✅ PASS

Premise-level review of c7d8191bbe23798958ca29646ca6a6d15df53261 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

All checks complete. The claims in the description survived verification: the two pre-existing reason=rotate archive segments (rich, sessions-a-few) are deliberately orphaned "synthesized archive slice[s]" for retention paths whose keys chain to no live session, so no existing scenario paginates; fixtures are auto-discovered by available_fixtures() so no registry edit was missing; all reader symbols and the 10 MiB/200-line budgets exist as described; the fixture sits well under the 64 KB cap that test_pod_seed_scenarios.py pins.

First-Principles-Verdict: PASS

The Green traces are route-level; the shipped tests pin the readers, not api_chat_slot_detail itself — worth one pod-seeded spot check before relying on it.

What this change ships

Intent: make a history-paging trace performable through the real routes by shipping a seed scenario whose session genuinely paginates — an ADDITION (test infrastructure), grounded in the #4100 stand-down.

Inventory (5 items)
  1. A new seedable scenario, sessions-long-history: one pinned session whose initial load advertises an earlier page — justified
  2. Its archive holds 60 real writer-rotated rows, so the cursor walk spans three pages and ends at the true first message — justified
  3. The scenario name now appears in the seed/pod vocabulary and its unknown-fixture error listings (auto-discovered) — justified
  4. A contract test pinning the paging design point and the shipped segment's shape against today's rotation writer — justified
  5. The worktree-verification guide now names this scenario for anything on the paging seam — justified

[FIRST-PRINCIPLES-REVIEWED] c7d8191

@github-actions

github-actions Bot commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ✅ no blocking findings

GPT 5.6 completed its review of c7d8191bbe23798958ca29646ca6a6d15df53261 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] c7d8191

False positive or not applicable? A repository writer can comment:
/ai-review override gpt c7d8191bbe23798958ca29646ca6a6d15df53261: <one-sentence reason>

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

readiness: action required A blocking check or review needs attention

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant