Give the typedmemeval verb a real stage-1 dry run

[joslat]

The three-stage run protocol covers every paid run, and the verb that runs most of them had no
stage 1.

What it does: loads the corpus through TypedMemEvalCorpus.Load with the same option mapping the
real run uses — so it proves the corpus the run will actually read rather than a simulation — and
reports vertical, arm token, seed, selected question count and shape distribution. An empty selection
aborts: a run that selects nothing measures nothing, and a window holding none of the phenomenon
has already cost a paid run once.

One thing worth noting. The first version sat inside RunVerticalAsync, so it booted a Neo4j
container and read Azure credentials before printing "nothing spent". A dry run that spends
anything — money, or two minutes of container start — isn't what the directive asked for. It now runs
before any client, container or credential read: 2.9s, no container.

Verified on the row 53 configuration — arithmetic, seed 20260821, arm default, 50 questions, shapes
count 14 / sum 14 / duration 12 / delta 10.

The drift guard now names --dry-run, so an advertised option that stops being carried fails the
suite. Release 0-warn; 730/730.

[Copilot]

🟡 Changes recommended

XML documentation comments around the new DryRun block contain duplicate/misattached <summary> tags that should be corrected to avoid misleading docs and potential XML-doc warnings.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds a real “stage 1” dry-run path to the typedmemeval tool so operators can validate corpus loading and question selection (including shape distribution) before any paid calls, Neo4j container startup, or credential reads.

Changes:

• Introduces a --dry-run option that exits early after loading the corpus via the same option-mapping path as the real run.
• Implements DryRun(...) output including vertical, arm token, seed, selected question count, and shape distribution; aborts if selection is empty.
• Extends the option drift-guard unit test to ensure --dry-run is carried on the options record.

File summaries

File	Description
tools/AgentMemory.LongMemEval/TypedMemEvalProgram.cs	Adds --dry-run, early-exit stage-1 execution, and the DryRun implementation for corpus/selection verification.
tests/AgentMemory.Tests.Unit.LongMemEval/TypedMemEvalCommandLineTests.cs	Updates the “advertised options are carried” drift guard to include --dry-run.
tests/AgentMemory.Tests.Unit.LongMemEval/TypedMemEvalArmProvenanceTests.cs	Updates construction of TypedMemEvalRunOptions to include the new DryRun field.

Review details

Suppressed comments (1)

tools/AgentMemory.LongMemEval/TypedMemEvalProgram.cs:546

• The summary that describes PrintSupersessionStore was moved above DryRun, leaving PrintSupersessionStore undocumented and DryRun partially mis-documented. Reattach the summary to PrintSupersessionStore.

    private static void PrintSupersessionStore(LongMemEvalSupersessionStore? store)
    {

• Files reviewed: 3/3 changed files
• Comments generated: 1
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.