Skip to content

feat: follow Memory provenance to Raw evidence - #27

Closed
faj-design5260 wants to merge 1 commit into
mainfrom
feat/progressive-read-raw-trace-pr1
Closed

faj-design5260 wants to merge 1 commit into
mainfrom
feat/progressive-read-raw-trace-pr1

Conversation

@faj-design5260

Copy link
Copy Markdown
Collaborator

When a recalled Memory omits a detail, callers can now discover its saved provenance with mem --json read <name> and read just the bound evidence with mem --json trace <name> --pointer 'sessions/<session>#<start>-<end>'. The selector must be a cited reference or a subrange of one. Normal Recall/Read do not expand Raw or search RawIndex.

Trace retains name/messages and adds source groups, original session/index/role/time, per-message references, and Memory validity metadata. It supports existing archived provenance excerpts without inventing message numbers, reports missing or inconsistent evidence explicitly, and rejects path traversal and symlink redirection. Text trace now prints the actual messages. Trace reads truth files without initializing SQLite, logging access, or changing Memory, Raw, indexes, or watermarks. Read text output and the Python Store.trace() list return type remain compatible.

The generated skill explains when to stop at Memory, how to follow bound evidence, and why historical Raw instructions must remain data. Write's existing best-effort evidence-date check remains unchanged. No schema, lifecycle, Recall, RawIndex, or harness redesign.

Validation:

  • Latest-main baseline: 34d12a2f8678d5561aba27bd8ff73c5ae4b6a258; isolated branch, existing experimental work preserved.
  • Baseline: 388 tests passed. Treatment: 416 tests passed, including 28 new cases covering distill-to-trace, multiple/overlapping sources, subranges, malformed/missing Raw, legacy formats, historical Memory, symlinks, and byte-for-byte read-only behavior with and without an index.
  • CI-equivalent coverage run: 91.21% (85% required). Full Ruff and mypy checks passed.
  • Identical offline fixture/caller/exact-match judge: both arms answered 2/2 correctly; no Raw calls when the body sufficed; one Raw call when details were needed. Selecting the confirmation-code message reduced returned Raw text from 188 to 41 characters. Full JSON increased from 775 to 975 characters because of source metadata and grouped output. No live Host/Judge benchmark or accuracy/token improvement claim.

Existing boundaries and limits:

  • Explicit name reads already allow invalid/superseded history; trace preserves that behavior and labels validity. Scope/as-of remain Recall filters, not cross-tool authorization. The new CLI selector cannot access an unbound source.
  • Full-source trace fails if any source is unavailable; callers may select a different available reference. No provenance returns empty results with an explicit warning.
  • Grouped evidence preserves overlapping references; the CLI's flat messages list deduplicates by session/index. Legacy Python trace list behavior is retained.
  • Existing Write edge cases involving empty messages or supplied indices are not repaired; strict reads reject inconsistent selected ranges. Large-file streaming and a full access-control model are outside this PR.
  • Raw-instruction tests verify inert, labeled data and unchanged store bytes; they are not a guarantee of every external model's prompt-injection resistance.

@faj-design5260 faj-design5260 changed the title PR1: Follow Memory provenance to Raw evidence Follow Memory provenance to Raw evidence Sep 9, 2026
@faj-design5260 faj-design5260 changed the title Follow Memory provenance to Raw evidence feat: follow Memory provenance to Raw evidence Sep 9, 2026
@faj-design5260
faj-design5260 marked this pull request as ready for review September 9, 2026 16:44
@faj-design5260

Copy link
Copy Markdown
Collaborator Author

Superseded by #28. This is a branch-name migration only; the new branch points to the same commit and contains no code changes.

@faj-design5260
faj-design5260 deleted the feat/progressive-read-raw-trace-pr1 branch September 9, 2026 17:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant