Skip to content

fix: memory-ingest partial-failure reporting + CRLF parity (iss-30, partial) - #42

Merged
REPPL merged 1 commit into
mainfrom
fix/iss-30-keep-original-partial-failure
Jul 12, 2026
Merged

fix: memory-ingest partial-failure reporting + CRLF parity (iss-30, partial)#42
REPPL merged 1 commit into
mainfrom
fix/iss-30-keep-original-partial-failure

Conversation

@REPPL

@REPPL REPPL commented Jul 12, 2026

Copy link
Copy Markdown
Owner

Drains two self-contained instances of ledger issue iss-30 memory-ingest-boundary, detector-first. The issue stays open for its untested-surface remainder (see below) — this is a partial, honest resolution, so there is no capture resolve in this branch.

What was broken

  1. --keep-original reported total failure over a durable write. storeOriginal runs after WritePages has durably written the pages and registry. A failure there returned IngestResult{}, err, so the caller saw total failure even though the pages and registry were already mutated on disk.
  2. CRLF documents were rejected by splitFileFrontmatter while parseFrontmatter accepted them — the closing --- delimiter was matched exactly, so "---\r" never matched and the whole document was rejected, silently degrading hashes and summaries.

The fix

  • A new IngestResult.KeepOriginalError field records a post-write keep-original failure; both ingest paths (ingested and registry_only) now return the successful result with that field set, and the CLI renders a warning and exits non-zero. The message is path-safe: keepOriginalErrorMessage strips the paths from *os.PathError and *os.LinkError (os.Rename carries two absolute paths), and the symlink/non-dir case names the repo-relative sourcesRelPath — so no absolute path reaches rendered output, including --json.
  • splitFileFrontmatter normalises line endings (\r\n/\r\n) at the top, matching its sibling parsers, so CRLF and LF forms split identically.

Detector-first evidence

  • TestIngestKeepOriginalFailureStillReportsIngest — a forced storeOriginal failure still reports the ingest, records the error, leaks no abspath.
  • TestKeepOriginalErrorMessageNoPathLeak — synthetic *PathError (1 path) and *LinkError (2 paths) both reduced to a path-free, repo-relative message.
  • TestSplitFileFrontmatterCRLFParity — a CRLF document splits identically to its LF twin.
  • TestMemoryIngestKeepOriginalPartialFailure — the CLI surface reports the warning, exits non-zero, writes the page, leaks no abspath (wired end-to-end).

make preflight clean (build, gofmt, vet, test, -race).

Review

  • Correctness (ruthless-reviewer): PROMOTE (after fixing an *os.LinkError HOLD and a stale doc comment).
  • Security (security-reviewer): initial BLOCKkeepOriginalErrorMessage only stripped *os.PathError, so an os.Rename *os.LinkError leaked two absolute paths into --json; fixed (LinkError branch + test) and re-reviewed PASS.

Remainder (keeps iss-30 open)

The untested ingest surfaces named in the detector — URL-ingest success path, content-type matrix, PDF extraction, and YAML block-scalar / double-quoted-escape parser cases. PDF extraction coverage may need a dependency and is gated on the no-new-dep STOP.

Branches from main (independent of PR #41, already merged). Do not merge / no auto-merge — maintainer merges fix: PRs.

Assisted-by: Claude:claude-opus-4-8

Two ingest-boundary instances from iss-30, each behind a watched-fail
detector; the issue stays open for its untested-surface remainder.

- --keep-original partial failure: storeOriginal runs after the durable
  page + registry write, so a failure there no longer returns total
  failure. Both ingest paths (ingested, registry_only) record the new
  IngestResult.KeepOriginalError and return the successful result; the
  CLI renders a warning and exits non-zero. The message is path-safe --
  keepOriginalErrorMessage strips *os.PathError and *os.LinkError paths
  (os.Rename carries two absolute paths) and the symlink/non-dir case
  names the repo-relative sourcesRelPath, so no absolute path reaches
  rendered output, including --json.
- CRLF parser parity: splitFileFrontmatter normalises line endings like
  parseFrontmatter/frontmatterKeyLine, so a CRLF closing '---' delimiter
  is no longer rejected and hashes/summaries no longer degrade.

Tests: TestIngestKeepOriginalFailureStillReportsIngest,
TestKeepOriginalErrorMessageNoPathLeak, TestSplitFileFrontmatterCRLFParity,
TestMemoryIngestKeepOriginalPartialFailure. make preflight green.

Assisted-by: Claude:claude-opus-4-8
@REPPL
REPPL merged commit 03188c2 into main Jul 12, 2026
12 checks passed
@REPPL
REPPL deleted the fix/iss-30-keep-original-partial-failure branch July 12, 2026 15:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant