fix: memory-ingest partial-failure reporting + CRLF parity (iss-30, partial) - #42
Merged
Merged
Conversation
Two ingest-boundary instances from iss-30, each behind a watched-fail detector; the issue stays open for its untested-surface remainder. - --keep-original partial failure: storeOriginal runs after the durable page + registry write, so a failure there no longer returns total failure. Both ingest paths (ingested, registry_only) record the new IngestResult.KeepOriginalError and return the successful result; the CLI renders a warning and exits non-zero. The message is path-safe -- keepOriginalErrorMessage strips *os.PathError and *os.LinkError paths (os.Rename carries two absolute paths) and the symlink/non-dir case names the repo-relative sourcesRelPath, so no absolute path reaches rendered output, including --json. - CRLF parser parity: splitFileFrontmatter normalises line endings like parseFrontmatter/frontmatterKeyLine, so a CRLF closing '---' delimiter is no longer rejected and hashes/summaries no longer degrade. Tests: TestIngestKeepOriginalFailureStillReportsIngest, TestKeepOriginalErrorMessageNoPathLeak, TestSplitFileFrontmatterCRLFParity, TestMemoryIngestKeepOriginalPartialFailure. make preflight green. Assisted-by: Claude:claude-opus-4-8
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Drains two self-contained instances of ledger issue iss-30
memory-ingest-boundary, detector-first. The issue stays open for its untested-surface remainder (see below) — this is a partial, honest resolution, so there is nocapture resolvein this branch.What was broken
--keep-originalreported total failure over a durable write.storeOriginalruns afterWritePageshas durably written the pages and registry. A failure there returnedIngestResult{}, err, so the caller saw total failure even though the pages and registry were already mutated on disk.splitFileFrontmatterwhileparseFrontmatteraccepted them — the closing---delimiter was matched exactly, so"---\r"never matched and the whole document was rejected, silently degrading hashes and summaries.The fix
IngestResult.KeepOriginalErrorfield records a post-write keep-original failure; both ingest paths (ingestedandregistry_only) now return the successful result with that field set, and the CLI renders a warning and exits non-zero. The message is path-safe:keepOriginalErrorMessagestrips the paths from*os.PathErrorand*os.LinkError(os.Renamecarries two absolute paths), and the symlink/non-dir case names the repo-relativesourcesRelPath— so no absolute path reaches rendered output, including--json.splitFileFrontmatternormalises line endings (\r\n/\r→\n) at the top, matching its sibling parsers, so CRLF and LF forms split identically.Detector-first evidence
TestIngestKeepOriginalFailureStillReportsIngest— a forced storeOriginal failure still reports the ingest, records the error, leaks no abspath.TestKeepOriginalErrorMessageNoPathLeak— synthetic*PathError(1 path) and*LinkError(2 paths) both reduced to a path-free, repo-relative message.TestSplitFileFrontmatterCRLFParity— a CRLF document splits identically to its LF twin.TestMemoryIngestKeepOriginalPartialFailure— the CLI surface reports the warning, exits non-zero, writes the page, leaks no abspath (wired end-to-end).make preflightclean (build, gofmt, vet, test,-race).Review
*os.LinkErrorHOLD and a stale doc comment).keepOriginalErrorMessageonly stripped*os.PathError, so anos.Rename*os.LinkErrorleaked two absolute paths into--json; fixed (LinkError branch + test) and re-reviewed PASS.Remainder (keeps iss-30 open)
The untested ingest surfaces named in the detector — URL-ingest success path, content-type matrix, PDF extraction, and YAML block-scalar / double-quoted-escape parser cases. PDF extraction coverage may need a dependency and is gated on the no-new-dep STOP.
Branches from
main(independent of PR #41, already merged). Do not merge / no auto-merge — maintainer mergesfix:PRs.Assisted-by: Claude:claude-opus-4-8