Repository navigation
Tests: corruption and forged-file tests for the segment format and the tier directory (refs #17) - #82
Merged
Merged
Conversation
Truncation at every length, one flipped bit in every byte that a CRC or a length check covers (header, footer, locator, every section), a wrong magic, appended bytes, and files that are no segment. Every case must be refused with a CorruptException (UnsupportedException for the magic and the major version) and never return a graph or fail in any other way. The zero fill that no CRC covers is pinned as "damage there changes nothing that is served". SegmentForge takes a good file apart and rebuilds it with chosen fields changed and the CRCs above the change recomputed; the next commit uses it. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Version rule (larger major refused, larger minor opens, newer header and footer metadata, unknown optional section ignored, unknown required section refused), the byte-order probe (every bit, a big-endian header), header fields a reader cannot use, and the section table, footer and locator rules, all with the CRCs above the damage recomputed so that only the rule meant to find the damage can fire. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…17) The claim is that damage is refused with a named exception. For the magic the reader currently throws UnsupportedException while the manifest throws CorruptException for the same kind of damage; SegmentFormatTest already pins the segment case, so these sweeps accept either named exception there. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Segment damage (cuts and bit flips in every section, header, footer and locator, of either segment), every bit and every cut of the manifest and the tombstone file, tombstone and segment versions (larger major refused and not skipped, larger minor opens and serves the writer's results, unknown required section refused), the fallback to the records, a failed open leaves the directory untouched, and damage to the newest of two generations. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds the corruption tests of #17 for the segment format and the tier directory. It changes
src/testonly;src/mainis untouched.SegmentCorruptionTest(23 tests): every truncation length, every section cut at its middle and at its last byte, every bit of the header, footer and locator flipped, every byte of every section flipped (the failure must name that section), wrong magic in every byte, and files that are no segment at all.SegmentForgedFileTest(124 tests): a helper,SegmentForge, rebuilds a file with one field changed and every CRC above it recomputed, so only the documented rule can fire. It covers major and minor versions, an unknown optional and an unknown required section, the endianness probe, header fields, and section-table, footer and locator inconsistencies.TierCorruptionTest(107 tests): the same damage throughTieredIndex.openon real checkpoints, the manifest and tombstone file bit by bit, and damage to the newest of two generations.Everything the issue lists is detected, and nothing other than
CorruptExceptionorUnsupportedExceptionescapes in any sweep. A failed open leaves the directory byte-identical.I checked that the tests can fail: with the reader temporarily broken in nine places (header, footer and section CRCs, the locator length, the required flag, the major check, the id order check, the required-section check, the manifest CRC), 80 of the 254 new tests failed, in every group. The mutations are not in this branch.
The full unit-test run on JDK 25 is 405 tests, 0 failures, 0 errors, 2 skipped (those two were skipped before this change). I did not run JDK 21 locally; CI does.
Not in this PR, so #17 stays open: the simulated file system test (crash after every publish operation, reopen as exactly N or N+1, no orphans) and the child-JVM
Runtime.halttest at every publish step.Bytes that no CRC covers (header bytes 52 to 4095, the gaps between sections, an unknown section) are not detected, by design; the tests pin that behaviour so a change is deliberate.
I will file the one open question about segment magic damage as a separate issue.
Refs #17
🤖 Generated with Claude Code