Repository navigation
fix(storage): prevent unpublished ready-pack entries from causing false dependency cycles (#723) - #737
Open
markankamaransheogarat-sys wants to merge 1 commit into
Conversation
…se dependency cycles (Ephemeral-AI-Lab#723)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #723.
Root Cause
During concurrent first-wins saves, an unpublished candidate in
Packer::readycan collide byObjectIdwith a record already committed by another writer inFetch, producing two false-positiveIntegrity("dependency cycle")paths:WaveSourceshadowing committed records (core/crates/layerfs-storage/src/save/source.rs):WaveSource::base_location(andWaveSource::locationoutside lost-candidate reconstruction) checkedself.packer.location(id)beforeself.fetch. If a committed object's base chain walked into anObjectIdthat also had an unpublished tentative copy inPacker::readypointing in the opposite direction,DepthCache::cost_of/Resolvermixed the committed and unpublished graphs and reported a cycle.Wave::resolvewalking from an unpublished root to the committed winner (core/crates/layerfs-storage/src/encoding/delta/read.rs): Whenacknowledge()loses first-wins onidand callsWave::resolve(id)to reconstruct its own candidate bytes fromPacker::ready, the root (chain[0]) haspack_id_a(unpublished) while its committed base chain inFetchcan legitimately reference the winner's committedidatpack_id_b. Checking onlyobject_idinResolver::read_with_cachestreatedpack_id_aandpack_id_bas the same node.Changes
WaveSource::base_locationnow queriesself.fetch.base_locationfirst and falls back toself.packer.locationonly if absent.WaveSource::locationqueriesself.fetch.locationfirst by default, and setsprefer_packer: trueonly inWave::resolve(reconstructing thisSave's own lost candidate).Resolver::read_with_cachescycle detection compares full record locators(object_id, pack_id, group_number, record_number)instead ofobject_idalone.core/crates/layerfs-storage/tests/concurrent_winner_chain.rs.Verification
Integrity("dependency cycle")failure with a 2-threadconcurrent_first_installsstress harness overMemoryMetadata(failed within 1–27 iterations before the fix; passed 100/100 iterations after the fix).cargo test -p layerfs-storage(all test targets pass).cargo clippy -p layerfs-storage --tests --no-deps -- -D warningsandrustfmt --checkon modified files pass.layerfs-persistenceintegration tests (concurrent_first_installs) are#![cfg(target_os = "linux")]and were not run locally on Windows.