Make source database operations crash-safe and skip unnecessary refreshes#17
Open
dinkelk wants to merge 1 commit into
Open
Make source database operations crash-safe and skip unnecessary refreshes#17dinkelk wants to merge 1 commit into
dinkelk wants to merge 1 commit into
Conversation
dinkelk
force-pushed
the
fix/source-db-crash-safety
branch
4 times, most recently
from
ae30e85 to
92c9975
Compare
When redo-ifchange encounters a source file from within a .do file, it previously called initializeSourceDatabase unconditionally — even if the file hadn't changed. That function deletes and recreates the entire database directory, which opens a corruption window: if the process is killed (e.g. Ctrl+C triggering SIGKILL via the process group handler) between the delete and the markSource write, the database is left without a source marker. This causes permanent "No rule to build" errors in projects with a catch-all default.do. Now we compare the current file stamp against the cached stamp first. If they match, we skip the refresh entirely — the database is already in the correct state. This eliminates the corruption window for the vast majority of source files on incremental builds (only files that actually changed need the refresh). Includes tests verifying: - Unchanged sources skip DB refresh (inode stability check) - Changed sources still trigger refresh and dependent rebuilds - New sources get properly initialized
dinkelk
force-pushed
the
fix/source-db-crash-safety
branch
from
92c9975 to
7c7443f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two hardening changes for the redo source database:
Commit 1: Skip unnecessary source database refresh for unchanged files
initializeSourceDatabasewas called unconditionally for every source file on every build, even when nothing changed. This deleted and recreated the database directory each time — unnecessary work that also opened a corruption window. Now we compare the current stamp against the cached stamp and skip the refresh when they match.Commit 2: Make initializeSourceDatabase crash-safe by writing source marker first
When
initializeSourceDatabasedoes need to run, it now writes the source marker (y) before any destructive operations. Previously it deleted the database first, then wrote the marker — if killed between those steps (e.g. Ctrl+C), the marker was lost. Now the marker is always present after the first write, and stale target entries are cleaned up individually afterward.