Skip unnecessary source database refresh for unchanged files#15
Closed
dinkelk wants to merge 3 commits into
Closed
Skip unnecessary source database refresh for unchanged files#15dinkelk wants to merge 3 commits into
dinkelk wants to merge 3 commits into
Conversation
When redo-ifchange encounters a source file from within a .do file, it previously called initializeSourceDatabase unconditionally — even if the file hadn't changed. That function deletes and recreates the entire database directory, which opens a corruption window: if the process is killed (e.g. Ctrl+C triggering SIGKILL via the process group handler) between the delete and the markSource write, the database is left without a source marker. This causes permanent "No rule to build" errors in projects with a catch-all default.do (like Adamant). Now we compare the current file stamp against the cached stamp first. If they match, we skip the refresh entirely — the database is already in the correct state. This eliminates the corruption window for the vast majority of source files on incremental builds (only files that actually changed need the refresh). Includes tests verifying: - Unchanged sources skip DB refresh (inode stability check) - Changed sources still trigger refresh and dependent rebuilds - New sources get properly initialized Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Previously, initializeSourceDatabase would delete the entire database
directory (refreshDatabase) before writing the source marker. If the
process was killed between the delete and the markSource write (e.g.
Ctrl+C triggering SIGKILL via the process group handler), the database
was left without a source marker — causing permanent corruption.
Now the source marker ("y" entry) is written FIRST, before any
destructive operations. Stale target entries (d, e, r, c, a, p) are
then cleaned up individually. If the process is killed at any point
after markSource, the source marker always exists and the file will
be correctly recognized as a source on the next build.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The C-level signal handler that sent SIGKILL to the entire process group on Ctrl+C was introduced to fix hangs where GHC's RTS couldn't deliver async exceptions while the main thread was blocked in waitpid. However, SIGKILL gives processes zero time to finish database operations, which can corrupt the redo database (e.g. source markers lost mid-write). This replaces the blocking waits with non-blocking polling loops: - Build.hs: waitForProcessInterruptible polls getProcessExitCode every 50ms instead of blocking in waitForProcess - JobServer.hs: waitOnJob and runJobs use non-blocking getProcessStatus with 50ms polling instead of blocking getProcessStatus Between polls, GHC's RTS can deliver async exceptions (UserInterrupt from SIGINT), so Ctrl+C is handled promptly without needing SIGKILL. The onException handler in runDoFile then terminates the child process group via interruptProcessGroupOf (children run in their own process group via create_group=True). Also removes: - installKillGroupHandler from Main.hs (no longer needed) - resetSignalHandlers from JobServer.hs (no SIGKILL handler to reset) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
redo-ifchangeencounters a source file from within a.dofile, it previously calledinitializeSourceDatabaseunconditionally — deleting and recreating the entire database directory even if nothing changed.SIGKILLvia the process group handler) between the delete and themarkSourcewrite, the database is left without a source marker, causing permanent "No rule to build" errors in projects with a catch-alldefault.do.Test plan
🤖 Generated with Claude Code