restore: Don't hang on a filesystem that never preallocates - #2
Open
msaggiorato wants to merge 1 commit into
Open
msaggiorato wants to merge 1 commit into
msaggiorato wants to merge 1 commit into
Conversation
This was referenced Sep 16, 2026
msaggiorato
force-pushed
the
upstream/preallocation-guard
branch
from
September 17, 2026 01:26
9c66b0b to
2d9eef8
Compare
msaggiorato
force-pushed
the
upstream/preallocation-guard
branch
from
September 17, 2026 01:30
2d9eef8 to
0b9ea9c
Compare
The restorer reserves space for each file before filling it, so that writing its blobs out of order does not fragment it. Some sandboxed filesystems, seen on managed WordPress hosting, accept that syscall and never answer it, and the consequences run deeper than a slow restore: the kernel holds the inode lock while it waits, so the file can no longer be written, shortened or unlinked — unlinking it blocks in turn, and takes its directory with it — and the waiting thread cannot be interrupted, not even by SIGKILL, so the process it belongs to can never exit. A restore stopped at the first file that needed data, left it empty, and printed nothing. Making the call is therefore an irreversible commitment, and no deadline around it can take one back: by the time the deadline passes, that file and that process are already lost. Ask the filesystem beforehand instead, with a throwaway file whose name is unlinked before the call, from a process that exists only to ask — so a filesystem that never answers strands that process instead of the restore. The answer is kept per filesystem. Whether the call succeeded is not the question, only whether it came back at all: one that fails does so promptly, and the caller tolerates a failure anyway. Both callers already treat a failed preallocation as harmless and let the writes extend the file, so where the question goes unanswered, restoring simply carries on without reserving space. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017d5y4KNmm2N5ySPPH1H49Z
msaggiorato
force-pushed
the
upstream/preallocation-guard
branch
from
September 17, 2026 01:34
0b9ea9c to
58b941c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Upstream submission candidate, off
master, one commit.The defect
The restorer reserves space for each file before filling it (upstream restic#2195, so that out-of-order blob writes don't fragment large files). On some sandboxed filesystems that syscall is accepted and never answered, and the damage goes well past a slow restore. Reproduced outside the original host — branch
repro/preallocation-hanghas a FUSE filesystem that never answersfallocate, plus the probes below:createFile→ensureSize→PreallocateFile→unix.Fallocate);pwrite,ftruncateand evenunlinkon that file block forever, and a blockedunlinkwedges its directory;FALLOC_FL_KEEP_SIZEbehaves identically, so no flag avoids it;state=D wchan=request_wait_answerand ignoresSIGKILL, so the process can never exit.Why a deadline is not the fix
An earlier version of this branch put a 5 s deadline around the call. The reproduction shows that cannot work: by the time it expires, that file is unwritable and unremovable and the process is unkillable. Issuing the call is an irreversible commitment.
The change
Ask the filesystem first, then use it: preallocate a throwaway file whose name is unlinked before the call, from a process that exists only to ask, so a filesystem that never answers strands that process rather than the restore. The answer is kept per filesystem. Whether the call succeeded is not the question, only whether it came back — one that fails does so promptly, and both callers already tolerate failure.
Evidence it works
Restored 8 files/dirs (73.009 MiB) in 0:05, exit 0state=D)Open before this goes upstream
pull/99999; restic wants prior discussion, and the real number replaces itCo-Authored-Bytrailer staysCI for this branch runs in #4, which adds the minio fix from #3, because
master's own test workflow cannot get pastGet programs.🤖 Generated with Claude Code
https://claude.ai/code/session_017d5y4KNmm2N5ySPPH1H49Z