Upstream tracking - #165
Draft
grahamc wants to merge 4109 commits into
Draft
Conversation
Flake lock file updates:
• Updated input 'nix':
'path:../..'
→ 'path:../..'
• Updated input 'nix/nixpkgs':
'https://api.flakehub.com/f/pinned/DeterminateSystems/secure-packages-25.11/0.1.915341%2Brev-e0a8747e95bf57d9bf61fc802e563675f7bf6d42/019f726f-f9de-7dc2-aa15-ca490d995c1c/source.tar.gz' (2026-07-17)
→ 'https://api.flakehub.com/f/pinned/DeterminateSystems/secure-packages-25.11/0.1.915354%2Brev-dca15ad98e9ae434bc8338133e9528120f5d35fc/019fae10-e6ee-7cc0-8f8a-fc79a3778326/source.tar.gz' (2026-07-29)
…d92-4bf8-4d1d-9811-2a844219633c Release v3.21.9
Flake lock file updates:
• Updated input 'nix':
'path:../..'
→ 'path:../..'
• Updated input 'nix/nixpkgs':
'https://api.flakehub.com/f/pinned/DeterminateSystems/secure-packages-25.11/0.1.915354%2Brev-dca15ad98e9ae434bc8338133e9528120f5d35fc/019fae10-e6ee-7cc0-8f8a-fc79a3778326/source.tar.gz' (2026-07-29)
→ 'https://api.flakehub.com/f/pinned/DeterminateSystems/secure-packages-26.05/0.1.1006668%2Brev-e87516da91e53b694a46eda0de22b782eb008d82/019fb45e-7a22-75f0-a9f9-a895378372a0/source.tar.gz' (2026-07-30)
Sync with upstream 2.35.1
LocalStore: Reduce number of SQLite calls
Also extend the symlinked-home functional NixOS VM test to catch such issues. (cherry picked from commit a58925d)
A forked nix-daemon worker ends with exit(0), which does not unwind the stack, so the LocalStore it owns is never destroyed and ~LocalStore() (which unlinks /nix/var/nix/temproots/<pid>) never runs. Each connection therefore leaks a stale temproots file until a later GC reaps it. Before 2.34.0 the store was a temporary passed straight into processConnection(), so it was destroyed at the end of that statement, before exit(0). c4e4084 hoisted it into a named local to call init(), extending its lifetime past exit(0). processConnection() takes the store by value, so std::move it in: the store is then destroyed when that call returns, restoring the cleanup. (cherry picked from commit f7f5c5d)
In 6fae3a2 I neglected the fact that a jobCategory == Build (and less so Substitution) doesn't necessarily use up and free up a build slot. This apparently led to NixOS#16005. From attached --debug logs in the issue: > skipping build of derivation '/nix/store/6gc23jl32lnaqv2ycch1m1igqm6hys35-unit-systemd-journald-.service.drv', someone beat us to it Which directly precedes the crash, seemingly caused by us failing to wake more goals waiting for build slots when a Build goal terminates while someone else has successfully built the same path. It's now the responsibility of the Worker::run loop to pick up just enough goals waiting for slots - overestimating a bit there is ok since the goal implementations avoid overconsuming slots by waiting more. This way we keep the bounded complexity on each iteration, while hopefully also fixing the root cause of the bug. (cherry picked from commit 3329769)
This surfaces the bug fixed by boostorg/context#337 in our test suite. (cherry picked from commit 88579a8)
… while abandoning the coroutine Applies a patch to our boost.context dependency to fix NixOS#16174. An alternative would be to apply basically the same workaround to our suspension points, but that would be more fragile. Other packagers might want to apply the boost patch too (maybe once that's merged upstream), since the issue is likely to affect more stuff than just nix. The bug is subtle enough that it went unnoticed for quite some time. (cherry picked from commit f8b102f)
…er ASan (cherry picked from commit 54aa888)
Fixes from upstream 2.35-maintenance
When useMaster=true, command SSHs through a live master socket do not
run LocalCommand. But when the master dies (ControlPersist=no) and the
command SSH falls back to a direct connection, LocalCommand fires and
'echo started' leaks into the nix protocol stream, causing:
error: protocol mismatch, got 'started'
This fix:
1. Replaces all -oLocalCommand=* args with -oLocalCommand=true on
command SSHs when useMaster=true (OpenSSH uses first-match-wins,
so appending would be silently ignored; prefix match handles
NIX_SSHOPTS and extraSshArgs injection).
2. Skips the readLine('started') readiness check when useMaster=true,
since LocalCommand is now a no-op. The command SSH either connects
through the live master or fails loudly via stderr.
3. Fixes startMaster() stale state: checks master process liveness
with waitpid(WNOHANG) before returning cached socket path. Uses
Pid::release() (not operator=) to avoid kill()/wait() on the
already-reaped child. Handles EINTR/ECHILD properly.
Refs: #441, NixOS#14132
std::function requires its callable to be copy-constructible, so work items that need to root a Value had to wrap the move-only RootValue in a std::shared_ptr, costing a heap-allocated control block and atomic reference counting per work item. Since we build with C++23, we can use std::move_only_function instead, allowing work items to capture a RootValue directly. Note that FutureVector::spawn() can no longer brace-initialize the WorkItems vector, since std::initializer_list requires copyable elements. Assisted-by: Claude Fable 5 <noreply@anthropic.com>
…e_only_function libc++ (as of version 21) doesn't implement std::move_only_function yet, breaking the libcxxStdenv build. Add nix::MoveOnlyFunction, which aliases std::move_only_function when available (per the __cpp_lib_move_only_function feature macro) and otherwise provides a minimal type-erased wrapper that only requires the callable to be move-constructible. Assisted-by: Claude Fable 5 <noreply@anthropic.com>
Use std::move_only_function for parallel evel work items
When the user suspends Nix (Ctrl-Z) and resumes it, the progress bar was not redrawn until the next logging event, and the cursor was left hidden while stopped. Generalize the InterruptCallbacks mechanism into a per-signal-type callback registry (createSignalCallback()), keyed by a portable SignalType enum. The signal handler thread now also handles: * SIGTSTP: run Stop callbacks (ProgressBar unhides the cursor), then stop the process explicitly via SIGSTOP, since sigwait() consumed the signal and suppressed its default action. Not on FreeBSD, where SIGTSTP doubles as NIX_SIG_MULTI_INT. * SIGCONT: run Cont callbacks (ProgressBar forces a redraw and re-hides the cursor). Also trigger Winch callbacks on SIGWINCH, so the progress bar reflows immediately on terminal resize instead of waiting for the next event. Assisted-by: Claude Fable 5 <noreply@anthropic.com>
These fail randomly in GHA with messages like builder-fast-fail.sh: line 3: /cancelled-builds-fifo/fifo: No such file or directory that suggest that the sandbox was not actually set up properly. This could happen if the clone() call in mountAndPidNamespacesSupported() fails under load. So let's ensure an explicit error message in that case.
Fix randomly failing build.sh test
The kernel only sends SIGWINCH to the terminal's foreground process group, so if the terminal is resized while we're stopped, we never hear about it. Re-query the window size on resume, like less(1) does. Assisted-by: Claude Fable 5 <noreply@anthropic.com>
Progress bar signal handling improvements
Refactor the inline waitpid(WNOHANG) logic in SSHMaster::startMaster() into a reusable Pid::isAlive() method (similar to Pid::wait()). If the child has exited, it is reaped and the Pid is reset so the destructor won't kill()/wait() an already-dead process. Assisted-by: Claude Fable 5 <noreply@anthropic.com>
…tocol-leak Prevent LocalCommand 'started' leak on stale SSH master socket
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
14 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Not intended to be merged directly. This PR is a convenience to show the diff between upstream Nix and Determinate Nix (the
mainbranch).Continuation of #4.