Repository navigation
procfs: hide host CPUs beyond num_cpus - #336
Merged
Merged
Conversation
num_cpus virtualized /proc/cpuinfo and sched_getaffinity, but glibc's sysconf(_SC_NPROCESSORS_ONLN) and _NPROCESSORS_CONF count the CPUs in /sys/devices/system/cpu/online and possible instead. Inside a 2-CPU sandbox on a 12-CPU host, nproc said 2 while os.cpu_count() and getconf said 12, so builds that size their parallelism with make -j$(getconf _NPROCESSORS_ONLN) still saw the host and stayed non-reproducible across runners. Serve online, possible and present as the same 0-(N-1) list. Signed-off-by: Cong Wang <cwang@multikernel.io>
/proc/stat kept a line per host CPU under num_cpus, so code that counts cpuN lines (glibc's fallback when sysfs is unavailable, top, psutil) still saw the host's core count. Keep only the first N per-CPU lines and sum the aggregate cpu line from them, so usage ratios computed from the total agree with the lines a reader can see. Signed-off-by: Cong Wang <cwang@multikernel.io>
Code that counts CPUs by listing the cpuN directories under /sys/devices/system/cpu (older glibc, hwloc, some build scripts) still saw every host core under num_cpus. Extend the getdents filter that hides foreign PIDs in /proc to that directory, dropping cpuN entries at or past N, so the listing agrees with the online, possible and present lists. Filter native directory records through a duplicated descriptor so EOF, seek cookies, rewinds, and cursors shared by dup and fork retain kernel semantics. Preserve the record layout of both getdents variants and restore the cursor when a read cannot be delivered. Cover both variants with regression tests for iteration, seeking, fd reuse, and read errors. Signed-off-by: Cong Wang <cwang@multikernel.io>
congwang-mk
force-pushed
the
num-cpus-sysfs
branch
from
October 30, 2026 17:01
b4ecdcd to
769b751
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
num_cpusvirtualized/proc/cpuinfoandsched_getaffinity, but glibc'ssysconf(_SC_NPROCESSORS_ONLN)andsysconf(_SC_NPROCESSORS_CONF)count CPUs from/sys/devices/system/cpu/onlineand/possibleinstead. Inside a 2-CPU sandbox on a 12-CPU host:With
num_cpusset,/sys/devices/system/cpu/{online,possible,present}now return the same0-(N-1)list, served by the handler that already serves/proc/cpuinfo.Why it matters
num_cpusis the determinism knob for CPU count (it does not limit CPU usage; that iscpu_coresfor pinning andmax_cpufor quota). Builds commonly size parallelism withmake -j$(getconf _NPROCESSORS_ONLN),os.cpu_count()orstd::thread::available_parallelismfallbacks, and the parallelism can leak into the output. reprotest, the Debian Reproducible Builds checker, has anum_cpusvariation for exactly this. Before this change the sandbox still exposed the runner's core count through sysconf, so builds remained non-reproducible across CI runners with different core counts.Other places the CPU count showed through
Each is a separate commit:
/proc/statkept a line per host CPU. Only the first NcpuNlines are kept now, and the aggregatecpuline is summed from them so usage ratios stay consistent. This covers glibc's fallback when sysfs is unavailable, plustopand psutil./sys/devices/system/cpu/showed every hostcpuNdirectory, which older glibc, hwloc and some build scripts count. Thegetdentsfilter that hides foreign PIDs in/procnow also dropscpuNentries at or past N there.Test plan
test_generate_cpu_list,test_filter_proc_stat,test_sysfs_cpu_id,test_build_filtered_direntstest_num_cpus_virtualizationnow also checksgetconf _NPROCESSORS_ONLN,getconf _NPROCESSORS_CONF,/sys/devices/system/cpu/present, thecpuNlines in/proc/statand thecpuNentries listed in/sys/devices/system/cpu; procfs integration suite passes (51 tests)os.cpu_count()🤖 Generated with Claude Code