Repository navigation
Release 5.0.1: bounded resources and simpler sample workers - #195
Merged
Merged
Conversation
Co-authored-by: mikessh <6215797+mikessh@users.noreply.github.com>
… into feature/container-thread-budget
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release 5.0.1 fixes unchecked CPU allocation and native thread-launch aborts. Kernels now receive one validated affinity/scheduler/cgroup budget; numerical import pools remain serial, and native workers are joined before exceptions propagate.
Sample/corpus execution uses bounded standard-library threads with shared read-only resources. Spawn helpers, global worker callables and environment mutation are removed. Serial mode starts no pool and keeps inner kernels serial; deferred readers may be closures. Callback errors propagate without a silent serial fallback.
Validation: final bounded release tier 1,255 passed / 12 documented skips; broader local tier 1,264 passed / 12 skips; clean-wheel import/ownership checks and both warning-free documentation builds passed. Native startup refusal raises a catchable exception instead of aborting. Across three fresh 48-vCPU synthetic repetitions, four sample threads scale 2.47x over serial, retain bit-identical outputs, and reduce peak RSS from 1,112 to 702 MiB and live OS threads from 47 to 17 compared with four processes. Four-thread throughput is 30% lower than four-process throughput; shallow workloads can be slower at high counts. Reference corpus sampling retains its seeded stream and remains GIL-limited.
The author confirmed that the hard ceiling applies to compute workers. The 48-worker probe respects that ceiling; its 56 live OS threads include idle runtime/control threads.
This review exposes the final release candidate while #194 handles integration into dev. Hold merge until CI and the author's review complete. The existing PyPI environment reviewer gate remains in place. mirpy's dependency floor will be pushed only after vdjtools 5.0.1 is listed in the PyPI simple index.