Repository navigation
Harden task validation and repair across the remaining FlyDSL tasks - #129
Merged
Merged
Conversation
irvineoy
force-pushed
the
codex/rl-task-gpu-quality
branch
from
October 20, 2026 22:28
9bbc7e7 to
2dae809
Compare
PR 127 hardened 37 FlyDSL task runtimes against candidates that reach evaluator state: importing the materialized timing helper, mutating imported modules such as torch.cuda.Event, reading frame or function globals, routing to sys through os or builtins, or disabling the PyTorch dispatch audit. The other 74 runtimes had no import closure at all, so a candidate could fabricate speedups there. Port the reviewed candidate_source_closure and check_candidate_control_plane bodies unchanged into those 74 copies, using the same check_dependencies edits PR 127 applied to the matching pre-image: the softmax_kernel form for the newer structure and the merge_state/rmsnorm2d form for the older one. 60 copies are now byte-identical to an already hardened runtime. The bundled flydsl_compat/meta.py in six flydsl2flydsl tasks walked frame.f_back for MLIR locations, so their shipped baselines failed the frame check. Select the same frame with traceback.walk_stack instead; a CPU test proves the selected frame and locations are unchanged. Document the protected state in each changed README. Extend the boundary tests to every FlyDSL runtime by family, add per-route and shipped-source checks, pin a full sha-bound inverse for the ported runtimes, and keep out-of-root candidate tests reaching their intended checks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The validator review rejected the harness boundary of two ported tasks: the guard accepts new non-protected files and the import closure runs them, yet the README only described the closure. Say explicitly that declared editable sources may import new task-local helper modules or packages, which are part of the candidate and checked with it, as the PR 127 closure tasks already document. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Keep the existing import and introspection restrictions while extending the candidate dependency closure. Use frame-free line-level locations in bundled compatibility helpers and preserve static buffer extent calculation. Update fixture roots and exact inverse checks for the current main sources.
irvineoy
force-pushed
the
codex/harden-remaining-task-runtimes
branch
from
October 28, 2026 06:10
4782181 to
e966fa7
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This change protects evaluator state across the remaining FlyDSL task runtimes and makes task maintenance fail closed. It also repairs the 39 packages that failed or warned in the initial qualification of the retained tasks. Original scored cases, seeds, numerical gates, warmups and sample counts are preserved.
Validation:
python3 -m pytest -q tests— 19,196 passed, 195 skipped, 35 subtests passed. After the final unscored MoE boundary additions, focused checks passed again: 97 direct controls and 52 task-contract/history regressions.make check-perf-helpersalso passed. The Linux process-cleanup test now handles a process disappearing while its/procstatus is being read; its full test module passed (45 tests), followed by 20 repeated runs of both real subprocess cleanup cases (40 checks). This is a test-only race fix; task and evaluator runtime sources are unchanged.No agent optimization campaign was run for these repairs. GPU qualification on other architectures is not claimed.
The source checks are structural controls, not a Python security sandbox. Submitted candidates still need their own full correctness/performance evaluation and independent semantic review.