Repository navigation
Build SIKL tasks from SIKL v5 checkouts - #130
zhenhuang12 wants to merge 12 commits into
Conversation
…l validate SIKL main (AMD-AGI/SIKL#52) dropped the flashinfer bundle export: kernels are committed v5 schema files with a destination-passing ABI and their semantics in the SIKL package. A --input-dir holding sikl/ops/ now takes a v5 path: one task per committed schema under tasks/SIKL-v5/<name>/<uuid>/, materialized deterministically, gated by the seven initial TaskSession actions and judged by python -m sikl validate at the pinned SIKL commit. The legacy bundle path keeps its behaviour; its run directory handling moves into the shared open_run.
mount-info reports v5 errors on stderr, which the Docker runner shows; a deadline that passes before an action starts exits the gate with its own code instead of matching TimeoutError in the log; an install race records the terminal output_conflict; unreadable baseline reports are listed rather than failing the gate; the runner unlinks a symlinked report directory, hashes start.json without attribute filters, strips GIT_* from SIKL's environment and validates the baseline at the provenance path; the export checks the copied candidate before the exit code, so an unchanged kernel always reports '0 candidate files'.
…p/aiter_configs in the runtime image Found by acceptance V5: every baseline bench failed with PermissionError on /tmp/aiter_configs/bf16_tuned_gemm.csv.lock, a root-owned directory of the base image that aiter's config merge writes to (aiter/jit/core.py).
…onfigs Acceptance V5 again: the base image ships root-owned merged configs there, which a non-root user cannot replace in a sticky directory, and which its own aiter, not the pinned checkout's, produced.
Found by acceptance V7: as a non-root user, git clone --shared /opt/sikl (root owned) fails with 'dubious ownership in repository at /opt/sikl/.git'; a local clone checks the source's git dir, which safe.directory did not list.
… trusts the runtime SIKL Acceptance V7: 9b2a813 was not enough. A local clone runs upload-pack on the source's git dir with GIT_CONFIG_PARAMETERS cleared, so no -c safe.directory reaches it (git 2.43 refuses even '*'). The runner now writes its own global config (hooks off, the runtime SIKL, its git dir and CK trusted) instead of /dev/null; the agent's ~/.gitconfig stays unread.
Inspection of committed schemas, materialized task contracts, the runner's verdicts over SIKL validate reports, gate run states and v5 config keys.
An aborted bench stage carries an empty detail, so the FAIL reason said only "validation aborted (bench): ". List each distinct finding after it.
Read schemas in checkout.py, emit tasks from materialize.py and gate them in validation.py, mirroring bundle.py, task_files and validate_task. run keeps its inline lock and state handling; only run-directory creation is shared. The v5 tests join the builder's test module. Generated tasks are byte-identical for all 56 schemas at SIKL 4858da2.
cursor
Bot
changed the title
|
Thanks for adding support for SIKL v5. I have two concerns about how this fits into AKA:
Could the builder import the necessary code and evaluation logic from a pinned SIKL snapshot, then produce tasks that run without a SIKL checkout or the The extraction needs to preserve input coverage and correctness checks, not just copy the timer. Any differences in timing behavior should also be documented and validated before treating the results as equivalent to SIKL's. |
SIKL
mainreplaced the definitions/workloads/solutions bundle imported by #96 with one committed schema per kernel,sikl/ops/<op>/impls/<language>/<name>/<uuid>.json(schema_version: 5). The reference, input distributions and error bounds now live in SIKL's Python contracts rather than in the file, so a generated task cannot carry its own semantics.sikl_task_buildernow also accepts a SIKL checkout and emits one Arena schema-v2 task per committed schema; each action rebuilds the schema fromsource/and is judged by SIKL's ownpython -m sikl validateat the pinned commit.Import and output contracts
checkout.pyreads committed schemas atHEADthrough git objects only, without importing SIKL or executing kernels, mirroringbundle.py. Uncommitted files never become tasks; malformed paths, modes, sizes, sources, entry points, targets and duplicate workloads fail with explicit codes.materialize.schema_task_filesemitsstart.json(the committed blob, protected), the editablesource/kernel files, a standard-libraryarena-eval-v1runner and asikl-schemaexport. The candidate starts as the committed kernel; the provided baseline isstart.json.sikl_v5_config.yaml; legacy policy, generator and validator keys are rejected. The legacy bundle path is unchanged.Validation and acceptance
Nothing in a v5 task is authored, so no generator or repair loop runs.
validation.validate_schema_taskcopies the draft to a fresh workspace and runs the framework's seven initial actions in one bounded subprocess; installation requires all of them to pass with an unchanged task digest. Gate failures map tobaseline_rejected,platform_deferred,output_conflictor a retriedfailedstate. Correctness fails on any SIKL finding other thannot_faster/regressed, located to a workload when the finding has axes; performance re-measures each time and reports SIKL's CUDA graph time per workload.sikl_runtime.Dockerfilebuilds the runtime from the image digest in SIKLci.yamlwith SIKL's Triton and FlyDSL pins and a root-owned/opt/sikl.validate-taskchecks the pinned commit, thestart.jsonblob, both pins, the CK submodule gitlink and that the SIKL root is not writable. The agent-driventask_validatoris not run for v5 tasks. Generated tasks follow SIKL commits and stay uncommitted (/tasks/SIKL-v5/is ignored).Verification
python3 -m pytest -q tests/test_sikl_task_builder.py tests/test_task_validator_v2.py tests/test_task_validator.py tests/test_task_run_v2.py: 243 passed, CPU only.4858da2were emitted before and after the module reorganization; the 367 generated files and theinspect/mount-infooutputs are byte-identical.4858da2onlmsysorg/sglang@sha256:424411da…:make docker-sikl-task-builderongemm_n32_k6144/653c074634fef840(FlyDSL, 13 workloads) passed all seven actions on 13/13 cases and installed the task. SIKL reported a worst violation fraction of 0 on every workload; the unchanged candidate measured 0.94–1.00× the baseline (4.0–23.0 µs, the low end at m ≤ 8).b8cffa0acrsuse2-m2m-204f7c8cf7d…069353ba…9deab255crsuse2-m2m-101f7c8cf7d…37557d04…1.01 * a @ b.Tfailedcandidate correctnesson 13/13 cases with per-workloadnumeric_mismatchandpost_timing_mismatch.make check-docker-runnerfails at the quality-node mount onmainas well, before it reaches the SIKL assertions. The remaining 55 schemas have not been GPU-qualified.