Skip to content

Harden private A100 SFT smoke preflight#3

Draft
Raidriar7170 wants to merge 5 commits into
mainfrom
codex/enable-private-a100-sft-smoke-v1
Draft

Harden private A100 SFT smoke preflight#3
Raidriar7170 wants to merge 5 commits into
mainfrom
codex/enable-private-a100-sft-smoke-v1

Conversation

@Raidriar7170

Copy link
Copy Markdown
Owner

边界

  • PR merge base(当前 origin/main):4c99e8d6cf1fe2782b2760e36437ff5fae164dff
  • 本轮 Changes Requested base:41c53b8eceaafef339e8984e861f16f749e85f38
  • Head:3a4bf38a574ed81534eaea1103b491b137da3049
  • 仅实现 Qwen2.5-7B-Instruct LoRA SFT 的 1-step/1–2 rows smoke 前置与证据门;不授权完整训练、DPO、GRPO、prediction、evaluation、lockbox 或模型改进声明。

Must Fix 实现位置

  1. Public result 严格 allowlist

    • src/voice2task/training.py:359 _public_sft_preflight_result
    • src/voice2task/training.py:460 public_training_result
    • src/voice2task/cli/train.py:100 CLI 只输出重建后的单个 JSON。
    • 顶层、preflight section/key/value 均 fail closed;adapter 仅公开 filename/size/SHA-256。
  2. Descriptor-relative output claim

    • src/voice2task/training.py:860 _open_bound_output_parent
    • src/voice2task/training.py:1036 _claim_sft_output_directory
    • exact parent 必须预存在;绑定 root/parent dev/ino/uid/gid/mode;使用 O_DIRECTORY/O_NOFOLLOW 与 dir_fd leaf mkdir;前后 identity check;不做 uncertainty pathname cleanup。
    • 用户授权的 threat model:防御所有 checkpoint-observable drift/symlink/inode exchange/concurrent leaf;不声称对同 UID 在最终 checkpoint→mkdirat syscall 窗口内的恶意 rename 具备内核原子性,部署必须保证该 namespace 前提。
  3. Shared GPU idle/free-memory gate

    • src/voice2task/training.py:1240 _probe_sft_gpu
    • 记录 free memory、minimum free memory、aggregate compute-process count、idle_verified。
    • 新 blocker:GPU_FREE_MEMORY_INSUFFICIENTGPU_BUSYGPU_OCCUPANCY_PROBE_FAILED
    • src/voice2task/training.py:4552 real runner 入口和权重加载紧前均复检。
  4. Assistant-only objective

    • src/voice2task/training.py:3944 assistant indices 独立从 tokenizer offsets/span 构造。
    • src/voice2task/training.py:3998 验证 prompt 全部 -100、assistant 目标非空且连续、labels/input_ids 对齐、三张量等长。
  5. Adapter 真实更新证据

    • src/voice2task/training.py:2341 before/after adapter tensor snapshot。
    • src/voice2task/training.py:2379 digest、changed tensor count、finite 证据。
    • src/voice2task/training.py:2401 smoke postconditions 和 adapter 文件 SHA-256。

新增重点回归

  • test_public_training_result_is_exact_allowlist_for_arbitrary_absolute_paths
  • test_public_training_result_rebuilds_preflight_without_unknown_nested_private_fields
  • test_public_training_result_rejects_private_paths_in_every_allowed_string_container
  • test_output_claim_blocks_root_same_path_inode_exchange_without_external_mutation
  • test_output_claim_blocks_parent_exchange_to_external_symlink_with_zero_external_mutation
  • test_output_claim_checks_parent_path_identity_after_open_before_leaf_mkdir
  • test_output_claim_rejects_concurrent_final_leaf_creation
  • test_gpu_probe_reports_free_memory_and_idle_aggregate
  • test_gpu_probe_blocks_insufficient_free_memory_or_busy_device
  • test_gpu_probe_fails_closed_when_occupancy_probe_fails
  • test_sft_preflight_rejects_one_prompt_label_leak_even_with_other_masked_tokens
  • test_real_sft_rechecks_gpu_idle_state_before_model_weight_load
  • test_adapter_state_evidence_requires_real_finite_change
  • test_smoke_postconditions_require_adapter_update_evidence

OpenSpec

  • openspec/specs/supervised-contract-tuning/spec.md 已同步。
  • Archived:openspec/changes/archive/2026-07-14-enable-private-a100-sft-smoke-v1/
  • openspec list --json:active changes = 0。
  • 未增加 active-change failure allowlist。

验证

  • PYTHONPATH=src pytest -q1413 passed
  • focused training suites → 203 passed
  • PYTHONPATH=src ruff check .PASS
  • OPENSPEC_TELEMETRY=0 openspec validate --all --strict15 passed, 0 failed
  • PYTHONPATH=src python scripts/check_current_truth_surface.pyexit 0
  • git diff --checkPASS
  • targeted mypy 命令仅报告两个未修改文件中的既有 imported-module 诊断:
    • slot_error_analysis.py:574
    • copy_backed_shadow_interface.py:357
      本次触及的 training.py / cli/train.py 无诊断、无新增错误。

Stop hook

本机 ~/.codex/hooks.json 已移除会因缺少 tree-sitter 而 exit 1 的 graph-on-stop.js 子 hook,保留 Hivemind stop.js;使用合法 hook JSON payload 直接验证 stop_hook_exit=0。这是本机配置修复,不属于仓库提交。

Committed-HEAD preflight

Exit code:1。stdout 为单个脱敏 JSON,stderr 无第二个结果:

{
  "blockers": [
    "CONFIG_FILE_MISSING"
  ],
  "config": {},
  "dataset": {},
  "git": {},
  "gpu": {},
  "model": {},
  "objective": {},
  "output": {},
  "ready": false,
  "runtime": {},
  "schema_version": "voice2task-sft-preflight-v1",
  "status": "blocked"
}
  • Private config:不存在;精确建议路径受 .gitignore 保护,未提交。
  • 真实 A100:未检测(preflight 在 missing config 处 fail closed,未进入 GPU probe)。
  • 本地 Qwen2.5-7B-Instruct:未检测(无合法 private model path)。
  • 真实 1-step smoke:未运行
  • 唯一真实 blocker:CONFIG_FILE_MISSING
  • 未使用 0.5B、mock、fixture 或 dry-run 冒充真实 7B smoke。

最终状态

  • git status --short:空
  • git diff --check:PASS
  • clean-evaluation truth surface 保持原值,execution_readiness=false
  • 没有执行完整训练、DPO、GRPO、prediction、evaluation,也没有作出模型改进、生产就绪或 clean-evaluation ready 声明。

@Raidriar7170

Copy link
Copy Markdown
Owner Author

A100 SFT smoke runtime follow-up (2026-07-14)

Status: PREFLIGHT_BLOCKED. This is not SMOKE_COMPLETED.

  • PR base SHA: 4c99e8d6cf1fe2782b2760e36437ff5fae164dff
  • Runtime-fix base SHA: 3a4bf38a574ed81534eaea1103b491b137da3049
  • Head SHA: f214b9be551e364742c2b4a07c341f5062dca552
  • Branch: codex/enable-private-a100-sft-smoke-v1
  • Draft PR remains open; no merge was performed.

Must Fix implementation

  • src/voice2task/training.py: _sample_sft_gpu_compute_process_count, _run_sft_gpu_fact_helper, _validated_sft_gpu_helper_result, _probe_sft_gpu
  • src/voice2task/_sft_gpu_probe_helper.py: isolated typed CUDA fact protocol and pre-import socket-denying audit hook
  • tests/test_sft_gpu_occupancy_helper.py: parent/helper ordering, typed blockers, strict protocol, minimal environment, real isolated child, real socket API denial, cross-field validation
  • tests/test_a100_sft_review_hardening.py and tests/test_private_a100_sft_preflight.py: shared probe regression coverage
  • openspec/specs/supervised-contract-tuning/spec.md: synced container PID namespace contract

Archived OpenSpec:
openspec/changes/archive/2026-07-14-fix-container-pid-aware-sft-gpu-occupancy-v1

Validation

  • Focused GPU/preflight suite: 241 passed
  • PYTHONPATH=src pytest -q: 1451 passed in 25.76s
  • PYTHONPATH=src ruff check .: PASS
  • OPENSPEC_TELEMETRY=0 openspec validate --all --strict: 15 passed, 0 failed
  • PYTHONPATH=src python scripts/check_current_truth_surface.py: exit 0
  • git diff --check: PASS
  • Stop hook with direct {} payload: exit 0
  • Targeted mypy: exit 1 only for two pre-existing diagnostics in untouched imported modules (slot_error_analysis.py:574, copy_backed_shadow_interface.py:357); no diagnostic in the touched GPU helper/probe code.

Runtime detection

  • Real A100 detected: yes (NVIDIA A100-SXM4-80GB, CC 8.0, BF16 supported, 79.151 GiB total, 78.734 GiB free, two sampled compute-process counts both zero)
  • Local Qwen2.5-7B-Instruct identity verified: yes (15,231,271,888-byte four-shard inventory; public identity and geometry matched)
  • Dedicated private venv: dependency imports pass; pip check reports no broken requirements
  • Private config: present, ignored, untracked, non-symlink
  • Real 1-step smoke executed: no
  • Unique real blocker: MAX_SEQUENCE_LENGTH_EXCEEDED
  • Preflight exit: 1; ready=false; stderr bytes: 0; final run directory was not created
  • Stage 2 development screen: not started

Complete sanitized preflight JSON

{"blockers":["MAX_SEQUENCE_LENGTH_EXCEEDED"],"config":{"allow_heavy_training":true,"base_model_public_id":"Qwen/Qwen2.5-7B-Instruct","bf16":true,"config_sha256":"2b0631a1ae11c3bff56313356c633bd844e4182a527bc125d876ff4e6ef3a8a6","dataset_split":"train","dtype_is_bfloat16":true,"fp16":false,"gradient_accumulation_steps":1,"gradient_checkpointing":true,"local_files_only":true,"logging_steps":1,"lora_policy_valid":true,"low_cpu_mem_usage":true,"max_seq_length":1024,"max_steps":1,"max_train_rows":2,"minimum_free_disk_gib":20.0,"per_device_train_batch_size":1,"private_file":{"git_ignored":true,"git_tracked":false,"nonsymlink":true,"under_private_runtime":true},"report_to_empty":true,"save_strategy":"no","seed":42,"tf32":true,"torch_dtype_is_bfloat16":true,"trust_remote_code":false,"use_cache":false},"dataset":{"manifest_file":"manifest_public_sample.json","manifest_id":"public-sample-20260619T090925Z","manifest_sha256":"f866c173795e97953b1dec85611b405867d0a29497910282f99d399f109cda95","non_train_rows_selected":0,"selected_row_count":2,"selected_row_ids_sha256":"0fdcbfe62867b87ededaf2c99a122438cb4eb89ef214c4e02fca703805738409","selected_split":"train","sft_file":"sft_public_sample.jsonl","sft_sha256":"4b677420f766555c04199f15f69f41f3b3ad36ad3cd5c33d2b40b0e3f8573587"},"git":{"commit_sha":"f214b9be551e364742c2b4a07c341f5062dca552","tracked_worktree_clean":true},"gpu":{"bf16_supported":true,"compute_capability":"8.0","compute_process_count":0,"cuda_version":"12.9","explicit_selection":true,"free_memory_gib":78.734,"idle_verified":true,"minimum_free_memory_gib":35.0,"minimum_memory_gib":35.0,"name":"NVIDIA A100-SXM4-80GB","total_memory_gib":79.151,"visible_device_count":1},"model":{"geometry_matches_qwen2_5_7b":true,"local_files_only":true,"minimum_weight_bytes":12884901888,"public_id":"Qwen/Qwen2.5-7B-Instruct","snapshot_revision_sha256":"2a1aaaf30e79bc9817f385e41260f822b4533f570a62f7253afc536e8f724363","stable_fingerprints":{"config.json":"7463bb0ea78315365e6c6b74de4e73bbcc8359dfb0c5a737584e077d42c0b03c","generation_config.json":"3a8f9087e486054c8a4a08dae2e5a3ba62e23da212b5b8c08bc42cb983c3459f","merges.txt":"599bab54075088774b1733fde865d5bd747cbcc7a547c5bc12610e874e26f5e3","model.safetensors.index.json":"624bf7c47cd12468fdc16e38a47cf4f19e0415b859a223ba3c027eed2f0e1028","tokenizer.json":"c0382117ea329cdf097041132f6d735924b697924d6f6fc3945713e96ce87539","tokenizer_config.json":"5b5d4f65d0acd3b2d56a35b56d374a36cbc1c8fa5cf3b3febbbfabf22f359583","vocab.json":"ca10d7e9fb3ed18575dd1e277a2579c16d108e32f27439684afa0e10b1440910"},"total_weight_bytes":15231271888,"weight_inventory":[{"name":"model-00001-of-00004.safetensors","size":3945441440},{"name":"model-00002-of-00004.safetensors","size":3864726352},{"name":"model-00003-of-00004.safetensors","size":3864726424},{"name":"model-00004-of-00004.safetensors","size":3556377672}]},"objective":{"assistant_target_present":false,"max_sequence_length":1024,"prompt_labels_masked":false,"records_checked":0},"output":{"blockers":[],"free_disk_gib":10142733.748,"minimum_free_disk_gib":20.0,"output_path_sha256":"6d5c0189521d027b29cdb81e4a139a419519312c4cd9a1ae6ac32630fde3b368","ready":true,"root_path_sha256":"c26d3afb706ca1995065c67df889a9c850ddf18072679c265574ec6bcfca84da","writable":true},"ready":false,"runtime":{"missing":[],"pip_check":"ok","python":"3.11.4","python_requirement":">=3.10","versions":{"accelerate":"1.13.0","datasets":"5.0.0","peft":"0.18.1","torch":"2.10.0+cu129","transformers":"5.5.0","trl":"1.5.1"}},"schema_version":"voice2task-sft-preflight-v1","status":"blocked"}

Repository status after push is clean and synchronized with the remote branch. No full training, DPO, GRPO, prediction, evaluation, lockbox access, or model-improvement claim was performed.

@Raidriar7170

Copy link
Copy Markdown
Owner Author

Real 1-step runtime evidence — postcondition failure (2026-07-15)

Authoritative external status: RUNTIME_POSTCONDITION_FAILED. Do not describe this run as SMOKE_COMPLETED.

The earlier MAX_SEQUENCE_LENGTH_EXCEEDED root cause was measured with the exact local tokenizer and shared assistant-only builder:

  • selected row 1: 1077 total tokens (1021 prompt, 56 assistant)
  • selected row 2: 1076 total tokens (1020 prompt, 56 assistant)
  • ignored private config was changed only from max_seq_length=1024 to the minimum safe value 1077

A fresh shared preflight returned exit 0, ready=true, blockers=[], checked both rows, and reported maximum_observed_tokens=1077.

Exactly one real sft --run-training process was then launched on the same explicitly selected GPU. It returned exit 0 and produced real adapter-update evidence:

{"adapter_files":[{"name":"README.md","sha256":"088fc697ae52cb14f1e808e5c967c8be13cb0256da184ad882dd41ba8e95c617","size":5262},{"name":"adapter_config.json","sha256":"5212eea727806033f85531cabb70ca59f6c93152aa3f783c543e169314644760","size":1029},{"name":"adapter_model.safetensors","sha256":"13fce8410c1c21d86c1dc628c9f20e9477294c707d1fe7235e4a111834a8fab7","size":20214760}],"clean_evaluation":{"acquisition_source_status":"UNAVAILABLE","authoritatively_bound_binding_count":0,"clean_population_status":"NOT_MATERIALIZED","execution_readiness":false,"freeze_authorized":false,"human_acceptance_status":"NOT_RECORDED","protocol_freeze_status":"NOT_FROZEN"},"observed_optimizer_steps":1,"smoke_status":"SMOKE_COMPLETED","stage":"sft","train_result_metrics":{"adapter_state_digest_after":"a2395bb971e32f30cf760f2da2fc9b06e727c82a2ffac6b753ac7b0b21f7105d","adapter_state_digest_before":"ba19fe10ce78816dbb3e3944dda2ae6d3e354277ad0ce1cad58279ec1ed68a52","adapter_tensor_count":224,"all_adapter_tensors_finite":true,"changed_adapter_tensor_count":112,"epoch":0.5,"train_loss":0.0034358606208115816,"train_runtime":0.9902,"train_samples_per_second":1.01,"train_steps_per_second":1.01,"trainable_parameter_count":5046272},"training_budget":{"configured_max_steps":1,"effective_batch_size":1,"gradient_accumulation_steps":1,"num_train_epochs":1.0,"observed_optimizer_steps":1,"per_device_train_batch_size":1,"scheduler_max_steps":1,"step_matched_not_token_matched":true,"step_matching_unit":"optimizer_steps","target_tokens_per_single_pass":112,"target_tokens_seen_estimate":56,"target_tokens_seen_status":"estimated_from_label_tokens_and_step_budget","theoretical_examples_seen":1,"train_row_count":2},"training_rows_used":2,"training_status":"training_completed"}

Additional verified postconditions:

  • adapter metadata exists and records the same update evidence
  • before/after adapter digests differ
  • 112 adapter tensors changed; all adapter tensors are finite
  • finite loss; no NaN/Inf metric
  • no full base-model weight file was written
  • selected GPU compute-process count returned to zero
  • one launch marker exists; no second run was attempted
  • tracked worktree remains clean; private config, logs, cache, metadata and adapter remain ignored/untracked
  • offline/local-only environment was used; no prediction, evaluation, DPO, GRPO or full training was run

Blocking postcondition

CLI stdout was not one JSON document:

  • stdout: 7054 bytes / 197 lines
  • final result JSON prefix: 298 non-JSON bytes
  • prefix consists of two Trainer Python mapping repr lines (step metrics and train summary)
  • one and only one JSON object contains training_status
  • stderr contains ordinary logs and no second result JSON

Therefore the embedded metadata value smoke_status=SMOKE_COMPLETED is not accepted as the external status. Independent review verdict: Changes Requested / Runtime Postcondition Failed.

Sanitized ready preflight JSON

{"blockers":[],"config":{"allow_heavy_training":true,"base_model_public_id":"Qwen/Qwen2.5-7B-Instruct","bf16":true,"config_sha256":"1a3932231d9c6df01167024bbfaf81ad549bf20ac672c15c5ee5b4a7b889f734","dataset_split":"train","dtype_is_bfloat16":true,"fp16":false,"gradient_accumulation_steps":1,"gradient_checkpointing":true,"local_files_only":true,"logging_steps":1,"lora_policy_valid":true,"low_cpu_mem_usage":true,"max_seq_length":1077,"max_steps":1,"max_train_rows":2,"minimum_free_disk_gib":20.0,"per_device_train_batch_size":1,"private_file":{"git_ignored":true,"git_tracked":false,"nonsymlink":true,"under_private_runtime":true},"report_to_empty":true,"save_strategy":"no","seed":42,"tf32":true,"torch_dtype_is_bfloat16":true,"trust_remote_code":false,"use_cache":false},"dataset":{"manifest_file":"manifest_public_sample.json","manifest_id":"public-sample-20260619T090925Z","manifest_sha256":"f866c173795e97953b1dec85611b405867d0a29497910282f99d399f109cda95","non_train_rows_selected":0,"selected_row_count":2,"selected_row_ids_sha256":"0fdcbfe62867b87ededaf2c99a122438cb4eb89ef214c4e02fca703805738409","selected_split":"train","sft_file":"sft_public_sample.jsonl","sft_sha256":"4b677420f766555c04199f15f69f41f3b3ad36ad3cd5c33d2b40b0e3f8573587"},"git":{"commit_sha":"f214b9be551e364742c2b4a07c341f5062dca552","tracked_worktree_clean":true},"gpu":{"bf16_supported":true,"compute_capability":"8.0","compute_process_count":0,"cuda_version":"12.9","explicit_selection":true,"free_memory_gib":78.734,"idle_verified":true,"minimum_free_memory_gib":35.0,"minimum_memory_gib":35.0,"name":"NVIDIA A100-SXM4-80GB","total_memory_gib":79.151,"visible_device_count":1},"model":{"geometry_matches_qwen2_5_7b":true,"local_files_only":true,"minimum_weight_bytes":12884901888,"public_id":"Qwen/Qwen2.5-7B-Instruct","snapshot_revision_sha256":"2a1aaaf30e79bc9817f385e41260f822b4533f570a62f7253afc536e8f724363","stable_fingerprints":{"config.json":"7463bb0ea78315365e6c6b74de4e73bbcc8359dfb0c5a737584e077d42c0b03c","generation_config.json":"3a8f9087e486054c8a4a08dae2e5a3ba62e23da212b5b8c08bc42cb983c3459f","merges.txt":"599bab54075088774b1733fde865d5bd747cbcc7a547c5bc12610e874e26f5e3","model.safetensors.index.json":"624bf7c47cd12468fdc16e38a47cf4f19e0415b859a223ba3c027eed2f0e1028","tokenizer.json":"c0382117ea329cdf097041132f6d735924b697924d6f6fc3945713e96ce87539","tokenizer_config.json":"5b5d4f65d0acd3b2d56a35b56d374a36cbc1c8fa5cf3b3febbbfabf22f359583","vocab.json":"ca10d7e9fb3ed18575dd1e277a2579c16d108e32f27439684afa0e10b1440910"},"total_weight_bytes":15231271888,"weight_inventory":[{"name":"model-00001-of-00004.safetensors","size":3945441440},{"name":"model-00002-of-00004.safetensors","size":3864726352},{"name":"model-00003-of-00004.safetensors","size":3864726424},{"name":"model-00004-of-00004.safetensors","size":3556377672}]},"objective":{"assistant_target_present":true,"max_sequence_length":1077,"maximum_observed_tokens":1077,"prompt_labels_masked":true,"records_checked":2},"output":{"blockers":[],"free_disk_gib":10142325.843,"minimum_free_disk_gib":20.0,"output_path_sha256":"3016709988fdcbd111576e8e41a21782e357575052879c9897de01cd98ad541b","ready":true,"root_path_sha256":"c26d3afb706ca1995065c67df889a9c850ddf18072679c265574ec6bcfca84da","writable":true},"ready":true,"runtime":{"missing":[],"pip_check":"ok","python":"3.11.4","python_requirement":">=3.10","versions":{"accelerate":"1.13.0","datasets":"5.0.0","peft":"0.18.1","torch":"2.10.0+cu129","transformers":"5.5.0","trl":"1.5.1"}},"schema_version":"voice2task-sft-preflight-v1","status":"ready"}

Stage 2 remains blocked and was not started. Fixing stdout and authorizing any second real smoke are separate future decisions.

@Raidriar7170

Copy link
Copy Markdown
Owner Author

Bounded stdout-contract fix closeout

  • Fix base: f214b9be551e364742c2b4a07c341f5062dca552
  • New head: df25648d1d837d7aa4af7757b9aea68783d2a5b6
  • Branch: codex/enable-private-a100-sft-smoke-v1
  • PR remains Draft; no merge was performed.

Must Fix implementation

  • Line-aware non-JSON stderr diagnostic stream: src/voice2task/cli/train.py:23-44
  • Unified backend-dispatch stdout isolation with final success/failure JSON outside the redirect: src/voice2task/cli/train.py:87-146
  • Real observed Trainer mapping, noisy exception, partial JSON, and mock-only DPO regressions: tests/test_a100_sft_review_hardening.py:122-207
  • Synced one-result contract: openspec/specs/supervised-contract-tuning/spec.md:154-171
  • Human Brief: docs/human-briefs/2026-07-15-fix-real-sft-cli-single-json-v1.html
  • Archived OpenSpec: openspec/changes/archive/2026-07-15-fix-real-sft-cli-single-json-v1/
  • Active OpenSpec changes: 0

TDD and review

  • RED: 3/3 new regressions failed on the unmodified CLI because backend output contaminated stdout.
  • GREEN focused: 3 passed.
  • Combined A100 review-hardening + private-preflight regression: 154 passed.
  • Independent read-only review: Pass; Must Fix 0; Should Fix 0; Re-plan not needed.
  • A real tqdm(..., file=sys.stdout) compatibility path was exercised without training.

Final validation

  • PYTHONPATH=src pytest -q: 1454 passed
  • PYTHONPATH=src ruff check .: PASS
  • targeted Mypy: no diagnostics in training.py or cli/train.py; only the two recorded imported-module baseline diagnostics remain
  • OPENSPEC_TELEMETRY=0 openspec validate --all --strict: 15/15 PASS
  • PYTHONPATH=src python scripts/check_current_truth_surface.py: exit 0
  • formal public validation: 696 SFT rows / 2100 DPO pairs, zero failures
  • DPO pair check: 2100
  • dataset/schema/DPO/evaluator focused suite: 88 passed
  • public leak scan: 0 findings
  • Human Brief links: 5/5
  • private-config guards: ignored, untracked, scoped status clean
  • direct Stop-hook probes: valid input exit 0; fatal-parse catch exit 0
  • git diff --check: PASS
  • git status --short: empty after commit

Runtime truth boundary

No second real smoke was run in this fix. The prior authorized one-step process still remains Runtime Postcondition Failed because its complete stdout was not a single JSON document. Although the prior run produced genuine one-step adapter-update evidence, this PR does not describe it as SMOKE_COMPLETED; the repaired protocol has not been revalidated by a separately authorized second real run.

No new GPU/model probe or preflight was run. The prior sanitized preflight and one-run evidence remain in the earlier audit comment: #3 (comment)

Clean-evaluation truth remains unchanged: source unavailable, authoritative binding count 0, human acceptance not recorded, protocol not frozen, clean population not materialized, freeze_authorized=false, and execution_readiness=false.

No full training, DPO, GRPO, prediction, evaluation, lockbox access, or model-improvement claim was performed.

@Raidriar7170

Raidriar7170 commented Jul 15, 2026

Copy link
Copy Markdown
Owner Author

Real 1-step A100 SFT smoke rerun after CLI stdout fix (2026-07-15)

Authoritative status: SMOKE_COMPLETED.

  • Executable code commit: df25648
  • Sanitized evidence commit: d86475b
  • Branch: codex/enable-private-a100-sft-smoke-v1
  • PR remains Draft/Open; no merge or archive was performed.
  • OpenSpec change remains active: rerun-real-a100-sft-smoke-after-cli-fix-v1

Fresh gates before the only launch

  • Remote tracked checkout clean at the executable commit; CLI stdout isolation present.
  • Private config: ignored, untracked, non-symlink.
  • Local-only Qwen/Qwen2.5-7B-Instruct identity and inventory verified: 15,231,271,888 total weight bytes; snapshot revision SHA-256 2a1aaaf30e79bc9817f385e41260f822b4533f570a62f7253afc536e8f724363.
  • Exactly one A100-SXM4-80GB selected explicitly: 0 compute processes; 78.734 GiB free against a 35 GiB threshold.
  • Fresh output identity gate passed.
  • Current formal manifest public-sample-20260619T090925Z, train split, 2 rows, max_steps=1.
  • Shared preflight: exit 0; stdout one JSON document; stderr empty; ready=true; blockers=[].

Only authorized launch

  • Launch count: 1; no retry.
  • Process exit: 0.
  • Stdout: 6,758 bytes and exactly one complete JSON document.
  • Stderr: 1,573 bytes; 0 whole-document JSON results and 0 JSON lines; no competing result.
  • training_status=training_completed.
  • smoke_status=SMOKE_COMPLETED.
  • observed_optimizer_steps=1; training_rows_used=2.
  • changed_adapter_tensor_count=112 of 224; all_adapter_tensors_finite=true; before/after adapter digests differ.
  • Adapter config: 1,029 bytes; adapter weights: 20,214,760 bytes.
  • Complete run tree: 4 files / 20,234,578 bytes; 0 base-shard matches and 0 suspicious full-weight files.

Independent review and validation

  • Read-only Reviewer: Pass; Must Fix 0; Should Fix 0.
  • Focused CLI/preflight/smoke tests: 206 passed.
  • No-training dataset/schema/DPO contract tests: 30 passed.
  • OpenSpec strict validation: 16/16 passed.
  • Public leak scan: 0 findings.
  • git diff --check: passed.

No prediction, evaluation, DPO, GRPO, public test, lockbox, clean-evaluation population access, extra seed, or metric experiment was run. This is bounded infrastructure-smoke evidence only; it is not a quality, held-out, generalization, safety, or production-readiness result.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant