Repository navigation
Evaluate supplied prediction files from the native Analysis workbench #69
Description
Native external-prediction evaluation — 6e5752b49b6f / production_slice
Researchers can now evaluate supplied predictions through the qualified CLI/job contribution in #57 / PR #65, but the native Analysis workbench still exposes only the identity-teacher simulation. At current main d43bdea867d6198a707a5f55e29216c76054517e, AnalysisJobExtras only supplies ml_bundle_job_id for Eval; it has no external mode or file choice. PR #65 head f6ff65d54756fef57ace4e693f52039e8ca59360 adds the existing extra.prediction_path dispatch and source-bound evaluator.
This contribution will let the operator explicitly choose Identity teacher (simulation) or External predictions file, choose an existing JSON file, and run that exact mode through the ordinary native QThread/job flow. Empty or refused external input must remain external and actionable rather than silently invoking identity mode. Cancellation retains the previously chosen input. The evaluator remains the sole authority for JSON/source/window/timestamp/value admission and numerical scoring; no model inference or new accuracy claim is introduced.
Bounded source ownership
- New
desktop/capture_desktop/widgets_analysis_predictions.pyfor the native mode/file control. - Thin construction, command, package-reset, extra-argument and validation hooks in
widgets_analysis_jobs.py. - If needed, one
screen_analysis.pybusy-state hook that disables only this new control during the already captured job. - Focused native tests under a new
tests/ui/test_analysis_predictions.py, operator instructions and an additive PROGRESS entry, with uniquely scoped source/receiving evidence.
The current public project issues, #57/#65, source-picker #62 and recent central HAMON #140 claims were read before this claim. Source picker #62, saved history #55, gallery discovery #64/export #59, checkpoint #54, kinematics, runner finalization #46 and sync admission #63 retain their owners and all existing behavior. This scope does not modify the evaluator, job dispatch, schemas, numerical masks, dependencies, workflows, source/raw data, capture or daemon. PR #65 retains its own implementation and receiving. The initial integration will be stacked on its exact qualified source if it has not merged.
Receiving
Use existing supported Python 3.12 and installed PySide6/dependencies in an isolated compact checkout/profile; no toolchain install or other worker checkout edits. Preserve original UI absence, test mode isolation, native chooser cancellation/reselection, missing/refused file behavior, package/command changes and immutable running-job inputs. Execute an actual synthetic sealed ML bundle plus external prediction JSON through the native controls/QThread and unchanged evaluator, then inspect its report/figures and exact retained source bytes. Retain identity-mode and unrelated-command controls. Keep source hashes, current-main preservation and independent receiving distinct from hosted Windows qualification.
Activity
Jacob-Met commentedon Oct 8, 2026
Scope extension for #69 after actual native receiving (HAMON 6e5752b49b6f / production_slice).
The independent receiver at exact input candidate 424298e30d6e0f08fe4124fa282328575c40d203 reproduced an inherited control-state failure: the reserved/running QThread disables the new evaluation input, but Run remains enabled and Cancel remains disabled. The unchanged d7459fa parent screen is reproduced byte-for-byte by removing the new one-line input hook, so this is a retained baseline defect, not an evaluator change.
The shared screen hunk will now derive busy from reserved thread presence, use the same value for evaluation input, and reject any start while a thread is reserved. The existing worker, cancellation event and job lifecycle stay in their owner's implementation. Receiving will exercise actual Cancel and subsequent re-enable once.
Actual 1280×960 screenshots also exposed vertically clipped new controls. The existing control column will gain a native scroll container while retaining its widgets/order, with explicit small-window keyboard reachability. Mac receiving uses the real AnalysisScreen. The unchanged MainWindow imports Windows named-pipe APIs before auto_connect=False, so its actual 1100×700 shell gate belongs to supported Windows CI; no transport stub or Mac MainWindow qualification will be claimed.
Source-picker/history, evaluator/JobParams/scoring/schema, gallery discovery/export, and raw capture remain with their existing owners. Original functional source, failed layout images and busy-state counterexample remain frozen alongside the corrected receiving.
Jacob-Met commentedon Oct 8, 2026
Implemented and published as draft PR84: #84.
Exact source/evidence head
1f0d19c17e760d4dd5c1146ce46075cc3f3ce45d, tree66ef9a19ffc8ab920e1be9c1d278ae0741d09af2, is independently received and public-tree verified. The native external/identity/refusal/retry, exact chooser paths, queued input/execution-time byte provenance and final minimum-size keyboard cancellation gates are preserved with their original source identities. Both demonstrated clipping and inherited busy-state counterexamples remain in the qualification packet.The current PR merges cleanly into main
11cc7829; all current unowned imported Eval dependencies are admitted. The remaining integration gate is actual supported Windows CI, including non-skippedtest_evaluation_controls_in_actual_minimum_main_window. This uses the existing full workflow; no Mac transport stub or skipped-case substitution. Root and independent receiving lane retain final hosted/merge acceptance. Other evaluator, source-picker, gallery/export and checkpoint ownership remains unchanged.Jacob-Met commentedon Oct 8, 2026
Windows receiving scope extension, based on actual unchanged-source evidence:
The second diagnostic (run37828302060 / job113486762104, exact e548 checkout) reaches the real MainWindow at1100×700 with unclipped controls, then shows Space on focused Clear leaves the retained path unchanged. Source review identifies the inherited application-wide checkpoint Space shortcut consuming that button key. Root and receiving peer confirmed this interaction.
The existing shared AnalysisScreen fence now includes only a ShortcutOverride handler for unmodified Space on the actual focused descendant QPushButton. Native button KeyPress/KeyRelease remain responsible for activation; no app.py/global shortcut policy changes. A real ApplicationShortcut countercontrol preserves C, Ctrl+Space and non-button Space, and the actual MainWindow keyboard assertion stays unchanged. Current merged source picker6ef303bb is preserved by exact composition.
Separately, the test-only chooser correction uses the independently received actual file-view row-selection method, with exact basename/path admission and retained watchdog. The read-only JUnit observer moves to one Python command to preserve the existing CI contract. The receiving peer owns the inherited scope-test event-loop correction; its actual failure/source identity remain distinct. No ordinary Windows acceptance is claimed until the combined exact checkout completes its unchanged300-second gate.