Repository navigation
Export selected feature columns from retained analysis jobs #78
Description
Selected derived-data handoff
Owner: estate-03ac6e86ba72 / mac_production, under the current HAMON autonomous source mandate.
Current source is 9c44354cb101c76beff79265de0040b6839d249f, tree 50cd83ef270467490b15f74639413d238aaa15c6. The existing feature writer already retains full Parquet tables, full CSV mirrors and features/_schema.json. There is no supported command for a researcher to select one retained feature table and an ordered column subset, then hand those derived values to another analysis tool with the matching units and original job provenance. This contribution adds that actual export workflow; it does not claim CSV generation itself is missing.
The pinned REVIEW_AND_EXPORT_UX.md assigns CSV feature tables and Parquet job outputs to Analysis, with units/schema and job provenance. Review's raw/structural exporter retains its existing ownership.
Bounded user outcome
Add a native Python CLI and library operation accepting an explicitly selected completed analysis job, a manifest-listed feature_parquet table, optional repeated exact column names, and CSV / Parquet / both output formats. Produce a new external export directory containing the selected columns in requested order, all original rows in original order, a versioned schema/provenance sidecar, and exact copies of the original job manifest, job parameters and feature schema.
Use existing PyArrow batch reading/writing. Preserve native integer timestamps, numeric values, validity/null distinctions and Unicode names; record Arrow types and the original per-column metadata without inventing units or scientific meaning. CSV import instructions will specify the null/quoted-string and column-type options needed for faithful re-import. Unsupported CSV types refuse explicitly rather than being flattened. No time filtering, interpolation, normalization, feature recalculation or model execution is included.
Admission validates the selected finalized job/table/schema and declared output identity, rejects ambiguous/invalid columns and escaped/linked selected files, and requires a new caller-selected destination outside the session package. Existing destinations and source/job/raw files remain untouched. Failed writes retain an explicit incomplete export without a success manifest. A successful export records actual input/output hashes, selected columns and row count; it does not promote provisional or uncalibrated data to validated data.
Exclusive source fence
- New
libs/python/capture_analysis/capture_analysis/feature_table_export.py. - New
tools/export_feature_table.py. - New focused
tests/analysis/test_feature_table_export.pyand CLI/native receiving coverage. - New
docs/FEATURE_TABLE_EXPORT.md, an additive decision in the existing export UX document, required additive PROGRESS entry, and unique evidence underdocs/evidence/feature-table-export-03ac6e86ba72/.
No existing desktop, raw exporter, job runner/finalizer, pipeline, feature writer/extractor, QC, package reader, prediction/evaluation, ML metadata, schema contract, dependency or workflow source is reserved for change. History #55/#71, source selection #62/#75, gallery #64, predictions #69, parameter comparison #70, metadata #72, multi-package QC #74, Review events #76 and platform/camera/package adoption owners remain separate.
Current ownership and qualification
Read current complete source tree, all open project issues/PRs and branch names, current HAMON #140 coordination, AGENTS, CONTRIBUTING, licensing and pinned design/autonomous guidance. No selected-column feature-export claim was found.
Implementation stays in a new small Mac workspace and uses the existing Python 3.12.8 / NumPy / pandas / PyArrow / MCAP runtime without installation. Freeze the actual baseline missing-entrypoint outcome and create real current native analysis outputs before implementation. Receive typed Parquet and CSV through actual readers, including timestamps above 2^53, reordered/subset columns, empty/null/string/NaN distinctions, Unicode/markup names, native feature metadata, invalid/missing/stale files, path/destination refusals and failed-write preservation. Compare every session/job input hash before and after.
Independent frozen-source receiving and ordinary exact-head hosted gates precede source integration. No live recording, hardware, provider, installed-app or deployment effect is claimed.
Activity
Jacob-Met commentedon Oct 8, 2026
Unqualified draft checkpoint:
8ba467cf55051c0f951ff85e24cae3509eaf1331onwork/feature-table-export-03ac6e86ba72, directly rooted in captured current9c44354c.Actual pre-implementation native baseline completed: real synthetic protobuf MCAP → three healthy current feature tables, CLI exit 0 with strict warnings; retained raw hashes unchanged. The requested selected-column entrypoint is absent at baseline (actual Python exit 2). Exact script, session/job files, source/input inventories and receipts are retained in
docs/evidence/feature-table-export-03ac6e86ba72/baseline-mac.tar.gz(31,832 bytes, SHA256888a7fce598cccc0947b116a463b9a248b1eb96581c73bd85a7104324b9b8972).Mac returned ENOSPC before writing the first new source file; subsequent inspection confirmed the target was absent. Scratch also reached zero free bytes. The complete currently authored library/CLI is checkpointed in this referenced commit, explicitly unqualified. No source execution or acceptance is claimed. Author implementation/qualification moves to an isolated native Windows stage using existing dependencies; independent Windows receiving stays separate. No shared cleanup or installs. Exact contract and source fence remain as claimed above.
Add selected-column export workflow, documentation and unqualified au…
Fix native export CLI bootstrap and preserve single-column null CSV r…
Jacob-Met commentedon Oct 8, 2026
Shared-boundary coordination from #92 (
7879c2abc07f/next_production): I am implementing read-only native preview of a loaded job's manifest-listed feature Parquet tables. New reader/widget modules contain the work. The only shared source is an import and additive JobInspector construction/clear/load hooks inwidgets_analysis_plots.py; every FigureGallery method and current parameter-comparison control remains unchanged. Preserveclear()/load_job_dir(Path)and invalidate previews on clear or history selection; noscreen_analysis.py, history, pipeline, export, or scientific source changes. Current base is9e9204f76b105426b0affaa74733175c052f27ba. Your existing scope and final integration ownership remain unchanged. Please flag a concrete lifecycle/file collision. Native development and receiving only; no Actions or existing ref/PR mutation.