Repository navigation
measurement(longitudinal): bind CWC coordinates to a versioned measurement contract #602
Description
Scientific / DDD finding
longitudinal_cwc_v1 now commits the exact cutoff-visible evidence population to admitted_evidence_sha256, but the admitted payload contains only opaque evidence/snapshot identity, cluster key, predictor/outcome binary64 coordinates, and availability time. LongitudinalClusterScore and LongitudinalCwcArtifact carry no identity for the measurement/scoring contract that produced those coordinates.
That is a different problem from #600. #600 proves which bytes/coordinates entered the run. It does not prove what those coordinates mean or which versioned measurement procedure produced them. The current ADR 0033 states this explicitly: the admitted-evidence digest is not source-generator attestation or construct-validity evidence.
Consequently, the same evidence identities and numeric coordinates can be interpreted under two incompatible upstream construct/scoring mappings while producing the same CWC artifact identity. More commonly, when a mapping changes and coordinates change, a consumer cannot distinguish a changed evidence population from a changed measurement procedure using the artifact alone. That prevents a buyer/auditor from tracing a longitudinal within/between/contextual result to the exact measurement definition whose validity, invariance, language handling, and scoring rules were in force.
This is a Measurement → Longitudinal Modeling ACL gap. It is not a reason to move scoring arithmetic into analysis_engine, and a measurement-contract identifier is traceability evidence rather than proof of construct validity.
Current source authority
At #372 exact 47b6a6c85c69983bd843d8bb6d19001cfd3629d8:
LongitudinalClusterScoreis already-mapped scientific input;admit_scores_at_cutoff()preserves evidence identity, snapshot, availability and coordinates;admitted_evidence_sha256domain-separates and commits those admitted values;AnalysisRunRequest.model_contract_versionidentifies the CWC execution/profile contract, not the upstream measurement/scoring definition;- ADR 0033 remains
Proposedand explicitly says the provenance digest is not source-generator attestation or construct-validity evidence.
No existing open TEPP issue found by a fresh repository search owns this CWC measurement-contract lineage.
Standards / scientific basis
The currently published joint AERA/APA/NCME Standards for Educational and Psychological Testing remains the 2014 edition; NCME continues to describe it as the joint current Testing Standards and emphasizes valid score interpretations and technology/automated-scoring transparency. TEPP's own protected-main research register applies those Standards to construct definition, score interpretation, validity evidence, longitudinal invariance, and automated measurement interpretation.
ISO 10667-1:2020 and ISO 10667-2:2020 remain the published assessment-service standards while Edition 3 is an approved work item under development as of 2026-05-27. The published/current and draft material both center assessment approach/procedure quality, validity, reliability, fairness and standardization. These sources support traceability of the assessment procedure; they do not by themselves prescribe this repository's wire shape or a cryptographic identifier.
Required RED before production repair
Add a public #372-or-successor contract with two executions that keep the accepted run, evidence IDs, snapshot, cutoff, cluster membership, predictor/outcome bits and availability identical but bind different valid measurement-contract identities. The digest-bound CWC artifact / terminal result must not be identical across those two contracts.
Also require fail-closed behavior for missing, empty or non-versioned/unresolvable measurement authority. Do not use a free-form human label as the scientific authority.
Repair boundary
Design one explicit ACL between Measurement and Longitudinal Modeling:
- bind the CWC input population to one canonical, versioned measurement/scoring contract identity (or a typed set only if heterogeneous mappings are scientifically supported and the composition rule is explicit);
- the identity must refer to immutable/released measurement authority before a production run is promoted;
- include that identity in the digest-bound artifact/provenance commitment so replay can distinguish measurement-version drift from evidence drift;
- keep the CWC arithmetic in its canonical psychometric owner;
- do not source-copy a scoring implementation, infer the measurement version from numeric values, or treat a hash/pointer as construct-validity evidence;
- define migration/schema-version behavior explicitly because adding a field to the artifact is a wire-contract change;
- preserve scientific(analysis): preserve evidence identity in longitudinal CWC composition #592/scientific(analysis): keep future-only CWC census metadata out of historical artifacts #593/scientific(analysis): exclude future-only cross-snapshot rows before provenance admission #595/scientific(longitudinal): make CWC cluster means permutation-stable through released binary64 owner #596/build(lockfile): prove CWC lock sync preserves registry checksums #597/fix(scientific): reject impossible all-singleton CWC success artifacts #599/validation(longitudinal): bind CWC artifacts to the exact cutoff-visible evidence payload #600/fix(temporal): require canonical knowledge-cutoff text in CWC artifacts #601 and scientific: establish longitudinal CWC recovery acceptance evidence #501 acceptance boundaries.
Acceptance
Keep this issue open until a surviving Analysis Run authority has the public RED, causal ACL repair, ADR/TRACEABILITY/product-gap decision record, exact-head Rust/rustdoc/line+branch+edge/security evidence, qualifying review, and an immutable measurement-contract authority suitable for the buyer path. #501 still owns true-parameter recovery/Monte Carlo acceptance; this issue adds traceability of the measurement procedure and does not substitute for that evidence.
No mutable PR-head measurement dependency, source copy, cross-service SQL, LLM-as-scoring/numerical authority, compatibility field with unverifiable semantics, skip/xfail, denominator manipulation, force push, destructive rebase, or predecessor-GREEN transfer.
Activity
seonghobae commentedon Sep 19, 2026
Fresh owner-path review resolves the previously unknown Measurement authority more narrowly.
TEPP protected-main issue #277 already states that Psychometrics Commons owns immutable result snapshots while TEPP owns longitudinal composition. A cross-org code search then found the live owner implementation at
ContextualWisdomLab/psychometrics-commons@e81441ce70c676992470afe2be469dd891ad3eb5:ScoringRequestpinsassessment_spec_ref,instrument_version_ref,scoring_version_ref,calibration_reference, optionalnorm_version_ref, and output schema;ScoringResultpins canonical engine-artifact SHA-256 and typedScoreObservation;ResultSnapshotcopies those values immutably and preservesScoreObservation.construct_ref()without recomputation. The owner repository's live GitHub release inventory is currently empty, so protected main is design/source authority but not a production-consumable immutable release.This also refines the required RED: a single measurement/scoring-contract ID is insufficient. One scoring contract can emit multiple construct observations, and CWC has distinct predictor and outcome coordinates. Therefore the consumer ACL must bind two coordinate authorities (predictor and outcome), each including the exact construct identity in addition to the released measurement/scoring provenance. Otherwise two subscales under one scoring contract—or a predictor/outcome construct swap with equal numeric bits—could remain artifact-identical.
Canonical owner follow-up is now Psychometrics Commons #448. It requires a small participant/source-text-free released projection that binds assessment spec, instrument, scoring version, calibration, optional norm, output schema, canonical engine artifact digest, and construct ref, plus immutable release evidence. TEPP must consume only the released projection/fingerprint through an ACL; it must not copy Psychometrics Commons structs, query its DB, or interpret the fingerprint as construct-validity evidence.
Until #448 lands and is immutably released, #602 remains intentionally without a production repair. The eventual TEPP RED should hold run/evidence/snapshot/cutoff/cluster/numeric bits/availability constant and vary only predictor or outcome coordinate authority; the digest-bound artifact/terminal result must change. Missing, unresolved, mutable, or unreleased authority must fail closed before promotion. #596 and #501 remain independent numerical/recovery gates.
seonghobae commentedon Sep 19, 2026
Owner path has advanced from design-only to a concrete Draft contract, but the TEPP consumer boundary is unchanged.
Psychometrics Commons #449 is now exact
c881c8de0e80113912e2d783de0db10e185ebf60on protectedmain@e81441ce70c676992470afe2be469dd891ad3eb5, ahead 11 / behind 0 and mergeable. Its current net diff is four intended files with zero deletions. It has owner-side REDs for construct-bound coordinate authority and canonical wire round-trip/alias rejection, plus a strict participant-freeMeasurementCoordinateProvenancedecoder/encoder. A temporary broadscoring.rsrewrite was detected by compare and ordinary-forward restored; current base→head diff contains no scoring API delta.This is still a mutable Draft. Exact-head Runtime CI, SAST, Security, CodeQL, SBOM/provenance, code-quality and required review workflows are queued, and Psychometrics Commons still has no immutable release. Therefore TEPP #602 must not add a dependency or duplicate the contract yet.
The eventual TEPP RED remains the same: hold run/evidence/snapshot/cutoff/cluster/numeric bits/availability fixed; vary only predictor or outcome released measurement-coordinate authority and require digest-bound CWC artifact/terminal identity to change. Missing, malformed, unsupported, mutable or unreleased coordinate authority must fail closed. #596 numerical permutation and #501 recovery acceptance remain independent gates.
seonghobae commentedon Sep 19, 2026
Measurement owner gate changed materially. Psychometrics Commons #449 remains exact
8ca24956ba5e2ebb5f9e3d32c5d905a2e3d13d1b, but its Supply chain provenance run35431439843is now a hosted RED: exact checkout succeeded, then repository contracttest_concurrency_never_cancels_a_different_protected_main_revisionfailed because protected main's provenance workflow used a unique run-id concurrency group with cancellation disabled.The shared owner defect is separated into Psychometrics Commons #450 and Draft #451 exact
e6dacffa77300fbd822fe02211d3bb07e973a580, which changes only provenance concurrency to same-PR bounded cancellation while keying protected-main pushes by exact SHA. #451 exact provenance/security/CodeQL/SBOM/Semgrep evidence is newly queued. #602 therefore remains fail closed: wait for #451 acceptance/review/ordinary landing, then #449 ordinary-forward restack and fresh exact-head acceptance/review, then immutable release before any TEPP consumer RED/ACL binds predictor/outcome coordinate authority.