Skip to content

measurement(longitudinal): bind CWC coordinates to a versioned measurement contract #602

Open

Description

@seonghobae

Scientific / DDD finding

longitudinal_cwc_v1 now commits the exact cutoff-visible evidence population to admitted_evidence_sha256, but the admitted payload contains only opaque evidence/snapshot identity, cluster key, predictor/outcome binary64 coordinates, and availability time. LongitudinalClusterScore and LongitudinalCwcArtifact carry no identity for the measurement/scoring contract that produced those coordinates.

That is a different problem from #600. #600 proves which bytes/coordinates entered the run. It does not prove what those coordinates mean or which versioned measurement procedure produced them. The current ADR 0033 states this explicitly: the admitted-evidence digest is not source-generator attestation or construct-validity evidence.

Consequently, the same evidence identities and numeric coordinates can be interpreted under two incompatible upstream construct/scoring mappings while producing the same CWC artifact identity. More commonly, when a mapping changes and coordinates change, a consumer cannot distinguish a changed evidence population from a changed measurement procedure using the artifact alone. That prevents a buyer/auditor from tracing a longitudinal within/between/contextual result to the exact measurement definition whose validity, invariance, language handling, and scoring rules were in force.

This is a Measurement → Longitudinal Modeling ACL gap. It is not a reason to move scoring arithmetic into analysis_engine, and a measurement-contract identifier is traceability evidence rather than proof of construct validity.

Current source authority

At #372 exact 47b6a6c85c69983bd843d8bb6d19001cfd3629d8:

  • LongitudinalClusterScore is already-mapped scientific input;
  • admit_scores_at_cutoff() preserves evidence identity, snapshot, availability and coordinates;
  • admitted_evidence_sha256 domain-separates and commits those admitted values;
  • AnalysisRunRequest.model_contract_version identifies the CWC execution/profile contract, not the upstream measurement/scoring definition;
  • ADR 0033 remains Proposed and explicitly says the provenance digest is not source-generator attestation or construct-validity evidence.

No existing open TEPP issue found by a fresh repository search owns this CWC measurement-contract lineage.

Standards / scientific basis

The currently published joint AERA/APA/NCME Standards for Educational and Psychological Testing remains the 2014 edition; NCME continues to describe it as the joint current Testing Standards and emphasizes valid score interpretations and technology/automated-scoring transparency. TEPP's own protected-main research register applies those Standards to construct definition, score interpretation, validity evidence, longitudinal invariance, and automated measurement interpretation.

ISO 10667-1:2020 and ISO 10667-2:2020 remain the published assessment-service standards while Edition 3 is an approved work item under development as of 2026-05-27. The published/current and draft material both center assessment approach/procedure quality, validity, reliability, fairness and standardization. These sources support traceability of the assessment procedure; they do not by themselves prescribe this repository's wire shape or a cryptographic identifier.

Required RED before production repair

Add a public #372-or-successor contract with two executions that keep the accepted run, evidence IDs, snapshot, cutoff, cluster membership, predictor/outcome bits and availability identical but bind different valid measurement-contract identities. The digest-bound CWC artifact / terminal result must not be identical across those two contracts.

Also require fail-closed behavior for missing, empty or non-versioned/unresolvable measurement authority. Do not use a free-form human label as the scientific authority.

Repair boundary

Design one explicit ACL between Measurement and Longitudinal Modeling:

Acceptance

Keep this issue open until a surviving Analysis Run authority has the public RED, causal ACL repair, ADR/TRACEABILITY/product-gap decision record, exact-head Rust/rustdoc/line+branch+edge/security evidence, qualifying review, and an immutable measurement-contract authority suitable for the buyer path. #501 still owns true-parameter recovery/Monte Carlo acceptance; this issue adds traceability of the measurement procedure and does not substitute for that evidence.

No mutable PR-head measurement dependency, source copy, cross-service SQL, LLM-as-scoring/numerical authority, compatibility field with unverifiable semantics, skip/xfail, denominator manipulation, force push, destructive rebase, or predecessor-GREEN transfer.

Activity

  1. seonghobae commented on Sep 19, 2026

    @seonghobae
    ContributorAuthor

    Fresh owner-path review resolves the previously unknown Measurement authority more narrowly.

    TEPP protected-main issue #277 already states that Psychometrics Commons owns immutable result snapshots while TEPP owns longitudinal composition. A cross-org code search then found the live owner implementation at ContextualWisdomLab/psychometrics-commons@e81441ce70c676992470afe2be469dd891ad3eb5: ScoringRequest pins assessment_spec_ref, instrument_version_ref, scoring_version_ref, calibration_reference, optional norm_version_ref, and output schema; ScoringResult pins canonical engine-artifact SHA-256 and typed ScoreObservation; ResultSnapshot copies those values immutably and preserves ScoreObservation.construct_ref() without recomputation. The owner repository's live GitHub release inventory is currently empty, so protected main is design/source authority but not a production-consumable immutable release.

    This also refines the required RED: a single measurement/scoring-contract ID is insufficient. One scoring contract can emit multiple construct observations, and CWC has distinct predictor and outcome coordinates. Therefore the consumer ACL must bind two coordinate authorities (predictor and outcome), each including the exact construct identity in addition to the released measurement/scoring provenance. Otherwise two subscales under one scoring contract—or a predictor/outcome construct swap with equal numeric bits—could remain artifact-identical.

    Canonical owner follow-up is now Psychometrics Commons #448. It requires a small participant/source-text-free released projection that binds assessment spec, instrument, scoring version, calibration, optional norm, output schema, canonical engine artifact digest, and construct ref, plus immutable release evidence. TEPP must consume only the released projection/fingerprint through an ACL; it must not copy Psychometrics Commons structs, query its DB, or interpret the fingerprint as construct-validity evidence.

    Until #448 lands and is immutably released, #602 remains intentionally without a production repair. The eventual TEPP RED should hold run/evidence/snapshot/cutoff/cluster/numeric bits/availability constant and vary only predictor or outcome coordinate authority; the digest-bound artifact/terminal result must change. Missing, unresolved, mutable, or unreleased authority must fail closed before promotion. #596 and #501 remain independent numerical/recovery gates.

  2. seonghobae commented on Sep 19, 2026

    @seonghobae
    ContributorAuthor

    Owner path has advanced from design-only to a concrete Draft contract, but the TEPP consumer boundary is unchanged.

    Psychometrics Commons #449 is now exact c881c8de0e80113912e2d783de0db10e185ebf60 on protected main@e81441ce70c676992470afe2be469dd891ad3eb5, ahead 11 / behind 0 and mergeable. Its current net diff is four intended files with zero deletions. It has owner-side REDs for construct-bound coordinate authority and canonical wire round-trip/alias rejection, plus a strict participant-free MeasurementCoordinateProvenance decoder/encoder. A temporary broad scoring.rs rewrite was detected by compare and ordinary-forward restored; current base→head diff contains no scoring API delta.

    This is still a mutable Draft. Exact-head Runtime CI, SAST, Security, CodeQL, SBOM/provenance, code-quality and required review workflows are queued, and Psychometrics Commons still has no immutable release. Therefore TEPP #602 must not add a dependency or duplicate the contract yet.

    The eventual TEPP RED remains the same: hold run/evidence/snapshot/cutoff/cluster/numeric bits/availability fixed; vary only predictor or outcome released measurement-coordinate authority and require digest-bound CWC artifact/terminal identity to change. Missing, malformed, unsupported, mutable or unreleased coordinate authority must fail closed. #596 numerical permutation and #501 recovery acceptance remain independent gates.

  3. seonghobae commented on Sep 19, 2026

    @seonghobae
    ContributorAuthor

    Measurement owner gate changed materially. Psychometrics Commons #449 remains exact 8ca24956ba5e2ebb5f9e3d32c5d905a2e3d13d1b, but its Supply chain provenance run 35431439843 is now a hosted RED: exact checkout succeeded, then repository contract test_concurrency_never_cancels_a_different_protected_main_revision failed because protected main's provenance workflow used a unique run-id concurrency group with cancellation disabled.

    The shared owner defect is separated into Psychometrics Commons #450 and Draft #451 exact e6dacffa77300fbd822fe02211d3bb07e973a580, which changes only provenance concurrency to same-PR bounded cancellation while keying protected-main pushes by exact SHA. #451 exact provenance/security/CodeQL/SBOM/Semgrep evidence is newly queued. #602 therefore remains fail closed: wait for #451 acceptance/review/ordinary landing, then #449 ordinary-forward restack and fresh exact-head acceptance/review, then immutable release before any TEPP consumer RED/ACL binds predictor/outcome coordinate authority.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions