Repository navigation
scientific: bind Rubin variance activation to draw-generation provenance #505
Description
Scientific finding
rubin_loading_uncertainty_v1 currently accepts arbitrary finite complete-data indicator draws once snapshot/availability/resource contracts pass. Protected-main psychometric_core::combine_draw_level_ols_loadings then evaluates Qbar, Ubar, B, and T = Ubar + (1 + 1/m)B. That arithmetic is correct for the supplied draws, but the Analysis Run contract does not by itself establish that the draw generator/analysis pairing is an approved inferential regime.
This matters because Rubin-style variance validity is not a property of the algebra alone. Rubin's proper-imputation framework conditions the combining rule on the imputation procedure/estimand, and later work shows Rubin variance can be biased under uncongenial imputer/analyst combinations. The current #504 Monte Carlo evidence deliberately demonstrates coverage only for one predeclared Gaussian draw generator; it must not authorize arbitrary caller-supplied draw sets as generally valid Rubin inference.
Required owner boundary
Keep reusable Qbar/Ubar/B/T arithmetic in psychometric_core. Do not copy the formula into analysis_engine and do not use an LLM judge for activation.
The Analysis Run / Validation Evidence boundary needs a versioned, digest-bound draw-generation provenance contract before an inferential Rubin claim can become authoritative. At minimum the surviving design binds:
- immutable generator/model contract identity and version;
- source snapshot and knowledge-cutoff/availability provenance already required by the profile;
- the estimand/analysis contract paired with the generated draws;
- validation-evidence identity and exact SHA-256 for that approved generator/analysis pairing, including recovery/coverage evidence and its design limits;
- enough provenance to distinguish supported posterior/imputation draws from arbitrary caller-constructed vectors without cross-service SQL or copied source truth.
The policy is fail closed: without an approved versioned draw-generation/analysis/evidence pairing, TEPP may expose bounded descriptive combination arithmetic but must not project it as generally validated interval/variance inference. Do not pretend to prove universal congeniality; record and validate the specific supported pairing and evidence envelope.
RED target
Two inputs with identical snapshot/cutoff/counts and finite draw matrices must remain distinguishable at the projection-authority boundary when one comes from an approved generator/evidence pairing and the other is arbitrary. The executable contract must show that provenance-free/unknown/mismatched/late authority cannot receive positive Rubin-inference activation, while an exact approved immutable pairing can be recognized by the pure decision algorithm.
The RED must not alter the numerical formula.
Acceptance evidence
- checked-in ADR/TRD/TRACEABILITY stating the activation rule and projection policy;
- bounded canonical activation receipt with exact generator/analysis/evidence identity, digest, snapshot, cutoff, availability, and design envelope;
- tests for missing, unknown, mismatched, late/stale/mutable, design-envelope mismatch, and approved pairing behavior;
- digest/provenance binding to the exact validation evidence used for activation;
- scientific: establish Rubin loading-uncertainty recovery and coverage evidence #503 repeated-sampling evidence retained as evidence for its declared generator only, not generalized to all draw sources;
- any new inferential claim remains leakage-safe across
AvailableTime/KnowledgeCutoffand preserves snapshot identity; - no mutable dependency, no source-copy of an external generator, no caller-supplied production approval registry, and no provider/LLM numerical authority.
Current implementation status
Draft #506 now implements the fail-closed negative artifact projection, RubinProjectionActivationReceiptV1, and a pure activation decision. The decision validates bounded canonical receipt metadata, snapshot/cutoff equality by typed instant, Validation Evidence availability, and exact generator/analysis/evidence/digest/design/indicator matching. A private test pairing exercises the positive algorithmic branch.
The production approved-pairing registry is intentionally empty. Therefore #506 does not yet emit an authoritative positive artifact state and does not promote Draft #504. This issue remains open until independently accepted Validation Evidence receives an immutable identity/digest, the exact pairing is promoted through the owner path, a positive artifact state is digest-bound to that receipt, shared TRACEABILITY/TRD/product-gap authorities are current, and exact-head quality/security/review/release gates are reacquired on the surviving vehicle.
Primary evidence
- Rubin, D. B. (1987). Multiple Imputation for Nonresponse in Surveys. Wiley. https://doi.org/10.1002/9780470316696
- Rubin, D. B. (1996). Multiple imputation after 18+ years. Journal of the American Statistical Association, 91(434), 473–489. https://doi.org/10.1080/01621459.1996.10476908
- Meng, X.-L. (1994). Multiple-imputation inferences with uncongenial sources of input. Statistical Science, 9(4), 538–558. https://doi.org/10.1214/ss/1177010269
- Xie, X., & Meng, X.-L. (2017). Dissecting multiple imputation from a multi-phase inference perspective: What happens when God's, imputer's and analyst's models are uncongenial? Statistica Sinica, 27(4), 1485–1594. https://doi.org/10.5705/ss.2014.067
Relationship to current work
Owner implementation lineage remains #374; candidate recovery/coverage evidence is Draft #504 and scientific gap #503. #506 is the activation/projection-policy child and remains Draft. It does not invalidate the checked arithmetic or the scoped #504 Monte Carlo results, and none of these children may be simple-closed until a verified successor inherits every valid source/test/evidence/contract delta.
Activity
seonghobae commentedon Sep 14, 2026
#506 now implements the fail-closed negative half of this finding on top of #504. RED
3ca865b21966659e60644e8d8ee7c81ef501d3bcexposes that provenance-free draw matrices lacked any projection-policy state; current head9ff4130643825e052d3d44d0e2b8956dd8c3db85emits and validates a digest-bound, read-onlydescriptive_only_unbound_draw_generation_provenancestatus without changingpsychometric_corearithmetic. #505 remains open: there is deliberately no positive authoritative activation path yet because #504 is not exact-head GREEN/independently approved. The remaining owner work is the versioned generator/analysis + exact Validation Evidence digest/availability binding and missing/unknown/mismatched/stale/approved activation contract.seonghobae commentedon Sep 14, 2026
#506 has advanced to exact head
7c036fb5b9eb9a24af5a2ee94783426a5c5d31d2on Draft #504. The negative/fail-closed half is implemented: provenance-free draw matrices remain executable only with digest-boundprojection_status = descriptive_only_unbound_draw_generation_provenance; there is still no positive inferential activation path.Two forward repairs are now part of that exact lineage.
696beea69b4e34eb9e391bd925ebdc4f5cd93362migrated the pre-existing integration test away from an external struct literal that became uncompilable whenprojection_statuswas made private; invalid Rubin-total and impossible-count cases now re-enter through boundedRubinLoadingUncertaintyArtifact::from_json.7c036fb5b9eb9a24af5a2ee94783426a5c5d31d2adds public-wire regressions proving that both a missing projection policy and a forgedvalidated_rubin_inferencepolicy fail closed.#505 remains open. Positive activation still requires a versioned generator/analysis pairing plus exact claim-specific Validation Evidence identity/digest and cutoff-safe provenance, with unknown/mismatched/stale/approved identity tests. It must not approve #504 while that exact-head evidence remains Draft or lacks its own GREEN exact-head evidence and qualifying independent approval. Any new #506 check receipts must be for
7c036fb5...; predecessor receipts do not transfer.seonghobae commentedon Sep 14, 2026
#506 advanced to exact head
52f099612815c3af4140956794087212fe99eb31with a checked-in branch-local activation design:docs/research/rubin-loading-projection-activation-contract.md. It specifies a future digest-bound receipt for exact generator contract/version + analysis contract/version + Validation Evidence identity/SHA-256 + evidenceAvailableTime+ source snapshot +KnowledgeCutoff+ design-envelope identity, with fail-closed missing/unknown/mismatch/digest-mismatch/late-evidence/cross-snapshot/mutable-locator/envelope-mismatch cases. Production approved-pairing registry remains intentionally empty; Draft #504 is not promoted. #505 stays open because the bounded receipt value object/pure decision function and an independently accepted immutable Validation Evidence package do not yet exist.seonghobae commentedon Sep 14, 2026
Implementation update: #506 exact head is now
c5ac44ef9907065fad27de730cf3249944ddf4ff. The issue's RED target has an executable bounded receipt and pure decision path: no receipt => descriptive-only; malformed/mutable/cross-snapshot/cutoff-mismatch/late-evidence/unknown-or-mismatched pairing => rejected; a private test-only exact pairing exercisesEligible. The production approved-pairing registry remains empty, so this is not positive product/scientific activation and #505 correctly remains open. The next owner-path step is independent promotion of an immutable Validation Evidence ID/digest, then exact pairing registration and receipt-digest binding into any positive artifact state on the surviving Analysis Run vehicle.seonghobae commentedon Sep 14, 2026
Fresh #506 review found one additional leakage-safe authority requirement, tracked in #507 and repaired by ordinary forward history.
The activation receipt already carried Validation Evidence
AvailableTime, but the approved pairing bound only evidence ID/digest. That made the future positive algorithm capable of trusting a caller-backdated availability once a production pairing existed. RED0a6a3abe0f66f28681294131cfb8e9a78111e08cexposes that case; repaird0434b61b272ccf88357f302a7d0e8cc4d4a5513binds canonical owner-controlled Validation Evidence availability in the approved pairing, requires receipt/authority instant equality, and requires that authoritative instant to be at or before the run cutoff. Research/ADR currentization is3b5d9b508eabcad65943473fb62a91588ae7b35e→5d2eb75a5041d3e44627a501248f52c7e5b94d8b.#505 remains open.
PRODUCTION_APPROVED_PAIRINGSis still empty; Draft #504 is not promoted; a future positive artifact state still needs independently promoted immutable Validation Evidence, receipt-digest binding, shared authority currentization, exact-head gates, and qualifying review.seonghobae commentedon Sep 14, 2026
Fresh #506 review found one additional pre-promotion immutability defect, tracked by #508. The checked-in activation contract already prohibited mutable branch/PR/check/comment/latest-release locators, but
is_mutable_authority_locatoronly recognized exact lowercaselatest/main/master,refs/, and HTTP(S). RED3cb07a51b65c3aba4b59c3ac1602070d23b02946exercises mutable aliases through the public bounded receipt wire; repairaebf54b2331bc2fb0c2d87554b0645f64a7f4312adds case-normalized classification for bounded default/latest aliases, PR/pull/issue numeric aliases, bare GitHub paths, and Git/SSH locators without changing original identifier bytes or the empty production registry. #505 remains open: positive artifact projection still requires independently promoted immutable Validation Evidence plus receipt-digest binding and surviving-vehicle gates.seonghobae commentedon Sep 14, 2026
#506 advanced to exact
e733e104c646172a08f7e12b9a4f74c88443fbc4with a new provenance repair tracked by #509. RED2dff30c9e544b9595a3ff5b73bf37b110f33da8dproved thatsource_snapshot_idstill admitted mutable branch/PR/repository aliases because it used only generic identifier syntax. Causal repair5acbb658f5fcf536b4a07478565244c7e6c78891moves the source snapshot into the activation-specific immutable-authority set and also rejects mutableexpected_snapshot_id;a2d788571ae3c744b08f249bb1d6b1c9286312fccurrentizes the integration contract, and3a16fbc...→e733e104...currentize the research contract/ADR. This does not promote #504: production approved pairings remain empty. Positive activation must continue to require immutable Validation Evidence plus an immutable source snapshot and cutoff-safe authority.seonghobae commentedon Sep 14, 2026
#510 adds one more prerequisite to the positive Rubin activation owner path. On #506, RED
616309e5665f964995ae45c5059b336d466d571eproved that the Rubin-specific receipt still admitted a syntactically valid different analysis ID/version; causal repairda6a2f9ddbec242826287ac94e0147744a1f30ecnow requiresRUBIN_LOADING_ANALYSIS_CONTRACT_IDplus the exactRUBIN_LOADING_MODEL_CONTRACT_VERSIONbefore any registry lookup. Research/ADR currentization isb328752c… → d6d8fd23…; #506 exact head isd6d8fd23507397874278333d08dc9b5274ea29b1. Production approved pairings remain empty. Any future positive activation here must consume that canonical-analysis invariant in addition to immutable generator/evidence/design identity, owner-controlled evidenceAvailableTime, cutoff safety, and receipt-digest binding.seonghobae commentedon Sep 14, 2026
#511에서 positive Rubin activation의 남은 provenance gap 하나를 확인해 #506에서 수리했습니다. 기존 receipt/runtime snapshot equality만으로는 approved Validation Evidence ID/digest가 어느 immutable source population을 검증했는지 owner authority가 고정되지 않았습니다.
RED
e2e4f4597a1404b272f88d4bcd689816890864f9→ causal fix32aa83da55c75ab22ebbca8a903790be119cc20e에서 privateApprovedRubinProjectionPairing이 authoritativesource_snapshot_id를 소유하고, 그 값이 immutable locator-safe이며 receipt snapshot과 정확히 일치해야만 eligible하도록 했습니다. Research/ADR currentization은c4572953382e50d9c12735a500227ed4c9900a05→85ab06bb948149fd4a93d96f91b75f25f1cbaec4입니다.따라서 #505의 positive activation prerequisite는 owner-controlled evidence ID/digest/AvailableTime뿐 아니라 그 evidence가 검증한 immutable source snapshot까지 포함해야 합니다. Production registry는 계속 비어 있고, #504를 promotion하지 않았습니다. #511과 #506은 exact-head hosted evidence 및 surviving #416(또는 verified successor)의 실제 승계가 확인될 때까지 open/Draft로 유지합니다.
seonghobae commentedon Sep 14, 2026
#512 adds a new prerequisite to positive Rubin activation: source population authority is now identity + canonical content digest, not
source_snapshot_idalone. Draft #506 RED940f271...→ source repair62306d5...→ contract propagation3b6419a.../cb7dd6e.../8c1e078...→ research/ADR1573d5c.../cc4442b.... Any future production pairing under #505 must carry owner-controlledsource_snapshot_sha256; runtime, receipt, and owner authority must match exactly beforeEligible. Production registry remains empty.seonghobae commentedon Sep 14, 2026
#515 closes a narrower but necessary positive-activation provenance gap on Draft #506. RED
db8e7b484007be535260a7b16458a8718b0f0eb3established that class-level generator/evidence approval plus snapshot/cutoff/design/counts cannot distinguish different complete-data matrices with the same shape. Current #506 head15fdd2c4d8808f29246bc9d0acd074190ce2bb32now exposes a draw-bound receipt and decision: the receipt commits to canonicalcomplete_data_draws_sha256, runtime supplies that digest independently, malformed/substituted payload digests fail closed before the private approved-pairing decision, and the public crate boundary no longer re-exports the class-level decision directly. Production approval remains empty and no positive artifact state is emitted. Important residual: a matching content digest prevents substitution/replay but is not proof that the approved generator actually produced those bytes. The trusted generator/artifact boundary (including canonical payload serialization/hashing and any execution attestation) remains a separate prerequisite before positive scientific activation can be promoted.seonghobae commentedon Sep 14, 2026
Follow-up to #515: #516 now makes the executor/artifact the canonical owner of the concrete draw-payload digest on #506 exact head
40fb4f6764afc0a32eb246bb27348c4c23ed7532. REDfcd1c0c97594dac319dc4da12a5c09625abf4d78→ repaird4404370d012d214891cf24b4a79688ecd58b845ensures the runtime hash is computed only from the cutoff-eligible, snapshot-admitted draw-major matrix actually passed topsychometric_core, with a versioned domain separator, dimensions, and exact IEEE-754 bits. This removes caller-defined JSON/serialization ambiguity. It still does not prove the approved generator executed: content identity and generator-execution attestation remain separate. Production approval stays empty and no positive artifact state is emitted; independently promoted #503/#504 evidence plus any required trusted generator/artifact attestation remain prerequisites.seonghobae commentedon Sep 14, 2026
Follow-up authority-integrity repair on Draft #506: issue #519 found that the new outer draw-bound receipt parsed through
serde_json::Value, which can collapse duplicate object member names before validation. RED75615fc5a3f6c2880bc8c572459ca3201088e5a6injects duplicate canonicalcomplete_data_draws_sha256members; fix75e72b19779d3b34ffdb2a5eedc4c3128c983937rejects duplicate top-level names before authority interpretation without imposing one formatting/order serialization. This is parser/evidence integrity only; production approval remains empty and Draft #504 is not promoted. #505 should retain this as part of the eventual immutable activation-receipt contract.seonghobae commentedon Sep 14, 2026
#522 adds a new prerequisite to positive Rubin activation. #516 originally committed only the indicator-draw matrix, but both protected loading estimators also consume the cutoff-admitted
factor_scoresvector. #506 REDe9e6a13427eb7f5ea66e8a2e719b8323ec71ae9dand causal repair7dc449f0d2ed8d1a070e5d9d9793e954f41660b1now make the executor-owned per-run commitment cover admitted dimensions + factor-score IEEE-754 bits + draw IEEE-754 bits undertepp.rubin_loading.analysis_payload.v1\0. Research/ADR/rustdoc are current through01c647665294d7290a4d4bf2c6862492397d72c5.Do not treat the content hash as generator or factor-mapping execution attestation. Production pairing remains empty. Before positive artifact projection, the unreleased Draft wire member
complete_data_draws_sha256also needs a semantic rename so the eventual v1 contract accurately names the full estimator payload.seonghobae commentedon Sep 14, 2026
#522 follow-up: the unreleased public contract has now been renamed precisely.
RubinLoadingUncertaintyArtifactandRubinProjectionActivationReceiptV1useestimator_payload_sha256; runtime decision takes the same semantic input. Artifact/source rename9101f6621a9ff26cb9a3ff21a527769040696d8d, public receipt/runtime renamef2596dbaffe0f3ffbae191c36ee7ac9b023b7875, research/ADR current through exact headf4a7eb5c018530e57a197ab2da56532a73e41456.Positive activation prerequisite is therefore the executor-owned commitment over admitted dimensions + factor-score vector + draw matrix, not a draw-only hash. This still does not attest generator or factor-mapping execution, and the production pairing registry remains empty.