Repository navigation
scientific(simulation): own true topic trajectories for realistic rolling-origin K recovery #689
Description
Finding
#680 now has the numerical pieces needed to evaluate candidate K without temporal preprocessing leakage: #684 fixed-training prevalence-mean prediction, #685 owner-admitted newly available partitions, #686 exact partition↔numerical row binding, #687 one-window predictive K selection, and #688 contiguous multi-window score aggregation.
The remaining acceptance cannot honestly be built from the current tepp_simulation::TruthManifest. The simulation owner currently records latent events, documents, availability, memberships, true/observed relations, method variants and digests, but it does not record a true topic count, topic-term probabilities, document logistic-normal coordinates, prevalence coefficients/covariance, topic trajectories, or generated term counts. SimulationConfig likewise has no topic-measurement DGP parameters. Consequently an apparent rolling-origin K-recovery test would have to hand-author counts outside the truth owner or infer "truth" from the estimator input, defeating the TEST_STRATEGY requirement for true-parameter recovery.
Required owner work
Extend tepp_simulation rather than adding a model-selection-local fixture. The owner contract must be deterministic/reproducible and expose enough truth to build realistic rolling-origin recovery without source rewriting:
- explicit true
Kand vocabulary size with bounded nondegenerate topic-term probability matrices; - declared prevalence trajectory parameters over EventTime, including stable/interpretable temporal drift and covariance rather than independent per-window topic identities;
- document latent logistic-normal coordinates and resulting topic mixtures generated from those declared parameters;
- weighted multilevel/cross-classified/multiple-membership contribution truth aligned with the generated document memberships;
- relation structure that can affect generated topic coordinates while retaining the existing event/revision/translation/copy truth and connected leakage groups;
- document term counts generated from the declared topic/content state with nonzero realistic document lengths and overlap/sparsity/imbalance controls;
- availability delays independent from EventTime semantics so deliberately future-available evidence can be constructed and rejected by
corpus_split; - immutable truth-manifest digest coverage for all topic DGP parameters, latent document states and term-count observations;
- deterministic CPU
f64generation with explicit seed domains and no LLM-generated numerical truth.
The public recovery fixture must not be a four-row micro-corpus. It should exercise multiple canonical rolling-origin windows, changing prevalence, overlapping topics, relation-connected observations and weighted memberships; candidate selection must consume #688 and each replication outcome must feed #680's denominator-preserving SelectedKRecoverySummary.
Required RED / acceptance
Before implementation, add a truth-owner RED demonstrating that a generated scenario exposes declared true_k, true topic/content/prevalence state and term counts whose identities align one-to-one with the manifest documents. The manifest digest must change when any scientific truth parameter or generated count changes.
Then add the #680 recovery acceptance separately: recover known K and declared trajectories/states over repeated realistic scenarios, report bias/RMSE/failure denominator/MCSE, and include a deliberately future-available document whose attempted admission is rejected/audited. Synthetic micro-data remains unit coverage; do not promote the owner generator itself as release acceptance without the full recovery experiment.
Boundary
tepp_simulation owns the known-truth DGP. topic_measurement owns the estimator. corpus_split owns rolling-origin/leakage admission. model_selection owns predictive K comparison/recovery summaries. Evidence, Membership and relation promotion remain #527/#658/#671, #605 and #675 respectively. Do not copy those owner contracts or treat simulation truth as production evidence authority.
Activity
test(simulation): require digest-bound known-topic truth (#689)
feat(simulation): add bounded known-topic DGP configuration (#689)
feat(simulation): generate known topic trajectories and counts (#689)
feat(simulation): bind topic truth to generated corpus (#689)
feat(simulation): digest and verify known-topic truth (#689)
test(simulation): require weighted cross-classified topic truth (#689)
feat(simulation): make memberships weighted and cross-classified (#689)
docs(changelog): record digest-bound known-topic simulation (#689)
docs(adr): bind known-topic recovery truth to TRSL-TM validation (#689)
seonghobae commentedon Sep 22, 2026
Ordinary-forward source lineage for the known-topic simulation owner is now on #639's canonical branch.
55ec09a462fe95e4c1141c4618d3ef7a5604148brequires a digest-bound true-K/topic/content/prevalence/document-state contract.e53486c0b406ba900d5a173372629875db16f381adds validated true K, vocabulary/document length, topic separation, temporal drift, logistic-normal residual scale, membership strength, and relation strength.fbe756d9df42d5e408da5504e0470042288eb473adds a separate topic seed domain, positive normalized topic-term truth, prevalence intercept/slope/covariance, per-document ALR coordinates/mixtures, membership/relation/method contributions, and fixed-length term counts.c788bf5536ad7c1c9ec5b5111af8735a16ad64f6generates topic truth from the already-created temporal/event/document/relation corpus and includes every DGP setting in the configuration fingerprint.87d347d64b221d5488d10585db3eecd322cd766abinds the entire topic truth intoTruthManifestverification and content hashing using exact f64 bit patterns plus term counts/document identities.ea989ec4c6eeff2f7f5e302ba623998894510bfcrequires exact 10,000-bps membership weights plus a recurring classification across events rather than atomized document-only groups.3d0d23be03bbd92a7384cf0d2b05a992d23c6d11makes role-specific group identities recur across independently crossed event classifications, preserves derivative-document grouping, and makes weights sum exactly to 10,000 bps.4b48bb5432af3d4c062284aa14ae9104d3bfac7d.03af74c6295674cb344b3b5649accf16179f1cb4records this as simulation/recovery truth only, not production Evidence/Membership/relation/source authority or scientific(selection): add leakage-safe rolling-origin K validation #680 acceptance.The first RED was an API-absence/source RED and the second a realistic behavioral RED; neither is being promoted as hosted RED evidence. Exact-head hosted checks for the current lineage are still non-terminal, so #689 remains open and no GREEN/merge/release claim is made.
test(simulation): cover all cross-classification lanes (#689)
seonghobae commentedon Sep 22, 2026
Follow-up: cross-classification edge coverage is now current-head
3f060b20824e1bf92d55e00ed69ca1cbfce79bfd, extending the integration fixture to exercise all four configured classification lanes. #639 body has also been currentized to this exact head and fresh base compare (ahead 245 / behind 0). This is test hardening only; the #689 claim boundary and no-hosted-GREEN status are unchanged.seonghobae commentedon Sep 22, 2026
Follow-up scientific hardening from #690 is now on the same #639 owner branch: RED
73f4c79a85071d5fe9804b47be746f1931385b95→ config repair7d4bf62e8dac8afa5edd07392e880d055c59497e→ CHANGELOG/current head9b4aef768ed77db8f1d6786e9e2731f2b24bd968.The finding was that
SimulationConfigaccepted more membership targets than the generator's six unique role labels, whileTruthManifestrequires per-document role uniqueness. The owner now fails closed above six instead of generating a manifest that violates its own invariant. This is #689 DGP integrity hardening only; it does not change the known-topic arithmetic, production Membership authority, or the still-outstanding #680 replicated rolling-origin recovery acceptance.