Skip to content

scientific(selection): score only owner-issued reference fits #677

Open

Description

@seonghobae

Finding

#676 now binds a statistical candidate's declared K to the fitted topic dimension, but model_selection::statistical_candidate_from_fit(input, candidate_k, model) still accepts ReferenceTopicInput and ReferenceTopicModel as detached public arguments.

ReferenceTopicFit was introduced in #665 specifically so an admitted input, deterministic configuration, and fitted model cannot be rebound after fitting. Model selection currently bypasses that nominal owner boundary: any dimension-compatible model can be scored against a different input, producing a finite in-sample likelihood and Schwarz-style score that looks statistically authoritative even though the model was not fitted to that corpus/design/membership/relation state.

The internal select_fitted_candidate_k(...) loop happens to pass the model it just fitted from the same input, but the public scorer remains a substitution path and the selector still uses the detached fit_reference_topic_model(...) result rather than the owner-issued aggregate.

Required RED

  • construct two dimension-compatible admitted inputs A/B whose term evidence differs;
  • fit or otherwise obtain an owner-issued reference fit for A;
  • scoring the owner-issued A fit succeeds;
  • substituting input B for the A model is impossible through the supported public scoring API or fails closed before emitting ModelCandidate;
  • candidate K is derived/bound to the fit, preserving scientific(selection): bind candidate K to fitted topic dimension #676;
  • select_fitted_candidate_k(...) uses the owner-issued fit path and retains existing fitted-selection recovery/LLM-refusal behavior.

Minimal causal direction

Make the public statistical scoring boundary consume ReferenceTopicFit rather than detached ReferenceTopicInput + ReferenceTopicModel. Migrate select_fitted_candidate_k(...) to ReferenceTopicFit::fit(...) and derive the candidate dimension from the retained model/config. Keep any detached likelihood arithmetic owner-internal or clearly non-authoritative; do not replace the owner aggregate with a locally computed digest.

If compatibility requires a staged API transition, the old detached scorer must not remain an equally authoritative public path indefinitely; record an explicit removal condition and keep CI warning-free.

Boundary

This is fit-local model-selection integrity. It does not prove Evidence source/vocabulary/event-time provenance (#527/#658/#671), relation promotion authority (#675), Membership authority (#605), fitted semantic topic identity (#663), or release projection (#664).

Refs #638 #639 #665 #670 #676.

Activity

  1. seonghobae commented on Sep 22, 2026

    @seonghobae
    ContributorAuthor

    Ordinary-forward repair on canonical #639 lane:

    • API RED b4aca694220d951f7d462f25ddb3e989efc3bc93: the public integration contract fits through ReferenceTopicFit::fit(...) and requires statistical_candidate_from_fit(&fit). The predecessor scorer still required detached (input, candidate_k, model), so this is a compile-level public-boundary RED. All five hosted runs on the superseded RED head were cancelled before useful execution; no hosted RED is claimed.
    • production repair 0a1f97a93d5d253f63e99325e8c756fc331ced95: the public scorer now accepts only &ReferenceTopicFit, reads input/model from the same aggregate, derives K from the retained fitted topic dimension, and select_fitted_candidate_k(...) now fits each candidate through ReferenceTopicFit::fit(...) before scoring. Likelihood, Schwarz penalty/free-parameter arithmetic, Pareto selection and LLM refusal are unchanged.
    • integration migration ef12e1dbda9161f0c496e319c3739a2fc9c36ad1: existing fitted-selection diagnostics now use owner-issued fits rather than hand-paired model/input state.
    • public rustdoc/export wording 1ca2a1f89a3a260f9db2dba0f91f27cd6ad9b777 names the owner-bound scoring contract.

    This closes the fit-local A-input/B-model substitution path for supported public model-selection scoring. It does not authenticate external Evidence/Temporal/Membership/topic semantics and does not close #658/#671/#675/#605/#663/#664.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions