Skip to content

Compare legacy prediction with conservative slot filling - #10

Closed
enaboapps wants to merge 4 commits into
mainfrom
experiment/legacy-slot-fill
Closed

enaboapps wants to merge 4 commits into
mainfrom
experiment/legacy-slot-fill

Conversation

@enaboapps

@enaboapps enaboapps commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Adds opt-in original, newer and combined prediction modes. Combined mode preserves newer suggestions and fills empty slots with normalized legacy suggestions, without mixing scores or personal counts. Existing APIs and default production model remain unchanged. No legacy database is distributed.

Closes #9.

The frozen comparison covers all six existing AAC, general-English and conversational partitions, prefixes zero through four, and 102,481 warmed queries per mode. Every newer suggestion retained its position. At two characters, test top-five accuracy is unchanged for AAC at 85.17%, improves general English 66.40% to 66.44%, and conversation 86.24% to 86.27%. At two characters gains are small because five newer suggestions usually occupy the list. At four characters, general English improves 83.24% to 87.79%, AAC 94.28% to 95.67%, and conversation 95.38% to 96.18%. Combined worst-partition p95 is 6.40 ms; cold loading is 6.6–6.8 seconds. Full metrics and limitations are in docs/legacy-comparison-results.md and .json.

Validation: Rust formatting, Clippy with warnings denied, 25 Rust tests, 14 Python tests and dependency audit passed locally. Hashes verify both source databases and all six partitions; databases are unchanged after evaluation. Historical ranking fixture and synthetic tests cover normalization, boundaries, corrupt data, personal-count exclusion and source immutability. Independent review cleared final head 63d7867 with no actionable findings and verified all 18 reports. All latest-head CI checks passed, including Windows/macOS/Linux checks, dependency audit, production database validation and packaged smoke tests on all three platforms.

No desktop integration, merge, model promotion or release is included.

@enaboapps enaboapps added this to the v0.2.0 milestone Oct 3, 2026
@enaboapps
enaboapps marked this pull request as ready for review October 3, 2026 09:01
@enaboapps enaboapps closed this Oct 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Compare legacy and modern predictors with conservative slot filling

2 participants