feat(history): snapshot actual LLM (provider/model) per review record (RENG-38) - #154
Open
Liewzheng wants to merge 1 commit into
Open
feat(history): snapshot actual LLM (provider/model) per review record (RENG-38)#154Liewzheng wants to merge 1 commit into
Liewzheng wants to merge 1 commit into
Conversation
…cord (RENG-38) (0.10.2)
Every history record now shows which LLM actually produced each expert
report, at expert granularity:
- CompletionResult carries the provider name; complete_with_fallback
attributes a success to the exact fallback-chain config that answered.
- ExpertReport/AggregatedReport gain #[serde(default)] Option
llm_provider/llm_model, backfilled at all expert/aggregator production
points (pipeline, orchestrator, expert runner); pre-0.10.2 JSON still
deserializes.
- Migration 0002_llm_snapshot.sql: expert_reports.llm_provider/llm_model
+ reviews.llm_summary (deduped [{provider, model}] JSON, written on
terminal completion so the list page never parses reviews.result).
Name snapshots, not FKs: llm_providers is rewritten DELETE+INSERT on
every config save, so foreign keys would dangle.
- API: GET /reviews/{id} experts gain llmProvider/llmModel; GET /reviews
items gain llmSummary. History page: per-expert provider/model tag in
the detail drawer + a compact LLM list column; records predating the
snapshot render as unknown/- instead of erroring.
Deferred: lead-overview/verifier/adjudicator passes call the LLM but
produce no expert_reports row, so their provider/model is not yet
snapshotted.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Records which LLM (provider/model) was actually used for each review in the history, instead of only showing the currently configured one.
Changes
expert_reportsandreviews.Test evidence
Plane: RENG-38