feat: record model parameters on generations - #25
Open
wierdbytes wants to merge 2 commits into
Open
wierdbytes wants to merge 2 commits into
wierdbytes wants to merge 2 commits into
Conversation
Closes langfuse#22. Set `modelParameters` on each generation from what pi resolves for the request — `ctx.model.maxTokens` (sent as the max output tokens) and, for reasoning models, `ctx.thinkingLevel` — so the Langfuse UI shows them next to the model name. The coding agent never sets temperature or other sampling parameters, so these two are the complete picture.
Unit tests pin the chip mapping (max tokens always, thinking level only for reasoning models and never for `off`); the integration tests check the exported attribute against the sandbox model as is and again with reasoning enabled and a default thinking level configured.
|
|
wierdbytes
marked this pull request as ready for review
September 3, 2026 19:19
Collaborator
|
Thanks you for raising the issue and creating this PR. I have adressed the bug in a different PR again which handles |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #22.
Change
Generations carried
model: ctx.model?.idandmetadata.providerand nothing about how the model was called.stop_reason: lengthwas recorded without the cap that produced it, and two runs at different thinking levels were indistinguishable.Each generation now gets
modelParametersfrom the two fields pi resolves every request from:max_tokensfromctx.model.maxTokens, which pi sends as the max output tokens (the coding agent never overrides it per request), andthinking_levelfromctx.thinkingLevel. The level is only reported whenctx.model.reasoningis true and it is notoff: pi clamps the level toofffor non-reasoning models before sending, so a stale level on the context must not produce a misleading value. Temperature and other sampling parameters are only sent when a caller sets them explicitly, which the coding agent does not, so these two are the complete picture.The values come from the context rather than the wire payload of
before_provider_request. The payload spells them differently per dialect (max_tokens,max_completion_tokens,max_output_tokens,maxOutputTokens;reasoning_effort,reasoning.effort,output_config.effort,thinkingConfig.thinkingLevel, …), and every one of them is derived from the same two context fields.extractModelParametersreturnsundefinedwhen there is nothing to report, and@langfuse/tracingdropsundefinedattributes, so the field is passed unconditionally.Before:
After:
Tests
pnpm typecheckis clean.pnpm testgives 70 of 70 passed, up from 62 onmain.off,undefinedwhen neither applies (pi's Agent starts withmaxTokens: 0).langfuse.observation.model.parametersoff every generation: the sandbox model as is gives{"max_tokens":8192}; withreasoning: trueinmodels.jsonanddefaultThinkingLevel: "high"insettings.jsonit gives{"max_tokens":8192,"thinking_level":"high"}. Onmainboth fail: the attribute is absent.