Skip to content

feat: record model parameters on generations - #25

Open
wierdbytes wants to merge 2 commits into
langfuse:mainfrom
wierdbytes:feat/model-parameters
Open

wierdbytes wants to merge 2 commits into
langfuse:mainfrom
wierdbytes:feat/model-parameters

Conversation

@wierdbytes

@wierdbytes wierdbytes commented Sep 3, 2026

Copy link
Copy Markdown

Closes #22.

Change

Generations carried model: ctx.model?.id and metadata.provider and nothing about how the model was called. stop_reason: length was recorded without the cap that produced it, and two runs at different thinking levels were indistinguishable.

Each generation now gets modelParameters from the two fields pi resolves every request from: max_tokens from ctx.model.maxTokens, which pi sends as the max output tokens (the coding agent never overrides it per request), and thinking_level from ctx.thinkingLevel. The level is only reported when ctx.model.reasoning is true and it is not off: pi clamps the level to off for non-reasoning models before sending, so a stale level on the context must not produce a misleading value. Temperature and other sampling parameters are only sent when a caller sets them explicitly, which the coding agent does not, so these two are the complete picture.

The values come from the context rather than the wire payload of before_provider_request. The payload spells them differently per dialect (max_tokens, max_completion_tokens, max_output_tokens, maxOutputTokens; reasoning_effort, reasoning.effort, output_config.effort, thinkingConfig.thinkingLevel, …), and every one of them is derived from the same two context fields.

extractModelParameters returns undefined when there is nothing to report, and @langfuse/tracing drops undefined attributes, so the field is passed unconditionally.

Before:

screen-2026-09-03-21 10 55

After:

screen-2026-09-03-21 16 09

Tests

  • pnpm typecheck is clean.
  • pnpm test gives 70 of 70 passed, up from 62 on main.
  • Six unit tests pin the mapping: max tokens always, thinking level only for reasoning models, never for off, undefined when neither applies (pi's Agent starts with maxTokens: 0).
  • Two integration runs read langfuse.observation.model.parameters off every generation: the sandbox model as is gives {"max_tokens":8192}; with reasoning: true in models.json and defaultThinkingLevel: "high" in settings.json it gives {"max_tokens":8192,"thinking_level":"high"}. On main both fail: the attribute is absent.

Closes langfuse#22.

Set `modelParameters` on each generation from what pi resolves for the
request — `ctx.model.maxTokens` (sent as the max output tokens) and, for
reasoning models, `ctx.thinkingLevel` — so the Langfuse UI shows them
next to the model name.

The coding agent never sets temperature or other sampling parameters, so
these two are the complete picture.
Unit tests pin the chip mapping (max tokens always, thinking level only for
reasoning models and never for `off`); the integration tests check the
exported attribute against the sandbox model as is and again with reasoning
enabled and a default thinking level configured.
@CLAassistant

CLAassistant commented Sep 3, 2026

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@wierdbytes
wierdbytes marked this pull request as ready for review September 3, 2026 19:19
@milanagm

milanagm commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator

Thanks you for raising the issue and creating this PR. I have adressed the bug in a different PR again which handles max_tokens differently.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Record model parameters (max tokens, thinking level) on generations

3 participants