Problem
pi-observability-plugin does not record any model parameters on generations, so Langfuse cannot show how a model was called — only which model.
Verified in src/index.ts (v0.1.2):
- the generation created in
before_provider_request gets model: ctx.model?.id and metadata.provider, nothing else — modelParameters is never set
rg "modelParameters|thinkingLevel|maxTokens" over src/ returns 0 matches
stop_reason is recorded on message_end, but the cap that produced a stop_reason: length is not
This hurts when:
- comparing runs of the same prompt at different thinking levels (
/thinking high vs low) — the traces are indistinguishable
- diagnosing truncated outputs — you see
stop_reason: length but not the max_tokens that was in effect
- reviewing traces later, once the pi settings that produced them have changed
Both values are already on the extension context, and pi resolves every request from the same two fields:
- max output tokens come from
model.maxTokens (clamped to the remaining context); the coding agent never overrides it per request, so ctx.model.maxTokens is what goes out
- reasoning comes from the thinking level, or nothing when it is
off; each provider maps that level to its own effort/budget field
- temperature and other sampling params are only sent when a caller sets them explicitly, which the coding agent does not
Expected
Every generation carries modelParameters with max_tokens (from ctx.model.maxTokens) and, for reasoning models with thinking on, thinking_level (from ctx.thinkingLevel), so the Langfuse UI shows them next to the model name.
Suggested implementation
- A pure
extractModelParameters(model, thinkingLevel) returning Record<string, string | number> | undefined, with unit tests for the reasoning / non-reasoning / off cases. pi clamps the level to off for non-reasoning models before sending, so a stale level on the context must not produce a misleading chip.
- In
before_provider_request, pass modelParameters: extractModelParameters(ctx.model, ctx.thinkingLevel) to startObservation. @langfuse/tracing drops undefined attributes, so no conditional spread is needed.
- Integration test on the sandbox model (
maxTokens: 8192, reasoning: false) → { "max_tokens": 8192 }; with reasoning: true and defaultThinkingLevel: "high" in settings.json → { "max_tokens": 8192, "thinking_level": "high" }.
Problem
pi-observability-plugin does not record any model parameters on generations, so Langfuse cannot show how a model was called — only which model.
Verified in
src/index.ts(v0.1.2):before_provider_requestgetsmodel: ctx.model?.idandmetadata.provider, nothing else —modelParametersis never setrg "modelParameters|thinkingLevel|maxTokens"oversrc/returns 0 matchesstop_reasonis recorded onmessage_end, but the cap that produced astop_reason: lengthis notThis hurts when:
/thinking highvslow) — the traces are indistinguishablestop_reason: lengthbut not themax_tokensthat was in effectBoth values are already on the extension context, and pi resolves every request from the same two fields:
model.maxTokens(clamped to the remaining context); the coding agent never overrides it per request, soctx.model.maxTokensis what goes outoff; each provider maps that level to its own effort/budget fieldExpected
Every generation carries
modelParameterswithmax_tokens(fromctx.model.maxTokens) and, for reasoning models with thinking on,thinking_level(fromctx.thinkingLevel), so the Langfuse UI shows them next to the model name.Suggested implementation
extractModelParameters(model, thinkingLevel)returningRecord<string, string | number> | undefined, with unit tests for the reasoning / non-reasoning /offcases. pi clamps the level toofffor non-reasoning models before sending, so a stale level on the context must not produce a misleading chip.before_provider_request, passmodelParameters: extractModelParameters(ctx.model, ctx.thinkingLevel)tostartObservation.@langfuse/tracingdropsundefinedattributes, so no conditional spread is needed.maxTokens: 8192,reasoning: false) →{ "max_tokens": 8192 }; withreasoning: trueanddefaultThinkingLevel: "high"insettings.json→{ "max_tokens": 8192, "thinking_level": "high" }.