Proposal: Braintrust backend should consume gen_ai.* OTEL GenAI semantic convention attributes

[brendangooden]

Problem

The Braintrust backend currently requires proprietary braintrust.* span attributes to render the dashboard (input/output panels, span type identification, token metrics, project routing). This means every Braintrust SDK instrumentation package must emit braintrust.* attributes — even when standard gen_ai.* OTEL GenAI semantic convention attributes are already present on spans from framework-level instrumentation.

Current State

What the backend requires

Backend Function	Required Attribute	Standard Equivalent
Route spans to project	braintrust.parent	(no equivalent — Braintrust-specific)
Render input panel	braintrust.input_json	gen_ai.input.messages
Render output panel	braintrust.output_json	gen_ai.output.messages
Identify span type	braintrust.span_attributes {"type":"llm"}	gen_ai.operation.name = "chat"
Token metrics	braintrust.metrics.prompt_tokens	gen_ai.usage.input_tokens
Token metrics	braintrust.metrics.completion_tokens	gen_ai.usage.output_tokens
Timing metrics	braintrust.metrics.time_to_first_token	(no direct equivalent)

What M.E.AI already provides

Microsoft.Extensions.AI's UseOpenTelemetry() middleware emits 20+ gen_ai.* attributes per OTEL semconv v1.40, including all the standard equivalents above. The upcoming M.E.AI release (post 10.4.0) also adds execute_tool spans with gen_ai.tool.call.arguments and gen_ai.tool.call.result.

The duplication problem

Today, an IChatClient user must stack two middleware layers that do overlapping work:

var client = new ChatClientBuilder(innerClient)
    .UseOpenTelemetry()         // emits gen_ai.* (standard OTEL)
    .UseBraintrustTracing()     // emits braintrust.* (Braintrust-specific, largely duplicative)
    .Build();

Both capture prompts, completions, token usage, and model info — just in different attribute namespaces. The Braintrust middleware exists solely because the backend doesn't read gen_ai.*.

Proposal

Update the Braintrust OTEL ingestion endpoint to also consume gen_ai.* attributes when braintrust.* equivalents are absent:

1. Input rendering: if braintrust.input_json is missing, fall back to gen_ai.input.messages
2. Output rendering: if braintrust.output_json is missing, fall back to gen_ai.output.messages
3. Span type: if braintrust.span_attributes is missing, derive type from gen_ai.operation.name ("chat" → "llm", "execute_tool" → "function_call", "invoke_agent" → "agent")
4. Token metrics: if braintrust.metrics.prompt_tokens is missing, fall back to gen_ai.usage.input_tokens (and similarly for completion tokens)
5. Model: read gen_ai.request.model and gen_ai.response.model

braintrust.parent remains required for routing (no OTEL equivalent).

Benefits

• Simpler SDK: UseBraintrustTracing() reduces to just adding braintrust.parent for routing — one tag instead of duplicating all content
• Zero-config for M.E.AI users: standard UseOpenTelemetry() spans render in Braintrust without any Braintrust-specific middleware for basic visibility
• Cross-platform: any OTEL-instrumented GenAI app (including Python, TypeScript SDKs using gen_ai.* conventions) gets basic Braintrust dashboard support
• Future-proof: as the OTEL GenAI spec stabilizes, Braintrust automatically benefits from framework improvements

Migration Path

1. Backend adds gen_ai.* fallback reading (non-breaking — braintrust.* still takes priority)
2. SDK deprecates braintrust.input_json / braintrust.output_json / braintrust.metrics.* on IChatClient packages in favor of gen_ai.*
3. SDK's UseBraintrustTracing() for IChatClient reduces to only braintrust.parent + braintrust.span_attributes
4. Raw SDK packages (OpenAI, Anthropic) continue emitting both until those SDKs add native OTEL

Context

• PR feat: OTEL GenAI semantic conventions + Braintrust.Sdk.Extensions.AI package #42 adds gen_ai.* attributes to tool spans and raw SDK packages
• dotnet/extensions main already has native execute_tool spans (unreleased, post 10.4.0)
• OTEL GenAI semantic conventions: https://opentelemetry.io/docs/specs/semconv/gen-ai/
• M.E.AI OpenTelemetryChatClient: https://learn.microsoft.com/en-us/dotnet/api/microsoft.extensions.ai.opentelemetrychatclient