You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
InstrumentedAzureOpenAIClient.GetEmbeddingClient() has an explicit TODO and delegates directly to the underlying client without wrapping, so all GenerateEmbedding / GenerateEmbeddings calls made through a Braintrust-instrumented Azure OpenAI client bypass telemetry entirely and produce no spans.
This is a separate call site from issue #35 (which covers InstrumentedOpenAIClient.GetEmbeddingClient() in the plain OpenAI integration). The fix to #35 would not automatically fix the Azure case — InstrumentedAzureOpenAIClient has its own GetEmbeddingClient override that also needs to be updated.
What is missing
In InstrumentedAzureOpenAIClient (src/Braintrust.Sdk.AzureOpenAI/InstrumentedAzureOpenAIClient.cs):
// lines 63–68/// <summary>/// Delegates GetEmbeddingClient to the wrapped client./// TODO: Add instrumentation for embeddings if needed./// </summary>publicoverrideEmbeddingClientGetEmbeddingClient(stringdeploymentName){return_client.GetEmbeddingClient(deploymentName);}
For comparison, GetChatClient in the same class is wrapped with InstrumentedChatClient and produces fully traced spans.
The EmbeddingClient returned by the unwrapped call exposes these virtual methods that bypass telemetry:
Users calling embeddings through BraintrustAzureOpenAI.WithBraintrust(client).GetEmbeddingClient("my-embedding-deployment") get no spans, no token usage, and no model metadata — even though chat completions from the same instrumented client are fully traced.
"Chat completion spans (Chat Completion), with input and output messages, request and response model, and token usage (prompt, completion, and total)."
Embeddings are not mentioned. The Braintrust proxy API does document an embeddings endpoint (/v1/proxy/embeddings), confirming embeddings are a recognized surface on the platform.
Status: not_found (embeddings instrumentation not documented for Azure OpenAI in Braintrust docs).
Upstream sources
Azure OpenAI .NET SDK: https://www.nuget.org/packages/Azure.AI.OpenAI — AzureOpenAIClient.GetEmbeddingClient(string deploymentName) returns an EmbeddingClient (from OpenAI.Embeddings) for generating text embeddings against Azure-deployed embedding models
src/Braintrust.Sdk.AzureOpenAI/InstrumentedAzureOpenAIClient.cs — GetEmbeddingClient() at lines 63–68 delegates without instrumentation; explicit TODO comment
src/Braintrust.Sdk.AzureOpenAI/InstrumentedAzureOpenAIClient.cs — GetChatClient() at lines 46–52 wraps with InstrumentedChatClient — reference pattern for the fix
src/Braintrust.Sdk.OpenAI/Braintrust.Sdk.OpenAI.csproj — InternalsVisibleTo includes Braintrust.Sdk.AzureOpenAI, so an InstrumentedEmbeddingClient added to the OpenAI package will be accessible here, but InstrumentedAzureOpenAIClient.GetEmbeddingClient() still needs to be updated separately
Summary
InstrumentedAzureOpenAIClient.GetEmbeddingClient()has an explicit TODO and delegates directly to the underlying client without wrapping, so allGenerateEmbedding/GenerateEmbeddingscalls made through a Braintrust-instrumented Azure OpenAI client bypass telemetry entirely and produce no spans.This is a separate call site from issue #35 (which covers
InstrumentedOpenAIClient.GetEmbeddingClient()in the plain OpenAI integration). The fix to #35 would not automatically fix the Azure case —InstrumentedAzureOpenAIClienthas its ownGetEmbeddingClientoverride that also needs to be updated.What is missing
In
InstrumentedAzureOpenAIClient(src/Braintrust.Sdk.AzureOpenAI/InstrumentedAzureOpenAIClient.cs):For comparison,
GetChatClientin the same class is wrapped withInstrumentedChatClientand produces fully traced spans.The
EmbeddingClientreturned by the unwrapped call exposes these virtual methods that bypass telemetry:GenerateEmbedding(string, EmbeddingGenerationOptions?, CancellationToken)→ClientResult<OpenAIEmbedding>GenerateEmbeddingAsyncGenerateEmbeddings(IEnumerable<string>, ...)→ClientResult<OpenAIEmbeddingCollection>GenerateEmbeddingsAsyncUsers calling embeddings through
BraintrustAzureOpenAI.WithBraintrust(client).GetEmbeddingClient("my-embedding-deployment")get no spans, no token usage, and no model metadata — even though chat completions from the same instrumented client are fully traced.Braintrust docs status
The Braintrust Azure integration docs at https://www.braintrust.dev/docs/integrations/ai-providers/azure document only chat completion instrumentation:
Embeddings are not mentioned. The Braintrust proxy API does document an embeddings endpoint (
/v1/proxy/embeddings), confirming embeddings are a recognized surface on the platform.Status: not_found (embeddings instrumentation not documented for Azure OpenAI in Braintrust docs).
Upstream sources
AzureOpenAIClient.GetEmbeddingClient(string deploymentName)returns anEmbeddingClient(fromOpenAI.Embeddings) for generating text embeddings against Azure-deployed embedding modelsLocal files inspected
src/Braintrust.Sdk.AzureOpenAI/InstrumentedAzureOpenAIClient.cs—GetEmbeddingClient()at lines 63–68 delegates without instrumentation; explicit TODO commentsrc/Braintrust.Sdk.AzureOpenAI/InstrumentedAzureOpenAIClient.cs—GetChatClient()at lines 46–52 wraps withInstrumentedChatClient— reference pattern for the fixsrc/Braintrust.Sdk.OpenAI/InstrumentedOpenAIClient.cs— same gap on the OpenAI (non-Azure) client, tracked in issue [BOT ISSUE] OpenAI embeddings calls are not instrumented #35src/Braintrust.Sdk.OpenAI/Braintrust.Sdk.OpenAI.csproj—InternalsVisibleToincludesBraintrust.Sdk.AzureOpenAI, so anInstrumentedEmbeddingClientadded to the OpenAI package will be accessible here, butInstrumentedAzureOpenAIClient.GetEmbeddingClient()still needs to be updated separately