Skip to content

[bot] AzureOpenAI EmbeddingClient calls are not instrumented #71

Description

@braintrust-bot

Summary

InstrumentedAzureOpenAIClient.GetEmbeddingClient() has an explicit TODO and delegates directly to the underlying client without wrapping, so all GenerateEmbedding / GenerateEmbeddings calls made through a Braintrust-instrumented Azure OpenAI client bypass telemetry entirely and produce no spans.

This is a separate call site from issue #35 (which covers InstrumentedOpenAIClient.GetEmbeddingClient() in the plain OpenAI integration). The fix to #35 would not automatically fix the Azure case — InstrumentedAzureOpenAIClient has its own GetEmbeddingClient override that also needs to be updated.

What is missing

In InstrumentedAzureOpenAIClient (src/Braintrust.Sdk.AzureOpenAI/InstrumentedAzureOpenAIClient.cs):

// lines 63–68
/// <summary>
/// Delegates GetEmbeddingClient to the wrapped client.
/// TODO: Add instrumentation for embeddings if needed.
/// </summary>
public override EmbeddingClient GetEmbeddingClient(string deploymentName)
{
    return _client.GetEmbeddingClient(deploymentName);
}

For comparison, GetChatClient in the same class is wrapped with InstrumentedChatClient and produces fully traced spans.

The EmbeddingClient returned by the unwrapped call exposes these virtual methods that bypass telemetry:

Method Signature Traced?
GenerateEmbedding (string, EmbeddingGenerationOptions?, CancellationToken)ClientResult<OpenAIEmbedding>
GenerateEmbeddingAsync async variant
GenerateEmbeddings (IEnumerable<string>, ...)ClientResult<OpenAIEmbeddingCollection>
GenerateEmbeddingsAsync async variant

Users calling embeddings through BraintrustAzureOpenAI.WithBraintrust(client).GetEmbeddingClient("my-embedding-deployment") get no spans, no token usage, and no model metadata — even though chat completions from the same instrumented client are fully traced.

Braintrust docs status

The Braintrust Azure integration docs at https://www.braintrust.dev/docs/integrations/ai-providers/azure document only chat completion instrumentation:

"Chat completion spans (Chat Completion), with input and output messages, request and response model, and token usage (prompt, completion, and total)."

Embeddings are not mentioned. The Braintrust proxy API does document an embeddings endpoint (/v1/proxy/embeddings), confirming embeddings are a recognized surface on the platform.

Status: not_found (embeddings instrumentation not documented for Azure OpenAI in Braintrust docs).

Upstream sources

Local files inspected

  • src/Braintrust.Sdk.AzureOpenAI/InstrumentedAzureOpenAIClient.csGetEmbeddingClient() at lines 63–68 delegates without instrumentation; explicit TODO comment
  • src/Braintrust.Sdk.AzureOpenAI/InstrumentedAzureOpenAIClient.csGetChatClient() at lines 46–52 wraps with InstrumentedChatClient — reference pattern for the fix
  • src/Braintrust.Sdk.OpenAI/InstrumentedOpenAIClient.cs — same gap on the OpenAI (non-Azure) client, tracked in issue [BOT ISSUE] OpenAI embeddings calls are not instrumented #35
  • src/Braintrust.Sdk.OpenAI/Braintrust.Sdk.OpenAI.csprojInternalsVisibleTo includes Braintrust.Sdk.AzureOpenAI, so an InstrumentedEmbeddingClient added to the OpenAI package will be accessible here, but InstrumentedAzureOpenAIClient.GetEmbeddingClient() still needs to be updated separately

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions