[bot] AWS Bedrock InvokeModel for embeddings not instrumented

[braintrust-bot]

Summary

The AWS Bedrock instrumentation in BraintrustBedrockInterceptor explicitly skips all operations other than Converse and ConverseStream. This means embedding model calls — which use the InvokeModel API, not Converse — produce no spans, no input capture, and no token metrics.

All embedding models on AWS Bedrock require InvokeModel:

• Amazon Titan Text Embeddings v2 (amazon.titan-embed-text-v2:0)
• Amazon Titan Text Embeddings v1 (amazon.titan-embed-text-v1)
• Amazon Titan Multimodal Embeddings (amazon.titan-embed-image-v1)
• Cohere Embed English v3 (cohere.embed-english-v3)
• Cohere Embed Multilingual v3 (cohere.embed-multilingual-v3)

None of these models support the Converse API — InvokeModel is the only invocation path for Bedrock embeddings.

What is missing

BraintrustBedrockInterceptor.beforeExecution() (lines 60–71) explicitly limits instrumentation to two operations:

private static final Set<String> INSTRUMENTED_OPERATIONS = Set.of("Converse", "ConverseStream");

// Only instrument Converse and ConverseStream — other Bedrock operations
// (InvokeModel, ApplyGuardrail, etc.) are not LLM calls we know how to tag.
if (!INSTRUMENTED_OPERATIONS.contains(operationName)) {
    return;
}

When a user calls client.invokeModel(...) with an embedding model (e.g. amazon.titan-embed-text-v2:0), the interceptor returns immediately without creating a span. No span is created, no input text is captured, and no embedding output or token metrics are recorded.

The BraintrustAWSBedrock.wrap() Javadoc on the sync builder explicitly states it traces "every converse call" and the async builder traces "every converseStream call" — embedding calls via InvokeModel are not in scope at all today.

A minimal InvokeModel embedding span should capture:

• braintrust.metadata: model ID (from the request URI path /model/{modelId}/invoke), provider
• braintrust.input_json: the inputText (or equivalent input field) from the request body
• braintrust.metrics: prompt_tokens from the response inputTextTokenCount (Titan) or meta.inputTokenCount (Cohere)
• braintrust.span_attributes: {"type": "embedding"}

Braintrust docs status

• AWS Bedrock listed as a supported cloud provider at https://www.braintrust.dev/docs/integrations/ai-providers: supported (for generative use)
• No mention of Bedrock embeddings or InvokeModel instrumentation in any Braintrust docs page: not_found
• For comparison, OpenAI and Google GenAI embedding instrumentation exists in this repo (though with detail gaps tracked in [BOT ISSUE] OpenAI embeddings spans missing output capture and incomplete input extraction #66 and [BOT ISSUE] Google GenAI embedContent spans lack embedding-specific input, metrics, and span type #65)

Upstream sources

• AWS Bedrock InvokeModel API reference: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModel.html — "You use model inference to generate text, images, and embeddings."
• Amazon Titan Text Embeddings v2 example: https://docs.aws.amazon.com/bedrock/latest/userguide/bedrock-runtime_example_bedrock-runtime_InvokeModel_TitanTextEmbeddings_section.html — uses client.invokeModel() with model ID amazon.titan-embed-text-v2:0; response includes embedding array and inputTextTokenCount
• Cohere Embed on Bedrock: https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-embed.html — uses InvokeModel with texts array; response includes embeddings and meta.inputTokenCount
• AWS SDK for Java v2 BedrockRuntimeClient: invokeModel(InvokeModelRequest) — operation name InvokeModel, same SDK client as Converse

Local files inspected

• braintrust-sdk/instrumentation/aws_bedrock_2_30_0/src/main/java/dev/braintrust/instrumentation/awsbedrock/v2_30_0/BraintrustBedrockInterceptor.java — line 60: Set.of("Converse", "ConverseStream"); lines 63–71: beforeExecution returns without span for any other operation name including InvokeModel
• braintrust-sdk/instrumentation/aws_bedrock_2_30_0/src/main/java/dev/braintrust/instrumentation/awsbedrock/v2_30_0/BraintrustAWSBedrock.java — Javadoc on both wrap() overloads confirms only Converse/ConverseStream are traced; no invokeModel wrapping exists
• braintrust-sdk/instrumentation/aws_bedrock_2_30_0/src/test/java/dev/braintrust/instrumentation/awsbedrock/v2_30_0/BraintrustAWSBedrockTest.java — all tests use ConverseRequest and ConverseStreamRequest; no InvokeModelRequest test exists
• braintrust-sdk/src/main/java/dev/braintrust/instrumentation/InstrumentationSemConv.java — tagBedrockRequest() and tagBedrockResponse() only handle Converse-shaped JSON (messages, output.message, usage.inputTokens); InvokeModel request/response bodies have entirely different schemas