diff --git a/hugo/content/en/llm_observability/instrumentation/sdk.md b/hugo/content/en/llm_observability/instrumentation/sdk.md index a508b2b379b..845ff118b41 100644 --- a/hugo/content/en/llm_observability/instrumentation/sdk.md +++ b/hugo/content/en/llm_observability/instrumentation/sdk.md @@ -1654,9 +1654,9 @@ The SDK provides several methods to annotate spans with inputs, outputs, metrics ### Annotating inputs and outputs -Use the `annotateIO()` member method of the `LLMObsSpan` interface to add structured input and output data to an `LLMObsSpan`. This includes optional arguments and LLM message objects. +The `LLMObsSpan` interface provides three methods for annotating span inputs and outputs. Use `annotateIO()` for LLM, task, agent, workflow, and tool spans. Use `annotateEmbeddingIO()` to annotate an embedding span's inputs with document objects, and `annotateRetrievalIO()` to annotate a retrieval span's outputs with document objects. `annotateEmbeddingIO()` and `annotateRetrievalIO()` require `dd-trace-java` 1.66.0 or later. -#### Arguments +#### annotateIO() arguments If an argument is null or empty, nothing happens. For example, if `inputData` is a non-empty string while `outputData` is null, then only `inputData` is recorded. @@ -1668,6 +1668,30 @@ If an argument is null or empty, nothing happens. For example, if `inputData` is : optional - _String_ or _List_
Either a string (for non-LLM spans) or a list of `LLMObs.LLMMessage`s for LLM spans. +#### annotateEmbeddingIO() arguments + +If an argument is null or empty, nothing happens. + +`inputDocuments` +: optional - _List_ +
A list of documents passed to the embedding model as input. + +`outputData` +: optional - _String_ +
The embedding output as a string. + +#### annotateRetrievalIO() arguments + +If an argument is null or empty, nothing happens. + +`inputData` +: optional - _String_ +
The retrieval query as a string. + +`outputDocuments` +: optional - _List_ +
A list of documents returned by the retrieval operation. + #### LLM Messages LLM spans must be annotated with LLM Messages using the `LLMObs.LLMMessage` object. @@ -1681,7 +1705,28 @@ The `LLMObs.LLMMessage` object can be instantiated by calling `LLMObs.LLMMessage : required - _String_
A string containing the content of the message. -#### Example +#### Documents +Embedding and retrieval spans use the `LLMObs.Document` class to represent individual documents. + +The `LLMObs.Document` object can be instantiated by calling `LLMObs.Document.from()` with the following arguments: + +`text` +: required - _String_ +
The text content of the document. + +`name` +: optional - _String_ +
A name for the document. + +`id` +: optional - _String_ +
A unique identifier for the document. + +`score` +: optional - _Double_ +
A relevance score for the document. + +#### Examples ```java import datadog.trace.api.llmobs.LLMObs; @@ -1706,6 +1751,42 @@ public class MyJavaClass { } ``` +```java +import datadog.trace.api.llmobs.LLMObs; + +public class MyJavaClass { + public float[] performEmbedding(String inputText) { + LLMObsSpan embeddingSpan = LLMObs.startEmbeddingSpan("embed-text", "text-embedding-3", "openai", null, "session-141"); + float[] embeddings = ... // user application logic to generate embeddings + embeddingSpan.annotateEmbeddingIO( + Arrays.asList(LLMObs.Document.from(inputText, null, null, null)), + null + ); + embeddingSpan.finish(); + return embeddings; + } +} +``` + +```java +import datadog.trace.api.llmobs.LLMObs; + +public class MyJavaClass { + public List retrieveDocuments(String query) { + LLMObsSpan retrievalSpan = LLMObs.startRetrievalSpan("retrieve-docs", null, "session-141"); + List docs = ... // user application logic to retrieve documents + retrievalSpan.annotateRetrievalIO( + query, + Arrays.asList( + LLMObs.Document.from(docs.get(0).text, docs.get(0).name, docs.get(0).id, docs.get(0).score) + ) + ); + retrievalSpan.finish(); + return docs; + } +} +``` + ### Adding metrics #### Bulk add metrics