Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 40 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,31 @@
# Braintrust Java SDK

An OpenTelemetry-based Braintrust SDK for Java 17 and up. Contains instrumentation for major AI vendors (OpenAI, Anthropic, etc)
An OpenTelemetry-based Braintrust SDK for Java 17 and up. Contains:

- Instrumentation for major AI vendors (OpenAI, Anthropic, etc)
- LLM Eval framework for sending Experiments to Braintrust
- Utils for sending Open Telemetry data to Braintrust

If you're simply looking to call the Braintrust API with Java code, see [https://github.com/braintrustdata/braintrust-java](https://github.com/braintrustdata/braintrust-java)

This SDK is currently is in BETA status and APIs may change

# Quickstart (Running examples and reporting data to Braintrust)
<!-- # Using the SDK in your code -->

<!-- *NOTE: The SDK has not published a release to maven yet. This section will not work until our first release is published* -->

<!-- build.gradle -->
<!-- ```gradle -->
<!-- dependencies { -->
<!-- implementation 'dev.braintrust:sdk:0.0.1' -->
<!-- } -->
<!-- ``` -->

# Examples

All examples can be found here: [./examples/src/main/java/dev/braintrust/examples](./examples/src/main/java/dev/braintrust/examples)

Prerequisites:
To run the examples from the command line you need:
- A Braintrust account and a valid BRAINTRUST_API_KEY
- Java 17 (or greater):
- macOS: `brew install openjdk@17`
Expand All @@ -25,21 +42,32 @@ If you wish to see all examples run:
./gradlew :examples:tasks --group="Braintrust SDK Examples"
```

Source code for all examples can be found here: [./examples/src/main/java/dev/braintrust/examples](./examples/src/main/java/dev/braintrust/examples)

If you wish to hack around with the examples you can modify source then re-run the gradle task.

# Using the SDK in your code
## Logging

The SDK uses a standard slf4j logger and will use the default log level (or not log at all if slf4j is not installed).

All Braintrust loggers will log into the `dev.braintrust` namespace. To adjust the log level, consult your logger documentation.

TODO: update this section when we have published our first release
For example, to enable debug logging for slf4j-simple you would set the system property `org.slf4j.simpleLogger.log.dev.braintrust=DEBUG`

# Developer Setup
# SDK Developer Docs

This section documents how to set up your machine to develop the SDK. This is not required if you simply wish to use the SDK or run examples.
The remaining sections document developing the SDK itself. Nothing below is required if you simply wish to use the SDK or run examples.

## Setup

TODO -- document sdkman, gradle, etc
- Install JDK 17
- Recommended to use SDK Man: https://sdkman.io/ and `sdk use java 17.0.16-tem`
- Ensure you can run all tests and checks: `./gradlew check build`
- IDE Setup
- Intellij Community
- Ubuntu: `sudo snap install intellij-idea-community`
- Other: https://www.jetbrains.com/idea/download/
- (Optional) Install pre-commit hooks: `./gradlew installGitHooks`
- These hooks automatically run common checks for you but CI also runs the same checks before merging to the main branch is allowed
- NOTE: this will overwrite existing hooks. Take backups before running

## Running a local OpenTelemetry collector

Expand All @@ -49,15 +77,12 @@ To run a local collector:

```
# Assumes you're in the repo root
docker run --rm -p 4318:4318 -v "$PWD/localcollector/collector.yaml:/etc/otelcol/config.yaml" otel/opentelemetry-collector:latest
docker run --rm -p 4318:4318 -v "$PWD/localcollector/collector.yaml:/etc/otelcol/config.yaml" otel/opentelemetry-collector:0.136.0 # latest release will probably also work
```

To send otel data to the local collector:
To send Braintrust otel data to the local collector:

```
# assumes you have BRAINTRUST_API_KEY and OPENAI_API_KEY exported
export BRAINTRUST_API_URL="http://localhost:4318" ; export BRAINTRUST_TRACES_PATH="/v1/traces"; export BRAINTRUST_LOGS_PATH="/v1/logs" ; ./gradlew :examples:runOpenAIInstrumentation
```


## TODO: finish these docs
8 changes: 0 additions & 8 deletions build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -149,11 +149,3 @@ task installGitHooks(type: Exec) {
group = 'Build Setup'
commandLine 'bash', 'scripts/install-hooks.sh'
}

// Run installGitHooks after project evaluation
afterEvaluate {
// Install hooks when building for the first time
tasks.named('build').configure {
dependsOn installGitHooks
}
}
4 changes: 4 additions & 0 deletions src/main/java/dev/braintrust/api/BraintrustApiClient.java
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,10 @@
import java.util.concurrent.ExecutionException;
import lombok.extern.slf4j.Slf4j;

/**
* Provides the necessary API calls for the Braintrust SDK. Users of the SDK should favor using
* {@link dev.braintrust.eval.Eval} or {@link dev.braintrust.trace.BraintrustTracing}
*/
public interface BraintrustApiClient {
/** Creates or gets a project by name. */
Project getOrCreateProject(String projectName);
Expand Down
8 changes: 7 additions & 1 deletion src/main/java/dev/braintrust/config/BraintrustConfig.java
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,13 @@
import lombok.Getter;
import lombok.experimental.Accessors;

/** Configuration for Braintrust SDK */
/**
* Configuration for Braintrust SDK with sane defaults.
*
* <p>Most SDK users will want to use envars to configure all Braintrust settings.
*
* <p>However, it's also possible to override any envar during config construction.
*/
@Getter
@Accessors(fluent = true)
public final class BraintrustConfig extends BaseConfig {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
import dev.braintrust.instrumentation.openai.otel.OpenAITelemetry;
import io.opentelemetry.api.OpenTelemetry;

/** Braintrust OpenAI client instrumentation. */
public class BraintrustOpenAI {
/** Instrument openai client with braintrust traces */
public static OpenAIClient wrapOpenAI(OpenTelemetry openTelemetry, OpenAIClient openAIClient) {
Expand Down