Skip to content

Add span origin environment provenance #40

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Stephen Belanger (Qard) wants to merge 1 commit into
base: main
Choose a base branch
from
+124 −19
Open

Add span origin environment provenance #40

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
78 changes: 70 additions & 8 deletions docs/features/environment-variables.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -101,14 +101,16 @@ SDKs may read only variables explicitly marked with `env_braintrust: true` in

The current `.env.braintrust` allowlist is:

| Variable | Purpose |
| ----------------------- | ------------------------------------------------ |
| `BRAINTRUST_API_KEY` | API key for authentication with Braintrust. |
| `BRAINTRUST_APP_URL` | Base URL for the Braintrust web application. |
| `BRAINTRUST_API_URL` | Base URL for the Braintrust API. |
| `BRAINTRUST_PROJECT` | Project name for Braintrust logging and tracing. |
| `BRAINTRUST_PROJECT_ID` | Project ID for logging spans. |
| `BRAINTRUST_ORG_NAME` | Organization name to use when logging in. |
| Variable | Purpose |
| ------------------------------ | ------------------------------------------------ |
| `BRAINTRUST_API_KEY` | API key for authentication with Braintrust. |
| `BRAINTRUST_APP_URL` | Base URL for the Braintrust web application. |
| `BRAINTRUST_API_URL` | Base URL for the Braintrust API. |
| `BRAINTRUST_PROJECT` | Project name for Braintrust logging and tracing. |
| `BRAINTRUST_PROJECT_ID` | Project ID for logging spans. |
| `BRAINTRUST_ORG_NAME` | Organization name to use when logging in. |
| `BRAINTRUST_ENVIRONMENT_TYPE` | Explicit span-origin environment type. |
| `BRAINTRUST_ENVIRONMENT_NAME` | Explicit span-origin environment name. |

If `.env.braintrust` contains other names, SDKs must ignore them. They must not
expose ignored values through SDK config APIs or process-environment helpers.
Expand Down Expand Up @@ -165,6 +167,66 @@ SDKs may start candidate file reads in parallel, but must still preserve
nearest-file-wins semantics. A higher parent may only win after all closer
candidates are known to be missing.

## Span origin environment

SDKs MAY populate `context.span_origin.environment` to identify the operating
environment where a span was captured. This is provenance about the caller's
runtime, not the Braintrust API or app URL environment.

SDKs SHOULD support explicit caller options equivalent to:

```json
{
"type": "ci",
"name": "github_actions"
}
```

SDKs SHOULD also support environment-variable overrides through the normal
Braintrust configuration precedence, including `.env.braintrust` fallback:

| Variable | Purpose |
| ------------------------------ | ------------------------------------------------------ |
| `BRAINTRUST_ENVIRONMENT_TYPE` | Explicit `context.span_origin.environment.type` value. |
| `BRAINTRUST_ENVIRONMENT_NAME` | Explicit `context.span_origin.environment.name` value. |

The environment type SHOULD be one of `ci`, `server`, `local`, or `gateway`,
but SDK type definitions SHOULD allow future string values. Environment names
SHOULD be normalized lower-snake-case labels.

Use this precedence:

1. A caller-provided SDK option wins over all ambient configuration.
2. `BRAINTRUST_ENVIRONMENT_TYPE` and `BRAINTRUST_ENVIRONMENT_NAME`, resolved
through process environment and `.env.braintrust` fallback, win over
automatic detection.
3. Trusted Braintrust Gateway/internal code may set `gateway`.
4. CI provider detection wins over server and language/framework detection.
5. Server/platform detection wins over language/framework detection.
6. Language/framework deployment-mode detection is a fallback.
7. If no reliable positive signal is present, omit `span_origin.environment`.

SDKs must not infer `local` from the absence of CI or server signals.
`gateway` is reserved for spans captured by Braintrust Gateway/internal code and
must not be inferred automatically by SDKs or from `metadata.provider`.

Reliable CI signals include `GITHUB_ACTIONS`, `GITLAB_CI`, `CIRCLECI`,
`BUILDKITE`, `JENKINS_URL`, `JENKINS_HOME`, `TF_BUILD`, `TEAMCITY_VERSION`,
`TRAVIS`, and `BITBUCKET_BUILD_NUMBER`. If no provider-specific signal is
present but `CI` is truthy, SDKs may emit type `ci` and name `ci`.

Reliable server/platform signals include `VERCEL`, `NETLIFY`,
`AWS_LAMBDA_FUNCTION_NAME`, Lambda `AWS_EXECUTION_ENV`, `K_SERVICE`,
`FUNCTION_TARGET`, `KUBERNETES_SERVICE_HOST`, `ECS_CONTAINER_METADATA_URI`,
`ECS_CONTAINER_METADATA_URI_V4`, `DYNO`, `FLY_APP_NAME`,
`RAILWAY_ENVIRONMENT`, and `RENDER_SERVICE_NAME`.

Language/framework variables such as `NODE_ENV`, `RAILS_ENV`, `RACK_ENV`,
`ASPNETCORE_ENVIRONMENT`, and `DOTNET_ENVIRONMENT` are weak fallback signals.
Map `production`, `prod`, `staging`, and `stage` to type `server`; map
`development`, `dev`, and `local` to type `local`; ignore `test` unless a
CI signal already classified the environment.

### Dotenv parsing

SDKs should use their runtime's standard dotenv parser when one is already
Expand Down
50 changes: 39 additions & 11 deletions docs/instrumentation-guide.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,14 +87,16 @@ The `metrics` field is an object of string keys to numeric values. Instrumentati

The `context` field is an object containing textual information about the code and systems that produced the span. SDKs MUST preserve the existing caller-location fields when available; caller-location fields are optional because not every runtime or instrumentation path can determine them.

| Field | Type | Description |
| ---------------------------------- | --------------- | ----------------------------------------------------------------------------------- |
| `caller_functionname` | string optional | Function or method that created the span |
| `caller_filename` | string optional | File where the span was created |
| `caller_lineno` | number optional | Line number where the span was created |
| `span_origin.name` | string optional | SDK, integration, Braintrust service, or exporter that emitted or exported the span |
| `span_origin.version` | string optional | Version of the SDK, integration, Braintrust service, or exporter when known |
| `span_origin.instrumentation.name` | string optional | Stable module, package, plugin, or OTel instrumentation scope that created the span |
| Field | Type | Description |
| ---------------------------------- | --------------- | -------------------------------------------------------------------------------------------- |
| `caller_functionname` | string optional | Function or method that created the span |
| `caller_filename` | string optional | File where the span was created |
| `caller_lineno` | number optional | Line number where the span was created |
| `span_origin.name` | string optional | SDK, integration, Braintrust service, or exporter that emitted or exported the span |
| `span_origin.version` | string optional | Version of the SDK, integration, Braintrust service, or exporter when known |
| `span_origin.instrumentation.name` | string optional | Stable module, package, plugin, or OTel instrumentation scope that created the span |
| `span_origin.environment.type` | string optional | Operating environment type where the span was captured: `ci`, `server`, `local`, or `gateway` |
| `span_origin.environment.name` | string optional | Normalized operating environment name, such as `github_actions`, `vercel`, or `development` |

Braintrust-created spans MUST also include span-origin provenance. Omit fields whose values are unknown.

Expand All @@ -104,21 +106,41 @@ Braintrust-created spans MUST also include span-origin provenance. Omit fields w
"caller_filename": "app.py",
"caller_lineno": 42,
"span_origin": {
"name": "braintrust.javascript",
"name": "braintrust.sdk.javascript",
"version": "1.2.3",
"instrumentation": {
"name": "openai-auto"
},
"environment": {
"type": "ci",
"name": "github_actions"
}
}
}
```

OTLP ingestion will map standard OTel code attributes into caller-location context when explicit context is not provided: `code.function.name` → `context.caller_functionname`, `code.file.path` → `context.caller_filename`, and `code.line.number` → `context.caller_lineno`.

`context.span_origin` identifies the SDK, integration, Braintrust service, or exporter that emitted or exported the span. For external OTLP spans that do not include Braintrust span-origin provenance, ingestion SHOULD fall back to OTel `telemetry.sdk.name` and `telemetry.sdk.version` resource attributes.
`context.span_origin` identifies the SDK, integration, service, or exporter that emitted or exported the span. For OTLP spans that do not include explicit Braintrust span-origin provenance, ingestion SHOULD set `context.span_origin.name` to `opentelemetry` and SHOULD fill `context.span_origin.version` from the OTel `telemetry.sdk.version` resource attribute when present.

Braintrust-maintained SDKs, plugins, services, and exporters SHOULD use stable `span_origin.name` values under the reserved `braintrust.` prefix when they provide explicit Braintrust span-origin provenance. Braintrust SDKs SHOULD use `braintrust.sdk.<language>` names, such as `braintrust.sdk.javascript`; Braintrust plugins SHOULD use `braintrust.plugin.<plugin>` names, such as `braintrust.plugin.codex`. User code and third-party integrations SHOULD NOT use the `braintrust.` prefix for caller-provided origin names.

`context.span_origin.instrumentation.name` identifies the stable module, package, plugin, or OTel instrumentation scope that directly created the span. Provider/client SDKs such as `openai` or `anthropic` are not the span origin and SHOULD continue to appear in `metadata.provider` when provider metadata is available.

`context.span_origin.environment` identifies the operating environment where the span was captured. The `type` field SHOULD be one of `ci`, `server`, `local`, or `gateway`; SDK type definitions SHOULD allow future string values. The `name` field is optional and SHOULD be a normalized lower-snake-case label. Emit this object only when the value comes from an explicit override or a reliable positive signal; do not infer `local` from the absence of CI or server signals. SDKs MUST NOT automatically classify spans as `gateway`; `gateway` is reserved for spans captured by Braintrust Gateway/internal code and MUST NOT be inferred from `metadata.provider`.

When SDKs and Braintrust-internal emitters resolve `context.span_origin.environment`, they SHOULD apply this precedence:

1. Caller-provided SDK option.
2. `BRAINTRUST_ENVIRONMENT_TYPE` / `BRAINTRUST_ENVIRONMENT_NAME`, resolved through process environment and `.env.braintrust` fallback.
3. Trusted Braintrust Gateway/internal configuration.
4. CI provider environment variables or generic `CI`.
5. Server/platform environment variables.
6. Language or framework deployment-mode variables.
7. Omit `environment`.

SDKs SHOULD use provider-specific CI variables when present, such as `GITHUB_ACTIONS`, `GITLAB_CI`, `CIRCLECI`, `BUILDKITE`, `JENKINS_URL`, `JENKINS_HOME`, `TF_BUILD`, `TEAMCITY_VERSION`, `TRAVIS`, or `BITBUCKET_BUILD_NUMBER`; if only generic `CI` is present, use `type: "ci"` and `name: "ci"`. SDKs MAY identify server environments from explicit platform variables such as `VERCEL`, `NETLIFY`, `AWS_LAMBDA_FUNCTION_NAME`, Lambda `AWS_EXECUTION_ENV`, `K_SERVICE`, `FUNCTION_TARGET`, `KUBERNETES_SERVICE_HOST`, `ECS_CONTAINER_METADATA_URI`, `ECS_CONTAINER_METADATA_URI_V4`, `DYNO`, `FLY_APP_NAME`, `RAILWAY_ENVIRONMENT`, or `RENDER_SERVICE_NAME`. SDKs MAY use language/framework variables such as `NODE_ENV`, `RAILS_ENV`, `RACK_ENV`, `ASPNETCORE_ENVIRONMENT`, or `DOTNET_ENVIRONMENT` only as weaker fallback signals: production/staging values map to `server`, development/local values map to `local`, and test values SHOULD be ignored unless CI was already detected.

---

## General Principles
Expand Down Expand Up @@ -1025,11 +1047,17 @@ Braintrust also consumes the standard OTel code attributes for caller location:
| Attribute | Type | Location | Description |
| ------------------------- | ------ | -------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
| `braintrust.context_json` | string | Span attribute | JSON-encoded context object. Extracted into the `context` field of the log row and deep-merged with backend-derived OTel provenance. |
| `braintrust.environment.type` | string | Resource or span attr | Operating environment type where the span was captured. Extracted into `context.span_origin.environment.type`. |
| `braintrust.environment.name` | string | Resource or span attr | Normalized operating environment name. Extracted into `context.span_origin.environment.name`. |

SDKs and exporters SHOULD encode Braintrust span-origin provenance in `braintrust.context_json` as `context.span_origin`. Ingestion MUST use those explicit `context.span_origin` values when present. When explicit Braintrust span-origin provenance is not provided, ingestion SHOULD derive `context.span_origin.name` and `context.span_origin.version` from OTel `telemetry.sdk.name` and `telemetry.sdk.version` resource attributes.
SDKs and exporters SHOULD encode Braintrust span-origin provenance in `braintrust.context_json` as `context.span_origin`. Ingestion MUST use those explicit `context.span_origin` values when present. When explicit Braintrust span-origin provenance is not provided on an OTLP span, ingestion SHOULD set `context.span_origin.name` to `opentelemetry` and SHOULD derive `context.span_origin.version` from the OTel `telemetry.sdk.version` resource attribute when present.

OTLP ingestion SHOULD derive `context.span_origin.instrumentation.name` from the OTLP instrumentation scope name.

OTLP ingestion SHOULD derive `context.span_origin.environment.type` and `context.span_origin.environment.name` from `braintrust.environment.type` and `braintrust.environment.name` resource attributes when present. If the SDK or instrumentation cannot set resource attributes, it MAY set the same attributes on individual spans. Explicit values in `braintrust.context_json` MUST win over backend-derived environment values.

Braintrust-internal OTLP ingest MAY derive `context.span_origin.environment.type = "gateway"` and `context.span_origin.environment.name = "braintrust_gateway"` from trusted Braintrust Gateway resource identity, such as the gateway service resource. SDKs and general OTLP ingest MUST NOT infer this gateway classification from public span attributes, routes, model/provider metadata, or `metadata.provider`.

### Span content

| Attribute | Type | Description |
Expand Down
10 changes: 10 additions & 0 deletions semconv/envar.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,3 +36,13 @@ config_envars:
required: false
env_braintrust: true
documentation: Organization name to use when logging in. Useful when account has multiple organizations.

- name: BRAINTRUST_ENVIRONMENT_TYPE
required: false
env_braintrust: true
documentation: Optional explicit span origin environment type. When set, SDKs use this value for context.span_origin.environment.type instead of automatic environment detection.

- name: BRAINTRUST_ENVIRONMENT_NAME
required: false
env_braintrust: true
documentation: Optional explicit normalized span origin environment name. When set, SDKs use this value for context.span_origin.environment.name instead of automatic environment detection.
4 changes: 4 additions & 0 deletions semconv/span_attributes.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,10 @@ span_attributes:
documentation: Serialized parent reference used to attach the span to a Braintrust project, experiment, dataset, prompt session, or parent span.
- name: braintrust.context_json
documentation: JSON-encoded context object extracted into the Braintrust row context field. Used for caller location and span-origin provenance.
- name: braintrust.environment.type
documentation: Operating environment type where the span was captured. Intended for OTel resource attributes when possible, span attributes otherwise, and extracted into context.span_origin.environment.type.
- name: braintrust.environment.name
documentation: Normalized operating environment name where the span was captured. Intended for OTel resource attributes when possible, span attributes otherwise, and extracted into context.span_origin.environment.name.
- name: braintrust.metrics
documentation: JSON-encoded metric annotations extracted into the Braintrust row metrics field.

Expand Down
1 change: 1 addition & 0 deletions test/IMPLEMENTATION-GUIDE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -360,6 +360,7 @@ Every expected LLM span asserts span-origin provenance in `context`:

- `context.span_origin.name` and `context.span_origin.version` MUST be non-empty strings.
- `context.span_origin.instrumentation.name` MUST identify the stable instrumentation module, package, plugin, or OTel instrumentation scope that created the span.
- `context.span_origin.environment` is optional. When present, `type` SHOULD identify the operating environment where the span was captured (`ci`, `server`, `local`, or `gateway`), and `name` SHOULD be a normalized lower-snake-case label.

Implementations MAY include additional context fields, including caller-location fields; recursive validation ignores extra keys.

Expand Down