Metergraph SDKs

Capture SDKs for Metergraph, which tracks LLM costs by application function and trace. Wrap your OpenAI, Anthropic, Gemini, or Python Vercel AI Gateway client—or add Metergraph middleware to a TypeScript Vercel AI SDK language model—and every call is attributed to the function that made it, with token counts (input/output, cache reads, aggregate and TTL-specific cache writes, reasoning), latency, and model. SDK rows contain usage counters, never embedded prices or client-computed cost. Metergraph sends scrubbed request and normalized response content to the hosted service by default; applications can opt out globally or around a sensitive route or trace. The SDKs have no runtime dependencies.

We recommend keeping content capture enabled because replayable requests and responses are what allow Metergraph to evaluate quality and produce optimization recommendations. Content is captured by default; the SDK removes common secret-bearing fields, supports application-specific redaction, and provides global and per-operation opt-outs. If your privacy requirements do not permit hosted content capture, contact Metergraph about deploying it privately in your VPC. The open-source server is also always available for self-hosting.

Package	Registry	Source
metergraph	PyPI	python/
metergraph	npm	typescript/

Set up with an AI coding agent

The fastest way to instrument an existing codebase is to paste this into Claude Code, Codex, Cursor, or whatever agent you use, from inside the repo you want instrumented. It covers both the Python and TypeScript SDKs:

Instrument this codebase's LLM API costs with Metergraph
(https://github.com/PioneerSquareLabs/metergraphsdk). It captures per-call
token usage (in/out, cached, reasoning), latency, model, scrubbed request, and
normalized response, attributed to the application function and logical trace
that made the call. Keep the default content capture enabled so Metergraph can
evaluate quality and produce recommendations. Set METERGRAPH_CAPTURE_TEXT=0
only when the deployment owner explicitly chooses metadata-only capture.

1. Install the SDK: `pip install metergraph` (Python) or `npm install
   metergraph` (TypeScript/JavaScript). Zero runtime dependencies.
2. Find every place an OpenAI, Anthropic, or Google Gemini client is
   constructed (OpenAI()/AsyncOpenAI(), Anthropic()/AsyncAnthropic(),
   genai.Client(), new OpenAI(), new Anthropic(), new GoogleGenAI()) and wrap
   it in place:
   - Python: `client = metergraph.wrap(OpenAI())` after `import metergraph`
   - TypeScript: `const client = mg.wrap(new OpenAI())` after
     `import * as mg from "metergraph"`
   wrap() returns the same client and initializes itself from the environment.
   Do not change any call sites, arguments, or error handling; streaming and
   async work unchanged.
   Python OpenAI/Anthropic clients whose base URL is
   https://ai-gateway.vercel.sh are Vercel AI Gateway clients and are detected
   automatically. Preserve their AI_GATEWAY_API_KEY / VERCEL_OIDC_TOKEN and
   creator-qualified model IDs; Metergraph must never capture those secrets.
   For Vercel AI SDK `generateText` / `streamText` calls, wrap each language
   model with `wrapLanguageModel({ model, middleware:
   mg.vercelAISDKMiddleware() })` instead of wrapping a provider client.
3. METERGRAPH_APP_TOKEN is required; the SDK warns and disables capture when
   it is missing. METERGRAPH_INGEST_URL is only needed when
   self-hosting the server from https://github.com/PioneerSquareLabs/metergraph.
   Document variable names with placeholders in `.env.example`, and put real
   values only in the deployment configuration. Repository identity enables
   repository-level attribution and [MeterGraph Bot](https://github.com/apps/metergraph).
   Configure it using any one of these sufficient options: the `repository`
   option to `init()` (`repository="owner/repository"` in Python or
   `{ repository: "owner/repository" }` in TypeScript),
   `METERGRAPH_REPOSITORY`, or a read-only
   `.metergraph/config.json` file containing `{"repository":"owner/repository"}`.
4. Attribution:
   - Python: automatic via stack walk. Optionally decorate key LLM-calling
     functions with @metergraph.track to pin a stable name.
   - TypeScript: wrap each LLM-calling function with
     mg.track("stable.name", fn), because stack-based attribution is
   unreliable under bundlers. Do this for every function that calls a
   wrapped client.
5. Wrap multi-call operations in metergraph.trace("stable-name") in Python or
   mg.trace("stable-name", fn) in TypeScript. Use capture_text=False /
   captureText: false around sensitive operations.
6. Serverless only (Lambda / Cloudflare Workers / Vercel): ensure delivery
   before the runtime freezes. Wrap handlers with mg.wrapHandler(handler),
   or call mg.bindWaitUntil(ctx) once per request, or await mg.flush() before
   returning. Long-running servers and scripts need nothing extra.
7. The SDK is fail-open: transport problems never break or slow LLM calls, so
   do not add defensive try/except around wrapping or the wrapped calls.

When done, list every client and Vercel AI SDK model you instrumented and where,
and flag any LLM calls made through other paths, since those are not captured.

Python

pip install metergraph
export METERGRAPH_APP_TOKEN=<token>

Initialize Metergraph once, then wrap each provider client. The token remains in the environment:

import metergraph

metergraph.init(repository="owner/repository")

# OpenAI
from openai import OpenAI
openai_client = metergraph.wrap(OpenAI())

# Anthropic
from anthropic import Anthropic
anthropic_client = metergraph.wrap(Anthropic())

# Gemini
from google import genai
gemini_client = metergraph.wrap(genai.Client())

Initialization is process-wide. The first init() configuration remains active; later explicit calls are ignored and produce one generic warning without option names, token values, or other secrets.

For Python applications using Vercel AI Gateway, configure the official OpenAI or Anthropic client with the Gateway URL and wrap that client. Metergraph detects the public Gateway URL automatically and uses the creator/model prefix for catalog pricing:

import os
import metergraph
from openai import OpenAI

metergraph.init(repository="owner/repository")

gateway = metergraph.wrap(OpenAI(
    api_key=os.getenv("AI_GATEWAY_API_KEY") or os.getenv("VERCEL_OIDC_TOKEN"),
    base_url="https://ai-gateway.vercel.sh/v1",
))

gateway.chat.completions.create(
    model="anthropic/claude-sonnet-4.6",
    messages=[{"role": "user", "content": "Hello"}],
)

Sync, async, streaming, tool calls, and Responses API requests use the same capture path. For a compatible client behind a custom gateway URL, pass provider="vercel" to metergraph.wrap().

For OpenRouter, wrap an ordinary OpenAI client pointed at https://openrouter.ai/api/v1; the host is auto-detected and captured rows gain served_model and, when OpenRouter supplies a valid usage.cost, the gateway-reported reported_cost_usd. A trusted custom domain uses metergraph.wrap(client, gateway="openrouter"). See the runnable Python OpenRouter example.

MeterGraph provides a standard OpenTelemetry GenAI span exporter that also reads OpenInference (Arize Phoenix), Langfuse SDK and LangSmith spans, so an app already instrumented for any of them captures by registering one exporter. See Capture from existing telemetry below. LiteLLM is the currently qualified integration and can attach the exporter without changing individual model calls. See the LiteLLM OpenTelemetry example.

Then use the wrapped client exactly as before:

@metergraph.track
def summarize_invoice(invoice):
    return openai_client.chat.completions.create(model="gpt-5.6-luna", messages=[...])

Attribution is automatic in Python: the SDK walks the stack to the nearest application function. @metergraph.track pins an explicit, stable name instead. Sync and async clients both work, streaming included. To configure in code rather than env vars, call metergraph.init(token=..., ...) before the first wrap().

Use with metergraph.trace("checkout"): or the equivalent decorator to group multiple provider calls. Set capture_text=False on init(), route(), or trace() when an operation must remain metadata-only.

For concurrent requests, background jobs, or reused workers, keep session and tag identity inside a bounded context:

with metergraph.context(
    session_id=run_id,
    tags={"customer": customer_id},
):
    run_job()

Context follows async work started inside the with block and is restored when the block exits. metergraph.session() and metergraph.tags() provide narrower scopes, and metergraph.set_default_tags() sets process-wide service metadata. See the Python SDK guide for nesting, legacy setter behavior, and the complete configuration reference.

TypeScript / JavaScript

npm install metergraph
export METERGRAPH_APP_TOKEN=<token>

Initialize Metergraph once, then wrap each provider client. The token remains in the environment:

import * as mg from "metergraph";

mg.init({ repository: "owner/repository" });

// OpenAI
import OpenAI from "openai";
const openai = mg.wrap(new OpenAI());

// Anthropic
import Anthropic from "@anthropic-ai/sdk";
const anthropic = mg.wrap(new Anthropic());

// Gemini
import { GoogleGenAI } from "@google/genai";
const gemini = mg.wrap(new GoogleGenAI({}));

For OpenRouter, wrap an ordinary OpenAI client pointed at https://openrouter.ai/api/v1; the host is auto-detected and captured rows gain served_model and, when OpenRouter supplies a valid usage.cost, the gateway-reported reported_cost_usd. A trusted custom domain uses mg.wrap(client, { gateway: "openrouter" }). See the runnable Node OpenRouter example.

const summarizeInvoice = mg.track("billing.summarize_invoice", async (invoice) => {
  return openai.chat.completions.create({ model: "gpt-5.6-luna", messages: [...] });
});

In TypeScript, use track() for attribution. It stays reliable across bundlers and minifiers, where stack parsing does not. Provider SDKs and the Vercel AI SDK are optional peer dependencies, and Metergraph itself has no runtime dependencies. To configure in code, call mg.init({ repository: "owner/repository", token, ... }) before the first wrap().

Use await mg.trace("checkout", async () => { ... }) to group multiple calls, and pass { captureText: false } for a metadata-only operation.

Vercel AI SDK models use the same capture and trace path through middleware:

import { generateText, wrapLanguageModel } from "ai";
import { openai } from "@ai-sdk/openai";

const model = wrapLanguageModel({
  model: openai("gpt-5.6-luna"),
  middleware: mg.vercelAISDKMiddleware({ repository: "owner/repository" }),
});

await mg.trace("support-answer", () =>
  generateText({ model, prompt: "Help this customer" })
);

Swap in a Vercel AI Gateway model the same way, with no other changes:

import { gateway } from "ai";

const model = wrapLanguageModel({
  model: gateway("anthropic/claude-sonnet-4.5"),
  middleware: mg.vercelAISDKMiddleware(),
});

Each provider request in a multi-step AI SDK tool loop becomes its own costed span. Provider options and transport headers are excluded from capture. The middleware initializes Metergraph itself; pass init() options to it as shown above, or call mg.init() first in applications with centralized setup. For multi-provider applications, use the examples chooser to select the provider-registry pattern or instrument a custom factory that your application already has. Each example clearly marks original application code versus MeterGraph additions.

For concurrent Node workers, bind request identity to a callback instead of mutable process state:

await mg.withContext({
  sessionId: runId,
  tags: { customer: customerId },
}, async () => {
  await runJob();
});

withSession() and withTags() provide narrower forms. Context follows async work created inside the callback and is restored afterward.

Vercel AI SDK	Metergraph middleware	Node.js
5	vercelAISDKMiddleware({ aiSdkVersion: 5 })	18+
6	vercelAISDKMiddleware()	18+
7	vercelAISDKMiddleware()	22+

Metergraph itself supports Node.js 18+; the AI SDK version you choose may require a newer runtime. See typescript/README.md for the specificationVersion advanced/backward-compatibility option.

Where the data goes

export METERGRAPH_INGEST_URL=http://localhost:8787   # your self-hosted server
export METERGRAPH_APP_TOKEN=<token>

Leave METERGRAPH_INGEST_URL unset to use the hosted service, or point it at a self-hosted Metergraph server. Without a token, capture is off and the SDK emits a warning. Hosted capture includes scrubbed request and response content by default because that content enables quality evaluation and optimization recommendations. Each field is capped at 1 MiB by default; set METERGRAPH_TEXT_MAX_BYTES, text_max_bytes, or textMaxBytes to accept larger prompts and responses. The SDK supports a custom redaction hook and explicit content opt-outs for additional privacy control. For content-aware analysis inside your own cloud boundary, contact Metergraph about a private deployment in your VPC. The public open-source self-hosted server remains available and discards content even when the SDK sends it. Transport problems never break or slow your LLM calls. When the collector is unreachable, capture drops and your application carries on.

See examples/ for runnable per-provider examples, including an offline fake-provider demo that needs no API keys. The instrumentation coverage contract is the source of truth for supported providers, frameworks, package anchors, and unsupported-path behavior.

Capture from existing telemetry (Phoenix, Langfuse, LangSmith)

Python applications that already trace LLM calls do not need wrap(). metergraph.opentelemetry.MetergraphGenAIExporter (pip install 'metergraph[otel]') reads OpenInference, langfuse.observation.* and LangSmith spans directly: attach it as one more span processor on the tracer provider that carries them.

For Phoenix, that is the provider phoenix.otel.register() returns:

from phoenix.otel import register
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from metergraph.opentelemetry import MetergraphGenAIExporter

tracer_provider = register(project_name="my-app")  # existing Phoenix setup
tracer_provider.add_span_processor(
    BatchSpanProcessor(MetergraphGenAIExporter()),
    # Required. phoenix.otel's TracerProvider shuts down ITS OWN exporter on
    # the first add_span_processor unless told otherwise, so without this
    # adding MeterGraph silently turns off your Phoenix tracing.
    replace_default_processor=False,
)

For the Langfuse Python SDK (v3/v4):

from opentelemetry import trace
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from metergraph.opentelemetry import MetergraphGenAIExporter

trace.get_tracer_provider().add_span_processor(
    BatchSpanProcessor(MetergraphGenAIExporter())
)

Verify the attachment against your installed Langfuse SDK version — whether Langfuse uses the global provider has changed across releases, and the client is a process-wide singleton, so the exporter must be on the provider the first Langfuse(...) was built with. Make one traced call and confirm a row arrives.

For the LangSmith SDK, set the tracer provider before the first traced call and turn on OTel export -- LangSmith's default tracing mode posts runs to its own API and emits no spans at all:

import os
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from metergraph.opentelemetry import MetergraphGenAIExporter

os.environ["LANGSMITH_TRACING_MODE"] = "otel"  # or "hybrid" to keep LangSmith
os.environ["LANGSMITH_TRACING"] = "true"

provider = TracerProvider()
provider.add_span_processor(BatchSpanProcessor(MetergraphGenAIExporter()))
# Install it BEFORE the first LangSmith client is built. LangSmith reuses an
# existing global provider, but builds its own private one if none is set --
# and spans on that provider never reach this exporter.
trace.set_tracer_provider(provider)

LangSmith exports every run as a GenAI span -- chains, tools and retrievers included -- so the exporter counts only those whose langsmith.span.kind is llm. It reports no cost (LangSmith prices server-side), and its spans carry no time-to-first-token.

Use exactly one capture path per call. Do not both wrap() a client and capture its spans through this exporter, or the call is counted twice. On a shared tracer provider, MetergraphGenAIExporter(include_scopes=[...], exclude_scopes=[...]) filters on span.instrumentation_scope.name (exclude wins). OpenInference instrumentors publish one scope per instrumented library, named after it — openinference.instrumentation.openai and so on; the Langfuse SDK publishes everything under the single scope langfuse-sdk. When nothing arrives, read the exporter's public exporter.skipped counters ("scope", "not-genai", "ineligible-kind", "no-model", plus the "parse-degraded" diagnostic). Runnable offline examples: examples/python-phoenix-otel/, examples/python-langfuse-otel/.

Vendor-reported cost lands on rows as reported_cost_usd with a fixed reported_cost_source; the SDK never computes cost itself. Note that Phoenix computes cost server-side — its instrumentors do not put llm.cost.total on spans — so in practice this evidence arrives from Langfuse.

Braintrust and other OTLP backends

Phoenix and Langfuse need their own dialects because they emit spans in a private vocabulary. A backend that only receives OTLP needs nothing: the spans on the provider are whatever the application's instrumentation emitted, and if that is gen_ai.* or OpenInference the exporter already reads it.

Braintrust is that second case. BraintrustSpanProcessor ships spans from a standard tracer provider to Braintrust, which implements the OpenTelemetry GenAI semantic conventions; the emitters it recommends are OpenLLMetry, the Vercel AI SDK, and raw OTLP. So an application already tracing to Braintrust captures by adding one more processor to the provider it has:

provider.add_span_processor(BraintrustSpanProcessor())  # existing, to Braintrust
provider.add_span_processor(BatchSpanProcessor(MetergraphGenAIExporter()))

The same reasoning applies to any OTLP backend. Two limits are worth knowing:

• Cache tokens need a dialect. The gen_ai.* extractor reads input and output tokens only, so gen_ai.usage.cache_read.input_tokens and gen_ai.usage.cache_creation.input_tokens do not reach the row. An OpenInference or Langfuse span carries cache counts; a pure gen_ai.* span does not.
• The braintrust.* attribute namespace is not read. It is an input convention for hand-written spans (braintrust.input_json, braintrust.metrics), not something an instrumentor emits, and a span carrying only those attributes is skipped as not-genai.

Applications using Braintrust's native SDK — wrap_openai(), @traced, start_span() — log straight to Braintrust's API and put nothing on a tracer provider, so there is no span to tee. Import those after the fact with metergraphrelay pull braintrust instead.

Batch-first execution (opt-in, not part of default capture)

Both SDKs also expose an explicit, separately opt-in batchFirst() / batch_first() API: submit one request through a provider's Batch API, wait up to a caller-chosen deadline, and fall back to exactly one direct call if the batch hasn't finished in time. This is a distinct code path from wrap()/capture — never enabled by wrap(), by default configuration, or by any environment variable — and it carries real cost and behavioral consequences a caller must accept explicitly before any provider call is made:

• Duplicate execution and cost on a missed deadline. The batch request is always submitted first; if it hasn't reached a terminal state by the deadline, batchFirst() also issues a direct call while the batch keeps running — the same prompt can be billed and executed twice. Both SDKs require an explicit, non-defaulted acknowledgement of this (acceptDuplicateProviderExecution: true in TypeScript, accept_duplicate_provider_execution=True in Python).
• Streaming is not supported. A request with stream: true / stream=True is rejected before any provider call.
• Tool calls need a separate acknowledgement. The batch result and the direct fallback are independent provider executions of the same prompt and may each choose a different tool-call plan. A request carrying tools is rejected unless the caller also sets allowDuplicateToolCallPlans: true / allow_duplicate_tool_call_plans=True — a caller whose tools have side effects must not assume the two plans agree. Neither path executes a tool call automatically; the caller receives the tool-call plan in the returned result and remains responsible for executing it, exactly as with a normal (non-batch-first) provider response.
• Exactly one canonical result, ever. Whichever result — batch or direct — settles first is the only one ever returned or executed; a batch result that arrives after a direct fallback already won is never returned, never executed, and never mutates the already-returned result. A losing batch's eventual outcome is observable only through an async, best-effort onLateBatchSettled / on_late_batch_settled callback, and only as whether it happened to contain a tool-call plan — never its content.
• Long-lived-process assumptions differ by language, and neither is fully solved yet. In TypeScript, the background poll that watches a losing batch for late telemetry uses an ordinary (non-unref'd) timer, so it keeps a Node process alive until the batch reaches a terminal state — which can take up to 24 hours. In Python, the equivalent background poll runs on a daemon thread, which does not keep the process alive — a short-lived script may exit before on_late_batch_settled ever fires, silently dropping that signal. Both are consequences of this feature only having been designed against a long-running-server assumption so far, not yet against short-lived scripts or serverless invocations.

Adapters exist for OpenAI, Anthropic, and Google Gemini in both SDKs, built against each provider's documented/introspected current Batch API shape and covered by fake-client contract tests — but none of the three has been exercised against a live provider Batch API from either SDK. Treat this as a beta-quality, code-reviewed-but-not-live-verified surface. See typescript/README.md and python/README.md for exact API shapes and examples.

License

Apache-2.0