PR: M1 — Drop-in Instrumentation & End-to-End Optimization

[mjehanzaib999]

PR: M1 — Drop-in Instrumentation & End-to-End Optimization

Branch: feature/M1-instrument-and-optimize → feature/M0-technical-plan

---

Summary

This PR delivers Milestone 1 of the LangGraph OTEL Instrumentation API: two function calls to instrument and optimize any LangGraph agent end-to-end.

• instrument_graph() — one-liner to wrap any StateGraph/CompiledGraph with full OTEL tracing, dual semantic conventions (param.* + gen_ai.*), and auto-derived Binding objects
• optimize_graph() — one-liner optimization loop: invoke → flush OTLP → TGJ → ingest_tgj → optimizer backward/step → apply_updates() via bindings → next iteration uses updated prompts
• 63 tests passing (StubLLM-only, CI-safe) including a 21-test E2E integration suite with a real LangGraph
• M1 notebook with dual mode (StubLLM deterministic + Live OpenRouter), Colab badge, and executed outputs

All 7 M1 acceptance gates from the reviewed technical plan are verified.

---

Architecture

User Code
─────────
ig = instrument_graph(graph, llm, initial_templates={...})
result = optimize_graph(ig, queries=[...], iterations=3)

Under the Hood
──────────────
┌─────────────────────────────────────────────────────────────┐
│  InstrumentedGraph                                          │
│  ├── .session    → TelemetrySession (TracerProvider +       │
│  │                  InMemorySpanExporter, flush_otlp/tgj)   │
│  ├── .tracing_llm → TracingLLM (dual semconv:               │
│  │                    param.* parent + gen_ai.* child spans) │
│  ├── .templates  → {param_key: template_str}                │
│  ├── .bindings   → {param_key: Binding(get, set)}           │
│  └── .invoke()   → delegates to compiled LangGraph          │
└─────────────────────────────────────────────────────────────┘

Optimization Loop (per iteration)
──────────────────────────────────
invoke() → flush_otlp() → OTLP→TGJ → ingest_tgj()
→ ParameterNode + MessageNode → backward(feedback)
→ optimizer.step() → apply_updates(bindings) → next invoke()

---

New Modules (opto/trace/io/)

File	Lines	Purpose
instrumentation.py	138	instrument_graph() + InstrumentedGraph dataclass
optimization.py	412	optimize_graph() + EvalResult, EvalFn, RunResult, OptimizationResult
telemetry_session.py	188	TelemetrySession — unified OTEL session with flush_otlp(), flush_tgj(), export_run_bundle()
bindings.py	105	Binding dataclass + apply_updates() + make_dict_binding()
otel_semconv.py	126	emit_reward(), emit_trace(), record_genai_chat(), set_span_attributes()

Modified Modules

File	Change
langgraph_otel_runtime.py	TracingLLM enhanced: dual semconv parent (param.*) + child (gen_ai.*) spans; child spans carry trace.temporal_ignore = "true" to protect TGJ chaining
__init__.py	Exports all new M1 public APIs (21 symbols)

Tests (63 passing)

File	Tests	Scope
test_bindings.py	10	Binding, apply_updates(), strict/non-strict modes
test_otel_semconv.py	5	emit_reward(), emit_trace(), record_genai_chat()
test_telemetry_session.py	6	flush_otlp(), flush_tgj(), record_spans, span_attribute_filter
test_instrumentation.py	10	instrument_graph(), child span emission, temporal chaining
test_optimization.py	11	EvalResult, _normalise_eval(), data contracts
test_e2e_m1_pipeline.py	21	Full E2E: real LangGraph + StubLLM → invoke → OTLP → TGJ → ParameterNode → mock optimizer → apply_updates() → re-invoke with updated template

Notebook and Docs

File	Description
examples/notebooks/01_m1_instrument_and_optimize.ipynb	10 sections, 20 code cells, dual mode (StubLLM + Live OpenRouter), Colab badge, 3 item dataset, temperature=0, max_tokens=256 budget guard, committed with executed outputs
docs/m1_README.md	Architecture diagrams, full API reference, data flow pipeline, semantic convention design, temporal chaining contract, file map, quick start, acceptance criteria status
requirements.txt	Pinned dependencies for uv/pip environments

---

Key Design Decisions

1. Generic — no hard-coded node names. trainable_keys=None means all nodes are trainable; explicit set to restrict. No "planner" or "synthesizer" baked into the optimization API.

2. Dual semantic conventions. Each TracingLLM.node_call() emits a parent span (param.* for optimization) and an optional child span (gen_ai.* for Agent Lightning observability). Child spans carry trace.temporal_ignore = "true".

3. Temporal chaining contract. The OTLP to TGJ adapter only advances the temporal chain on root spans (those without parentSpanId). This prevents child LLM spans from corrupting the planner to synthesizer ordering that the optimizer relies on. Verified by 3 dedicated tests.

4. Explicit Binding layer. Optimizer output keys map to Binding(get, set) objects. apply_updates() pushes values through the bindings. Auto-derived from initial_templates by default, but users can supply custom bindings for class attributes, DB rows, etc.

5. Flexible EvalFn contract. eval_fn can return float, str, dict, or EvalResult — all auto-normalized. This avoids forcing users into a rigid evaluation interface.

6. Closure-based wiring. Node functions close over ig.tracing_llm and ig.templates dict. When apply_updates() modifies the dict, the next invoke() automatically reads the updated values — no globals needed.

---

M1 Acceptance Gates

#	Gate	Status
1	OTLP export works — flush_otlp(clear=True) returns at least 1 span; second flush returns 0	PASS
2	TGJ conversion works — flush_tgj() docs consumable by ingest_tgj()	PASS
3	Temporal chaining — child spans do NOT advance TGJ temporal chain	PASS
4	Bindings apply deterministically — strict=True raises on missing keys	PASS
5	E2E update path (StubLLM) — optimize_graph(iterations>=2) changes at least 1 prompt	PASS
6	Notebook live validation — OTLP+TGJ with param.* from real provider	PASS
7	Tests + notebook gate — all new APIs have at least 1 pytest; Colab badge present	PASS

---

Test Plan

• Run python -m pytest tests/unit_tests/test_bindings.py tests/unit_tests/test_otel_semconv.py tests/unit_tests/test_telemetry_session.py tests/unit_tests/test_instrumentation.py tests/unit_tests/test_optimization.py tests/features_tests/test_e2e_m1_pipeline.py -v — all 63 pass
• Notebook runs end-to-end in StubLLM mode without API keys
• Notebook live section runs when OPENROUTER_API_KEY is set (Colab Secrets or .env)
• No secrets committed in notebook outputs
• notebook_outputs/m1/ artifacts contain valid OTLP JSON and TGJ JSON