[RFC] Proposal: Standardized Intermediate Representation Strategy for UCP

[gsmith85]

RFC: Standardized Intermediate Representation Strategy for UCP

1. Summary

The objective of this RFC is to produce a standardized intermediate representation for UCP that enables downstream code generation.

This is in part to support UCP's managed SDKs (js-sdk, python-sdk) which are costly to maintain, particularly in lockstep with schema changes. Moreover, this is an approach to enable merchants and platform builders to self-generate types for their own systems, tailored to their specific UCP profile in a format (OpenAPI 3.1) that has broad industry adoption, potentially eliminating the need for UCP to maintain a large number of first-party SDKs.

This is not new ground. Rather, it is the outcome of previous Technical Council discussions—most notably Issue #484 (Proposal: UCP Open Source Unified Toolchain and SDK Generation), alongside key PRs from the 2026-08-25 release cycle (e.g. PR #688 on tagged union scaffolding and PR #684 on generator class breakage) where the TC navigated the tension between maintaining a fully-featured JSON Schema authoring surface and comprehension.

Proposed Solution: Extension of ucp-schema Toolchain

We propose extending the shared Rust compiler toolchain (ucp-schema) with an export-openapi command.

OpenAPI 3.1 natively adopts the JSON Schema 2020-12 specification, eliminating dialect translation loss and keyword stripping ($defs, validation rules, etc.). As such, it is well suited to handle the dynamicism and compositional extensibility of UCP.

The export-openapi command produces a fully-resolved IDL from UCP source schema and specified extensions producing:

1. Canonical Types & DTOs (components.schemas) for Type Generation:
   • Directional Models: Slices schemas into explicit request/resource models (e.g. CheckoutCreateRequest, CheckoutSession) based on (direction, op) annotations.
   • Tagged Unions: Maps union variants (such as PR fix!: make destination types explicit #688 fulfillment destinations) to standard OpenAPI discriminators so code generators emit real typed unions instead of Any/unknown.
   • Recursive Constraint Normalization: Resolves # self-references in constraint_expression.json and distributes bare anyOf properties so downstream tools don't drop fields.
2. Normative REST Binding (paths & parameters) for Service Stub Generation:
   • Codifies existing normative REST endpoints (POST /checkout-sessions, PUT /checkout-sessions/{id}, POST /checkout-sessions/{id}/complete).
   • Binds standard protocol parameters (UCP-Agent, Idempotency-Key), RFC 9421 security schemes, and standard ErrorResponse definitions as a helpful affordance for service stub generation.

Beyond human developers, the self-contained output produced by export-openapi is a pre-resolved contract. We believe there is an opportunity to leverage this output to make UCP significantly easier for LLMs and autonomous agents to grok rather than chasing $ref graphs across dozens of fragmented files.

flowchart TD
    subgraph "1. Schema Sources"
        Core["Core UCP Repository (ucp/)<br/>Canonical Schemas (Draft 2020-12)"]
        Extensions["External Repositories<br/>Merchant & Domain Extensions"]
    end

    subgraph "2. IDL Compilation (ucp-schema export-openapi)"
        Core -->|"export-openapi"| BaseSpec["Canonical Base Spec<br/>(shopping.openapi.json)"]
        Core & Extensions -->|"export-openapi --profile"| ProfileSpec["Profile-Resolved Spec<br/>(custom_profile.openapi.json)"]
    end

    subgraph "3. Downstream Consumers"
        BaseSpec --> BaseConsumers["Official SDKs & Quick-Starts<br/>(js-sdk, python-sdk, docs, mocks)"]
        ProfileSpec --> CustomTypegen["Custom Tech Stacks<br/>(Typegen for Merchant/Platform ERP & OMS)"]
        ProfileSpec --> AgentTools["Autonomous Agent Tooling<br/>(MCP Servers, Dynamic Tool Calling)"]
    end

Loading

---

2. Problem Statements

2.1 SDK / Schema Drift

Rather than evolving continuously with the protocol, UCP SDKs are updated in batch after formal version cuts. This lag stalls merchants wanting to pilot draft capabilities and creates drift in our reference samples. An OpenAPI IDL provides the intermediate representation needed to automate SDK generation directly in CI as schemas evolve.

2.2 Per Language SDK Maintenance

Without an intermediate IDL, each downstream language repository is independently
attempting to parse and post-process raw UCP schemas. The IDL is essentially centralized preprocessing that spares downstream consumers, and spares them needing to understand the translation from UCP JSON Schema to their target language.

js-sdk#61, the JavaScript SDK rev for the 2026-08-25 release, is illustrative:

• It required revision of three custom scripts to interpret the raw schemas.
• Manually resolving recursive schema references in constraint_expression.json for topological sort stability.
• Adding .passthrough() on ConstraintExpressionSchema in Zod as an escape hatch because the code generator dropped open constraint fields.
• Independently duplicating the same interpretation logic in Python (preprocess_schemas.py, postprocess_models.py).

2.3 The Static SDK Limit & Ecosystem Self-Service

In UCP's mix-and-match model, merchants advertise differing subsets of capabilities alongside proprietary or domain extensions.

Pre-packaged SDKs published to public package registries can only serve the baseline, extension-less core configuration. While this lowest-common-denominator SDK is valuable for platforms seeking a turnkey solution, real e-commerce merchants operate bespoke backend stacks (ERP, OMS, custom routing). For these implementers, static pre-compiled SDKs fail to provide the exact static types needed for their active UCP profile.

At the same time, attempting to maintain first-party SDKs across multiple languages (TypeScript, Python, Go, Java, Rust, PHP) and verticals is both lagging and unsustainable for UCP Dev Ops.

Compiling an intermediate IDL enables ecosystem self-service to resolve both sides:

• Merchant Profile Precision: Implementers can self-generate types tailored to the exact capabilities and extensions their stack supports.
• Reduces Expansion Urgency: UCP Dev Ops is relieved of the pressure to author and publish official client packages across an expanding matrix of languages and verticals.
• Refocuses Existing SDKs on Quick-Starts: The official SDKs we do maintain become lightweight, quick-start-focused developer kits for demonstration, experimentation, and reference validation.

2.4 Bridging Schema Authoring and Tooling Interoperability

In PR #688, the consensus was clear: we do not want to artificially restrict the authoring grammar of UCP JSON Schema. Source schemas must remain open-world, allowing decentralized extensions via allOf and unconstrained polymorphic variants.

At the same time, code generators and agent tool calling require closed, unambiguous contracts. A runtime service or client does not operate against an unbounded universe; it implements a specific capability profile. Compiling to OpenAPI 3.1 reifies these open polymorphic hierarchies into concrete, profile-scoped tagged unions (oneOf with explicit discriminator.mapping).

This establishes a clean two-plane architecture:

• Protocol Authoring Plane (JSON Schema 2020-12): Open-world extensibility for canonical types and decentralized custom extensions.
• Operational Interface Plane (OpenAPI 3.1): Profile-reified closed tagged unions for deterministic code generation and runtime routing, while preserving property-level extensibility (additionalProperties: true) across both planes for forward compatibility.

2.5 Persona Considerations: Implementers vs. Autonomous Agents

The developer ecosystem interacts with the protocol across two distinct parties:

• Service Implementers (Merchants & Platforms): Benefit from strongly typed contracts matching their specific capability profile to scaffold service interfaces, route handlers, and DTOs into existing backends.
• Autonomous Agents (Buyer Agents & Tools): Benefit less from compiling static client libraries in their runtime environment and more from pre-resolved, discoverable contracts to dynamically generate runtime tool definitions and inspect payloads on the fly.

A compiled OpenAPI IDL serves both needs fundamentally better than runtime trial-and-error:

• For Implementers: Acts as standard input for generators to emit strongly typed SDKs and server stubs in any language.
• For Agents: While runtime schema validators allow trial-and-error rejection sampling (guess $\rightarrow$ validate $\rightarrow$ retry), an OpenAPI IDL provides full request/response shapes and operational context—routes, HTTP verbs, and headers—eliminating multi-turn validation latency and context-window bloat from chasing external $ref graphs.

---

3. Contract Lifecycle & Verification Pipeline

To guarantee specification stability and eliminate version drift (Issue #484), this RFC establishes three core pipeline invariants:

1. CI Verification Gate (export-openapi --check):
   • Integrated into .github/workflows/ in Universal-Commerce-Protocol/ucp.
   • Any pull request altering source/schemas/ or source/services/ will automatically execute export-openapi --check.
   • Compilation Closure Check: Enforces complete schema closure, failing PRs on dangling $refs, conflicting discriminator mappings, or composite directional contradictions.
2. Committed Artifact & PR Review Visibility (dist/shopping.openapi.json):
   • The compiled specification is committed to dist/ under version control, guarded by --check.
   • Reviewer Visibility: Reviewers can inspect the exact projected wire contract diff directly in the GitHub PR review UI, preventing regressions and unintended schema changes from reaching main.
   • Low-Friction Consumption: Hobbyist implementers and SDK generators can consume canonical specs as a convenient, pre-resolved artifact.
3. Decoupled Tooling & AST Script Retirement: Retires >3,500 LOC of AST pre/post-processing scripts across python-sdk and js-sdk which will now rely directly on the output of export-openapi.

---

4. Alternatives Considered

Alternative	Advantages	Disadvantages
Status Quo (Language-Specific Preprocessors)	Zero core tool changes.	Externalizes schema parsing complexity onto all downstream consumers; proven by >3,500 LOC of custom AST scripts in python-sdk and js-sdk; delays releases; breaks on custom merchant extensions.
Rewrite Source to TypeSpec / Smithy	Rich IDL syntax.	Requires discarding existing JSON Schemas and retraining all ecosystem contributors; breaks backwards compatibility.
Pure Runtime Dynamic Clients	No ahead-of-time codegen.	Loses static type safety and autocomplete; heavy startup parsing latency; incompatible with enterprise production air-gap policies.
OpenAPI 3.1 via ucp-schema (Proposed)	Native JSON Schema 2020-12 parity; supports composition; empowers arbitrary typegen; proven by working prototype.	Requires maintaining compiler module in Rust ucp-schema.

---

5. Rollout Plan & Milestones

• Milestone 1: Technical Council review and adoption of this RFC.
• Milestone 2: Merge companion Draft PR into Universal-Commerce-Protocol/ucp-schema, releasing ucp-schema export-openapi.
• Milestone 3: Update js-sdk and python-sdk to generate types directly from the compiled OpenAPI 3.1 spec; delete legacy pre/post-processing scripts.
• Milestone 4: Activate atomic export-openapi --check gate on all PRs in Universal-Commerce-Protocol/ucp.

---

Appendix: Companion Implementation & Upstream References

• Companion Draft PR: Universal-Commerce-Protocol/ucp-schema#73 (Branch: feat/export-openapi)
• Related Upstream Items:
  • Issue #484 (Proposal: UCP Open Source Unified Toolchain and SDK Generation)
  • PR #125 (fix(schema): publish UCP annotated schemas & runtime resolution)
  • PR #684 (Refactor totals.json from top-level array to top-level object)
  • PR #688 (Fulfillment destination tagged unions & open typed contracts)
  • PR js-sdk#61 (Update SDK models for UCP 2026-08-25: Downstream maintenance case study)