Route AI features through bb Cloud

[ymichael]

Summary

• expand the existing getbb.app connection into one user-facing Cloud plugin while preserving the compatibility id connect and the bb connect setup/management command
• split the plugin settings page into Remote access and Cloud AI sections; keep the sidebar phone as a capability shortcut that deep-links to Remote access while the plugin itself uses a cloud icon
• add authenticated Cloudflare endpoints for structured inference and voice transcription, with gate-owned model selection, request limits, and per-account daily budget enforcement
• add typed @bb/connect-client calls and an experimental plugin provider seam so the server can try bb Cloud before locally configured AI providers
• route thread-title generation, commit-message generation, and voice transcription through the builtin Cloud plugin while paired and enabled

Responsibility boundaries

server inference / transcription
  -> experimental plugin provider slot
  -> builtin Cloud plugin (compatibility id: connect)
  -> @bb/connect-client
  -> getbb.app Cloudflare worker
  -> OpenAI

Cloudflare worker -> D1 ai_usage metering

• Connect gate: credential authentication, cloud model and cost policy, input caps, upstream calls, quota reservation/refunds, sanitized errors
• Connect client: request encoding, credential header, response validation, stable transport errors
• Plugin SDK/server: provider-neutral contract, cloud-first/local-fallback routing, host-owned timeouts, final structured-output validation
• Cloud plugin: pairing-aware availability, remote tunnel and port sharing, persisted Cloud AI preference, rejected-credential latch, quota cooldown, UI/CLI/RPC integration
• App host: plugin-branded settings identity plus action-specific sidebar icons and section-targeted settings navigation

Contract changes

• add POST /api/connect/ai/inference with { prompt, schema } -> { value }
• add POST /api/connect/ai/transcription with multipart file/optional prompt -> { text }
• require header-only machine or paired-server credential authentication; session cookies are not accepted
• add fetchAiInference, fetchAiTranscription, and ConnectAiError to @bb/connect-client
• add CloudAiProvider and bb.experimental_registerCloudAiProvider(...) to the public plugin SDK
• allow sidebar actions to call openSettings({ sectionId }); action glyphs are distinct from the plugin branding icon
• add required cloudAiEnabled to connect status plus setCloudAi RPC and bb connect ai [on|off]
• preserve the internal plugin id, RPC paths, storage namespace, and bb connect CLI compatibility
• no host-daemon wire/protocol change

Data model

• add D1 ai_usage, keyed by (user_id, UTC day), containing estimated aggregate spend in micro-USD
• share one $2/day abuse ceiling across inference and transcription
• reserve usage atomically before the upstream call and refund failed upstream calls best-effort
• store no prompts, schemas, responses, diffs, or audio
• add plugin KV preference cloudAiEnabled, defaulting to true when absent

Lifecycle and fallback behavior

• the Cloud plugin registers its provider during load and unregisters it through the existing plugin-dispose lifecycle
• cloud routing is available only while enabled, paired, outside a quota cooldown, and using a credential the gate has not rejected
• cloud failures and provider bugs fall back to locally configured providers; timeouts preserve the existing host timeout semantics without stacking another attempt
• a rejected credential remains latched until re-pairing supplies a different credential
• quota exhaustion pauses cloud attempts for five minutes
• bb connect off unlinks the shared Cloud pairing, stopping remote access and making Cloud AI unavailable
• disabling the Cloud plugin cuts off remote access and Cloud AI together

Test coverage

• gate auth, cookie rejection, budget concurrency/rollover/refunds, payload limits, strict schemas, and upstream error mapping
• connect-client request contracts, typed errors, malformed responses, and abort propagation
• server routing priority, fallback, validation, timeouts, and registration disposal
• Cloud-plugin preference persistence, latches, RPC, CLI, status propagation, separate settings sections, branding, and Remote access deep-link behavior
• direct local Wrangler inference/transcription E2E and source bb CLI title-generation E2E completed
• affected Turbo tests and package typechecks pass after the latest rebase

Draft blockers / follow-ups

• Included packages/connect-db/migrations/0006_ai_usage.sql in commit a66961246.
• Rebased onto origin/main at 85c49bc3c.
• Updated the draft PR to the Cloud-plugin/Remote-access-section product model.