Epic: OpenPets v4 — conversational Pet Assistant

[alvinunreal]

Goal

Deliver OpenPets v4 as a conversational desktop pet. A user can speak or type to the pet, continue one shared conversation, invoke capabilities from enabled plugins, and receive responses based on authoritative plugin results.

Product direction lives in docs/openpets_v4.md. This epic tracks delivery; child issues own implementation details.

Architecture invariants

• The host owns conversation lifecycle, provider integration, microphone state, capability routing, transcript/history, and user-facing feedback.
• Plugins own domain state, scheduling, notifications, and registered capability handlers.
• Voice and chat share one canonical conversation/turn/event contract.
• Capability results are authoritative; the assistant never invents successful actions.
• Provider-specific wire formats never enter capability or conversation contracts.
• The model receives only registered capabilities from enabled, healthy plugins.
• No arbitrary plugin commands, filesystem, shell, or unrestricted SDK access is exposed.
• Personality affects communication only; it cannot override host rules, permissions, or capability outcomes.
• Focus and Reminder actions execute without confirmation in v4.
• Recent local history is bounded and user-deletable; v4 has no semantic long-term memory.

Delivery tracker

Foundation

• feat(plugins): add Pet Assistant capability contract and runtime lifecycle #137 Plugin assistant capability SDK/runtime
• feat(assistant): add provider-neutral Pet Assistant conversation and tool loop #138 Provider-neutral Pet Assistant conversation/tool loop
• feat(focus-buddy): expose deterministic Pet Assistant capabilities #143 Focus Buddy assistant capabilities
• feat(reminders): expose Pet Assistant reminder capabilities #144 Quick Reminders assistant capabilities

Providers and prompting

• feat(providers): configure independent text, STT, and TTS profiles #145 Independent text, STT, and TTS provider profiles
• feat(assistant): add pet personality and layered prompt composition #146 Pet personality and layered prompt composition

Conversation adapters

• feat(voice): add composable STT → Pet Assistant → TTS conversations #147 Composable STT → Pet Assistant → TTS conversations
• feat(voice): add an OpenAI Realtime adapter for Pet Assistant conversations #139 OpenAI Realtime Pet Assistant adapter

Product surfaces and retention

• feat(desktop): add shared Pet Assistant chat and live transcript UI #148 Shared chat and live transcript UI
• feat(assistant): persist and manage recent local conversation history #149 Recent local conversation history and deletion
• feat(desktop): add pet Talk controls, shortcut, and assistant state feedback #150 Pet Talk controls, shortcut, and state feedback

Completion criteria

• Text chat and voice contribute to the same active conversation and transcript.
• Supported text, STT, and TTS providers can be independently selected.
• The generic voice path honors those independent selections.
• OpenAI Realtime is an optional optimized adapter, not the provider-neutral architecture.
• Focus Buddy and Quick Reminders work through natural-language requests using their existing domain state and scheduling.
• Talk starts from a pet-owned control or keyboard shortcut and can be muted or ended explicitly.
• The pet visibly represents listening, thinking, acting, speaking, missing information, success, and failure.
• Current-session transcript and bounded recent history are viewable and deletable.
• Required unit/contract tests, maintained docs, and real desktop validation are complete.

Coordination

Before starting a v4 child issue, claim it in an epic comment with its scope, assign it to yourself, and request a handoff if another developer has already claimed it. Link the focused PR to the child issue. After the PR is validated, reviewed, committed, pushed, and merged, comment here with the PR, delivered behavior, and validation; only then check the item off.

Out of scope

• Calendar capabilities
• Wake words or background listening
• Long-term semantic memory
• Arbitrary local-machine agent access
• A universal confirmation framework
• SDK capability progress streaming without a concrete v4 use case

[alvinunreal]

Completed: #143 and #144 merged in #152 (a77d374).

• Focus Buddy now exposes typed start, status, pause, resume, and end capabilities.
• Quick Reminders now exposes typed create, list, complete, snooze, and remove capabilities with serialized durable mutations.
• Validation: pnpm check, pnpm test, pnpm plugins:test, and Greptile (5/5, no findings).
• docs/openpets_v4.md now defines the two-developer claim → assign → PR → validate/review → commit/push → merge → epic-update flow.

#143 and #144 are closed. Next work is #145 and #146; claim one here and assign it before starting.

[bferanmi806-sketch]

I’ll take #146 — pet personality and layered prompt composition. I’ll keep the work scoped to persisted personality preferences, deterministic prompt layering, Settings UI/defaults, boundary-focused tests, and matching docs. Assigning #146 to myself before starting so we don’t overlap.

[alvinunreal]

Claiming #145: host-owned, independently persisted text/reasoning, STT, and TTS profiles; routing for existing assistant/plugin voice/realtime paths; Control Center settings, diagnostics, contract tests, and docs. Excludes chat/voice surfaces, personality, and Realtime tool calling.

[alvinunreal]

Claiming #147: provider-neutral STT → Pet Assistant → TTS multi-turn lifecycle, shared microphone/privacy ownership, normalized transcripts, pet activity mapping, deterministic adapter tests, and matching docs. Excludes Talk controls/shortcut (#150), chat UI (#148), retained history (#149), and OpenAI Realtime protocol work (#139).

[bferanmi806-sketch]

I’ll take #148 shared Pet Assistant chat and live transcript UI. I’ll keep this focused on the Control Center Conversation surface, host-owned current-session projection, narrow preload/main IPC, typed Pet Assistant turns, canonical assistant event rendering, cancellation, and a clean integration seam for #147’s normalized voice transcript events. I’ll keep persisted history (#149), Talk controls/shortcuts (#150), provider UI, and provider-specific voice protocol work out of this PR.

[alvinunreal]

Claiming #150 — pet-owned Talk entry point, configurable global shortcut with lifecycle feedback, Conversation controls, and adapter-neutral pet activity feedback. No provider protocols, capability execution, retained history, or Realtime.

[alvinunreal]

Claiming #149 — host-owned atomic local conversation archive with documented retention/corruption recovery; explicit active-context vs archive separation; bounded archived context in prompt composition; narrow Conversation list/open/delete-one/delete-all UI/IPC; deterministic storage, context, UI/IPC, and no-provider-call tests. Excludes embeddings, summaries, semantic retrieval, inferred preferences, long-term memory, provider protocol, Realtime (#139), and unrelated Talk work.

[bferanmi806-sketch]

I’ll take #139 OpenAI Realtime Pet Assistant adapter. I’ll keep this scoped to bounded/correlated realtime tool-event routing through PetAssistantService, capability discovery/execution, structured tool-result return, canonical transcript/activity mapping, OpenAI provider-profile selection, stale-session/generation cleanup, deterministic fake-transport coverage, and matching docs.