Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,11 @@ GOOGLE_CONNECTOR_UID=
# optional click-to-message shortcut in the workspace.
LINQ_CONNECTOR=
LINQ_PHONE_NUMBER=
# Optional per-user Link wallet connection. Register the exact callback URL
# https://<BETTER_AUTH_URL host>/api/auth/callback/link with Stripe.
LINK_CLIENT_ID=
LINK_CLIENT_SECRET=
STRIPE_PUBLISHABLE_KEY=
# Development benchmarks only (pnpm bench:browser).
BROWSER_BENCH_LABEL=self-hosted
BROWSER_BENCH_REPETITIONS=1
47 changes: 47 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,53 @@ Gotchas:
- Sending email and creating confirmed calendar events always require approval.
Calendar events with attendees send Google invitations.

## Link wallet

The root agent mounts `@stripe/link-integrations-eve` in
`agent/extensions/link.ts`. It uses Stripe's bundled wallet tools, skills, and
approval policies, with per-user authorization through the existing Better Auth
account. It does not use a shared wallet token.

To enable it, register a Link OAuth client and configure `LINK_CLIENT_ID`,
`LINK_CLIENT_SECRET`, and `STRIPE_PUBLISHABLE_KEY` on the server. Register the
exact redirect URI `https://<your-app-host>/api/auth/callback/link` with Stripe,
including a separate localhost URI when developing locally. Set
`BETTER_AUTH_URL` to the canonical application origin. These values are optional
for installations that do not use Link.

Apply the database migration with `pnpm db:migrate` before enabling Link. It
enforces one Link wallet per OpenInstinct account. Disconnect the current wallet
before connecting a different one; reconnecting the same wallet refreshes its
grant.

Users connect or disconnect their wallet from **Link wallet** in the sidebar.
An agent request that needs a wallet opens the same connection flow and resumes
through Eve's authorization callback. Connection attempts expire after ten
minutes and belong to the signed-in user. Better Auth stores encrypted grants
and refreshes tokens; disconnection revokes the Link grant before removing it.
Phone sign-in continues to work after disconnecting a wallet.

Wallet access is available in interactive conversations, not scheduled workers
or scheduled result delivery. The extension's default Eve approval for creating
spend requests remains enabled, separately from approval in Link. Its tools can
return payment credentials into stored Eve tool results; the bundled skills
instruct the agent not to repeat them in chat. Financial-data tools also require
the corresponding Link grant scopes; the default grant requests
`payment_methods.agentic` and `userinfo:read`.

## Eve upgrades

This application uses Eve `0.69.0`. Browser work runs as a resumable task:
`run_browser` returns a task receipt, subsequent calls use `taskId`, and
`task_cancel` stops the task's current work while keeping its browser session
available for another assignment. Task results arrive within the open turn.

Eve rejects session handoffs across the 0.69 upgrade boundary. Keep an existing
session on its owning deployment until it finishes, and start a new conversation
on the upgraded deployment. This migration does not cancel or reset existing
production sessions. Update remote-agent deployments before their callers when
using remote agents.

## Local development

The **Deploy with Vercel** flow above is the simplest way to run OpenInstinct. It
Expand Down
8 changes: 8 additions & 0 deletions agent/channels/eve.ts
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,14 @@ const channel = eveChannel({
auth: authenticate,
events: {
async "action.result"(event, _channel, session) {
if (
event.status === "completed" &&
event.result.kind === "tool-result" &&
event.result.toolName === "no_reply"
) {
await finalizeScheduledReportDelivery(session, "suppressed");
return;
}
if (
event.status === "completed" &&
sendMessageToolResultSchema.safeParse(event.result).success
Expand Down
8 changes: 8 additions & 0 deletions agent/channels/linq.ts
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,14 @@ export default linqChannel({
credentials,
events: {
async "action.result"(event, context, session) {
if (
event.status === "completed" &&
event.result.kind === "tool-result" &&
event.result.toolName === "no_reply"
) {
await finalizeScheduledReportDelivery(session, "suppressed");
return;
}
const reaction = reactToMessageToolResultSchema.safeParse(event.result);
if (event.status === "completed" && reaction.success) {
if (!context.thread) {
Expand Down
4 changes: 4 additions & 0 deletions agent/extensions/link.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
import link from "@stripe/link-integrations-eve";
import { linkAuth } from "@agent/lib/link-auth";

export default link({ auth: linkAuth });
12 changes: 3 additions & 9 deletions agent/hooks/background-reply-target.ts
Original file line number Diff line number Diff line change
@@ -1,17 +1,11 @@
import { defineHook } from "eve/hooks";
import { registerBackgroundReplyTarget } from "@agent/lib/reply-targets";
import { browserTaskReceiptSchema } from "@agent/lib/browser-task";

export default defineHook({
events: {
"action.result"(event, context) {
if (event.data.status !== "completed") return;
const task = browserTaskReceiptSchema.safeParse(event.data.result);
if (!task.success) return;
registerBackgroundReplyTarget(
task.data.output.taskId,
context.session.auth
);
"task.started"(event, context) {
if (event.data.name !== "run_browser") return;
registerBackgroundReplyTarget(event.data.taskId, context.session.auth);
},
},
});
24 changes: 1 addition & 23 deletions agent/hooks/scheduled-run-completion.ts
Original file line number Diff line number Diff line change
@@ -1,10 +1,8 @@
import { defineHook } from "eve/hooks";
import { browserTaskReceiptSchema } from "@agent/lib/browser-task";
import { scheduledRunIdentity } from "@agent/lib/schedules/identity";
import { scheduledRunOutcomeSchema } from "@shared/schedules/outcome";
import {
completeScheduledAgentRun,
deferScheduledAgentRunCompletion,
markScheduledAgentRunStarted,
releaseScheduledAgentRun,
waitForScheduledAgentRunInput,
Expand Down Expand Up @@ -62,26 +60,6 @@ export default defineHook({
sessionId: ctx.session.id,
});
},
async "action.result"(event, ctx) {
if (event.data.status !== "completed") return;
const task = browserTaskReceiptSchema.safeParse(event.data.result);
if (!task.success) return;
const identity = scheduledRunIdentity(ctx.session.auth);
if (!identity) return;
const deferred = await deferScheduledAgentRunCompletion(
identity.runId,
identity.leaseToken,
ctx.session.turn.id,
new Date(event.meta.at)
);
console.info("[scheduled-run] worker delegated background work", {
deferred,
runId: identity.runId,
sessionId: ctx.session.id,
taskId: task.data.output.taskId,
turnId: ctx.session.turn.id,
});
},
async "message.completed"(event, ctx) {
const identity = scheduledRunIdentity(ctx.session.auth);
if (!identity) return;
Expand All @@ -102,7 +80,7 @@ export default defineHook({
logDeadLetterReportQueued(status, identity.runId, ctx.session.id);
return;
}
const message = event.data.message?.trim().slice(0, 4_000);
const message = event.data.message.trim().slice(0, 4_000);
const outcome = scheduledRunOutcomeSchema.parse(
message
? {
Expand Down
10 changes: 5 additions & 5 deletions agent/instructions/content/role/interactive.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,15 +56,15 @@ The main conversation is the control plane. Coordinate the user's work there and
- Eve and Linq own read receipts, typing indicators, delivery state, authorization prompts, and approval/input cards. Let those native control-plane features operate normally; do not duplicate them as prose unless the user needs an explanation.
- After a `send_message` or `react_to_message` call, never repeat or summarize it in assistant text. If the runtime requires terminal assistant text after the last delivery, emit only `DELIVERY_COMPLETE`; ordinary assistant text is not delivered to Linq.
- The worker's structured result is coordinator-facing only. Rewrite it into a concise user-facing response; never imply that the worker spoke to the user.
- Start a background worker without a separate preamble. Once its working receipt arrives, send one short acknowledgment saying what is underway. Treat the receipt as acceptance, not completion.
- Keep intermediate background-task wakes silent unless the user must act. When the worker settles, compose the result according to the iMessage composition rules, including a separate follow-up message when the user has a distinct choice or approval to make.
- End the turn whenever the root conversation genuinely needs an answer. Before surfacing a `Needs user input:` blocker for routine contact, traveler, or address fields, confirm the worker explicitly reported checking compatible vault items. If it did not, continue the same worker once and instruct it to call `list_vault` and `fill_from_vault`; only surface the question if no compatible item exists or secure fill still fails. After the user replies, continue that worker with its `agentId` and the supplied answer so it retains its browser state and context. If the answer is an OTP, pass it immediately without echoing it in assistant text.
- Start a browser task without a separate preamble. Once its working receipt arrives, send one short acknowledgment saying what is underway. Treat the receipt as acceptance, not completion.
- Tasks report results within the open turn. Keep intermediate updates silent unless the user must act. When the worker settles, compose the result according to the iMessage composition rules, including a separate follow-up message when the user has a distinct choice or approval to make.
- End the turn whenever the root conversation genuinely needs an answer. Before surfacing a `Needs user input:` blocker for routine contact, traveler, or address fields, confirm the worker explicitly reported checking compatible vault items. If it did not, continue the same worker once and instruct it to call `list_vault` and `fill_from_vault`; only surface the question if no compatible item exists or secure fill still fails. After the user replies, continue that worker with its `taskId` and the supplied answer so it retains its browser state and context. If the answer is an OTP, pass it immediately without echoing it in assistant text.

# Worker coordination

- Delegate to `browser-agent` only when the task requires interaction with a known site, authenticated or browser-local state, visual inspection, form entry, or content that `web_search` and `web_fetch` could not retrieve. Do not use a generic agent copy or any browser-execution tool yourself. When public discovery is needed before interaction, complete it with `web_search` first and include the useful target URLs and facts in the worker assignment.
- Worker images are private, user-scoped artifacts. Ask the worker for and include visual artifacts when they materially help the user verify a completed outcome, compare genuinely visual options, or understand a concrete result. Choose the smallest useful set: one image for a single result, or two to four distinct images when a compact gallery adds real comparison or verification value. Useful cases include showing a purchased item or safe, non-sensitive confirmation state and comparing clothing, food, product, or other visually differentiated options. Put two to four selected artifact references in the same `send_message` call so Linq can present them as one native media gallery; keep its text to one short caption, identify images by order only when the mapping matters, and use a separate short selection question when needed. Do not use a gallery for text-heavy comparisons such as flights, schedules, or prices unless the images materially improve the decision. Skip decorative, repetitive, text-only, or sensitive visuals. Render every selected artifact in order with its exact descriptor as `![label](/artifacts/id)`. Never invent an artifact, change its id or URL, expose a private Blob URL, or claim an internal screenshot was delivered.
- When a visually selected product is unavailable, treat images as part of the replacement outcome even if the user does not ask for them again. Find the strongest viable alternatives, ask the worker for one clear artifact per recommendation, and return a compact two-to-four-image gallery with the concrete availability details.
- When the worker returns a missing vault item, call `request_vault_setup` and give its link to the user before ending the turn; do not surface any live browser URL from that result. For a purchase decision, authentication challenge after vault fill, unresolved CAPTCHA after Kernel's managed solver wait, ambiguous choice, or human-takeover blocker, ask the user in the root conversation. Preserve the worker's `agentId` and live browser URL when available, then continue that same parked worker after the user responds so it re-reads the current page before acting.
- Treat a new user message as current steering. Preserve unrelated work. When it revises an active worker task, cancel its `run_browser` task with `task_cancel`, wait for cancellation to settle, then call `run_browser` with the parked worker's `agentId` and the complete revised assignment. Do not send another workflow call to a worker that is still running. Use `task_cancel` when the user wants work stopped without a replacement assignment. Reuse an available worker only when its prior browser state and context remain useful. Cancellation is cooperative and does not roll back external effects, so do not promise atomic interruption.
- When the worker returns a missing vault item, call `request_vault_setup` and give its link to the user before ending the turn; do not surface any live browser URL from that result. For a purchase decision, authentication challenge after vault fill, unresolved CAPTCHA after Kernel's managed solver wait, ambiguous choice, or human-takeover blocker, ask the user in the root conversation. Preserve the worker's `taskId` and live browser URL when available, then continue that same parked worker after the user responds so it re-reads the current page before acting.
- Treat a new user message as current steering. Preserve unrelated work. When it revises an active worker task, cancel its `run_browser` task with `task_cancel`, wait for cancellation to settle, then call `run_browser` with the parked worker's `taskId` and the complete revised assignment. Do not send another workflow call to a worker that is still running. Use `task_cancel` when the user wants work stopped without a replacement assignment. Reuse an available worker only when its prior browser state and context remain useful. Cancellation is cooperative and does not roll back external effects, so do not promise atomic interruption.
- Do not create overlapping workers for the same assignment. Do not delegate non-browser work merely to create activity.
2 changes: 1 addition & 1 deletion agent/instructions/content/role/scheduled-report.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,6 @@ You are OpenInstinct evaluating the completed outcome of a background scheduled
- Consider the outcome together with the current conversation and the time for which it was scheduled.
- This turn exists only to report the outcome, resume its run with context already present in the conversation, or create a secure setup link for a vault item the run needs. Never invoke another agent, alter a schedule or profile, read or change vault contents, access an account, or perform any other external action.
- When the run is waiting for input, answer it with `schedules-answer` if the existing conversation resolves the request clearly, then do not send a message. If it needs a supported vault item, call `request_vault_setup` with only the safe metadata in the request, then call `send_message` exactly once with the returned setup link and ask the user to reply when finished. For any other unresolved request, call `send_message` exactly once with its question. Preserve the internal run ID in context for a later `schedules-answer` call, and never guess or expose it to the user.
- For a completed outcome, call `send_message` exactly once only when it is still useful, actionable, time-sensitive, or materially changes what the user knows. Otherwise finish with exactly `<eve-empty-delivery/>` and do not call `send_message`.
- For a completed outcome, call `send_message` exactly once only when it is still useful, actionable, time-sensitive, or materially changes what the user knows. Otherwise call `no_reply` to end the turn silently.
- Rewrite useful information as a natural message from OpenInstinct. Never mention the internal worker, handoff, reporting state, or implementation details.
- After `send_message`, emit only `DELIVERY_COMPLETE` if the runtime requires terminal assistant text. Never repeat or summarize the delivered message.
2 changes: 1 addition & 1 deletion agent/instructions/content/worker-coordination.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,6 @@

- Use `run_browser` to give `browser-agent` one bounded browser outcome, the exact target URL when known, all relevant non-secret context, the user's constraints, and any exact transaction approval already granted. The worker does not see the parent conversation. Preserve the requested visual deliverable in the assignment: treat a photo, pic, or image of a specific webpage item, product, dish, or listing as the item's actual page image, while a screenshot or page view means rendered screen context. When authentication may be required, tell it to return `Needs vault setup:` with safe login metadata if no compatible login exists, without a live-view URL or a request for manual credential entry. Tell it to return `Needs user input:` asking for the code when an OTP blocks progress; do not make live view the OTP path.
- Begin every worker `message` with `Task: <a concise summary of at most 10 words>`, followed by a blank line and the complete assignment. Keep the summary free of credentials and sensitive personal details, and reuse it when continuing the same worker.
- The `run_browser` tool supplies the required structured completion schema on every initial or resumed call, including when passing an existing `agentId`. Supply only the assignment and optional `agentId`; do not supply an `outputSchema`.
- The `run_browser` tool supplies the required structured completion schema on every initial or resumed call, including when passing an existing `taskId`. Supply only the assignment and optional `taskId`; do not supply an `outputSchema`.
- The worker must finish each browser assignment by calling Eve's native `final_output` tool exactly once with a result matching the required `outputSchema`, then stop without prose, JSON text, another tool, or a second completion. Treat `success` as achieved only when its message includes a verified outcome. Treat `failure` as a blocker or incomplete outcome, not proof that no progress occurred.
- Worker images are private, user-scoped artifacts. Ask for visual artifacts only when they materially help verify an outcome or compare genuinely visual options, and choose the smallest useful set. Preserve each selected artifact's exact descriptor as `![label](/artifacts/id)`; never invent an artifact, change its id or URL, expose a private Blob URL, or claim an internal screenshot was delivered.
7 changes: 0 additions & 7 deletions agent/lib/browser-task.ts

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,7 @@ function toolContext() {
async getSandbox() {
throw new Error("Sandbox access is outside this focused test.");
},
messages: [],
abortSignal: new AbortController().signal,
callId: "call-1",
getToken,
Expand Down
Loading
Loading