Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .changeset/agent-safety-long-tasks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
'@roomote/worker': patch
'@roomote/cloud-agents': patch
---

Long tasks are no longer stopped after 200 tool calls or 100 model steps, and replying to a task stopped for repeated activity resumes it from its retained work. The 1,000-request lifetime ceiling and repetition detection still apply.
19 changes: 10 additions & 9 deletions apps/docs/tasks.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -110,11 +110,11 @@ The read-only task-update polling tool has a larger, finite allowance of 20
identical completions. Shell commands and search tools have no polling exception.

Subagent passes, including review, judge, explore, and custom subagents, also
stop at 100 model steps, 200 completed tools, or 20 minutes. Activity does not
extend the deadline. The main task also has repetition and completed-tool
protection. A task has a lifetime ceiling of 1,000 admitted provider requests,
shared across its models, children, retries, and resumed runs. Unknown prices
do not disable that request ceiling.
stop after 20 minutes. Activity does not extend the deadline. The main task
also has repetition protection. A task has a lifetime ceiling of 1,000 admitted
provider requests, shared across its models, children, retries, and resumed
runs. Unknown prices do not disable that request ceiling. Long tasks are not
stopped for the number of tool calls or model steps alone.

A safety stop cancels the task's agent tree instead of asking the parent agent
to retry the stopped pass. A notice is saved in the task and its owning session,
Expand All @@ -123,10 +123,11 @@ if a notice has not appeared in an already-open view. Partial work is retained;
an interrupted review must not be treated as a clean review. An already-admitted
provider request can still finish and incur cost while cancellation settles.

Automatic retry, model fallback, restart, and resume do not reset a stopped
task's admission state. Inspect the partial work and the stop reason before
explicitly starting a new task with a narrower request. There is no automatic
budget reset or in-product allowance-extension control.
Automatic retry, model fallback, and restart do not continue a stopped task.
Resuming the task, for example by replying to it, continues from the retained
work after a repetition or time stop. The request count is never reset: a task
that reached the 1,000-request ceiling stays stopped, and continuing that work
requires a new task. There is no in-product allowance-extension control.

These are request and execution bounds, not a dollar spending limit. Use your
inference provider's own billing controls for a financial cap; see
Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

50 changes: 50 additions & 0 deletions packages/cloud-agents/src/server/__tests__/enqueue-task.test.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 4 additions & 0 deletions packages/cloud-agents/src/server/task-run-queue.ts
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ import {
type TaskRun,
type DatabaseTransaction,
db,
releaseAgentSafetyStopForResume,
deploymentSettings,
ensureAutomationRowsOnce,
ensureSessionForTask,
Expand Down Expand Up @@ -3092,6 +3093,9 @@ async function enqueueSnapshotResume(
await options.afterCreateInTransaction(tx, insertedRun);
}

// Resuming continues a pass-level safety stop; the request ceiling holds.
await releaseAgentSafetyStopForResume(tx, sourceRun.taskId);

// The new resume run is non-terminal (pending/dequeued), so re-derive the
// task state: a task that had gone terminal on a prior attempt flips back
// to 'active' now that another attempt is queued.
Expand Down
24 changes: 15 additions & 9 deletions packages/db/agent-safety.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,13 @@
The task row is the durable admission authority. `inference_request_limit`
defaults to 1,000; `inference_requests` counts admitted attempts, not successful
responses or accounting events. `safety_stopped_at` and `safety_stop_reason`
are an irreversible application-level latch for the task. No resume, new run,
provider fallback, or ordinary message clears these fields. There is no UI or
public API to grant more allowance in this release; explicit continuation is a
new task after inspecting the retained work. Operators should not clear a latch
while an old runtime is alive.
are the task's stop latch. Provider fallback, in-run retries, and ordinary
admission never clear it. Creating a snapshot-resume run releases a pass-level
latch (`releaseAgentSafetyStopForResume`, in the run-creation transaction)
unless the reason is `task_request_limit` or the counter is already at the
ceiling; `inference_requests` is never reset. There is no UI or public API to
grant more allowance; continuing past the ceiling is a new task. Operators
should not clear a latch while an old runtime is alive.

## Admission guarantees

Expand Down Expand Up @@ -49,9 +51,12 @@ updates for the same invocation are ignored. The exact read-only
`manage_tasks`/`roomote_manage_tasks` action `get_updates` gets 20 identical
completions; it is not an unlimited polling exception.

Each live child gets its own 100 unique assistant-message starts, 200 unique
terminal tool completions, and 20-minute wall deadline, independent of progress
and siblings. A pending task-tool spawn is time-bounded even before its child ID
Each live child gets its own 20-minute wall deadline, independent of progress
and siblings. There is no per-pass tool-completion or step cap: legitimate long
passes routinely exceed any fixed count, and the lifetime request ceiling plus
repetition detection bound runaway spend. `pass_step_limit` and
`pass_tool_limit` remain accepted reasons only so older workers can report
them (N-1). A pending task-tool spawn is time-bounded even before its child ID
arrives. The worker checks wall deadlines every second. Root tool completions
are guarded too. Control-plane prompts retain their normal shorter deadlines
(120 seconds by default); `timeoutMs: null` and larger timeouts cannot disable
Expand All @@ -62,7 +67,8 @@ retry/fallback queues, aborts the root and discovered descendants with bounded
abort requests, emits the terminal task event to release worker/parent waits,
and refuses late local commands/events. This intentionally does not let the
parent spawn a replacement pass. The durable task latch refuses all subsequent
managed provider dispatch even if the worker restarts. The existing parent
managed provider dispatch even if the worker restarts, until a resume run
releases a pass-level stop. The existing parent
settle/error relays do not wake parent inference for a safety-stopped task.

## Boundaries to keep in mind
Expand Down
43 changes: 43 additions & 0 deletions packages/db/src/lib/__tests__/agent-safety.test.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

34 changes: 31 additions & 3 deletions packages/db/src/lib/agent-safety.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { and, desc, eq } from 'drizzle-orm';
import { and, desc, eq, isNotNull, isNull, lt, ne, or } from 'drizzle-orm';
import {
AGENT_SAFETY_STOP_TEXT,
ACP_ENVELOPE_EVENT_TYPES,
Expand Down Expand Up @@ -81,8 +81,9 @@ async function stopInTransaction(
.insert(fastAgentMessages)
.values({
conversationId: parentConversation.id,
eventId: `agent-safety:${taskId}`,
turnId: `agent-safety:${taskId}`,
// Per run: a resume can release the latch and a later run stop again.
eventId: `agent-safety:${taskId}:${run.id}`,
turnId: `agent-safety:${taskId}:${run.id}`,
turnSeq: 0,
ts,
eventType: ACP_ENVELOPE_EVENT_TYPES.AgentSafetyStop,
Expand Down Expand Up @@ -142,3 +143,30 @@ export async function admitTaskInferenceRequest(
return true;
});
}

/**
* A new resume run is an explicit continuation, so it releases a pass-level
* safety stop (for example an identical-tool loop). The lifetime request
* counter is never reset, and a `task_request_limit` stop stays latched.
*/
export async function releaseAgentSafetyStopForResume(
tx: Transaction,
taskId: string,
): Promise<boolean> {
const released = await tx
.update(tasks)
.set({ safetyStoppedAt: null, safetyStopReason: null })
.where(
and(
eq(tasks.id, taskId),
isNotNull(tasks.safetyStoppedAt),
or(
isNull(tasks.safetyStopReason),
ne(tasks.safetyStopReason, 'task_request_limit'),
),
lt(tasks.inferenceRequests, tasks.inferenceRequestLimit),
),
)
.returning({ id: tasks.id });
return released.length > 0;
}
18 changes: 8 additions & 10 deletions packages/types/src/__tests__/agent-safety.test.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 1 addition & 6 deletions packages/types/src/agent-safety.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,13 @@
export const AGENT_SAFETY = {
identicalCompletions: 5,
pollingCompletions: 20,
passSteps: 100,
passTools: 200,
passTimeoutMs: 20 * 60_000,
taskRequests: 1_000,
} as const;

export type AgentSafetyReason =
| 'identical_tool_loop'
// N-1: no longer raised; kept so older workers can still report them.
| 'pass_step_limit'
| 'pass_tool_limit'
| 'pass_time_limit'
Expand Down Expand Up @@ -89,8 +88,6 @@ export class AgentSafetyPass {
step(id: string): AgentSafetyReason | undefined {
if (this.stopped) return this.stopped;
this.steps.add(id);
if (this.steps.size >= AGENT_SAFETY.passSteps)
this.stopped ??= 'pass_step_limit';
return this.stopped;
}

Expand Down Expand Up @@ -122,8 +119,6 @@ export class AgentSafetyPass {
: AGENT_SAFETY.identicalCompletions)
)
this.stopped = 'identical_tool_loop';
if (this.completedCalls.size >= AGENT_SAFETY.passTools)
this.stopped ??= 'pass_tool_limit';
return this.stopped;
}
}
Loading