From f64ce3f1922fe8ddcd8c88db1d766de49c325f47 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E9=83=91=E8=BE=BE=E5=A5=87?= <> Date: Mon, 27 Jul 2026 16:01:08 +0800 Subject: [PATCH 1/2] feat(agent): preview post-tool convergence before boundaries --- ...CK_CONVERGENCE_V24R15_POST_TOOL_PREVIEW.md | 164 ++++++++++++++++ .../legal/plugins/legal-coverage/hook.mjs | 77 +++++++- .../plugins/legal-coverage/hooks/hooks.json | 2 +- src/agent/convergence/ProgressLease.ts | 39 +++- src/agent/loop/AgentLoop.ts | 74 ++++++- src/agent/protocol/events.ts | 6 + src/extension/hooks/execution/HookRuntime.ts | 11 +- .../hooks/execution/parseHookOutput.ts | 34 ++++ src/extension/hooks/protocol/output.ts | 15 ++ src/gateway/client/InProcessGateway.ts | 6 + src/lifecycle/protocol/effects.ts | 2 + src/observability/agentEventBridge.ts | 15 ++ src/tool/execution/ToolRuntime.ts | 14 +- .../agent/agent-loop-runtime-controls.spec.ts | 180 +++++++++++++++++- tests/agent/progress-lease.spec.ts | 56 ++++++ .../hooks/model-request-patch.spec.ts | 88 +++++++++ tests/gateway/map-agent-event-runid.spec.ts | 14 ++ .../local-gateway-progress-handoff.spec.ts | 80 ++++++-- tests/observability/recorder.spec.ts | 25 +++ tests/products/legal-coverage.spec.ts | 46 ++++- tests/tool/bash-result-display.spec.ts | 58 ++++++ 21 files changed, 964 insertions(+), 42 deletions(-) create mode 100644 docs/PILOTDECK_CONVERGENCE_V24R15_POST_TOOL_PREVIEW.md diff --git a/docs/PILOTDECK_CONVERGENCE_V24R15_POST_TOOL_PREVIEW.md b/docs/PILOTDECK_CONVERGENCE_V24R15_POST_TOOL_PREVIEW.md new file mode 100644 index 00000000..30067fd2 --- /dev/null +++ b/docs/PILOTDECK_CONVERGENCE_V24R15_POST_TOOL_PREVIEW.md @@ -0,0 +1,164 @@ +# PilotDeck V24R15 post-tool convergence preview + +## Problem contract + +The preserved V24R14 Case 09 run did useful work but reached the 2100-second +machine deadline. O1 recorded 67 model requests, 113 tool pairs, and 19 full +compactions. Most forced compactions began at only 49-55 percent context use. + +The dominant ordering failure was: + +1. `PreModelRequest` reported stagnation and scheduled a boundary. +2. The model wrote a valid, state-bound legal proposal. +3. The Legal Coverage plugin could report the new handoff only at the next + `PreModelRequest`. +4. AgentLoop ran the already-scheduled full compaction before that hook. +5. The hook then confirmed `handoff_grace`, after the expensive boundary had + already consumed time. + +One observed proposal write took about 14 seconds, while the unnecessary full +summary immediately after it took about 45 seconds. The run spent an estimated +10-15 minutes on this repeated ordering penalty. + +## Scope and ownership + +V24R15 adds one domain-neutral Core protocol and one Legal Coverage adapter. + +Core owns: + +- a bounded `convergencePreview` PostToolUse output; +- internal transport from HookRuntime through ToolRuntime to AgentLoop; +- pre-boundary eligibility checks; +- mandatory confirmation before the next model request; +- fail-closed behavior and O1/Gateway projection. + +The Legal Coverage plugin owns: + +- deciding which legal tool operations may change durable legal state; +- re-running the unchanged legal validator after those operations; +- deriving the same opaque convergence ordinals used by `PreModelRequest`; +- emitting no legal work items, commands, prompts, or validator payload in the + preview. + +Core does not understand legal phases, ledgers, proposal schemas, authorities, +or matrix transactions. The plugin cannot alter Progress Lease policy. + +## Bounded protocol + +`convergencePreview` accepts only: + +- `schemaVersion`; +- `scope`, `phase`, and `stateHash`; +- optional `blockingCode`; +- `remainingCount`; +- optional progress, repair, repair-preparation, and handoff ordinals. + +Strings and ordinals are length/range checked. Unknown fields such as +`nextBatch`, commands, templates, or private domain payload are discarded. +HookRuntime produces the effect only for `PostToolUse`. ToolRuntime stores it +only under internal lifecycle metadata; the tool result text shown to the model +and user is unchanged. + +## Boundary decision + +At the next loop boundary, Progress Lease may defer a required full compaction +only when exactly one tracked scope requires a boundary and its newest preview +shows one of: + +- `remainingCount` decreased; +- `progressOrdinal` increased; +- completion (`remainingCount = 0` and no blocker); +- `handoffOrdinal` increased within the existing per-progress-epoch budget. + +A preview cannot defer for opaque state-hash churn, repair-only changes, +repair-preparation-only changes, replayed/lower ordinals, an exhausted handoff +budget, or multiple simultaneously forcing scopes. Lease thresholds remain +`maxInitialStagnantObservations=8` and `maxStagnantObservations=2`; no new grace +decision is added. + +The preview is advisory and consumes no lease state. AgentLoop clears it after +one boundary decision. + +## Confirmation and failure mode + +After a boundary is deferred, the immediately following `PreModelRequest` is +still authoritative. It must report the same scope and produce `renewed`, +`completed`, or `handoff_grace` under the existing Progress Lease rules. + +If the report is missing, stale, replayed, repair-only, over budget, or belongs +to another scope, AgentLoop stops before sending the model request with: + +- error: `agent_convergence_stalled`; +- reason: `boundary_preview_unconfirmed`. + +This prevents a buggy or stale PostToolUse hook from converting an advisory +preview into unbounded execution. + +## Legal adapter performance boundary + +The Legal Coverage hook does not validate after every tool call. It computes a +preview only after operations that may change durable legal state: + +- `write_file` inside `.pilotdeck/work/legal-coverage`; +- the plugin's explicit state-changing CLI commands, including initialization, + source preparation/apply, matrix apply, issue apply, authority apply, and + coverage batch apply. + +`read_file`, writes outside the legal state directory, and read-only legal CLI +commands do not produce a preview. Existing repair-preparation tracking remains +separate and cannot defer a boundary. + +The adapter computes ordinals without writing progress or handoff state. +`PreModelRequest` remains the only persistence and confirmation point. Writing +a proposal can expose a bounded handoff, but cannot manufacture semantic +progress. + +## Observability + +Every deferral emits a separate O1 decision: + +- type: `harness.decision`; +- component: `progress-boundary`; +- policy: `progress-boundary/v1`; +- decision: `deferred`; +- reason: `post_tool_convergence_preview`. + +The following authoritative result remains a normal +`progress-lease/v5` decision. This preserves the existing policy stream while +making ordering and confirmation independently auditable. Gateway consumers +also receive `progress_boundary_deferred` status with only scope names. + +## Code gate + +The required code evidence is: + +- parser whitelist and size/range rejection; +- HookRuntime event restriction; +- internal-only ToolRuntime transport; +- Progress Lease acceptance and counterexamples; +- multi-scope and exhausted-budget rejection; +- AgentLoop success with zero forced full compactions; +- stale-preview failure before a third model request; +- real local Gateway run with complete O1 pairing and the ordered deferred then + confirmed decisions; +- unchanged Legal Coverage validator and transaction tests; +- full repository test suite and diff checks. + +## Product gate + +Code success does not prove Case 09 success. After the code gate, create a new +immutable V24R15 campaign from the same V24R14 runner, candidate baseline, +model, Skills, corpus, Router/Memory controls, O1 diagnostic profile, 2100-second +deadline, and Lease `8/2`. + +Run Gate 1, Case 05, then Case 09 exactly once. Freeze the campaign on failure. +V25 and the 85-case campaign remain blocked until Case 09 produces a complete +O1 trajectory, unchanged-validator proof, completion proof, and substantive +report artifacts. No result from V24R14 is overwritten or reclassified. + +## Non-goals + +V24R15 does not change validators, prompt content, domain transactions, Router, +Memory, model selection, compaction capacity thresholds, runner deadlines, or +the corpus. It does not add Case 09 text matching and does not let plugins +cancel boundaries directly. diff --git a/products/legal/plugins/legal-coverage/hook.mjs b/products/legal/plugins/legal-coverage/hook.mjs index 3c3c6a2a..1207a30f 100644 --- a/products/legal/plugins/legal-coverage/hook.mjs +++ b/products/legal/plugins/legal-coverage/hook.mjs @@ -1,6 +1,6 @@ import { createHash } from "node:crypto"; import { mkdir, readFile, rm, stat, writeFile } from "node:fs/promises"; -import { dirname } from "node:path"; +import { dirname, isAbsolute, relative } from "node:path"; import { fileURLToPath } from "node:url"; import { PROOF_PATH, @@ -75,20 +75,29 @@ try { || await pathExists(input.cwd, `${STATE_DIRECTORY}/config.json`) || await pathExists(input.cwd, PROOF_PATH)); if (active && input.hookEventName === "PostToolUse" - && ["read_file", "write_file"].includes(input.toolName)) { + && ["bash", "read_file", "write_file"].includes(input.toolName)) { const preparation = await advanceRepairPreparation( input.cwd, sessionState, input.toolName, input.toolInput, ); + const previewSessionState = preparation.changed + ? { + ...sessionState, + active: true, + repairPreparationOrdinal: preparation.ordinal, + repairPreparationCheckpointDigests: preparation.seenCheckpointDigests, + } + : sessionState; if (preparation.changed) { - await writeSessionState(input.cwd, sessionPath, { - ...sessionState, - active: true, - repairPreparationOrdinal: preparation.ordinal, - repairPreparationCheckpointDigests: preparation.seenCheckpointDigests, - }); + await writeSessionState(input.cwd, sessionPath, previewSessionState); + } + if (await toolMayChangeConvergence(input.cwd, input.toolName, input.toolInput)) { + output.hookSpecificOutput.convergencePreview = await postToolConvergencePreview( + input.cwd, + previewSessionState, + ); } } @@ -237,6 +246,58 @@ async function dynamicWorkItems(workspaceRoot, result) { return undefined; } +async function postToolConvergencePreview(workspaceRoot, sessionState) { + const result = await validateWorkspace({ workspaceRoot, writeProof: false }); + const workItems = await dynamicWorkItems(workspaceRoot, result); + const digest = milestoneDigest(result, workItems); + const repairCheckpointDigest = checkpointDigest(legalRepairCheckpoint(workItems)); + const progressState = advanceProgressState(sessionState, digest, { + phase: result.passed ? "complete" : result.errors[0]?.phase ?? "incomplete", + blockingCode: result.errors[0]?.code ?? null, + remainingCount: result.errors.length, + }, legalProgressCheckpointDigest(workItems), repairCheckpointDigest); + const handoffState = advanceHandoffState(sessionState, legalHandoffCheckpointDigest(workItems)); + const report = convergenceReport( + result, + workItems, + convergenceStateHash(result, workItems), + progressState.ordinal, + progressState.repairOrdinal, + repairPreparationOrdinal(sessionState), + handoffState.ordinal, + ); + return { + schemaVersion: 1, + scope: report.scope, + phase: report.phase, + stateHash: report.stateHash, + ...(report.blockingCode ? { blockingCode: report.blockingCode } : {}), + remainingCount: report.remainingCount, + progressOrdinal: report.progressOrdinal, + repairOrdinal: report.repairOrdinal, + repairPreparationOrdinal: report.repairPreparationOrdinal, + handoffOrdinal: report.handoffOrdinal, + }; +} + +async function toolMayChangeConvergence(workspaceRoot, toolName, toolInput) { + if (!toolInput || typeof toolInput !== "object" || Array.isArray(toolInput)) return false; + if (toolName === "write_file") { + const inputPath = toolInput.file_path; + if (typeof inputPath !== "string" || inputPath.length === 0 || inputPath.length > 2048) return false; + const stateRoot = await resolveSafeWorkspacePath(workspaceRoot, STATE_DIRECTORY, { allowMissing: true }); + const writtenPath = await resolveSafeWorkspacePath(workspaceRoot, inputPath, { allowMissing: true }); + const stateRelativePath = relative(stateRoot, writtenPath); + return stateRelativePath === "" + || (!stateRelativePath.startsWith("..") && !isAbsolute(stateRelativePath)); + } + if (toolName !== "bash") return false; + const command = toolInput.command; + if (typeof command !== "string" || command.length === 0 || command.length > 32768) return false; + if (!command.includes("legal-coverage.mjs")) return false; + return /(?:^|\s)(?:init|bootstrap-sources|reference|fragment-slice|source-merge-prepare|source-merge-apply|source-repair-apply|matrix-selection-apply|matrix-proposal-apply|issue-closure-apply|authority-closure-apply|apply-batch)(?:\s|$)/u.test(command); +} + function convergenceReport( result, workItems, diff --git a/products/legal/plugins/legal-coverage/hooks/hooks.json b/products/legal/plugins/legal-coverage/hooks/hooks.json index 3c8dc016..ed19c245 100644 --- a/products/legal/plugins/legal-coverage/hooks/hooks.json +++ b/products/legal/plugins/legal-coverage/hooks/hooks.json @@ -21,7 +21,7 @@ ], "PostToolUse": [ { - "matcher": "read_file|write_file", + "matcher": "bash|read_file|write_file", "hooks": [ { "type": "command", diff --git a/src/agent/convergence/ProgressLease.ts b/src/agent/convergence/ProgressLease.ts index f823403a..6abb010b 100644 --- a/src/agent/convergence/ProgressLease.ts +++ b/src/agent/convergence/ProgressLease.ts @@ -33,6 +33,11 @@ export type ProgressBoundaryOutcome = { rejectionReason?: string; }; +export type ProgressBoundaryPlan = { + requested: boolean; + deferredScopes: string[]; +}; + export type ProgressLeaseObservation = { scope: string; phase: string; @@ -73,12 +78,29 @@ export class ProgressLease { constructor(private readonly config?: ProgressLeaseConfig) {} - shouldForceBoundary(): boolean { - if (!this.config?.enabled) return false; - return [...this.scopes.values()].some((state) => + shouldForceBoundary(previews: readonly ConvergenceReport[] = []): boolean { + return this.planBoundary(previews).requested; + } + + planBoundary(previews: readonly ConvergenceReport[] = []): ProgressBoundaryPlan { + if (!this.config?.enabled) return { requested: false, deferredScopes: [] }; + const required = [...this.scopes.entries()].filter(([, state]) => !state.awaitingPostBoundaryProgress && state.stagnantObservations >= this.stagnationLimit(state) - 1 ); + if (required.length === 0) return { requested: false, deferredScopes: [] }; + if (required.length !== 1) return { requested: true, deferredScopes: [] }; + + const previewByScope = new Map(); + for (const preview of previews) previewByScope.set(preview.scope, preview); + const deferredScopes = required.flatMap(([scope, state]) => { + const preview = previewByScope.get(scope); + return preview && this.previewCanDeferBoundary(state, preview) ? [scope] : []; + }); + if (deferredScopes.length !== required.length) { + return { requested: true, deferredScopes: [] }; + } + return { requested: false, deferredScopes: deferredScopes.sort() }; } observe( @@ -270,6 +292,17 @@ export class ProgressLease { private handoffLimit(): number { return this.config!.maxInitialStagnantObservations ?? this.config!.maxStagnantObservations; } + + private previewCanDeferBoundary(state: ScopeState, preview: ConvergenceReport): boolean { + if (preview.remainingCount === 0 && preview.blockingCode === undefined) return true; + const progressed = preview.remainingCount < state.remainingCount + || (preview.progressOrdinal !== undefined + && (state.progressOrdinal === undefined || preview.progressOrdinal > state.progressOrdinal)); + if (progressed) return true; + const handoffAdvanced = preview.handoffOrdinal !== undefined + && (state.handoffOrdinal === undefined || preview.handoffOrdinal > state.handoffOrdinal); + return handoffAdvanced && state.handoffsUsedSinceProgress < this.handoffLimit(); + } } export function parseConvergenceReport(value: unknown): ConvergenceReport | undefined { diff --git a/src/agent/loop/AgentLoop.ts b/src/agent/loop/AgentLoop.ts index 8d050889..e416c6cb 100644 --- a/src/agent/loop/AgentLoop.ts +++ b/src/agent/loop/AgentLoop.ts @@ -76,6 +76,7 @@ import { CONVERGENCE_METADATA_KEY, ProgressLease, parseConvergenceReport, + type ConvergenceReport, type ProgressBoundaryOutcome, } from "../convergence/ProgressLease.js"; import { observationHash } from "../../observability/index.js"; @@ -277,6 +278,7 @@ export class AgentLoop { let transientPromptCounter = 0; const activeTransientPromptIds = new Set(); const progressLease = new ProgressLease(this.config.progressLease); + let pendingConvergencePreviews: ConvergenceReport[] = []; const pushTransientSyntheticPrompt = (prompt: string, purpose: string): void => { const transientId = this.dependencies.uuid?.() ?? `transient-${++transientPromptCounter}`; @@ -377,7 +379,18 @@ export class AgentLoop { let pendingContextBudget: TokenBudgetSnapshot | undefined; const ctx = this.dependencies.context; - const forceProgressBoundary = progressLease.shouldForceBoundary(); + const boundaryPlan = progressLease.planBoundary(pendingConvergencePreviews); + const forceProgressBoundary = boundaryPlan.requested; + const deferredBoundaryScopes = boundaryPlan.deferredScopes; + pendingConvergencePreviews = []; + if (deferredBoundaryScopes.length > 0) { + yield { + type: "progress_boundary_deferred", + sessionId: input.sessionId, + turnId: input.turnId, + scopes: deferredBoundaryScopes, + }; + } const progressBoundary: ProgressBoundaryOutcome = { requested: forceProgressBoundary, attempted: false, @@ -488,8 +501,10 @@ export class AgentLoop { request = transformedRequest.request; promptInjections = [...promptInjections, ...preModelInjections]; const convergenceReport = parseConvergenceReport(request.metadata?.[CONVERGENCE_METADATA_KEY]); + let progressDecision: ReturnType = undefined; if (convergenceReport) { const progress = progressLease.observe(convergenceReport, progressBoundary); + progressDecision = progress; if (progress) { yield { type: "progress_lease_evaluated", @@ -521,6 +536,36 @@ export class AgentLoop { } } } + const deferredBoundaryConfirmed = deferredBoundaryScopes.length === 0 + || (progressDecision !== undefined + && deferredBoundaryScopes.includes(progressDecision.scope) + && ["renewed", "completed", "handoff_grace"].includes(progressDecision.decision)); + if (!deferredBoundaryConfirmed) { + const error = agentError( + "agent_convergence_stalled", + "A post-tool convergence preview deferred a required boundary but the next model request did not confirm progress or a bounded handoff.", + { + reason: "boundary_preview_unconfirmed", + deferredScopes: deferredBoundaryScopes, + observedDecision: progressDecision?.decision, + }, + "Inspect the preserved post-tool preview and PreModelRequest convergence report before retrying.", + ); + const result = this.createTurnResult(input, { + type: "error", + stopReason: "unsupported_recovery", + usage, + permissionDenials, + turns: turnCount, + startedAt, + finalMessage, + errors: [error], + }); + yield { type: "turn_failed", sessionId: input.sessionId, turnId: input.turnId, error }; + await captureTurn(true); + yield { type: "turn_completed", sessionId: input.sessionId, turnId: input.turnId, result }; + return { result, messages }; + } yield { type: "model_request_started", sessionId: input.sessionId, @@ -1584,6 +1629,7 @@ export class AgentLoop { lastToolFailureFingerprint, ); pairedResults = annotateRepeatedToolFailures(pairedResults, repeatedFailure.repeatedKeys); + pendingConvergencePreviews = convergencePreviewsFromToolResults(pairedResults); lastToolFailureFingerprint = repeatedFailure.currentFingerprint; const toolResultRepair = largeFileRepair.analyzeToolResults(pairedResults, { outputTruncated: assembled.finishReason === "length" || assembled.hasRepairedToolCalls === true, @@ -2638,6 +2684,32 @@ function findToolLifecycleBlock(results: PilotDeckToolResult[]): { reason: strin return undefined; } +function convergencePreviewsFromToolResults(results: PilotDeckToolResult[]): ConvergenceReport[] { + const candidates: Array<{ + report: ConvergenceReport; + completedAt: string; + resultIndex: number; + previewIndex: number; + }> = []; + for (const [resultIndex, result] of results.entries()) { + const lifecycle = result.metadata?.lifecycle; + if (!isRecord(lifecycle) || !Array.isArray(lifecycle.convergencePreviews)) continue; + for (const [previewIndex, value] of lifecycle.convergencePreviews.slice(0, 16).entries()) { + const report = parseConvergenceReport(value); + if (!report) continue; + candidates.push({ report, completedAt: result.completedAt, resultIndex, previewIndex }); + } + } + candidates.sort((left, right) => + left.completedAt.localeCompare(right.completedAt) + || left.resultIndex - right.resultIndex + || left.previewIndex - right.previewIndex + ); + const latestByScope = new Map(); + for (const candidate of candidates) latestByScope.set(candidate.report.scope, candidate.report); + return [...latestByScope.values()].sort((left, right) => left.scope.localeCompare(right.scope)); +} + function textFromMessage(message: CanonicalMessage): string { return message.content .filter((block) => block.type === "text") diff --git a/src/agent/protocol/events.ts b/src/agent/protocol/events.ts index 6a1b1c79..9fc58f59 100644 --- a/src/agent/protocol/events.ts +++ b/src/agent/protocol/events.ts @@ -50,6 +50,12 @@ export type AgentEvent = postRatio: number; } | { type: "context_budget"; sessionId: string; turnId: string; snapshot: TokenBudgetSnapshot } + | { + type: "progress_boundary_deferred"; + sessionId: string; + turnId: string; + scopes: string[]; + } | { type: "progress_lease_evaluated"; sessionId: string; diff --git a/src/extension/hooks/execution/HookRuntime.ts b/src/extension/hooks/execution/HookRuntime.ts index bcc2da86..a330d392 100644 --- a/src/extension/hooks/execution/HookRuntime.ts +++ b/src/extension/hooks/execution/HookRuntime.ts @@ -105,7 +105,7 @@ export class HookRuntime { }); } - effects.push(...effectsFromHookOutput(result.output, hookName)); + effects.push(...effectsFromHookOutput(result.output, hookName, input.event)); } return { effects, events, blockingErrors, nonBlockingErrors }; @@ -167,7 +167,11 @@ export class HookRuntime { } } -function effectsFromHookOutput(output: PilotDeckHookOutput, hookName: string): PilotDeckHookEffect[] { +function effectsFromHookOutput( + output: PilotDeckHookOutput, + hookName: string, + hookEvent: PilotDeckHookEvent, +): PilotDeckHookEffect[] { if (output.type === "async") { return []; } @@ -219,6 +223,9 @@ function effectsFromHookOutput(output: PilotDeckHookOutput, hookName: string): P if (specific.modelRequestPatch) { effects.push({ type: "model_request_patch", patch: specific.modelRequestPatch }); } + if (specific.convergencePreview && hookEvent === "PostToolUse") { + effects.push({ type: "convergence_preview", report: specific.convergencePreview }); + } if (specific.artifactContracts?.length) { effects.push({ type: "artifact_contracts", sourcePluginId: hookName, contracts: specific.artifactContracts }); } diff --git a/src/extension/hooks/execution/parseHookOutput.ts b/src/extension/hooks/execution/parseHookOutput.ts index 86d9215a..ea5eb678 100644 --- a/src/extension/hooks/execution/parseHookOutput.ts +++ b/src/extension/hooks/execution/parseHookOutput.ts @@ -63,11 +63,37 @@ function parseSpecificOutput(value: unknown): PilotDeckHookSpecificOutput | unde retry: booleanOrUndefined(record.retry), worktreePath: stringOrUndefined(record.worktreePath), modelRequestPatch: parseModelRequestPatch(record.modelRequestPatch), + convergencePreview: parseConvergencePreview(record.convergencePreview), artifactContracts: parseArtifactContracts(record.artifactContracts), dynamicContext: parseDynamicContext(record.dynamicContext), }; } +function parseConvergencePreview(value: unknown): PilotDeckHookSpecificOutput["convergencePreview"] { + if (!isRecord(value) || value.schemaVersion !== 1) return undefined; + if (!boundedString(value.scope, 128) || !boundedString(value.phase, 128)) return undefined; + if (!boundedString(value.stateHash, 256)) return undefined; + if (!nonNegativeInteger(value.remainingCount)) return undefined; + if (value.blockingCode !== undefined && !boundedString(value.blockingCode, 256)) return undefined; + for (const key of ["progressOrdinal", "repairOrdinal", "repairPreparationOrdinal", "handoffOrdinal"] as const) { + if (value[key] !== undefined && !nonNegativeInteger(value[key])) return undefined; + } + return { + schemaVersion: 1, + scope: value.scope, + phase: value.phase, + stateHash: value.stateHash, + ...(value.blockingCode !== undefined ? { blockingCode: value.blockingCode } : {}), + remainingCount: value.remainingCount, + ...(value.progressOrdinal !== undefined ? { progressOrdinal: value.progressOrdinal } : {}), + ...(value.repairOrdinal !== undefined ? { repairOrdinal: value.repairOrdinal } : {}), + ...(value.repairPreparationOrdinal !== undefined + ? { repairPreparationOrdinal: value.repairPreparationOrdinal } + : {}), + ...(value.handoffOrdinal !== undefined ? { handoffOrdinal: value.handoffOrdinal } : {}), + } as PilotDeckHookSpecificOutput["convergencePreview"]; +} + function parseDynamicContext(value: unknown): PilotDeckHookSpecificOutput["dynamicContext"] { if (!Array.isArray(value)) return undefined; const entries = value.slice(0, 64).flatMap((item) => { @@ -153,3 +179,11 @@ function stringOrUndefined(value: unknown): string | undefined { function booleanOrUndefined(value: unknown): boolean | undefined { return typeof value === "boolean" ? value : undefined; } + +function boundedString(value: unknown, maxLength: number): value is string { + return typeof value === "string" && value.length > 0 && value.length <= maxLength; +} + +function nonNegativeInteger(value: unknown): value is number { + return Number.isSafeInteger(value) && (value as number) >= 0; +} diff --git a/src/extension/hooks/protocol/output.ts b/src/extension/hooks/protocol/output.ts index 4075e34c..b22664aa 100644 --- a/src/extension/hooks/protocol/output.ts +++ b/src/extension/hooks/protocol/output.ts @@ -10,6 +10,19 @@ export type PilotDeckPermissionHookDecision = interrupt?: boolean; }; +export type PilotDeckConvergencePreview = { + schemaVersion: 1; + scope: string; + phase: string; + stateHash: string; + blockingCode?: string; + remainingCount: number; + progressOrdinal?: number; + repairOrdinal?: number; + repairPreparationOrdinal?: number; + handoffOrdinal?: number; +}; + export type PilotDeckHookSpecificOutput = { hookEventName: string; additionalContext?: string; @@ -30,6 +43,8 @@ export type PilotDeckHookSpecificOutput = { temperature?: number; metadata?: Record; }; + /** Bounded post-tool preview; Core must confirm it on PreModelRequest. */ + convergencePreview?: PilotDeckConvergencePreview; artifactContracts?: Array<{ id: string; path: string; diff --git a/src/gateway/client/InProcessGateway.ts b/src/gateway/client/InProcessGateway.ts index 6fdd8772..f9890a75 100644 --- a/src/gateway/client/InProcessGateway.ts +++ b/src/gateway/client/InProcessGateway.ts @@ -1659,6 +1659,12 @@ function mapAgentEventForTurn(event: AgentEvent, runId: string): GatewayEvent[] reason: event.reason, }, }]; + case "progress_boundary_deferred": + return [{ + type: "agent_status", + event: "progress_boundary_deferred", + detail: { scopes: event.scopes }, + }]; case "warning": return [{ type: "agent_status", diff --git a/src/lifecycle/protocol/effects.ts b/src/lifecycle/protocol/effects.ts index adc7b43a..3b58f559 100644 --- a/src/lifecycle/protocol/effects.ts +++ b/src/lifecycle/protocol/effects.ts @@ -9,6 +9,7 @@ export type PilotDeckModelRequestPatch = { }; import type { ArtifactContract } from "../../artifact/index.js"; +import type { PilotDeckConvergencePreview } from "../../extension/hooks/protocol/output.js"; export type PilotDeckPermissionRequestResult = | { @@ -33,6 +34,7 @@ export type PilotDeckHookEffect = } | { type: "system_message"; content: string } | { type: "model_request_patch"; patch: PilotDeckModelRequestPatch } + | { type: "convergence_preview"; report: PilotDeckConvergencePreview } | { type: "artifact_contracts"; sourcePluginId: string; contracts: readonly ArtifactContract[] } | { type: "block"; reason: string; stopReason?: string } | { type: "permission_decision"; behavior: PilotDeckHookPermissionBehavior; reason?: string } diff --git a/src/observability/agentEventBridge.ts b/src/observability/agentEventBridge.ts index 09cea66e..0ead06e7 100644 --- a/src/observability/agentEventBridge.ts +++ b/src/observability/agentEventBridge.ts @@ -135,6 +135,21 @@ export function observeAgentEvent(recorder: ObservationRecorder | undefined, eve priority: "important", }); return; + case "progress_boundary_deferred": + recorder.emit({ + ...base, + type: "harness.decision", + payload: { + component: "progress-boundary", + policyVersion: "progress-boundary/v1", + decision: "deferred", + reasonCode: "post_tool_convergence_preview", + observed: { scopes: event.scopes }, + forceBoundaryNext: false, + }, + priority: "important", + }); + return; case "subagent_started": recorder.emit({ ...base, diff --git a/src/tool/execution/ToolRuntime.ts b/src/tool/execution/ToolRuntime.ts index 169b29fa..ed80456e 100644 --- a/src/tool/execution/ToolRuntime.ts +++ b/src/tool/execution/ToolRuntime.ts @@ -550,14 +550,20 @@ function lifecycleMetadata(result: { effects: PilotDeckHookEffect[] }): Record effect.type === "block"); const additionalContext = result.effects.filter((effect) => effect.type === "additional_context"); const updatedMcpOutput = result.effects.find((effect) => effect.type === "updated_mcp_tool_output"); - if (!blocking && additionalContext.length === 0 && !updatedMcpOutput) { + const convergencePreviews = result.effects + .filter((effect) => effect.type === "convergence_preview") + .map((effect) => effect.report); + if (!blocking && additionalContext.length === 0 && !updatedMcpOutput && convergencePreviews.length === 0) { return undefined; } return { lifecycle: { - blocked: blocking ? { reason: blocking.reason, stopReason: blocking.stopReason } : undefined, - additionalContext: additionalContext.map((effect) => effect.content), - updatedMcpToolOutput: updatedMcpOutput?.output, + ...(blocking ? { blocked: { reason: blocking.reason, stopReason: blocking.stopReason } } : {}), + ...(additionalContext.length > 0 + ? { additionalContext: additionalContext.map((effect) => effect.content) } + : {}), + ...(updatedMcpOutput ? { updatedMcpToolOutput: updatedMcpOutput.output } : {}), + ...(convergencePreviews.length > 0 ? { convergencePreviews } : {}), }, }; } diff --git a/tests/agent/agent-loop-runtime-controls.spec.ts b/tests/agent/agent-loop-runtime-controls.spec.ts index 63092c52..35c6f317 100644 --- a/tests/agent/agent-loop-runtime-controls.spec.ts +++ b/tests/agent/agent-loop-runtime-controls.spec.ts @@ -523,11 +523,13 @@ test("AgentLoop permits one prepared-target request and then requires genuine pr test("AgentLoop carries a bounded handoff through the required boundary to genuine progress", async () => { const requests: CanonicalModelRequest[] = []; const defaultContext = new DefaultContextRuntime(); + let forcedFullCompactions = 0; const context: AgentContextRuntime = { prepareForModel: (input) => defaultContext.prepareForModel(input), commitPreparedContext: (input) => defaultContext.commitPreparedContext(input), async tryAutoCompact(input) { if (!input.forceFull) return { type: "skipped", snapshot: budgetSnapshot(10_000) }; + forcedFullCompactions += 1; return { type: "compacted", messages: input.messages, @@ -575,6 +577,7 @@ test("AgentLoop carries a bounded handoff through the required boundary to genui }, }; let preModelCalls = 0; + let toolBatches = 0; const dependencies = createDependencies(requests, { context, router, @@ -618,6 +621,13 @@ test("AgentLoop carries a bounded handoff through the required boundary to genui registry: new ToolRegistry(), scheduler: { async executeAll(calls) { + toolBatches += 1; + const nextReport = [ + { progressOrdinal: 8, handoffOrdinal: 0, stateHash: "first-matrix" }, + { progressOrdinal: 8, handoffOrdinal: 1, stateHash: "apply-ready" }, + { progressOrdinal: 8, handoffOrdinal: 2, stateHash: "next-page" }, + { progressOrdinal: 9, handoffOrdinal: 2, stateHash: "finalized" }, + ][toolBatches - 1]; return calls.map((call) => ({ type: "success" as const, toolCallId: call.id, @@ -625,6 +635,20 @@ test("AgentLoop carries a bounded handoff through the required boundary to genui content: [{ type: "text" as const, text: "synthetic handoff completed" }], startedAt: "2026-07-27T00:00:00.000Z", completedAt: "2026-07-27T00:00:00.000Z", + ...(nextReport ? { + metadata: { + lifecycle: { + convergencePreviews: [{ + schemaVersion: 1, + scope: "synthetic-validation", + phase: "coverage", + blockingCode: "missing_rows", + remainingCount: 4, + ...nextReport, + }], + }, + }, + } : {}), })); }, }, @@ -640,7 +664,7 @@ test("AgentLoop carries a bounded handoff through the required boundary to genui }, }), dependencies); - const events: Array<{ type?: string; decision?: string; handoffOrdinal?: number }> = []; + const events: Array<{ type?: string; decision?: string; handoffOrdinal?: number; scopes?: string[] }> = []; const iterator = loop.run({ sessionId: "session-bounded-handoff", turnId: "turn-bounded-handoff", @@ -653,11 +677,12 @@ test("AgentLoop carries a bounded handoff through the required boundary to genui completed = next.value; break; } - events.push(next.value as { type?: string; decision?: string; handoffOrdinal?: number }); + events.push(next.value as { type?: string; decision?: string; handoffOrdinal?: number; scopes?: string[] }); } assert.equal(completed.result.type, "success"); assert.equal(requests.length, 5); + assert.equal(forcedFullCompactions, 0); assert.match(messageText(requests[2]?.messages ?? []), /state-bound apply command/u); assert.match(messageText(requests[3]?.messages ?? []), /next bounded evidence page/u); assert.deepEqual( @@ -667,10 +692,159 @@ test("AgentLoop carries a bounded handoff through the required boundary to genui ["baseline", 0], ["renewed", 0], ["handoff_grace", 1], - ["boundary_grace", 2], + ["handoff_grace", 2], ["renewed", 2], ], ); + assert.deepEqual( + events.filter((event) => event.type === "progress_boundary_deferred").map((event) => event.scopes), + [["synthetic-validation"], ["synthetic-validation"]], + ); +}); + +test("AgentLoop fails closed before the model when a deferred preview is not confirmed", async () => { + const requests: CanonicalModelRequest[] = []; + const defaultContext = new DefaultContextRuntime(); + let forcedFullCompactions = 0; + const context: AgentContextRuntime = { + prepareForModel: (input) => defaultContext.prepareForModel(input), + commitPreparedContext: (input) => defaultContext.commitPreparedContext(input), + async tryAutoCompact(input) { + if (!input.forceFull) return { type: "skipped", snapshot: budgetSnapshot(10_000) }; + forcedFullCompactions += 1; + return { + type: "compacted", + messages: input.messages, + tier: "full", + snapshot: budgetSnapshot(5_000), + }; + }, + }; + const router = { + async decide(input: { request: CanonicalModelRequest }): Promise { + return { + provider: input.request.provider, + model: input.request.model, + scenarioType: "default" as const, + isSubagent: false, + orchestrating: false, + resolvedFrom: "scenario" as const, + mutations: {}, + }; + }, + async *execute(_decision: RouterDecision, request: CanonicalModelRequest): AsyncIterable { + requests.push(request); + const toolCall = { id: `stale-preview-${requests.length}`, name: "noop", input: {} }; + yield { type: "message_start", role: "assistant" }; + yield { type: "tool_call_start", id: toolCall.id, name: toolCall.name }; + yield { type: "tool_call_end", toolCall }; + yield { type: "message_end", finishReason: "tool_call" }; + }, + async *stream(): AsyncIterable { + throw new Error("stream fallback should not be used"); + }, + }; + let preModelCalls = 0; + let toolBatches = 0; + const dependencies = createDependencies(requests, { + context, + router, + lifecycle: { + async dispatch(input: { event: string }) { + if (input.event !== "PreModelRequest") return emptyLifecycleResult(); + preModelCalls += 1; + return { + ...emptyLifecycleResult(), + effects: [{ + type: "model_request_patch" as const, + patch: { + metadata: { + pilotdeckConvergence: { + schemaVersion: 1, + scope: "synthetic-validation", + phase: "coverage", + stateHash: "unchanged", + blockingCode: "missing_rows", + remainingCount: 4, + progressOrdinal: 8, + handoffOrdinal: 0, + }, + }, + }, + }], + }; + }, + } as never, + tools: { + registry: new ToolRegistry(), + scheduler: { + async executeAll(calls) { + toolBatches += 1; + return calls.map((call) => ({ + type: "success" as const, + toolCallId: call.id, + toolName: call.name, + content: [{ type: "text" as const, text: "synthetic state unchanged" }], + startedAt: "2026-07-27T00:00:00.000Z", + completedAt: "2026-07-27T00:00:00.000Z", + ...(toolBatches === 2 ? { + metadata: { + lifecycle: { + convergencePreviews: [{ + schemaVersion: 1, + scope: "synthetic-validation", + phase: "coverage", + stateHash: "claimed-handoff", + blockingCode: "missing_rows", + remainingCount: 4, + progressOrdinal: 8, + handoffOrdinal: 1, + }], + }, + }, + } : {}), + })); + }, + }, + }, + }); + const loop = new AgentLoop(createConfig(process.cwd(), { + maxContextTokens: 10_000, + progressLease: { + enabled: true, + mode: "evaluation", + maxStagnantObservations: 2, + maxInitialStagnantObservations: 2, + }, + }), dependencies); + + const events: Array<{ type?: string }> = []; + const iterator = loop.run({ + sessionId: "session-stale-preview", + turnId: "turn-stale-preview", + messages: [userMessage("complete the synthetic matrix")], + }); + let completed: AgentLoopRunResult | undefined; + while (true) { + const next = await iterator.next(); + if (next.done) { + completed = next.value; + break; + } + events.push(next.value as { type?: string }); + } + + assert.equal(completed.result.type, "error"); + assert.equal(completed.result.errors?.[0]?.code, "agent_convergence_stalled"); + assert.equal( + completed.result.errors?.[0]?.details + && (completed.result.errors[0].details as { reason?: string }).reason, + "boundary_preview_unconfirmed", + ); + assert.equal(requests.length, 2); + assert.equal(preModelCalls, 3); + assert.equal(forcedFullCompactions, 0); + assert.equal(events.filter((event) => event.type === "progress_boundary_deferred").length, 1); }); test("artifact failure injects one bounded correction turn and succeeds after validation", async () => { diff --git a/tests/agent/progress-lease.spec.ts b/tests/agent/progress-lease.spec.ts index f6de9cc1..8d877ff2 100644 --- a/tests/agent/progress-lease.spec.ts +++ b/tests/agent/progress-lease.spec.ts @@ -60,6 +60,62 @@ test("two stagnant observations require a boundary and allow exactly one post-bo assert.equal(failed?.reason, "post_boundary_stagnation"); }); +test("a post-tool progress or bounded handoff preview defers but does not consume a required boundary", () => { + const progressLease = configuredLease(); + progressLease.observe(report({ progressOrdinal: 8, handoffOrdinal: 0 }), none); + progressLease.observe(report({ progressOrdinal: 8, handoffOrdinal: 0 }), none); + + assert.deepEqual(progressLease.planBoundary([ + report({ progressOrdinal: 9, handoffOrdinal: 0, remainingCount: 3 }), + ]), { + requested: false, + deferredScopes: ["domain-validation"], + }); + assert.deepEqual(progressLease.planBoundary([ + report({ progressOrdinal: 8, handoffOrdinal: 1 }), + ]), { + requested: false, + deferredScopes: ["domain-validation"], + }); + assert.equal( + progressLease.observe(report({ progressOrdinal: 8, handoffOrdinal: 1 }), none)?.decision, + "handoff_grace", + ); +}); + +test("replayed, repair-only, and over-budget previews cannot defer a required boundary", () => { + const replayLease = configuredLease(); + replayLease.observe(report({ progressOrdinal: 8, repairOrdinal: 0, handoffOrdinal: 1 }), none); + replayLease.observe(report({ progressOrdinal: 8, repairOrdinal: 0, handoffOrdinal: 1 }), none); + assert.deepEqual(replayLease.planBoundary([ + report({ progressOrdinal: 8, repairOrdinal: 0, handoffOrdinal: 1 }), + ]), { requested: true, deferredScopes: [] }); + assert.deepEqual(replayLease.planBoundary([ + report({ progressOrdinal: 8, repairOrdinal: 1, handoffOrdinal: 1 }), + ]), { requested: true, deferredScopes: [] }); + + const budgetLease = configuredLease(); + budgetLease.observe(report({ progressOrdinal: 8, handoffOrdinal: 0 }), none); + budgetLease.observe(report({ progressOrdinal: 8, handoffOrdinal: 1 }), none); + budgetLease.observe(report({ progressOrdinal: 8, handoffOrdinal: 2 }), none); + assert.deepEqual(budgetLease.planBoundary([ + report({ progressOrdinal: 8, handoffOrdinal: 3 }), + ]), { requested: true, deferredScopes: [] }); +}); + +test("previews cannot defer when more than one scope requires a boundary", () => { + const lease = configuredLease(); + for (const scope of ["domain-a", "domain-b"]) { + lease.observe(report({ scope, progressOrdinal: 1 }), none); + lease.observe(report({ scope, progressOrdinal: 1 }), none); + } + + assert.deepEqual(lease.planBoundary([ + report({ scope: "domain-a", progressOrdinal: 2 }), + report({ scope: "domain-b", progressOrdinal: 2 }), + ]), { requested: true, deferredScopes: [] }); +}); + test("new repair feedback after a boundary gets one delivery turn without renewing progress", () => { const lease = configuredLease(); lease.observe(report({ progressOrdinal: 1, repairOrdinal: 0 }), none); diff --git a/tests/extension/hooks/model-request-patch.spec.ts b/tests/extension/hooks/model-request-patch.spec.ts index 3bdc6b11..d519aeab 100644 --- a/tests/extension/hooks/model-request-patch.spec.ts +++ b/tests/extension/hooks/model-request-patch.spec.ts @@ -1,6 +1,7 @@ import test from "node:test"; import assert from "node:assert/strict"; import { parseHookOutput } from "../../../src/extension/hooks/execution/parseHookOutput.js"; +import { HookRuntime } from "../../../src/extension/hooks/execution/HookRuntime.js"; test("parses only the supported model request patch fields", () => { const output = parseHookOutput(JSON.stringify({ @@ -47,3 +48,90 @@ test("parses bounded dynamic context controls for hook-driven injection", () => { id: "invalid", content: "ignored priority", priority: undefined, ttlMs: 86_400_000 }, ]); }); + +test("parses only bounded convergence preview fields", () => { + const output = parseHookOutput(JSON.stringify({ + hookSpecificOutput: { + hookEventName: "PostToolUse", + convergencePreview: { + schemaVersion: 1, + scope: "synthetic-validation", + phase: "matrices", + stateHash: "state-b", + blockingCode: "matrix_pending", + remainingCount: 4, + progressOrdinal: 7, + repairOrdinal: 1, + repairPreparationOrdinal: 1, + handoffOrdinal: 3, + nextBatch: { privateDomainPayload: true }, + }, + }, + })); + + assert.equal(output.type, "sync"); + if (output.type !== "sync") return; + assert.deepEqual(output.specific?.convergencePreview, { + schemaVersion: 1, + scope: "synthetic-validation", + phase: "matrices", + stateHash: "state-b", + blockingCode: "matrix_pending", + remainingCount: 4, + progressOrdinal: 7, + repairOrdinal: 1, + repairPreparationOrdinal: 1, + handoffOrdinal: 3, + }); +}); + +test("HookRuntime accepts convergence previews only from PostToolUse", async () => { + const convergencePreview = { + schemaVersion: 1 as const, + scope: "synthetic-validation", + phase: "matrices", + stateHash: "state-b", + blockingCode: "matrix_pending", + remainingCount: 4, + progressOrdinal: 7, + handoffOrdinal: 3, + }; + const runtime = new HookRuntime({ + PreModelRequest: [{ hooks: [{ type: "command", command: "ignored" }] }], + PostToolUse: [{ hooks: [{ type: "command", command: "ignored" }] }], + }, { + async execute(options: { hookInput: { hookEventName: string } }) { + return { + stdout: "", + stderr: "", + exitCode: 0, + outcome: "success" as const, + output: { + type: "sync" as const, + specific: { + hookEventName: options.hookInput.hookEventName, + convergencePreview, + }, + }, + }; + }, + } as never); + const baseInput = { sessionId: "session-1", transcriptPath: "", cwd: process.cwd() }; + + const preModel = await runtime.run({ + event: "PreModelRequest", + hookInput: { ...baseInput, hookEventName: "PreModelRequest" }, + cwd: process.cwd(), + }); + const postTool = await runtime.run({ + event: "PostToolUse", + hookInput: { ...baseInput, hookEventName: "PostToolUse", toolName: "write_file" }, + cwd: process.cwd(), + }); + + assert.equal(preModel.effects.some((effect) => effect.type === "convergence_preview"), false); + assert.deepEqual( + postTool.effects.filter((effect) => effect.type === "convergence_preview"), + [{ type: "convergence_preview", report: convergencePreview }], + ); +}); diff --git a/tests/gateway/map-agent-event-runid.spec.ts b/tests/gateway/map-agent-event-runid.spec.ts index 98111410..26a24883 100644 --- a/tests/gateway/map-agent-event-runid.spec.ts +++ b/tests/gateway/map-agent-event-runid.spec.ts @@ -73,3 +73,17 @@ test("mapAgentEvent projects every convergence ordinal", () => { reason: undefined, }); }); + +test("mapAgentEvent exposes a post-tool boundary deferral without domain payload", () => { + const [status] = mapAgentEvent({ + type: "progress_boundary_deferred", + sessionId: "session-1", + turnId: "turn-1", + scopes: ["domain-validation"], + }, "run-1"); + + assert.equal(status?.type, "agent_status"); + if (status?.type !== "agent_status") return; + assert.equal(status.event, "progress_boundary_deferred"); + assert.deepEqual(status.detail, { scopes: ["domain-validation"] }); +}); diff --git a/tests/observability/local-gateway-progress-handoff.spec.ts b/tests/observability/local-gateway-progress-handoff.spec.ts index 17efde2c..1b8d6903 100644 --- a/tests/observability/local-gateway-progress-handoff.spec.ts +++ b/tests/observability/local-gateway-progress-handoff.spec.ts @@ -17,6 +17,7 @@ test("local Gateway records a bounded handoff through boundary to progress", asy const pilotHome = join(root, "home"); const pluginRoot = join(projectRoot, ".pilotdeck", "plugins", "handoff-qa"); const requests: CanonicalModelRequest[] = []; + const modelMetrics = { compactions: 0 }; await mkdir(join(pluginRoot, "hooks"), { recursive: true }); await mkdir(pilotHome, { recursive: true }); await writeFile(join(pilotHome, "pilotdeck.yaml"), TEST_CONFIG); @@ -27,6 +28,7 @@ test("local Gateway records a bounded handoff through boundary to progress", asy })); await writeFile(join(pluginRoot, "hooks", "hooks.json"), JSON.stringify({ PreModelRequest: [{ hooks: [{ type: "command", command: "node hook.mjs" }] }], + PostToolUse: [{ matcher: "bash", hooks: [{ type: "command", command: "node hook.mjs" }] }], })); await writeFile(join(pluginRoot, "hook.mjs"), HOOK_SCRIPT); @@ -35,7 +37,7 @@ test("local Gateway records a bounded handoff through boundary to progress", asy fallbackProjectRoot: projectRoot, pilotHome, env: { ...process.env, PILOT_HOME: pilotHome, PILOTDECK_BUILD_SHA: "test-build" }, - __testModelFactory: () => handoffModelRuntime(requests), + __testModelFactory: () => handoffModelRuntime(requests, modelMetrics), }); try { const gatewayEvents = []; @@ -59,9 +61,17 @@ test("local Gateway records a bounded handoff through boundary to progress", asy ["baseline", 0], ["renewed", 0], ["handoff_grace", 1], - ["boundary_grace", 2], + ["handoff_grace", 2], ["renewed", 2], ]); + assert.equal(modelMetrics.compactions, 0); + assert.deepEqual( + gatewayEvents.flatMap((event) => event.type === "agent_status" + && event.event === "progress_boundary_deferred" + ? [event.detail?.scopes] + : []), + [["handoff-qa"], ["handoff-qa"]], + ); assert.equal( gatewayEvents.some((event) => event.type === "turn_completed" && event.finishReason === "completed"), true, @@ -81,9 +91,22 @@ test("local Gateway records a bounded handoff through boundary to progress", asy event.payload.decision, (event.payload.observed as { handoffOrdinal?: number } | undefined)?.handoffOrdinal, ]); + const decisionSequence = observations + .filter((event) => event.type === "harness.decision" + && ["progress-boundary", "progress-lease"].includes(String(event.payload.component))) + .map((event) => [event.payload.component, event.payload.decision]); const integrity = JSON.parse(await readFile(join(storage.observabilityDir, "integrity.json"), "utf8")); assert.deepEqual(decisions, gatewayDecisions); + assert.deepEqual(decisionSequence, [ + ["progress-lease", "baseline"], + ["progress-lease", "renewed"], + ["progress-lease", "handoff_grace"], + ["progress-boundary", "deferred"], + ["progress-lease", "handoff_grace"], + ["progress-boundary", "deferred"], + ["progress-lease", "renewed"], + ]); assert.equal(integrity.status, "complete"); assert.equal(integrity.checks.modelRequestsPaired, true); assert.equal(integrity.checks.toolCallsPaired, true); @@ -96,7 +119,10 @@ test("local Gateway records a bounded handoff through boundary to progress", asy } }); -function handoffModelRuntime(requests: CanonicalModelRequest[]): ModelRuntime { +function handoffModelRuntime( + requests: CanonicalModelRequest[], + metrics: { compactions: number }, +): ModelRuntime { return { async *stream(request) { if (isCompactionRequest(request)) { @@ -125,7 +151,8 @@ function handoffModelRuntime(requests: CanonicalModelRequest[]): ModelRuntime { yield { type: "usage", usage: { inputTokens: 10, outputTokens: 5, totalTokens: 15 } }; yield { type: "message_end", finishReason: "stop" }; }, - async complete() { + async complete(request) { + if (isCompactionRequest(request)) metrics.compactions += 1; return { role: "assistant", content: [{ type: "text", text: "Bounded context summary." }], @@ -186,9 +213,6 @@ const input = JSON.parse(body); const counterPath = join(input.cwd, ".pilotdeck", "handoff-qa-counter"); let count = 0; try { count = Number(await readFile(counterPath, "utf8")) || 0; } catch {} -count += 1; -await mkdir(dirname(counterPath), { recursive: true }); -await writeFile(counterPath, String(count)); const reports = [ { progressOrdinal: 7, handoffOrdinal: 0, stateHash: "prior" }, { progressOrdinal: 8, handoffOrdinal: 0, stateHash: "first-matrix" }, @@ -196,16 +220,34 @@ const reports = [ { progressOrdinal: 8, handoffOrdinal: 2, stateHash: "next-page" }, { progressOrdinal: 9, handoffOrdinal: 2, stateHash: "finalized" } ]; -const report = reports[Math.min(count - 1, reports.length - 1)]; -console.log(JSON.stringify({ hookSpecificOutput: { - hookEventName: input.hookEventName, - modelRequestPatch: { metadata: { pilotdeckConvergence: { - schemaVersion: 1, - scope: "handoff-qa", - phase: "coverage", - blockingCode: "missing_rows", - remainingCount: 4, - ...report - } } } -} })); +if (input.hookEventName === "PostToolUse") { + const report = reports[Math.min(count, reports.length - 1)]; + console.log(JSON.stringify({ hookSpecificOutput: { + hookEventName: input.hookEventName, + convergencePreview: { + schemaVersion: 1, + scope: "handoff-qa", + phase: "coverage", + blockingCode: "missing_rows", + remainingCount: 4, + ...report + } + } })); +} else { + count += 1; + await mkdir(dirname(counterPath), { recursive: true }); + await writeFile(counterPath, String(count)); + const report = reports[Math.min(count - 1, reports.length - 1)]; + console.log(JSON.stringify({ hookSpecificOutput: { + hookEventName: input.hookEventName, + modelRequestPatch: { metadata: { pilotdeckConvergence: { + schemaVersion: 1, + scope: "handoff-qa", + phase: "coverage", + blockingCode: "missing_rows", + remainingCount: 4, + ...report + } } } + } })); +} `; diff --git a/tests/observability/recorder.spec.ts b/tests/observability/recorder.spec.ts index 4ac91c17..d2a27602 100644 --- a/tests/observability/recorder.spec.ts +++ b/tests/observability/recorder.spec.ts @@ -154,6 +154,31 @@ test("repair preparation grace is observable without becoming progress", () => { }); }); +test("post-tool boundary deferral is independently observable and domain-bounded", () => { + const drafts: ObservationEventDraft[] = []; + const recorder = { + emit: (draft: ObservationEventDraft) => { + drafts.push(draft); + return undefined; + }, + } as unknown as ObservationRecorder; + + observeAgentEvent(recorder, { + type: "progress_boundary_deferred", + sessionId: "session-1", + turnId: "turn-1", + scopes: ["legal-coverage"], + }); + + assert.equal(drafts.length, 1); + assert.equal(drafts[0]?.type, "harness.decision"); + assert.equal(drafts[0]?.payload?.component, "progress-boundary"); + assert.equal(drafts[0]?.payload?.policyVersion, "progress-boundary/v1"); + assert.equal(drafts[0]?.payload?.decision, "deferred"); + assert.deepEqual(drafts[0]?.payload?.observed, { scopes: ["legal-coverage"] }); + assert.equal(JSON.stringify(drafts).includes("nextBatch"), false); +}); + test("recorder finalizes a complete hash-only trajectory", async () => { const root = await mkdtemp(join(tmpdir(), "pilotdeck-observation-recorder-")); try { diff --git a/tests/products/legal-coverage.spec.ts b/tests/products/legal-coverage.spec.ts index 7e701a9a..09206295 100644 --- a/tests/products/legal-coverage.spec.ts +++ b/tests/products/legal-coverage.spec.ts @@ -3753,6 +3753,46 @@ test("legal coverage risk-signal issue closure exposes one bounded state-bound t const valid = issueClosureProposal(envelope.workItems.proposal.template); await writeJson(join(workspace, envelope.workItems.proposal.path), valid); + const postWrite = await runHook({ + hookEventName: "PostToolUse", + sessionId: "issue-closure", + transcriptPath: "", + cwd: workspace, + toolName: "write_file", + toolInput: { file_path: envelope.workItems.proposal.path }, + }); + assert.equal(postWrite.hookSpecificOutput.modelRequestPatch, undefined); + assert.equal( + postWrite.hookSpecificOutput.convergencePreview?.handoffOrdinal, + initialConvergence.handoffOrdinal + 1, + ); + const unrelatedWrite = await runHook({ + hookEventName: "PostToolUse", + sessionId: "issue-closure", + transcriptPath: "", + cwd: workspace, + toolName: "write_file", + toolInput: { file_path: "notes.md" }, + }); + assert.equal(unrelatedWrite.hookSpecificOutput.convergencePreview, undefined); + const proposalRead = await runHook({ + hookEventName: "PostToolUse", + sessionId: "issue-closure", + transcriptPath: "", + cwd: workspace, + toolName: "read_file", + toolInput: { file_path: envelope.workItems.proposal.path }, + }); + assert.equal(proposalRead.hookSpecificOutput.convergencePreview, undefined); + const readOnlyCli = await runHook({ + hookEventName: "PostToolUse", + sessionId: "issue-closure", + transcriptPath: "", + cwd: workspace, + toolName: "bash", + toolInput: { command: `node ${JSON.stringify(CLI)} status --workspace .` }, + }); + assert.equal(readOnlyCli.hookSpecificOutput.convergencePreview, undefined); const validHook = await runHook({ hookEventName: "PreModelRequest", sessionId: "issue-closure", @@ -4063,7 +4103,7 @@ test("legal product plugin loads one skill and contains no benchmark-specific co assert.equal(plugin.skills?.[0]?.name, "legal-coverage:conduct-legal-due-diligence"); assert.equal(plugin.hooksConfig?.PreModelRequest?.length, 1); assert.equal(plugin.hooksConfig?.PostToolUse?.length, 1); - assert.equal(plugin.hooksConfig?.PostToolUse?.[0]?.matcher, "read_file|write_file"); + assert.equal(plugin.hooksConfig?.PostToolUse?.[0]?.matcher, "bash|read_file|write_file"); assert.equal(plugin.hooksConfig?.PostCompact?.length, 1); const files = await collectFiles(PLUGIN_ROOT); @@ -4501,6 +4541,10 @@ async function runHook(input: Record): Promise<{ dynamicContext?: unknown[]; artifactContracts?: Array<{ path: string }>; modelRequestPatch?: { metadata?: Record }; + convergencePreview?: { + progressOrdinal?: number; + handoffOrdinal?: number; + }; }; }> { const result = await runHookProcess(input); diff --git a/tests/tool/bash-result-display.spec.ts b/tests/tool/bash-result-display.spec.ts index ba70b0e0..cce6f0f8 100644 --- a/tests/tool/bash-result-display.spec.ts +++ b/tests/tool/bash-result-display.spec.ts @@ -97,6 +97,64 @@ test("bash allows hyphenated commands whose names merely start with next", async ]); }); +test("PostToolUse carries bounded convergence previews only in internal lifecycle metadata", async () => { + const registry = new ToolRegistry(); + registry.register(createBashTool({ + runner: { + async run() { + return { exitCode: 0, stdout: "applied\n", stderr: "", timedOut: false, durationMs: 2 }; + }, + }, + })); + const runtime = new ToolRuntime(registry, new PermissionRuntime(), { + async dispatch(input: { event: string }) { + if (input.event !== "PostToolUse") { + return { effects: [], messages: [], events: [], blockingErrors: [], nonBlockingErrors: [] }; + } + return { + effects: [{ + type: "convergence_preview" as const, + report: { + schemaVersion: 1, + scope: "synthetic-validation", + phase: "matrices", + stateHash: "apply-ready", + blockingCode: "matrix_pending", + remainingCount: 4, + progressOrdinal: 8, + handoffOrdinal: 1, + }, + }], + messages: [], + events: [], + blockingErrors: [], + nonBlockingErrors: [], + }; + }, + } as never); + + const result = await runtime.execute( + { id: "call-preview", name: "bash", input: { command: "true" } }, + context(), + ); + + assert.equal(result.type, "success"); + assert.deepEqual(result.metadata?.lifecycle, { + convergencePreviews: [{ + schemaVersion: 1, + scope: "synthetic-validation", + phase: "matrices", + stateHash: "apply-ready", + blockingCode: "matrix_pending", + remainingCount: 4, + progressOrdinal: 8, + handoffOrdinal: 1, + }], + }); + const text = result.content[0]?.type === "text" ? result.content[0].text : ""; + assert.doesNotMatch(text, /synthetic-validation|apply-ready/u); +}); + test("bash still rejects exact framework CLI tokens that start long-lived processes", async () => { const commands: string[] = []; const runtime = createRuntime({ From 2313f01fd6de22b2ca35fb04879288745a101b65 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E9=83=91=E8=BE=BE=E5=A5=87?= <> Date: Mon, 27 Jul 2026 16:01:15 +0800 Subject: [PATCH 2/2] test(agent): record V24R15 QA evidence --- .../QA.md | 89 + .../diff-check.log | 0 .../focused-gate.log | 633 ++++++ .../full-suite.log | 2001 +++++++++++++++++ 4 files changed, 2723 insertions(+) create mode 100644 .omo/evidence/20260727-convergence-v24r15-post-tool-preview/QA.md create mode 100644 .omo/evidence/20260727-convergence-v24r15-post-tool-preview/diff-check.log create mode 100644 .omo/evidence/20260727-convergence-v24r15-post-tool-preview/focused-gate.log create mode 100644 .omo/evidence/20260727-convergence-v24r15-post-tool-preview/full-suite.log diff --git a/.omo/evidence/20260727-convergence-v24r15-post-tool-preview/QA.md b/.omo/evidence/20260727-convergence-v24r15-post-tool-preview/QA.md new file mode 100644 index 00000000..674406a3 --- /dev/null +++ b/.omo/evidence/20260727-convergence-v24r15-post-tool-preview/QA.md @@ -0,0 +1,89 @@ +# V24R15 post-tool convergence preview QA + +## What was tested + +### Focused cross-layer gate + +Command: + +```bash +npm run build +node --test --test-force-exit --test-timeout 120000 \ + dist/tests/extension/hooks/model-request-patch.spec.js \ + dist/tests/agent/progress-lease.spec.js \ + dist/tests/tool/bash-result-display.spec.js \ + dist/tests/agent/agent-loop-runtime-controls.spec.js \ + dist/tests/gateway/map-agent-event-runid.spec.js \ + dist/tests/observability/recorder.spec.js \ + dist/tests/observability/local-gateway-progress-handoff.spec.js \ + dist/tests/products/legal-coverage.spec.js +``` + +Artifact: `focused-gate.log` + +This drove the bounded parser, HookRuntime event restriction, ToolRuntime +internal transport, Progress Lease planner, AgentLoop ordering, Gateway event +projection, O1 recorder, a real local Gateway, and the unchanged Legal Coverage +validator/transaction surface. + +### Full repository regression + +Command: + +```bash +npm test +``` + +Artifact: `full-suite.log` + +### Patch hygiene + +Command: + +```bash +git diff --check +``` + +Artifact: `diff-check.log` + +## What was observed + +- Focused gate: 103 tests passed, 0 failed. +- Full repository suite: 325 tests passed, 0 failed. +- TypeScript build completed in both gates. +- The synthetic AgentLoop handoff trajectory reached genuine progress with two + `progress_boundary_deferred` events and zero forced full compactions. +- A stale preview stopped with `boundary_preview_unconfirmed` before a third + model request was sent and without running a forced full compaction. +- Replay, repair-only, exhausted-budget, and multiple-forcing-scope previews + did not defer the required boundary. +- The real local Gateway emitted the ordered O1 sequence + `progress-boundary/deferred` then the authoritative + `progress-lease/v5` confirmation for both deferred boundaries. +- The real Gateway trajectory finalized with complete model/tool/turn pairing, + zero recorder drops, and no fake API key in the observation log. +- Legal Coverage produced previews for legal-state writes, but not for reads, + unrelated writes, or read-only legal CLI commands. +- `git diff --check` produced no findings. + +## Why this is enough for the code gate + +The tests cover both halves of the protocol. Positive tests prove the original +ordering defect is removed in AgentLoop and in a real Gateway runtime. Negative +tests prove the preview remains advisory: it cannot carry domain payload, +consume lease state, bypass the handoff budget, handle multiple forcing scopes, +or survive a mismatched authoritative report. Full-suite coverage checks the +new hook effect and agent event against every existing product and runtime test. + +This evidence completes only the code gate. It does not claim that the 35-minute +Case 09 task now completes semantically. + +## What was omitted + +- No real LLM credentials, auth headers, environment dumps, or private corpus + content were copied into evidence. +- No validator, model, Router, Memory, deadline, corpus, or Progress Lease + threshold was changed. +- No V24R14 campaign artifact was modified or rerun. +- The V24R15 product campaign remains a separate immutable gate after this + commit and PR are available to the isolated candidate instance. diff --git a/.omo/evidence/20260727-convergence-v24r15-post-tool-preview/diff-check.log b/.omo/evidence/20260727-convergence-v24r15-post-tool-preview/diff-check.log new file mode 100644 index 00000000..e69de29b diff --git a/.omo/evidence/20260727-convergence-v24r15-post-tool-preview/focused-gate.log b/.omo/evidence/20260727-convergence-v24r15-post-tool-preview/focused-gate.log new file mode 100644 index 00000000..4883c0bd --- /dev/null +++ b/.omo/evidence/20260727-convergence-v24r15-post-tool-preview/focused-gate.log @@ -0,0 +1,633 @@ +TAP version 13 +# (node:9458) ExperimentalWarning: SQLite is an experimental feature and might change at any time +# (Use `node --trace-warnings ...` to show where the warning was created) +# Subtest: PreModelRequest mutations survive a post-routing request rebuild and remain model-only +ok 1 - PreModelRequest mutations survive a post-routing request rebuild and remain model-only + --- + duration_ms: 13.36275 + type: 'test' + ... +# Subtest: AgentLoop exposes explicit subagent identity to lifecycle hooks +ok 2 - AgentLoop exposes explicit subagent identity to lifecycle hooks + --- + duration_ms: 0.77375 + type: 'test' + ... +# Subtest: evaluation progress lease stops before a third unchanged model request when full compaction is rejected +ok 3 - evaluation progress lease stops before a third unchanged model request when full compaction is rejected + --- + duration_ms: 3.625875 + type: 'test' + ... +# Subtest: AgentLoop delivers newly surfaced repair feedback once after an applied boundary +ok 4 - AgentLoop delivers newly surfaced repair feedback once after an applied boundary + --- + duration_ms: 5.7745 + type: 'test' + ... +# Subtest: AgentLoop permits one prepared-target request and then requires genuine progress +ok 5 - AgentLoop permits one prepared-target request and then requires genuine progress + --- + duration_ms: 2.2025 + type: 'test' + ... +# Subtest: AgentLoop carries a bounded handoff through the required boundary to genuine progress +ok 6 - AgentLoop carries a bounded handoff through the required boundary to genuine progress + --- + duration_ms: 3.981042 + type: 'test' + ... +# Subtest: AgentLoop fails closed before the model when a deferred preview is not confirmed +ok 7 - AgentLoop fails closed before the model when a deferred preview is not confirmed + --- + duration_ms: 1.036417 + type: 'test' + ... +# Subtest: artifact failure injects one bounded correction turn and succeeds after validation +ok 8 - artifact failure injects one bounded correction turn and succeeds after validation + --- + duration_ms: 8.516375 + type: 'test' + ... +# Subtest: required artifact failure wins over max-turn completion after a final tool call +ok 9 - required artifact failure wins over max-turn completion after a final tool call + --- + duration_ms: 4.427917 + type: 'test' + ... +# Subtest: progress lease stays inert unless explicitly enabled +ok 10 - progress lease stays inert unless explicitly enabled + --- + duration_ms: 0.680625 + type: 'test' + ... +# Subtest: opaque hash churn is stagnant while a smaller remaining count renews the lease +ok 11 - opaque hash churn is stagnant while a smaller remaining count renews the lease + --- + duration_ms: 0.13625 + type: 'test' + ... +# Subtest: only a strictly increasing domain progress ordinal renews the lease +ok 12 - only a strictly increasing domain progress ordinal renews the lease + --- + duration_ms: 0.055459 + type: 'test' + ... +# Subtest: a lower remaining count cannot roll the stored progress ordinal backward +ok 13 - a lower remaining count cannot roll the stored progress ordinal backward + --- + duration_ms: 0.080709 + type: 'test' + ... +# Subtest: two stagnant observations require a boundary and allow exactly one post-boundary turn +ok 14 - two stagnant observations require a boundary and allow exactly one post-boundary turn + --- + duration_ms: 0.065125 + type: 'test' + ... +# Subtest: a post-tool progress or bounded handoff preview defers but does not consume a required boundary +ok 15 - a post-tool progress or bounded handoff preview defers but does not consume a required boundary + --- + duration_ms: 0.613167 + type: 'test' + ... +# Subtest: replayed, repair-only, and over-budget previews cannot defer a required boundary +ok 16 - replayed, repair-only, and over-budget previews cannot defer a required boundary + --- + duration_ms: 0.128333 + type: 'test' + ... +# Subtest: previews cannot defer when more than one scope requires a boundary +ok 17 - previews cannot defer when more than one scope requires a boundary + --- + duration_ms: 0.052167 + type: 'test' + ... +# Subtest: new repair feedback after a boundary gets one delivery turn without renewing progress +ok 18 - new repair feedback after a boundary gets one delivery turn without renewing progress + --- + duration_ms: 0.204333 + type: 'test' + ... +# Subtest: genuine progress after repair feedback renews the lease +ok 19 - genuine progress after repair feedback renews the lease + --- + duration_ms: 0.245583 + type: 'test' + ... +# Subtest: a newly prepared repair target gets one non-progress request after feedback +ok 20 - a newly prepared repair target gets one non-progress request after feedback + --- + duration_ms: 0.092417 + type: 'test' + ... +# Subtest: genuine progress immediately after repair preparation renews the lease +ok 21 - genuine progress immediately after repair preparation renews the lease + --- + duration_ms: 0.04175 + type: 'test' + ... +# Subtest: repair feedback co-delivered by a boundary preserves the later preparation turn +ok 22 - repair feedback co-delivered by a boundary preserves the later preparation turn + --- + duration_ms: 0.033917 + type: 'test' + ... +# Subtest: a boundary co-delivering repair feedback and preparation grants no replay turn +ok 23 - a boundary co-delivering repair feedback and preparation grants no replay turn + --- + duration_ms: 0.028416 + type: 'test' + ... +# Subtest: preparation observed before feedback cannot be replayed after the boundary +ok 24 - preparation observed before feedback cannot be replayed after the boundary + --- + duration_ms: 0.027334 + type: 'test' + ... +# Subtest: a second repair revision cannot replace missing progress after feedback +ok 25 - a second repair revision cannot replace missing progress after feedback + --- + duration_ms: 0.027459 + type: 'test' + ... +# Subtest: repair feedback already delivered before a boundary cannot be replayed as grace +ok 26 - repair feedback already delivered before a boundary cannot be replayed as grace + --- + duration_ms: 0.026 + type: 'test' + ... +# Subtest: a new operational handoff gets one bounded request without renewing progress +ok 27 - a new operational handoff gets one bounded request without renewing progress + --- + duration_ms: 0.043833 + type: 'test' + ... +# Subtest: a post-boundary handoff is bounded and its replay still fails closed +ok 28 - a post-boundary handoff is bounded and its replay still fails closed + --- + duration_ms: 0.028583 + type: 'test' + ... +# Subtest: simultaneous repair and handoff revisions cannot be redeemed on separate turns +ok 29 - simultaneous repair and handoff revisions cannot be redeemed on separate turns + --- + duration_ms: 0.029167 + type: 'test' + ... +# Subtest: a lower handoff ordinal cannot be replayed as grace +ok 30 - a lower handoff ordinal cannot be replayed as grace + --- + duration_ms: 0.021917 + type: 'test' + ... +# Subtest: handoff allowance has a hard per-progress-epoch limit and resets only on progress +ok 31 - handoff allowance has a hard per-progress-epoch limit and resets only on progress + --- + duration_ms: 0.034291 + type: 'test' + ... +# Subtest: handoff cannot bypass an unavailable required boundary +ok 32 - handoff cannot bypass an unavailable required boundary + --- + duration_ms: 0.024042 + type: 'test' + ... +# Subtest: sanitized Case 09 matrix handoff trajectory reaches the next semantic checkpoint +ok 33 - sanitized Case 09 matrix handoff trajectory reaches the next semantic checkpoint + --- + duration_ms: 0.31025 + type: 'test' + ... +# Subtest: cold-start allowance expires after the first explicit domain progress +ok 34 - cold-start allowance expires after the first explicit domain progress + --- + duration_ms: 0.039583 + type: 'test' + ... +# Subtest: evaluation mode fails closed when the required boundary is rejected or unavailable +ok 35 - evaluation mode fails closed when the required boundary is rejected or unavailable + --- + duration_ms: 0.033 + type: 'test' + ... +# Subtest: a completed report releases the tracked scope +ok 36 - a completed report releases the tracked scope + --- + duration_ms: 0.02675 + type: 'test' + ... +# Subtest: convergence metadata parser rejects malformed and oversized reports +ok 37 - convergence metadata parser rejects malformed and oversized reports + --- + duration_ms: 0.085042 + type: 'test' + ... +# Subtest: parses only the supported model request patch fields +ok 38 - parses only the supported model request patch fields + --- + duration_ms: 1.547625 + type: 'test' + ... +# Subtest: parses bounded dynamic context controls for hook-driven injection +ok 39 - parses bounded dynamic context controls for hook-driven injection + --- + duration_ms: 0.125417 + type: 'test' + ... +# Subtest: parses only bounded convergence preview fields +ok 40 - parses only bounded convergence preview fields + --- + duration_ms: 0.141458 + type: 'test' + ... +# Subtest: HookRuntime accepts convergence previews only from PostToolUse +ok 41 - HookRuntime accepts convergence previews only from PostToolUse + --- + duration_ms: 0.298333 + type: 'test' + ... +# Subtest: mapAgentEvent propagates runId to streaming lifecycle boundaries +ok 42 - mapAgentEvent propagates runId to streaming lifecycle boundaries + --- + duration_ms: 1.117084 + type: 'test' + ... +# Subtest: mapAgentEvent projects every convergence ordinal +ok 43 - mapAgentEvent projects every convergence ordinal + --- + duration_ms: 1.061 + type: 'test' + ... +# Subtest: mapAgentEvent exposes a post-tool boundary deferral without domain payload +ok 44 - mapAgentEvent exposes a post-tool boundary deferral without domain payload + --- + duration_ms: 0.144917 + type: 'test' + ... +# (node:9462) ExperimentalWarning: SQLite is an experimental feature and might change at any time +# (Use `node --trace-warnings ...` to show where the warning was created) +# [session-title] generation skipped (invalid_json): Unexpected token 'B', "Bounded co"... is not valid JSON +# Subtest: local Gateway records a bounded handoff through boundary to progress +ok 45 - local Gateway records a bounded handoff through boundary to progress + --- + duration_ms: 1027.830709 + type: 'test' + ... +# Subtest: subagent tool detection and result pair at the model projection layer +ok 46 - subagent tool detection and result pair at the model projection layer + --- + duration_ms: 7.597917 + type: 'test' + ... +# Subtest: main tool detection and result produce one exact O1 pair +ok 47 - main tool detection and result produce one exact O1 pair + --- + duration_ms: 0.732792 + type: 'test' + ... +# Subtest: repair preparation grace is observable without becoming progress +ok 48 - repair preparation grace is observable without becoming progress + --- + duration_ms: 0.08175 + type: 'test' + ... +# Subtest: post-tool boundary deferral is independently observable and domain-bounded +ok 49 - post-tool boundary deferral is independently observable and domain-bounded + --- + duration_ms: 0.055542 + type: 'test' + ... +# Subtest: recorder finalizes a complete hash-only trajectory +ok 50 - recorder finalizes a complete hash-only trajectory + --- + duration_ms: 35.3905 + type: 'test' + ... +# Subtest: queue overflow is explicit and makes integrity partial +ok 51 - queue overflow is explicit and makes integrity partial + --- + duration_ms: 10.696667 + type: 'test' + ... +# Subtest: verifier rejects reused model request identity and duplicate terminals +ok 52 - verifier rejects reused model request identity and duplicate terminals + --- + duration_ms: 14.775083 + type: 'test' + ... +# Subtest: verifier rejects duplicate identity and secret-bearing keys +ok 53 - verifier rejects duplicate identity and secret-bearing keys + --- + duration_ms: 0.120083 + type: 'test' + ... +# Subtest: legal coverage validator creates a current proof and removes it when the deliverable changes +ok 54 - legal coverage validator creates a current proof and removes it when the deliverable changes + --- + duration_ms: 351.085709 + type: 'test' + ... +# Subtest: legal coverage initializer creates a text skeleton before source review and remains idempotent +ok 55 - legal coverage initializer creates a text skeleton before source review and remains idempotent + --- + duration_ms: 248.933042 + type: 'test' + ... +# Subtest: legal coverage initializer binds trusted manifests to original inputs +ok 56 - legal coverage initializer binds trusted manifests to original inputs + --- + duration_ms: 353.734209 + type: 'test' + ... +# Subtest: legal coverage source bootstrap creates only deterministic pending manifest rows +ok 57 - legal coverage source bootstrap creates only deterministic pending manifest rows + --- + duration_ms: 354.240916 + type: 'test' + ... +# Subtest: legal coverage injects deterministic disjoint worker batches for large pending source rooms +ok 58 - legal coverage injects deterministic disjoint worker batches for large pending source rooms + --- + duration_ms: 5116.800709 + type: 'test' + ... +# Subtest: legal coverage CLI exposes bundled guidance through stable named references +ok 59 - legal coverage CLI exposes bundled guidance through stable named references + --- + duration_ms: 127.834875 + type: 'test' + ... +# Subtest: legal coverage initializer creates only explicit text formats and preserves existing content +ok 60 - legal coverage initializer creates only explicit text formats and preserves existing content + --- + duration_ms: 50.274625 + type: 'test' + ... +# Subtest: legal coverage initializer rejects traversal and symlink ancestors without external writes +ok 61 - legal coverage initializer rejects traversal and symlink ancestors without external writes + --- + duration_ms: 135.975292 + type: 'test' + ... +# Subtest: legal coverage validator binds runner originals and derivations into its proof +ok 62 - legal coverage validator binds runner originals and derivations into its proof + --- + duration_ms: 198.847042 + type: 'test' + ... +# Subtest: legal coverage validator rejects missing lineage and mutable or derived input roots +ok 63 - legal coverage validator rejects missing lineage and mutable or derived input roots + --- + duration_ms: 251.136541 + type: 'test' + ... +# Subtest: legal coverage next-batch exposes one bounded deterministic repair slice +ok 64 - legal coverage next-batch exposes one bounded deterministic repair slice + --- + duration_ms: 197.346542 + type: 'test' + ... +# Subtest: legal coverage apply-batch atomically updates only the current bounded slice +ok 65 - legal coverage apply-batch atomically updates only the current bounded slice + --- + duration_ms: 290.908167 + type: 'test' + ... +# Subtest: legal coverage apply-batch rejects record and byte limit violations before writing +ok 66 - legal coverage apply-batch rejects record and byte limit violations before writing + --- + duration_ms: 187.116625 + type: 'test' + ... +# Subtest: legal coverage validator rejects orphaned conflicts and incomplete final disclosure +ok 67 - legal coverage validator rejects orphaned conflicts and incomplete final disclosure + --- + duration_ms: 96.387959 + type: 'test' + ... +# Subtest: legal coverage validator detects un-inventoried sources and cross-fact timeline collisions +ok 68 - legal coverage validator detects un-inventoried sources and cross-fact timeline collisions + --- + duration_ms: 101.897459 + type: 'test' + ... +# Subtest: legal coverage validator rejects reused generic quotes and unsupported fact coverage +ok 69 - legal coverage validator rejects reused generic quotes and unsupported fact coverage + --- + duration_ms: 98.755667 + type: 'test' + ... +# Subtest: legal coverage validator requires authority links for critical issues and legal-authority matrices +ok 70 - legal coverage validator requires authority links for critical issues and legal-authority matrices + --- + duration_ms: 96.913333 + type: 'test' + ... +# Subtest: legal coverage validator rejects non-object canonical JSON documents +ok 71 - legal coverage validator rejects non-object canonical JSON documents + --- + duration_ms: 676.479667 + type: 'test' + ... +# Subtest: legal coverage validator binds reviewed source bytes to the ledger and state hash +ok 72 - legal coverage validator binds reviewed source bytes to the ledger and state hash + --- + duration_ms: 194.96725 + type: 'test' + ... +# Subtest: legal coverage validator rejects ancestor symlinks and a symlinked proof +ok 73 - legal coverage validator rejects ancestor symlinks and a symlinked proof + --- + duration_ms: 147.642541 + type: 'test' + ... +# Subtest: legal coverage validator does not read or write through a symlinked state ancestor +ok 74 - legal coverage validator does not read or write through a symlinked state ancestor + --- + duration_ms: 92.308542 + type: 'test' + ... +# Subtest: legal coverage validator requires unresolved disclosure for unverified material facts +ok 75 - legal coverage validator requires unresolved disclosure for unverified material facts + --- + duration_ms: 96.99575 + type: 'test' + ... +# Subtest: legal coverage validator rejects unique but irrelevant issue and authority quotes +ok 76 - legal coverage validator rejects unique but irrelevant issue and authority quotes + --- + duration_ms: 97.060083 + type: 'test' + ... +# Subtest: legal coverage validator enforces reciprocal and same-entry ledger relationships +ok 77 - legal coverage validator enforces reciprocal and same-entry ledger relationships + --- + duration_ms: 97.732917 + type: 'test' + ... +# Subtest: legal coverage validator accepts null for an optional threshold assessment +ok 78 - legal coverage validator accepts null for an optional threshold assessment + --- + duration_ms: 96.992042 + type: 'test' + ... +# Subtest: legal coverage validator uses locators without quotes for binary deliverables +ok 79 - legal coverage validator uses locators without quotes for binary deliverables + --- + duration_ms: 97.772542 + type: 'test' + ... +# Subtest: legal coverage validator rejects empty-fact shortcuts and material facts outside matrices +ok 80 - legal coverage validator rejects empty-fact shortcuts and material facts outside matrices + --- + duration_ms: 206.208 + type: 'test' + ... +# Subtest: legal coverage hook activates only legal work and injects one observable milestone +ok 81 - legal coverage hook activates only legal work and injects one observable milestone + --- + duration_ms: 528.1675 + type: 'test' + ... +# Subtest: legal coverage hook groups repeated validator errors into one bounded milestone +ok 82 - legal coverage hook groups repeated validator errors into one bounded milestone + --- + duration_ms: 520.367042 + type: 'test' + ... +# Subtest: legal coverage pending-matrix selection is bounded across pages and invalid revisions cannot manufacture progress +ok 83 - legal coverage pending-matrix selection is bounded across pages and invalid revisions cannot manufacture progress + --- + duration_ms: 996.7945 + type: 'test' + ... +# Subtest: legal coverage progress ordinal advances once for a new legal phase +ok 84 - legal coverage progress ordinal advances once for a new legal phase + --- + duration_ms: 203.208209 + type: 'test' + ... +# Subtest: legal coverage permits not-applicable only after exhaustive selection and rejects stale or changed matrix proposals +ok 85 - legal coverage permits not-applicable only after exhaustive selection and rejects stale or changed matrix proposals + --- + duration_ms: 493.921792 + type: 'test' + ... +# Subtest: legal coverage rejects oversized matrix evidence before making a selection receipt immutable +ok 86 - legal coverage rejects oversized matrix evidence before making a selection receipt immutable + --- + duration_ms: 151.756292 + type: 'test' + ... +# Subtest: legal coverage fails closed when one matrix index item exceeds the bounded page +ok 87 - legal coverage fails closed when one matrix index item exceeds the bounded page + --- + duration_ms: 159.338125 + type: 'test' + ... +# Subtest: legal coverage injects a bounded deterministic matrix relation-closure batch +ok 88 - legal coverage injects a bounded deterministic matrix relation-closure batch + --- + duration_ms: 206.827542 + type: 'test' + ... +# Subtest: legal coverage authority closure uses one bounded state-bound transaction +ok 89 - legal coverage authority closure uses one bounded state-bound transaction + --- + duration_ms: 571.168708 + type: 'test' + ... +# Subtest: legal coverage risk-signal issue closure exposes one bounded state-bound transaction +ok 90 - legal coverage risk-signal issue closure exposes one bounded state-bound transaction + --- + duration_ms: 776.784125 + type: 'test' + ... +# Subtest: legal coverage rejects changed and symlinked issue closure proposals without ledger mutation +ok 91 - legal coverage rejects changed and symlinked issue closure proposals without ledger mutation + --- + duration_ms: 294.590417 + type: 'test' + ... +# Subtest: legal coverage rejects changed and symlinked authority closure proposals without ledger mutation +ok 92 - legal coverage rejects changed and symlinked authority closure proposals without ledger mutation + --- + duration_ms: 243.057833 + type: 'test' + ... +# Subtest: legal coverage milestone digest ignores opaque incomplete-state hash churn +ok 93 - legal coverage milestone digest ignores opaque incomplete-state hash churn + --- + duration_ms: 0.251959 + type: 'test' + ... +# Subtest: legal coverage hook activates a malformed configured workspace so the agent can repair it +ok 94 - legal coverage hook activates a malformed configured workspace so the agent can repair it + --- + duration_ms: 102.851791 + type: 'test' + ... +# Subtest: legal coverage hook rejects a symlinked session-state ancestor without writing outside the workspace +ok 95 - legal coverage hook rejects a symlinked session-state ancestor without writing outside the workspace + --- + duration_ms: 49.473333 + type: 'test' + ... +# Subtest: legal product plugin loads one skill and contains no benchmark-specific controls +ok 96 - legal product plugin loads one skill and contains no benchmark-specific controls + --- + duration_ms: 3.246875 + type: 'test' + ... +# Subtest: bash success result is formatted with assertions, stdout, and stderr +ok 97 - bash success result is formatted with assertions, stdout, and stderr + --- + duration_ms: 7.63925 + type: 'test' + ... +# Subtest: bash allows hyphenated commands whose names merely start with next +ok 98 - bash allows hyphenated commands whose names merely start with next + --- + duration_ms: 4.195667 + type: 'test' + ... +# Subtest: PostToolUse carries bounded convergence previews only in internal lifecycle metadata +ok 99 - PostToolUse carries bounded convergence previews only in internal lifecycle metadata + --- + duration_ms: 0.547459 + type: 'test' + ... +# Subtest: bash still rejects exact framework CLI tokens that start long-lived processes +ok 100 - bash still rejects exact framework CLI tokens that start long-lived processes + --- + duration_ms: 1.9225 + type: 'test' + ... +# Subtest: bash failure tool result includes raw stdout and stderr tail for UI and model +ok 101 - bash failure tool result includes raw stdout and stderr tail for UI and model + --- + duration_ms: 19.576709 + type: 'test' + ... +# Subtest: bash failure diagnostic extracts Python traceback location and exception +ok 102 - bash failure diagnostic extracts Python traceback location and exception + --- + duration_ms: 1.221042 + type: 'test' + ... +# Subtest: bash success keeps full output for ToolResultBudget persistence +ok 103 - bash success keeps full output for ToolResultBudget persistence + --- + duration_ms: 8.925875 + type: 'test' + ... +1..103 +# tests 103 +# suites 0 +# pass 103 +# fail 0 +# cancelled 0 +# skipped 0 +# todo 0 +# duration_ms 15456.426292 diff --git a/.omo/evidence/20260727-convergence-v24r15-post-tool-preview/full-suite.log b/.omo/evidence/20260727-convergence-v24r15-post-tool-preview/full-suite.log new file mode 100644 index 00000000..84f3e44e --- /dev/null +++ b/.omo/evidence/20260727-convergence-v24r15-post-tool-preview/full-suite.log @@ -0,0 +1,2001 @@ + +> pilotdeck@0.1.0 test +> npm run build && node --test --test-force-exit --test-timeout 60000 "dist/tests/**/*.test.js" "dist/tests/**/*.spec.js" + + +> pilotdeck@0.1.0 prebuild +> npm run check:runtime && node scripts/bootstrap-pilotdeck-config.mjs && cd src/context/memory/edgeclaw-memory-core && npm run build + + +> pilotdeck@0.1.0 check:runtime +> node scripts/check-node-runtime.mjs + + +> edgeclaw-memory-core@0.0.0 build +> node -e "require('fs').rmSync('lib',{recursive:true,force:true})" && tsc -p tsconfig.json + + +> pilotdeck@0.1.0 build +> node -e "require('fs').rmSync('dist',{recursive:true,force:true})" && tsc -p tsconfig.json && node -e "require('fs').cpSync('src/extension/plugins/builtin','dist/src/extension/plugins/builtin',{recursive:true})" + +TAP version 13 +# Subtest: Feishu handles permission replies before the active chat drain finishes +ok 1 - Feishu handles permission replies before the active chat drain finishes + --- + duration_ms: 1.269959 + type: 'test' + ... +# Subtest: ImPermissionHelper resolves all pending permission requests for a chat +ok 2 - ImPermissionHelper resolves all pending permission requests for a chat + --- + duration_ms: 1.0765 + type: 'test' + ... +# Subtest: ImPermissionHelper keeps pending requests when the reply is invalid +ok 3 - ImPermissionHelper keeps pending requests when the reply is invalid + --- + duration_ms: 0.057416 + type: 'test' + ... +# (node:10209) ExperimentalWarning: SQLite is an experimental feature and might change at any time +# (Use `node --trace-warnings ...` to show where the warning was created) +# Subtest: PreModelRequest mutations survive a post-routing request rebuild and remain model-only +ok 4 - PreModelRequest mutations survive a post-routing request rebuild and remain model-only + --- + duration_ms: 11.243 + type: 'test' + ... +# Subtest: AgentLoop exposes explicit subagent identity to lifecycle hooks +ok 5 - AgentLoop exposes explicit subagent identity to lifecycle hooks + --- + duration_ms: 2.258792 + type: 'test' + ... +# Subtest: evaluation progress lease stops before a third unchanged model request when full compaction is rejected +ok 6 - evaluation progress lease stops before a third unchanged model request when full compaction is rejected + --- + duration_ms: 5.41725 + type: 'test' + ... +# Subtest: AgentLoop delivers newly surfaced repair feedback once after an applied boundary +ok 7 - AgentLoop delivers newly surfaced repair feedback once after an applied boundary + --- + duration_ms: 9.823875 + type: 'test' + ... +# Subtest: AgentLoop permits one prepared-target request and then requires genuine progress +ok 8 - AgentLoop permits one prepared-target request and then requires genuine progress + --- + duration_ms: 5.893792 + type: 'test' + ... +# Subtest: AgentLoop carries a bounded handoff through the required boundary to genuine progress +ok 9 - AgentLoop carries a bounded handoff through the required boundary to genuine progress + --- + duration_ms: 5.147208 + type: 'test' + ... +# Subtest: AgentLoop fails closed before the model when a deferred preview is not confirmed +ok 10 - AgentLoop fails closed before the model when a deferred preview is not confirmed + --- + duration_ms: 5.639917 + type: 'test' + ... +# Subtest: artifact failure injects one bounded correction turn and succeeds after validation +ok 11 - artifact failure injects one bounded correction turn and succeeds after validation + --- + duration_ms: 9.349833 + type: 'test' + ... +# Subtest: required artifact failure wins over max-turn completion after a final tool call +ok 12 - required artifact failure wins over max-turn completion after a final tool call + --- + duration_ms: 3.895375 + type: 'test' + ... +# Subtest: AgentSession restores terminal state and dispatches SessionEnd when the runner throws +ok 13 - AgentSession restores terminal state and dispatches SessionEnd when the runner throws + --- + duration_ms: 1.500583 + type: 'test' + ... +# Subtest: AgentSession cleanup runs when a consumer stops reading early +ok 14 - AgentSession cleanup runs when a consumer stops reading early + --- + duration_ms: 0.09875 + type: 'test' + ... +# (node:10211) ExperimentalWarning: SQLite is an experimental feature and might change at any time +# (Use `node --trace-warnings ...` to show where the warning was created) +# Subtest: real gateway drives legal plugin milestones through bounded artifact correction +ok 15 - real gateway drives legal plugin milestones through bounded artifact correction + --- + duration_ms: 1805.579875 + type: 'test' + ... +# Subtest: real gateway blocks completion when the legal Stop hook cannot read session state +ok 16 - real gateway blocks completion when the legal Stop hook cannot read session state + --- + duration_ms: 1116.047417 + type: 'test' + ... +# Subtest: real gateway executes the state-bound legal authority closure with complete O1 evidence +ok 17 - real gateway executes the state-bound legal authority closure with complete O1 evidence + --- + duration_ms: 1035.242083 + type: 'test' + ... +# Subtest: real gateway executes the state-bound legal issue closure with complete O1 evidence +ok 18 - real gateway executes the state-bound legal issue closure with complete O1 evidence + --- + duration_ms: 788.426375 + type: 'test' + ... +# Subtest: real gateway hands off a validated ordinary source proposal before renewing from its durable receipt +ok 19 - real gateway hands off a validated ordinary source proposal before renewing from its durable receipt + --- + duration_ms: 1533.12 + type: 'test' + ... +# Subtest: real gateway applies an immutable source repair before renewing legal progress +ok 20 - real gateway applies an immutable source repair before renewing legal progress + --- + duration_ms: 1361.073209 + type: 'test' + ... +# (node:10212) ExperimentalWarning: SQLite is an experimental feature and might change at any time +# (Use `node --trace-warnings ...` to show where the warning was created) +# Subtest: local gateway applies project hook context and enforces artifact correction +ok 21 - local gateway applies project hook context and enforces artifact correction + --- + duration_ms: 1146.531458 + type: 'test' + ... +# Subtest: ordinary edit failures after a successful write do not enter large-file repair +ok 22 - ordinary edit failures after a successful write do not enter large-file repair + --- + duration_ms: 0.788834 + type: 'test' + ... +# Subtest: an explicit post-draft large-file failure starts bounded recovery +ok 23 - an explicit post-draft large-file failure starts bounded recovery + --- + duration_ms: 0.175625 + type: 'test' + ... +# Subtest: successful focused write clears an active large-file repair episode +ok 24 - successful focused write clears an active large-file repair episode + --- + duration_ms: 0.142916 + type: 'test' + ... +# Subtest: permission failures are never reclassified as large-file recovery +ok 25 - permission failures are never reclassified as large-file recovery + --- + duration_ms: 0.042 + type: 'test' + ... +# Subtest: progress lease stays inert unless explicitly enabled +ok 26 - progress lease stays inert unless explicitly enabled + --- + duration_ms: 0.672417 + type: 'test' + ... +# Subtest: opaque hash churn is stagnant while a smaller remaining count renews the lease +ok 27 - opaque hash churn is stagnant while a smaller remaining count renews the lease + --- + duration_ms: 0.137208 + type: 'test' + ... +# Subtest: only a strictly increasing domain progress ordinal renews the lease +ok 28 - only a strictly increasing domain progress ordinal renews the lease + --- + duration_ms: 0.067833 + type: 'test' + ... +# Subtest: a lower remaining count cannot roll the stored progress ordinal backward +ok 29 - a lower remaining count cannot roll the stored progress ordinal backward + --- + duration_ms: 0.081167 + type: 'test' + ... +# Subtest: two stagnant observations require a boundary and allow exactly one post-boundary turn +ok 30 - two stagnant observations require a boundary and allow exactly one post-boundary turn + --- + duration_ms: 0.066875 + type: 'test' + ... +# Subtest: a post-tool progress or bounded handoff preview defers but does not consume a required boundary +ok 31 - a post-tool progress or bounded handoff preview defers but does not consume a required boundary + --- + duration_ms: 0.612 + type: 'test' + ... +# Subtest: replayed, repair-only, and over-budget previews cannot defer a required boundary +ok 32 - replayed, repair-only, and over-budget previews cannot defer a required boundary + --- + duration_ms: 0.12825 + type: 'test' + ... +# Subtest: previews cannot defer when more than one scope requires a boundary +ok 33 - previews cannot defer when more than one scope requires a boundary + --- + duration_ms: 0.053584 + type: 'test' + ... +# Subtest: new repair feedback after a boundary gets one delivery turn without renewing progress +ok 34 - new repair feedback after a boundary gets one delivery turn without renewing progress + --- + duration_ms: 0.207125 + type: 'test' + ... +# Subtest: genuine progress after repair feedback renews the lease +ok 35 - genuine progress after repair feedback renews the lease + --- + duration_ms: 0.246875 + type: 'test' + ... +# Subtest: a newly prepared repair target gets one non-progress request after feedback +ok 36 - a newly prepared repair target gets one non-progress request after feedback + --- + duration_ms: 0.092708 + type: 'test' + ... +# Subtest: genuine progress immediately after repair preparation renews the lease +ok 37 - genuine progress immediately after repair preparation renews the lease + --- + duration_ms: 0.042042 + type: 'test' + ... +# Subtest: repair feedback co-delivered by a boundary preserves the later preparation turn +ok 38 - repair feedback co-delivered by a boundary preserves the later preparation turn + --- + duration_ms: 0.03525 + type: 'test' + ... +# Subtest: a boundary co-delivering repair feedback and preparation grants no replay turn +ok 39 - a boundary co-delivering repair feedback and preparation grants no replay turn + --- + duration_ms: 0.028042 + type: 'test' + ... +# Subtest: preparation observed before feedback cannot be replayed after the boundary +ok 40 - preparation observed before feedback cannot be replayed after the boundary + --- + duration_ms: 0.027125 + type: 'test' + ... +# Subtest: a second repair revision cannot replace missing progress after feedback +ok 41 - a second repair revision cannot replace missing progress after feedback + --- + duration_ms: 0.025791 + type: 'test' + ... +# Subtest: repair feedback already delivered before a boundary cannot be replayed as grace +ok 42 - repair feedback already delivered before a boundary cannot be replayed as grace + --- + duration_ms: 0.025375 + type: 'test' + ... +# Subtest: a new operational handoff gets one bounded request without renewing progress +ok 43 - a new operational handoff gets one bounded request without renewing progress + --- + duration_ms: 0.029458 + type: 'test' + ... +# Subtest: a post-boundary handoff is bounded and its replay still fails closed +ok 44 - a post-boundary handoff is bounded and its replay still fails closed + --- + duration_ms: 0.027083 + type: 'test' + ... +# Subtest: simultaneous repair and handoff revisions cannot be redeemed on separate turns +ok 45 - simultaneous repair and handoff revisions cannot be redeemed on separate turns + --- + duration_ms: 0.027166 + type: 'test' + ... +# Subtest: a lower handoff ordinal cannot be replayed as grace +ok 46 - a lower handoff ordinal cannot be replayed as grace + --- + duration_ms: 0.021542 + type: 'test' + ... +# Subtest: handoff allowance has a hard per-progress-epoch limit and resets only on progress +ok 47 - handoff allowance has a hard per-progress-epoch limit and resets only on progress + --- + duration_ms: 0.034375 + type: 'test' + ... +# Subtest: handoff cannot bypass an unavailable required boundary +ok 48 - handoff cannot bypass an unavailable required boundary + --- + duration_ms: 0.023292 + type: 'test' + ... +# Subtest: sanitized Case 09 matrix handoff trajectory reaches the next semantic checkpoint +ok 49 - sanitized Case 09 matrix handoff trajectory reaches the next semantic checkpoint + --- + duration_ms: 0.487958 + type: 'test' + ... +# Subtest: cold-start allowance expires after the first explicit domain progress +ok 50 - cold-start allowance expires after the first explicit domain progress + --- + duration_ms: 0.060167 + type: 'test' + ... +# Subtest: evaluation mode fails closed when the required boundary is rejected or unavailable +ok 51 - evaluation mode fails closed when the required boundary is rejected or unavailable + --- + duration_ms: 0.041375 + type: 'test' + ... +# Subtest: a completed report releases the tracked scope +ok 52 - a completed report releases the tracked scope + --- + duration_ms: 0.031583 + type: 'test' + ... +# Subtest: convergence metadata parser rejects malformed and oversized reports +ok 53 - convergence metadata parser rejects malformed and oversized reports + --- + duration_ms: 0.094958 + type: 'test' + ... +# Subtest: explore subagent does not probe tool safety before execution +ok 54 - explore subagent does not probe tool safety before execution + --- + duration_ms: 25.027667 + type: 'test' + ... +# Subtest: explore registry ignores an unallowed dynamic execute_code tool without probing it +ok 55 - explore registry ignores an unallowed dynamic execute_code tool without probing it + --- + duration_ms: 0.335541 + type: 'test' + ... +# Subtest: read-only subagent evaluates bash safety from the real command +ok 56 - read-only subagent evaluates bash safety from the real command + --- + duration_ms: 2.658792 + type: 'test' + ... +# Subtest: read-only execute_code checks the real code instead of crashing on registry setup +ok 57 - read-only execute_code checks the real code instead of crashing on registry setup + --- + duration_ms: 0.241583 + type: 'test' + ... +# Subtest: subagent budget defaults to ten minutes without a parent deadline +ok 58 - subagent budget defaults to ten minutes without a parent deadline + --- + duration_ms: 2.035875 + type: 'test' + ... +# Subtest: subagent budget preserves the parent handoff window +ok 59 - subagent budget preserves the parent handoff window + --- + duration_ms: 0.417625 + type: 'test' + ... +# Subtest: subagent budget directive exposes the effective bound and diminishing-return rule +ok 60 - subagent budget directive exposes the effective bound and diminishing-return rule + --- + duration_ms: 0.113375 + type: 'test' + ... +# Subtest: composed subagent timeout aborts with a typed reason +ok 61 - composed subagent timeout aborts with a typed reason + --- + duration_ms: 6.726875 + type: 'test' + ... +# Subtest: subagent operation returns control even when the callee ignores abort +ok 62 - subagent operation returns control even when the callee ignores abort + --- + duration_ms: 5.287208 + type: 'test' + ... +# Subtest: parent abort wins without being misclassified as a child timeout +ok 63 - parent abort wins without being misclassified as a child timeout + --- + duration_ms: 0.396959 + type: 'test' + ... +# Subtest: filterIncompleteToolCalls tolerates malformed messages without content +ok 64 - filterIncompleteToolCalls tolerates malformed messages without content + --- + duration_ms: 2.633875 + type: 'test' + ... +# Subtest: fork timeout closes child lifecycle and lets the parent finish +ok 65 - fork timeout closes child lifecycle and lets the parent finish + --- + duration_ms: 53.412917 + type: 'test' + ... +# Subtest: turn environment provides an isolated PilotDeck-owned work directory +ok 66 - turn environment provides an isolated PilotDeck-owned work directory + --- + duration_ms: 1.277709 + type: 'test' + ... +# Subtest: turn environment inherits the process environment when no override is configured +ok 67 - turn environment inherits the process environment when no override is configured + --- + duration_ms: 0.205125 + type: 'test' + ... +# Subtest: validates a required artifact and keeps contracts isolated by session +ok 68 - validates a required artifact and keeps contracts isolated by session + --- + duration_ms: 3.468041 + type: 'test' + ... +# Subtest: missing or wrong-format artifacts fail with reviewer-readable issues +ok 69 - missing or wrong-format artifacts fail with reviewer-readable issues + --- + duration_ms: 1.015875 + type: 'test' + ... +# Subtest: empty required artifacts fail validation +ok 70 - empty required artifacts fail validation + --- + duration_ms: 2.061917 + type: 'test' + ... +# Subtest: rejects traversal and symlink escapes before a domain validator runs +ok 71 - rejects traversal and symlink escapes before a domain validator runs + --- + duration_ms: 1.992833 + type: 'test' + ... +# Subtest: legal-specific validator data stays in the plugin validator +ok 72 - legal-specific validator data stays in the plugin validator + --- + duration_ms: 0.650041 + type: 'test' + ... +# Subtest: validator exceptions become structured failures instead of escaping the runtime +ok 73 - validator exceptions become structured failures instead of escaping the runtime + --- + duration_ms: 1.167916 + type: 'test' + ... +# Subtest: duplicate validator ids cannot override an existing validator +ok 74 - duplicate validator ids cannot override an existing validator + --- + duration_ms: 0.215125 + type: 'test' + ... +# Subtest: contract registration is atomic when one contract is invalid +ok 75 - contract registration is atomic when one contract is invalid + --- + duration_ms: 0.052291 + type: 'test' + ... +# Subtest: extension watcher ignores generated Python and Office runtime files +ok 76 - extension watcher ignores generated Python and Office runtime files + --- + duration_ms: 0.52475 + type: 'test' + ... +# Subtest: config bootstrap does not copy bundled skills into user storage +ok 77 - config bootstrap does not copy bundled skills into user storage + --- + duration_ms: 70.063625 + type: 'test' + ... +# Subtest: Office attachments are reported unsupported before size checks +ok 78 - Office attachments are reported unsupported before size checks + --- + duration_ms: 4.845 + type: 'test' + ... +# Subtest: auto compaction reports summary success separately from rejected oversized output +ok 79 - auto compaction reports summary success separately from rejected oversized output + --- + duration_ms: 1.22025 + type: 'test' + ... +# Subtest: auto compaction marks a budget-clearing summary as applied +ok 80 - auto compaction marks a budget-clearing summary as applied + --- + duration_ms: 0.077583 + type: 'test' + ... +# Subtest: progress policy can force a full boundary while the token budget is still healthy +ok 81 - progress policy can force a full boundary while the token budget is still healthy + --- + duration_ms: 0.089375 + type: 'test' + ... +# Subtest: full compaction distinguishes a fully retained trajectory from model failure +ok 82 - full compaction distinguishes a fully retained trajectory from model failure + --- + duration_ms: 583.198083 + type: 'test' + ... +# Subtest: auto compaction propagates no-summarizable as a stable rejection reason without claiming a summary attempt +ok 83 - auto compaction propagates no-summarizable as a stable rejection reason without claiming a summary attempt + --- + duration_ms: 0.335834 + type: 'test' + ... +# Subtest: bounded protected-prefix retention summarizes old agent turns and preserves every kept tool pair +ok 84 - bounded protected-prefix retention summarizes old agent turns and preserves every kept tool pair + --- + duration_ms: 2.790583 + type: 'test' + ... +# Subtest: full compaction shares one token budget across the exact tail and protected prefix +ok 85 - full compaction shares one token budget across the exact tail and protected prefix + --- + duration_ms: 50.932167 + type: 'test' + ... +# Subtest: full compaction keeps one oversized newest atomic frame intact +ok 86 - full compaction keeps one oversized newest atomic frame intact + --- + duration_ms: 0.479667 + type: 'test' + ... +# Subtest: full compaction summarizes a real-shaped single-prompt Agent trajectory +ok 87 - full compaction summarizes a real-shaped single-prompt Agent trajectory + --- + duration_ms: 1.288458 + type: 'test' + ... +# Subtest: full compaction aligns a token-budget tail boundary to complete tool turns +ok 88 - full compaction aligns a token-budget tail boundary to complete tool turns + --- + duration_ms: 1.239583 + type: 'test' + ... +# Subtest: sanitized Case 09 replay retains enough evidence to detect rejected blocking compactions +ok 89 - sanitized Case 09 replay retains enough evidence to detect rejected blocking compactions + --- + duration_ms: 10.925542 + type: 'test' + ... +# Subtest: lifecycle context is injected as a transient model-only message +ok 90 - lifecycle context is injected as a transient model-only message + --- + duration_ms: 6.10075 + type: 'test' + ... +# Subtest: lifecycle without a dynamic store preserves legacy hook messages +ok 91 - lifecycle without a dynamic store preserves legacy hook messages + --- + duration_ms: 0.096459 + type: 'test' + ... +# Subtest: SessionEnd clears session-scoped runtime state even when a hook throws +ok 92 - SessionEnd clears session-scoped runtime state even when a hook throws + --- + duration_ms: 0.268041 + type: 'test' + ... +# Subtest: merges pending context by priority and consumes it once +ok 93 - merges pending context by priority and consumes it once + --- + duration_ms: 2.178 + type: 'test' + ... +# Subtest: re-registering the same source and id replaces stale content without changing order +ok 94 - re-registering the same source and id replaces stale content without changing order + --- + duration_ms: 0.122375 + type: 'test' + ... +# Subtest: expired context is pruned without affecting another session +ok 95 - expired context is pruned without affecting another session + --- + duration_ms: 0.263 + type: 'test' + ... +# Subtest: blank context is ignored +ok 96 - blank context is ignored + --- + duration_ms: 0.116416 + type: 'test' + ... +# Subtest: clearing a session does not disturb another session with a shared prefix +ok 97 - clearing a session does not disturb another session with a shared prefix + --- + duration_ms: 0.341833 + type: 'test' + ... +# Subtest: context budgets retain higher-priority entries and cap merged prompt size +ok 98 - context budgets retain higher-priority entries and cap merged prompt size + --- + duration_ms: 1.002708 + type: 'test' + ... +# Subtest: source and id tuples cannot collide through delimiter characters +ok 99 - source and id tuples cannot collide through delimiter characters + --- + duration_ms: 0.105083 + type: 'test' + ... +# Subtest: default prompt does not require Markdown links for generated files +ok 100 - default prompt does not require Markdown links for generated files + --- + duration_ms: 1.86575 + type: 'test' + ... +# Subtest: default prompt does not advertise web search when the tool is unavailable +ok 101 - default prompt does not advertise web search when the tool is unavailable + --- + duration_ms: 0.146833 + type: 'test' + ... +# Subtest: available skills include the resolved SKILL.md path and bounded lookup guidance +ok 102 - available skills include the resolved SKILL.md path and bounded lookup guidance + --- + duration_ms: 2.094 + type: 'test' + ... +# Subtest: budget snapshot uses conservative budget tokens for ratio while preserving real tokens +ok 103 - budget snapshot uses conservative budget tokens for ratio while preserving real tokens + --- + duration_ms: 4.579666 + type: 'test' + ... +# Subtest: tool text under token budget remains inline even when over legacy byte threshold +ok 104 - tool text under token budget remains inline even when over legacy byte threshold + --- + duration_ms: 7195.904542 + type: 'test' + ... +# Subtest: tool text over token budget is persisted with expanded grep-first preview +ok 105 - tool text over token budget is persisted with expanded grep-first preview + --- + duration_ms: 9.173958 + type: 'test' + ... +# Subtest: large tool error references preserve error semantics for model replay +ok 106 - large tool error references preserve error semantics for model replay + --- + duration_ms: 3.223125 + type: 'test' + ... +# Subtest: multibyte truncated tool result references advertise read_file access +ok 107 - multibyte truncated tool result references advertise read_file access + --- + duration_ms: 1.658541 + type: 'test' + ... +# Subtest: parses declarative artifact contracts without domain-specific knowledge +ok 108 - parses declarative artifact contracts without domain-specific knowledge + --- + duration_ms: 1.080167 + type: 'test' + ... +# (node:10282) ExperimentalWarning: SQLite is an experimental feature and might change at any time +# (Use `node --trace-warnings ...` to show where the warning was created) +# Subtest: an active domain plugin contributes only generic runtime contracts +ok 109 - an active domain plugin contributes only generic runtime contracts + --- + duration_ms: 4.031042 + type: 'test' + ... +# Subtest: internal prompts do not reactivate domain detection +ok 110 - internal prompts do not reactivate domain detection + --- + duration_ms: 0.127209 + type: 'test' + ... +# Subtest: parses only the supported model request patch fields +ok 111 - parses only the supported model request patch fields + --- + duration_ms: 1.242292 + type: 'test' + ... +# Subtest: parses bounded dynamic context controls for hook-driven injection +ok 112 - parses bounded dynamic context controls for hook-driven injection + --- + duration_ms: 0.118042 + type: 'test' + ... +# Subtest: parses only bounded convergence preview fields +ok 113 - parses only bounded convergence preview fields + --- + duration_ms: 0.135833 + type: 'test' + ... +# Subtest: HookRuntime accepts convergence previews only from PostToolUse +ok 114 - HookRuntime accepts convergence previews only from PostToolUse + --- + duration_ms: 0.301083 + type: 'test' + ... +# Subtest: standalone skills expose only their slug without a parent-directory namespace +ok 115 - standalone skills expose only their slug without a parent-directory namespace + --- + duration_ms: 21.738667 + type: 'test' + ... +# Subtest: a plugin skill directory used as the configured base never derives a parent namespace +ok 116 - a plugin skill directory used as the configured base never derives a parent namespace + --- + duration_ms: 0.664542 + type: 'test' + ... +# Subtest: standalone skill precedence is project > user > builtin without legacy aliases +ok 117 - standalone skill precedence is project > user > builtin without legacy aliases + --- + duration_ms: 22.748417 + type: 'test' + ... +# Subtest: legacy bundled migration backs up only byte-identical bootstrap copies +ok 118 - legacy bundled migration backs up only byte-identical bootstrap copies + --- + duration_ms: 35.31825 + type: 'test' + ... +# Subtest: legacy bundled migration leaves user skills untouched when the release bundle is missing +ok 119 - legacy bundled migration leaves user skills untouched when the release bundle is missing + --- + duration_ms: 5.2475 + type: 'test' + ... +# Subtest: SkillManager lists built-ins separately and describes override relationships +ok 120 - SkillManager lists built-ins separately and describes override relationships + --- + duration_ms: 30.268791 + type: 'test' + ... +# Subtest: SkillManager permits reading but rejects mutations of built-in skills +ok 121 - SkillManager permits reading but rejects mutations of built-in skills + --- + duration_ms: 3.393958 + type: 'test' + ... +# Subtest: registered plain-text attachments with non-whitelisted names are described as read_file inspectable +ok 122 - registered plain-text attachments with non-whitelisted names are described as read_file inspectable + --- + duration_ms: 33.058625 + type: 'test' + ... +# Subtest: registered Office attachments are still described as not directly inspectable +ok 123 - registered Office attachments are still described as not directly inspectable + --- + duration_ms: 3.321375 + type: 'test' + ... +# Subtest: startPilotDeckServer listens before a background channel finishes starting +ok 124 - startPilotDeckServer listens before a background channel finishes starting + --- + duration_ms: 31.503458 + type: 'test' + ... +# (node:10304) ExperimentalWarning: SQLite is an experimental feature and might change at any time +# (Use `node --trace-warnings ...` to show where the warning was created) +# Subtest: browser-use args do not inherit generic proxy env by default +ok 125 - browser-use args do not inherit generic proxy env by default + --- + duration_ms: 1.475917 + type: 'test' + ... +# Subtest: browser-use args use explicit browser proxy server +ok 126 - browser-use args use explicit browser proxy server + --- + duration_ms: 0.0985 + type: 'test' + ... +# Subtest: browser-use args only inherit generic proxy env when opted in +ok 127 - browser-use args only inherit generic proxy env when opted in + --- + duration_ms: 0.069584 + type: 'test' + ... +# Subtest: browser-use args use config proxy when browser env proxy is absent +ok 128 - browser-use args use config proxy when browser env proxy is absent + --- + duration_ms: 0.369416 + type: 'test' + ... +# Subtest: browser-use args allow explicit direct mode to disable config proxy +ok 129 - browser-use args allow explicit direct mode to disable config proxy + --- + duration_ms: 0.168875 + type: 'test' + ... +# Subtest: internal prompt dispatch waits for the user turn and fully drains the synthetic turn +ok 130 - internal prompt dispatch waits for the user turn and fully drains the synthetic turn + --- + duration_ms: 11.6 + type: 'test' + ... +# Subtest: mapAgentEvent propagates runId to streaming lifecycle boundaries +ok 131 - mapAgentEvent propagates runId to streaming lifecycle boundaries + --- + duration_ms: 1.60125 + type: 'test' + ... +# Subtest: mapAgentEvent projects every convergence ordinal +ok 132 - mapAgentEvent projects every convergence ordinal + --- + duration_ms: 0.703708 + type: 'test' + ... +# Subtest: mapAgentEvent exposes a post-tool boundary deferral without domain payload +ok 133 - mapAgentEvent exposes a post-tool boundary deferral without domain payload + --- + duration_ms: 0.081375 + type: 'test' + ... +# Subtest: defer returns immediately while a user turn is busy +ok 134 - defer returns immediately while a user turn is busy + --- + duration_ms: 1.084958 + type: 'test' + ... +# Subtest: enqueue waits for idle and dispatches exactly once +ok 135 - enqueue waits for idle and dispatches exactly once + --- + duration_ms: 1.709708 + type: 'test' + ... +# Subtest: same semantic key coalesces in-flight and dedupes after completion +ok 136 - same semantic key coalesces in-flight and dedupes after completion + --- + duration_ms: 0.16475 + type: 'test' + ... +# Subtest: an idle-settle race retries when the gateway reports busy +ok 137 - an idle-settle race retries when the gateway reports busy + --- + duration_ms: 0.161917 + type: 'test' + ... +# Subtest: abort while queued does not run an internal turn +ok 138 - abort while queued does not run an internal turn + --- + duration_ms: 0.103208 + type: 'test' + ... +# Subtest: turn timeouts are reported distinctly from generic execution failures +ok 139 - turn timeouts are reported distinctly from generic execution failures + --- + duration_ms: 0.04975 + type: 'test' + ... +# Subtest: a zero recent-dedupe window disables completed-result deduplication +ok 140 - a zero recent-dedupe window disables completed-result deduplication + --- + duration_ms: 0.148 + type: 'test' + ... +# Subtest: dispose prevents a queued prompt from starting after shutdown +ok 141 - dispose prevents a queued prompt from starting after shutdown + --- + duration_ms: 0.113583 + type: 'test' + ... +# Subtest: the public WebSocket boundary strips trusted in-process turn controls +ok 142 - the public WebSocket boundary strips trusted in-process turn controls + --- + duration_ms: 1.939375 + type: 'test' + ... +# Subtest: waitForIdle resolves only after the matching turn slot is released +ok 143 - waitForIdle resolves only after the matching turn slot is released + --- + duration_ms: 1.458292 + type: 'test' + ... +# Subtest: waitForIdle honors AbortSignal without releasing the turn +ok 144 - waitForIdle honors AbortSignal without releasing the turn + --- + duration_ms: 0.554584 + type: 'test' + ... +# Subtest: mapAgentEvent bounds live tool result previews at the gateway +ok 145 - mapAgentEvent bounds live tool result previews at the gateway + --- + duration_ms: 1.465417 + type: 'test' + ... +# Subtest: mapAgentEvent bounds large strings inside successful tool data +ok 146 - mapAgentEvent bounds large strings inside successful tool data + --- + duration_ms: 0.302959 + type: 'test' + ... +# Subtest: mapAgentEvent bounds subagent tool result content and preview +ok 147 - mapAgentEvent bounds subagent tool result content and preview + --- + duration_ms: 0.133416 + type: 'test' + ... +# Subtest: Gateway propagates its absolute turn deadline into the Agent session +ok 148 - Gateway propagates its absolute turn deadline into the Agent session + --- + duration_ms: 1.675875 + type: 'test' + ... +# Subtest: WeixinChannel.start returns while QR login is waiting +ok 149 - WeixinChannel.start returns while QR login is waiting + --- + duration_ms: 24.790041 + type: 'test' + ... +# Subtest: channel runtime status reporter persists the latest channel state +ok 150 - channel runtime status reporter persists the latest channel state + --- + duration_ms: 7.656917 + type: 'test' + ... +# Subtest: UI weixin QR route only reads runtime status +ok 151 - UI weixin QR route only reads runtime status + --- + duration_ms: 1.102625 + type: 'test' + ... +# Subtest: UI weixin QR begin route delegates to gateway prepare RPC +ok 152 - UI weixin QR begin route delegates to gateway prepare RPC + --- + duration_ms: 0.18 + type: 'test' + ... +# Subtest: Gateway settings keeps existing status rendered during silent refresh +ok 153 - Gateway settings keeps existing status rendered during silent refresh + --- + duration_ms: 0.362583 + type: 'test' + ... +# Subtest: Gateway settings starts weixin QR by begin route and ignores stale runtime errors +ok 154 - Gateway settings starts weixin QR by begin route and ignores stale runtime errors + --- + duration_ms: 0.250708 + type: 'test' + ... +# Subtest: Gateway protocol exposes prepare_weixin_login RPC +ok 155 - Gateway protocol exposes prepare_weixin_login RPC + --- + duration_ms: 0.448083 + type: 'test' + ... +# Subtest: applies context, system messages, and restricted model request patches +ok 156 - applies context, system messages, and restricted model request patches + --- + duration_ms: 0.978834 + type: 'test' + ... +# Subtest: blocks before any request mutation is used +ok 157 - blocks before any request mutation is used + --- + duration_ms: 0.053833 + type: 'test' + ... +# Subtest: McpClient keeps stdio clients idle before connection +ok 158 - McpClient keeps stdio clients idle before connection + --- + duration_ms: 1.177333 + type: 'test' + ... +# Subtest: McpClient constructs streamable_http transport without requiring stdio fields +ok 159 - McpClient constructs streamable_http transport without requiring stdio fields + --- + duration_ms: 0.131625 + type: 'test' + ... +# Subtest: McpClient routes streamable_http fetches with bounded timeouts +ok 160 - McpClient routes streamable_http fetches with bounded timeouts + --- + duration_ms: 1.607334 + type: 'test' + ... +# Subtest: expandMcpString expands ${env:*} placeholders +ok 161 - expandMcpString expands ${env:*} placeholders + --- + duration_ms: 0.601542 + type: 'test' + ... +# Subtest: expandMcpString expands ${userHome} placeholder +ok 162 - expandMcpString expands ${userHome} placeholder + --- + duration_ms: 0.093375 + type: 'test' + ... +# Subtest: expandMcpString expands ~ prefix +ok 163 - expandMcpString expands ~ prefix + --- + duration_ms: 0.0395 + type: 'test' + ... +# Subtest: expandMcpString leaves unknown env vars as empty string +ok 164 - expandMcpString leaves unknown env vars as empty string + --- + duration_ms: 0.032125 + type: 'test' + ... +# Subtest: expandMcpString handles combined placeholders +ok 165 - expandMcpString handles combined placeholders + --- + duration_ms: 0.03825 + type: 'test' + ... +# Subtest: expandMcpConfig recursively expands objects and arrays +ok 166 - expandMcpConfig recursively expands objects and arrays + --- + duration_ms: 0.533833 + type: 'test' + ... +# Subtest: parsePluginMcpServers expands ${env:*} in stdio env +ok 167 - parsePluginMcpServers expands ${env:*} in stdio env + --- + duration_ms: 0.147541 + type: 'test' + ... +# Subtest: parsePluginMcpServers expands ${env:*} in stdio command before transport detection +ok 168 - parsePluginMcpServers expands ${env:*} in stdio command before transport detection + --- + duration_ms: 0.043208 + type: 'test' + ... +# Subtest: parsePluginMcpServers drops empty expanded stdio command +ok 169 - parsePluginMcpServers drops empty expanded stdio command + --- + duration_ms: 0.188042 + type: 'test' + ... +# Subtest: parsePluginMcpServers expands ${userHome} in stdio cwd +ok 170 - parsePluginMcpServers expands ${userHome} in stdio cwd + --- + duration_ms: 0.2305 + type: 'test' + ... +# Subtest: parsePluginMcpServers expands ~ in stdio args (backward compat) +ok 171 - parsePluginMcpServers expands ~ in stdio args (backward compat) + --- + duration_ms: 0.104541 + type: 'test' + ... +# Subtest: parsePluginMcpServers expands ${env:*} in streamable_http url +ok 172 - parsePluginMcpServers expands ${env:*} in streamable_http url + --- + duration_ms: 0.042 + type: 'test' + ... +# Subtest: parsePluginMcpServers expands ${env:*} in streamable_http headers +ok 173 - parsePluginMcpServers expands ${env:*} in streamable_http headers + --- + duration_ms: 0.036125 + type: 'test' + ... +# Subtest: user config and plugin config resolve same placeholders consistently +ok 174 - user config and plugin config resolve same placeholders consistently + --- + duration_ms: 0.0385 + type: 'test' + ... +# Subtest: Anthropic-compatible terminated SSE errors remain retryable +ok 175 - Anthropic-compatible terminated SSE errors remain retryable + --- + duration_ms: 1.873042 + type: 'test' + ... +# Subtest: catalog provider resolves api key from default env var when apiKey is omitted +ok 176 - catalog provider resolves api key from default env var when apiKey is omitted + --- + duration_ms: 0.760375 + type: 'test' + ... +# Subtest: catalog provider resolves api key from default env var when apiKey is blank +ok 177 - catalog provider resolves api key from default env var when apiKey is blank + --- + duration_ms: 0.081167 + type: 'test' + ... +# Subtest: normalizeModelError classifies common network failures +ok 178 - normalizeModelError classifies common network failures + --- + duration_ms: 2.896291 + type: 'test' + ... +# Subtest: model request failure preserves provider raw message before action guidance +ok 179 - model request failure preserves provider raw message before action guidance + --- + duration_ms: 1.718792 + type: 'test' + ... +# Subtest: model_not_found guidance points users to local model settings +ok 180 - model_not_found guidance points users to local model settings + --- + duration_ms: 0.779708 + type: 'test' + ... +# Subtest: stream idle timeout is classified as timeout with network and timeoutMs guidance +ok 181 - stream idle timeout is classified as timeout with network and timeoutMs guidance + --- + duration_ms: 0.271667 + type: 'test' + ... +# Subtest: provider terminated stream is classified as a retryable connection reset +ok 182 - provider terminated stream is classified as a retryable connection reset + --- + duration_ms: 0.192833 + type: 'test' + ... +# Subtest: billing and rate limit guidance distinguish provider-side fixes +ok 183 - billing and rate limit guidance distinguish provider-side fixes + --- + duration_ms: 0.083625 + type: 'test' + ... +# Subtest: unknown provider errors still give actionable settings and provider checks +ok 184 - unknown provider errors still give actionable settings and provider checks + --- + duration_ms: 0.040375 + type: 'test' + ... +# Subtest: OpenAI Responses usage reads cached tokens from input token details +ok 185 - OpenAI Responses usage reads cached tokens from input token details + --- + duration_ms: 0.537333 + type: 'test' + ... +# Subtest: Gemini usage counts thoughts tokens as output consumption +ok 186 - Gemini usage counts thoughts tokens as output consumption + --- + duration_ms: 0.082584 + type: 'test' + ... +# Subtest: Ollama provider uses catalog defaults and does not require apiKey +ok 187 - Ollama provider uses catalog defaults and does not require apiKey + --- + duration_ms: 0.727041 + type: 'test' + ... +# Subtest: Ollama provider builds OpenAI-compatible chat completions body +ok 188 - Ollama provider builds OpenAI-compatible chat completions body + --- + duration_ms: 0.849709 + type: 'test' + ... +# Subtest: model request builders tolerate malformed messages with missing content +ok 189 - model request builders tolerate malformed messages with missing content + --- + duration_ms: 6.588667 + type: 'test' + ... +# Subtest: cloneMessages normalizes malformed messages with missing content +ok 190 - cloneMessages normalizes malformed messages with missing content + --- + duration_ms: 1.147125 + type: 'test' + ... +# Subtest: streamModel retries Anthropic-compatible terminated SSE errors +ok 191 - streamModel retries Anthropic-compatible terminated SSE errors + --- + duration_ms: 18.157334 + type: 'test' + ... +# Subtest: networkFetch retries retryable status responses and then succeeds +ok 192 - networkFetch retries retryable status responses and then succeeds + --- + duration_ms: 5.166292 + type: 'test' + ... +# Subtest: networkFetch uses retry-after when calculating retry delay +ok 193 - networkFetch uses retry-after when calculating retry delay + --- + duration_ms: 0.158667 + type: 'test' + ... +# Subtest: networkFetch caps retry-after delays with maxDelayMs +ok 194 - networkFetch caps retry-after delays with maxDelayMs + --- + duration_ms: 0.068209 + type: 'test' + ... +# Subtest: networkFetch normalizes DNS and reset errors +ok 195 - networkFetch normalizes DNS and reset errors + --- + duration_ms: 0.193 + type: 'test' + ... +# Subtest: networkFetch times out requests +ok 196 - networkFetch times out requests + --- + duration_ms: 12.314625 + type: 'test' + ... +# Subtest: networkFetch honors init.signal abort reasons without options.signal +ok 197 - networkFetch honors init.signal abort reasons without options.signal + --- + duration_ms: 0.671625 + type: 'test' + ... +# Subtest: networkFetch preserves parent NetworkFetchError reasons passed through options.signal +ok 198 - networkFetch preserves parent NetworkFetchError reasons passed through options.signal + --- + duration_ms: 0.603834 + type: 'test' + ... +# (node:10333) ExperimentalWarning: SQLite is an experimental feature and might change at any time +# (Use `node --trace-warnings ...` to show where the warning was created) +# [session-title] generation skipped (invalid_json): Unexpected token 'Q', "QA" is not valid JSON +# Subtest: local gateway writes a complete shadow bundle linked to the final request +ok 199 - local gateway writes a complete shadow bundle linked to the final request + --- + duration_ms: 976.805875 + type: 'test' + ... +# [session-title] generation skipped (invalid_json): Unexpected token 'Q', "QA" is not valid JSON +# [session-title] generation skipped (invalid_json): Unexpected token 'Q', "QA" is not valid JSON +# Subtest: enabling O1 does not change Agent-visible model input +ok 200 - enabling O1 does not change Agent-visible model input + --- + duration_ms: 463.208792 + type: 'test' + ... +# (node:10335) ExperimentalWarning: SQLite is an experimental feature and might change at any time +# (Use `node --trace-warnings ...` to show where the warning was created) +# [session-title] generation skipped (invalid_json): Unexpected token 'B', "Bounded co"... is not valid JSON +# Subtest: local Gateway records a bounded handoff through boundary to progress +ok 201 - local Gateway records a bounded handoff through boundary to progress + --- + duration_ms: 1685.554084 + type: 'test' + ... +# (node:10336) ExperimentalWarning: SQLite is an experimental feature and might change at any time +# (Use `node --trace-warnings ...` to show where the warning was created) +# Subtest: local Gateway records a complete O1 trajectory when a bounded child times out +ok 202 - local Gateway records a complete O1 trajectory when a bounded child times out + --- + duration_ms: 1852.941459 + type: 'test' + ... +# Subtest: subagent tool detection and result pair at the model projection layer +ok 203 - subagent tool detection and result pair at the model projection layer + --- + duration_ms: 11.903083 + type: 'test' + ... +# Subtest: main tool detection and result produce one exact O1 pair +ok 204 - main tool detection and result produce one exact O1 pair + --- + duration_ms: 0.612958 + type: 'test' + ... +# Subtest: repair preparation grace is observable without becoming progress +ok 205 - repair preparation grace is observable without becoming progress + --- + duration_ms: 0.075667 + type: 'test' + ... +# Subtest: post-tool boundary deferral is independently observable and domain-bounded +ok 206 - post-tool boundary deferral is independently observable and domain-bounded + --- + duration_ms: 0.055041 + type: 'test' + ... +# Subtest: recorder finalizes a complete hash-only trajectory +ok 207 - recorder finalizes a complete hash-only trajectory + --- + duration_ms: 27.819084 + type: 'test' + ... +# Subtest: queue overflow is explicit and makes integrity partial +ok 208 - queue overflow is explicit and makes integrity partial + --- + duration_ms: 28.56875 + type: 'test' + ... +# Subtest: verifier rejects reused model request identity and duplicate terminals +ok 209 - verifier rejects reused model request identity and duplicate terminals + --- + duration_ms: 24.7175 + type: 'test' + ... +# Subtest: verifier rejects duplicate identity and secret-bearing keys +ok 210 - verifier rejects duplicate identity and secret-bearing keys + --- + duration_ms: 0.224583 + type: 'test' + ... +# [PilotDeck] transientRetry: rate_limit (attempt 1/1, delay=1ms) +# Subtest: router observation closes every fallback and retry attempt +ok 211 - router observation closes every fallback and retry attempt + --- + duration_ms: 29.239792 + type: 'test' + ... +# Subtest: router request identity separates sessions sharing a turn id +ok 212 - router request identity separates sessions sharing a turn id + --- + duration_ms: 6.893333 + type: 'test' + ... +# Subtest: router request identity advances across sequential calls in one turn +ok 213 - router request identity advances across sequential calls in one turn + --- + duration_ms: 8.74925 + type: 'test' + ... +# Subtest: observability is disabled by default and preserves the O1 diagnostic profile +ok 214 - observability is disabled by default and preserves the O1 diagnostic profile + --- + duration_ms: 43.191167 + type: 'test' + ... +# Subtest: O1 rejects unsupported profiles and unsafe labels +ok 215 - O1 rejects unsupported profiles and unsafe labels + --- + duration_ms: 7.753083 + type: 'test' + ... +# Subtest: web search can be explicitly disabled without discarding provider config +ok 216 - web search can be explicitly disabled without discarding provider config + --- + duration_ms: 1.454458 + type: 'test' + ... +# Subtest: web search enabled remains optional for backwards compatibility +ok 217 - web search enabled remains optional for backwards compatibility + --- + duration_ms: 0.060416 + type: 'test' + ... +# Subtest: web search enabled must be a boolean +ok 218 - web search enabled must be a boolean + --- + duration_ms: 0.053625 + type: 'test' + ... +# Subtest: progress lease is disabled by default and opt-in evaluation config is preserved +ok 219 - progress lease is disabled by default and opt-in evaluation config is preserved + --- + duration_ms: 21.834417 + type: 'test' + ... +# Subtest: progress lease preserves an explicit cold-start allowance +ok 220 - progress lease preserves an explicit cold-start allowance + --- + duration_ms: 1.507083 + type: 'test' + ... +# Subtest: progress lease rejects non-evaluation modes +ok 221 - progress lease rejects non-evaluation modes + --- + duration_ms: 6.940667 + type: 'test' + ... +# Subtest: progress lease rejects a cold-start allowance below the steady-state limit +ok 222 - progress lease rejects a cold-start allowance below the steady-state limit + --- + duration_ms: 4.306333 + type: 'test' + ... +# Subtest: legal coverage validator creates a current proof and removes it when the deliverable changes +ok 223 - legal coverage validator creates a current proof and removes it when the deliverable changes + --- + duration_ms: 472.254 + type: 'test' + ... +# Subtest: legal coverage initializer creates a text skeleton before source review and remains idempotent +ok 224 - legal coverage initializer creates a text skeleton before source review and remains idempotent + --- + duration_ms: 384.280625 + type: 'test' + ... +# Subtest: legal coverage initializer binds trusted manifests to original inputs +ok 225 - legal coverage initializer binds trusted manifests to original inputs + --- + duration_ms: 477.36375 + type: 'test' + ... +# Subtest: legal coverage source bootstrap creates only deterministic pending manifest rows +ok 226 - legal coverage source bootstrap creates only deterministic pending manifest rows + --- + duration_ms: 445.925291 + type: 'test' + ... +# Subtest: legal coverage injects deterministic disjoint worker batches for large pending source rooms +ok 227 - legal coverage injects deterministic disjoint worker batches for large pending source rooms + --- + duration_ms: 6896.769291 + type: 'test' + ... +# Subtest: legal coverage CLI exposes bundled guidance through stable named references +ok 228 - legal coverage CLI exposes bundled guidance through stable named references + --- + duration_ms: 152.118459 + type: 'test' + ... +# Subtest: legal coverage initializer creates only explicit text formats and preserves existing content +ok 229 - legal coverage initializer creates only explicit text formats and preserves existing content + --- + duration_ms: 58.032416 + type: 'test' + ... +# Subtest: legal coverage initializer rejects traversal and symlink ancestors without external writes +ok 230 - legal coverage initializer rejects traversal and symlink ancestors without external writes + --- + duration_ms: 391.294542 + type: 'test' + ... +# Subtest: legal coverage validator binds runner originals and derivations into its proof +ok 231 - legal coverage validator binds runner originals and derivations into its proof + --- + duration_ms: 250.527042 + type: 'test' + ... +# Subtest: legal coverage validator rejects missing lineage and mutable or derived input roots +ok 232 - legal coverage validator rejects missing lineage and mutable or derived input roots + --- + duration_ms: 302.683417 + type: 'test' + ... +# Subtest: legal coverage next-batch exposes one bounded deterministic repair slice +ok 233 - legal coverage next-batch exposes one bounded deterministic repair slice + --- + duration_ms: 230.945375 + type: 'test' + ... +# Subtest: legal coverage apply-batch atomically updates only the current bounded slice +ok 234 - legal coverage apply-batch atomically updates only the current bounded slice + --- + duration_ms: 335.899041 + type: 'test' + ... +# Subtest: legal coverage apply-batch rejects record and byte limit violations before writing +ok 235 - legal coverage apply-batch rejects record and byte limit violations before writing + --- + duration_ms: 217.33875 + type: 'test' + ... +# Subtest: legal coverage validator rejects orphaned conflicts and incomplete final disclosure +ok 236 - legal coverage validator rejects orphaned conflicts and incomplete final disclosure + --- + duration_ms: 114.81825 + type: 'test' + ... +# Subtest: legal coverage validator detects un-inventoried sources and cross-fact timeline collisions +ok 237 - legal coverage validator detects un-inventoried sources and cross-fact timeline collisions + --- + duration_ms: 119.736958 + type: 'test' + ... +# Subtest: legal coverage validator rejects reused generic quotes and unsupported fact coverage +ok 238 - legal coverage validator rejects reused generic quotes and unsupported fact coverage + --- + duration_ms: 114.2025 + type: 'test' + ... +# Subtest: legal coverage validator requires authority links for critical issues and legal-authority matrices +ok 239 - legal coverage validator requires authority links for critical issues and legal-authority matrices + --- + duration_ms: 116.535208 + type: 'test' + ... +# Subtest: legal coverage validator rejects non-object canonical JSON documents +ok 240 - legal coverage validator rejects non-object canonical JSON documents + --- + duration_ms: 775.704208 + type: 'test' + ... +# Subtest: legal coverage validator binds reviewed source bytes to the ledger and state hash +ok 241 - legal coverage validator binds reviewed source bytes to the ledger and state hash + --- + duration_ms: 228.496167 + type: 'test' + ... +# Subtest: legal coverage validator rejects ancestor symlinks and a symlinked proof +ok 242 - legal coverage validator rejects ancestor symlinks and a symlinked proof + --- + duration_ms: 252.17625 + type: 'test' + ... +# Subtest: legal coverage validator does not read or write through a symlinked state ancestor +ok 243 - legal coverage validator does not read or write through a symlinked state ancestor + --- + duration_ms: 120.840708 + type: 'test' + ... +# Subtest: legal coverage validator requires unresolved disclosure for unverified material facts +ok 244 - legal coverage validator requires unresolved disclosure for unverified material facts + --- + duration_ms: 116.507875 + type: 'test' + ... +# Subtest: legal coverage validator rejects unique but irrelevant issue and authority quotes +ok 245 - legal coverage validator rejects unique but irrelevant issue and authority quotes + --- + duration_ms: 119.680542 + type: 'test' + ... +# Subtest: legal coverage validator enforces reciprocal and same-entry ledger relationships +ok 246 - legal coverage validator enforces reciprocal and same-entry ledger relationships + --- + duration_ms: 115.977 + type: 'test' + ... +# Subtest: legal coverage validator accepts null for an optional threshold assessment +ok 247 - legal coverage validator accepts null for an optional threshold assessment + --- + duration_ms: 119.171917 + type: 'test' + ... +# Subtest: legal coverage validator uses locators without quotes for binary deliverables +ok 248 - legal coverage validator uses locators without quotes for binary deliverables + --- + duration_ms: 116.352875 + type: 'test' + ... +# Subtest: legal coverage validator rejects empty-fact shortcuts and material facts outside matrices +ok 249 - legal coverage validator rejects empty-fact shortcuts and material facts outside matrices + --- + duration_ms: 228.331167 + type: 'test' + ... +# Subtest: legal coverage hook activates only legal work and injects one observable milestone +ok 250 - legal coverage hook activates only legal work and injects one observable milestone + --- + duration_ms: 595.177167 + type: 'test' + ... +# Subtest: legal coverage hook groups repeated validator errors into one bounded milestone +ok 251 - legal coverage hook groups repeated validator errors into one bounded milestone + --- + duration_ms: 588.061542 + type: 'test' + ... +# Subtest: legal coverage pending-matrix selection is bounded across pages and invalid revisions cannot manufacture progress +ok 252 - legal coverage pending-matrix selection is bounded across pages and invalid revisions cannot manufacture progress + --- + duration_ms: 1081.491208 + type: 'test' + ... +# Subtest: legal coverage progress ordinal advances once for a new legal phase +ok 253 - legal coverage progress ordinal advances once for a new legal phase + --- + duration_ms: 232.90525 + type: 'test' + ... +# Subtest: legal coverage permits not-applicable only after exhaustive selection and rejects stale or changed matrix proposals +ok 254 - legal coverage permits not-applicable only after exhaustive selection and rejects stale or changed matrix proposals + --- + duration_ms: 520.601 + type: 'test' + ... +# Subtest: legal coverage rejects oversized matrix evidence before making a selection receipt immutable +ok 255 - legal coverage rejects oversized matrix evidence before making a selection receipt immutable + --- + duration_ms: 178.593917 + type: 'test' + ... +# Subtest: legal coverage fails closed when one matrix index item exceeds the bounded page +ok 256 - legal coverage fails closed when one matrix index item exceeds the bounded page + --- + duration_ms: 194.366875 + type: 'test' + ... +# Subtest: legal coverage injects a bounded deterministic matrix relation-closure batch +ok 257 - legal coverage injects a bounded deterministic matrix relation-closure batch + --- + duration_ms: 239.511958 + type: 'test' + ... +# Subtest: legal coverage authority closure uses one bounded state-bound transaction +ok 258 - legal coverage authority closure uses one bounded state-bound transaction + --- + duration_ms: 700.367 + type: 'test' + ... +# Subtest: legal coverage risk-signal issue closure exposes one bounded state-bound transaction +ok 259 - legal coverage risk-signal issue closure exposes one bounded state-bound transaction + --- + duration_ms: 949.249375 + type: 'test' + ... +# Subtest: legal coverage rejects changed and symlinked issue closure proposals without ledger mutation +ok 260 - legal coverage rejects changed and symlinked issue closure proposals without ledger mutation + --- + duration_ms: 331.433417 + type: 'test' + ... +# Subtest: legal coverage rejects changed and symlinked authority closure proposals without ledger mutation +ok 261 - legal coverage rejects changed and symlinked authority closure proposals without ledger mutation + --- + duration_ms: 280.793 + type: 'test' + ... +# Subtest: legal coverage milestone digest ignores opaque incomplete-state hash churn +ok 262 - legal coverage milestone digest ignores opaque incomplete-state hash churn + --- + duration_ms: 0.259083 + type: 'test' + ... +# Subtest: legal coverage hook activates a malformed configured workspace so the agent can repair it +ok 263 - legal coverage hook activates a malformed configured workspace so the agent can repair it + --- + duration_ms: 113.5185 + type: 'test' + ... +# Subtest: legal coverage hook rejects a symlinked session-state ancestor without writing outside the workspace +ok 264 - legal coverage hook rejects a symlinked session-state ancestor without writing outside the workspace + --- + duration_ms: 56.635541 + type: 'test' + ... +# Subtest: legal product plugin loads one skill and contains no benchmark-specific controls +ok 265 - legal product plugin loads one skill and contains no benchmark-specific controls + --- + duration_ms: 3.943792 + type: 'test' + ... +# Subtest: file artifacts include every meaningful workspace change without an extension allowlist +ok 266 - file artifacts include every meaningful workspace change without an extension allowlist + --- + duration_ms: 39.565167 + type: 'test' + ... +# Subtest: file artifact fingerprints are reused for unchanged files across scans and turns +ok 267 - file artifact fingerprints are reused for unchanged files across scans and turns + --- + duration_ms: 6.867875 + type: 'test' + ... +# Subtest: TurnRunner emits and persists file artifacts before completing the turn +ok 268 - TurnRunner emits and persists file artifacts before completing the turn + --- + duration_ms: 36.0885 + type: 'test' + ... +# Subtest: TurnRunner does not collect generated files when artifacts are disabled +ok 269 - TurnRunner does not collect generated files when artifacts are disabled + --- + duration_ms: 13.011125 + type: 'test' + ... +# Subtest: bash success result is formatted with assertions, stdout, and stderr +ok 270 - bash success result is formatted with assertions, stdout, and stderr + --- + duration_ms: 9.538208 + type: 'test' + ... +# Subtest: bash allows hyphenated commands whose names merely start with next +ok 271 - bash allows hyphenated commands whose names merely start with next + --- + duration_ms: 3.294959 + type: 'test' + ... +# Subtest: PostToolUse carries bounded convergence previews only in internal lifecycle metadata +ok 272 - PostToolUse carries bounded convergence previews only in internal lifecycle metadata + --- + duration_ms: 1.561959 + type: 'test' + ... +# Subtest: bash still rejects exact framework CLI tokens that start long-lived processes +ok 273 - bash still rejects exact framework CLI tokens that start long-lived processes + --- + duration_ms: 1.407166 + type: 'test' + ... +# Subtest: bash failure tool result includes raw stdout and stderr tail for UI and model +ok 274 - bash failure tool result includes raw stdout and stderr tail for UI and model + --- + duration_ms: 51.372084 + type: 'test' + ... +# Subtest: bash failure diagnostic extracts Python traceback location and exception +ok 275 - bash failure diagnostic extracts Python traceback location and exception + --- + duration_ms: 1.66725 + type: 'test' + ... +# Subtest: bash success keeps full output for ToolResultBudget persistence +ok 276 - bash success keeps full output for ToolResultBudget persistence + --- + duration_ms: 36.842792 + type: 'test' + ... +# Subtest: agent tool accepts explorer as an alias for explore +ok 277 - agent tool accepts explorer as an alias for explore + --- + duration_ms: 4.026708 + type: 'test' + ... +# Subtest: agent tool defaults general-purpose to explore in ask mode +ok 278 - agent tool defaults general-purpose to explore in ask mode + --- + duration_ms: 1.986875 + type: 'test' + ... +# Subtest: agent tool injects and forwards the parent-bounded timeout +ok 279 - agent tool injects and forwards the parent-bounded timeout + --- + duration_ms: 0.947666 + type: 'test' + ... +# Subtest: agent tool rejects a launch that would consume the parent handoff window +ok 280 - agent tool rejects a launch that would consume the parent handoff window + --- + duration_ms: 0.293667 + type: 'test' + ... +# Subtest: agent fallback path enforces the same timeout +ok 281 - agent fallback path enforces the same timeout + --- + duration_ms: 13.069083 + type: 'test' + ... +# Subtest: agent tool preserves unknown custom fallback subagent names +ok 282 - agent tool preserves unknown custom fallback subagent names + --- + duration_ms: 0.3245 + type: 'test' + ... +# Subtest: execute_code read-only probe handles missing input +ok 283 - execute_code read-only probe handles missing input + --- + duration_ms: 0.536542 + type: 'test' + ... +# Subtest: disabling web search removes it from the registry but keeps web fetch +ok 284 - disabling web search removes it from the registry but keeps web fetch + --- + duration_ms: 1.090792 + type: 'test' + ... +# Subtest: execute_code rejects nested web search calls when web search is disabled +ok 285 - execute_code rejects nested web search calls when web search is disabled + --- + duration_ms: 0.123375 + type: 'test' + ... +# Subtest: read_skill returns the resolved SKILL.md path with the skill body +ok 286 - read_skill returns the resolved SKILL.md path with the skill body + --- + duration_ms: 0.993792 + type: 'test' + ... +# Subtest: read_skill preserves legacy content-only loading when metadata is unavailable +ok 287 - read_skill preserves legacy content-only loading when metadata is unavailable + --- + duration_ms: 0.164542 + type: 'test' + ... +# Subtest: web_search retries transient provider failures +ok 288 - web_search retries transient provider failures + --- + duration_ms: 631.077875 + type: 'test' + ... +# Subtest: web_search turns request timeout into tool_timeout +ok 289 - web_search turns request timeout into tool_timeout + --- + duration_ms: 3.119 + type: 'test' + ... +# Subtest: web_search turns network timeout errors into tool_timeout +ok 290 - web_search turns network timeout errors into tool_timeout + --- + duration_ms: 1.137834 + type: 'test' + ... +# Subtest: write_file freshness error tells the model to read the file first +ok 291 - write_file freshness error tells the model to read the file first + --- + duration_ms: 0.963042 + type: 'test' + ... +# Subtest: write_file missing content points at the missing field and chunked recovery +ok 292 - write_file missing content points at the missing field and chunked recovery + --- + duration_ms: 0.242459 + type: 'test' + ... +# Subtest: invalid tool input keeps a bounded original error block +ok 293 - invalid tool input keeps a bounded original error block + --- + duration_ms: 0.197709 + type: 'test' + ... +# Subtest: edit_file old_string miss tells the model to reread and copy exact text +ok 294 - edit_file old_string miss tells the model to reread and copy exact text + --- + duration_ms: 0.1225 + type: 'test' + ... +# Subtest: read_file invalid range preserves the concrete issue +ok 295 - read_file invalid range preserves the concrete issue + --- + duration_ms: 0.170042 + type: 'test' + ... +# Subtest: bash missing command reports the required command parameter +ok 296 - bash missing command reports the required command parameter + --- + duration_ms: 0.132875 + type: 'test' + ... +# Subtest: bash timeout over max keeps foreground and task_wait guidance +ok 297 - bash timeout over max keeps foreground and task_wait guidance + --- + duration_ms: 0.145333 + type: 'test' + ... +# Subtest: bash background rejection keeps background-specific guidance +ok 298 - bash background rejection keeps background-specific guidance + --- + duration_ms: 0.085083 + type: 'test' + ... +# Subtest: write_file can create a new file without a prior read_file call +ok 299 - write_file can create a new file without a prior read_file call + --- + duration_ms: 7.767209 + type: 'test' + ... +# Subtest: edit_file can create a new file with empty old_string without a prior read_file call +ok 300 - edit_file can create a new file with empty old_string without a prior read_file call + --- + duration_ms: 4.980709 + type: 'test' + ... +# Subtest: read_file auto-pages large text files instead of failing +ok 301 - read_file auto-pages large text files instead of failing + --- + duration_ms: 2518.178333 + type: 'test' + ... +# Subtest: read_file rejects Office container files during validation +ok 302 - read_file rejects Office container files during validation + --- + duration_ms: 0.97825 + type: 'test' + ... +# Subtest: read_file explicit limit reads a large file range without auto paging +ok 303 - read_file explicit limit reads a large file range without auto paging + --- + duration_ms: 5.733292 + type: 'test' + ... +# Subtest: read_file auto-shrinks oversized persisted tool-result ref ranges +ok 304 - read_file auto-shrinks oversized persisted tool-result ref ranges + --- + duration_ms: 25396.376792 + type: 'test' + ... +# Subtest: read_file keeps explicit oversized ordinary file ranges strict +ok 305 - read_file keeps explicit oversized ordinary file ranges strict + --- + duration_ms: 2.913542 + type: 'test' + ... +# Subtest: read_file explicit limit records a ranged snapshot for follow-up edits +ok 306 - read_file explicit limit records a ranged snapshot for follow-up edits + --- + duration_ms: 2.549375 + type: 'test' + ... +# Subtest: read_file auto-paged large files record a ranged snapshot for follow-up edits +ok 307 - read_file auto-paged large files record a ranged snapshot for follow-up edits + --- + duration_ms: 1664.990084 + type: 'test' + ... +# Subtest: read_file returns a head-tail preview for a single oversized line +ok 308 - read_file returns a head-tail preview for a single oversized line + --- + duration_ms: 2.80875 + type: 'test' + ... +# Subtest: read_file classifies common Office formats as binary +ok 309 - read_file classifies common Office formats as binary + --- + duration_ms: 0.254708 + type: 'test' + ... +# Subtest: read_file rejects ranged binary input instead of hanging +ok 310 - read_file rejects ranged binary input instead of hanging + --- + duration_ms: 1.054292 + type: 'test' + ... +# Subtest: read_file range honors an aborted signal +ok 311 - read_file range honors an aborted signal + --- + duration_ms: 0.730167 + type: 'test' + ... +# Subtest: glob result text tells the model when more files are available +ok 312 - glob result text tells the model when more files are available + --- + duration_ms: 56.064 + type: 'test' + ... +# Subtest: grep result text includes next offset when paginated +ok 313 - grep result text includes next offset when paginated + --- + duration_ms: 21.233291 + type: 'test' + ... +# Subtest: todo_write returns the actual todo list in model-visible content +ok 314 - todo_write returns the actual todo list in model-visible content + --- + duration_ms: 0.6795 + type: 'test' + ... +# Subtest: task_list returns model-visible status and next action hints +ok 315 - task_list returns model-visible status and next action hints + --- + duration_ms: 1.9425 + type: 'test' + ... +# Subtest: applyResultSizeLimit keeps both head and tail when truncating text output +ok 316 - applyResultSizeLimit keeps both head and tail when truncating text output + --- + duration_ms: 0.705042 + type: 'test' + ... +# Subtest: large tool results are persisted under workspace .pilotdeck and readable by read_file +ok 317 - large tool results are persisted under workspace .pilotdeck and readable by read_file + --- + duration_ms: 13.420458 + type: 'test' + ... +# Subtest: large tool result read_file aliases are short and sequential +ok 318 - large tool result read_file aliases are short and sequential + --- + duration_ms: 5.453292 + type: 'test' + ... +# Subtest: history replay restores structured agent file artifacts +ok 319 - history replay restores structured agent file artifacts + --- + duration_ms: 10.736042 + type: 'test' + ... +# Subtest: history replay hides Agent file artifacts in general conversations +ok 320 - history replay hides Agent file artifacts in general conversations + --- + duration_ms: 3.847958 + type: 'test' + ... +# Subtest: history replay preserves agent status i18n metadata and user hint +ok 321 - history replay preserves agent status i18n metadata and user hint + --- + duration_ms: 12.113875 + type: 'test' + ... +# Subtest: history token usage restores latest non-empty turn past latest empty turn result +ok 322 - history token usage restores latest non-empty turn past latest empty turn result + --- + duration_ms: 6.053291 + type: 'test' + ... +# Subtest: history token usage prefers persisted context budget snapshot +ok 323 - history token usage prefers persisted context budget snapshot + --- + duration_ms: 4.080333 + type: 'test' + ... +# Subtest: web reducer merges persisted tool result detail path into existing tool result +ok 324 - web reducer merges persisted tool result detail path into existing tool result + --- + duration_ms: 1.3965 + type: 'test' + ... +# Subtest: web reducer bounds huge live tool result previews +ok 325 - web reducer bounds huge live tool result previews + --- + duration_ms: 0.188208 + type: 'test' + ... +1..325 +# tests 325 +# suites 0 +# pass 325 +# fail 0 +# cancelled 0 +# skipped 0 +# todo 0 +# duration_ms 32545.227041