Skip to content

Commit 222ba07

Browse files
authored
fix(api): attribute a polled run's failure only to the block that caused it (#9916)
* fix(api): attribute a polled run's failure only to the block that caused it A run's stored error was paired with any failed block in its trace when the message classified as a generic failure, so a run-level fault or a branch that failed after the settling error reported an unrelated block. The trace read now names a block only when its name heads the run error or its recorded error is exactly it, which keeps unprefixed loop and parallel failures attributed. * fix(cli-auth): say when a requested key never expires
1 parent b4bee71 commit 222ba07

8 files changed

Lines changed: 99 additions & 29 deletions

File tree

‎apps/docs/content/docs/api-reference/build-and-run-workflows.mdx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -259,7 +259,7 @@ Run failures are reported in the result, not as HTTP errors: branch on `status`
259259

260260
## 10. Debug a failed run [#debugging]
261261

262-
A synchronous execute response names the failing block in `error.blockId`, `error.blockName`, and `error.blockType`. The polled run resource cannot attribute a block; read the run's trace spans instead:
262+
Both the execute response and the polled run resource name the failing block in `error.blockId`, `error.blockName`, and `error.blockType`. The run resource reads these from the run's trace, so they are absent when the trace is unavailable or hidden by your organization's permission groups, when no failed block in the trace recorded the run error, and for a timeout, cancellation, usage-limit, or invalid-input failure. Read the trace spans for the full picture:
263263

264264
<Tabs items={['curl', 'CLI']}>
265265
<Tab value="curl">

‎apps/docs/openapi-v2-workflows.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15539,7 +15539,7 @@
1553915539
"description": "Stable machine-readable execution failure code. `BLOCK_EXECUTION_FAILED` and `CHILD_WORKFLOW_FAILED` are reported only where block attribution is available; elsewhere a block-level failure is reported as `EXECUTION_FAILED`."
1554015540
},
1554115541
"blockId": {
15542-
"description": "Identifier of the failing block, when attributable. Get workflow run reads it from the run trace, so it is absent there when the trace is unavailable or hidden by organization permission groups, and for a timeout, cancellation, usage-limit, or invalid-input failure.",
15542+
"description": "Identifier of the failing block, when attributable. Get workflow run reads it from the run trace, so it is absent there when the trace is unavailable or hidden by organization permission groups, when no failed block in the trace recorded the run error, and for a timeout, cancellation, usage-limit, or invalid-input failure.",
1554315543
"type": "string"
1554415544
},
1554515545
"blockName": {

‎apps/sim/app/cli/auth/cli-auth-view.tsx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -101,7 +101,7 @@ export function CliAuthView() {
101101
*/
102102
const keyLifetime =
103103
request.expiresInDays === null
104-
? ''
104+
? ' that never expires'
105105
: ` that expires in ${request.expiresInDays} ${request.expiresInDays === 1 ? 'day' : 'days'}`
106106
const keyDescription = `${request.access === 'read' ? 'a read-only' : 'a'} personal key tied to your account${keyLifetime}`
107107

‎apps/sim/executor/utils/errors.test.ts‎

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,8 @@
11
import { describe, expect, it } from 'vitest'
22
import { v2ExecutionErrorSchema } from '@/lib/api/contracts/v2/workflows'
33
import { HttpError } from '@/lib/core/utils/http-error'
4+
import { findFailedTraceBlock } from '@/lib/logs/execution/trace-spans/trace-spans'
5+
import type { TraceSpan } from '@/lib/logs/types'
46
import type { ExecutionResult } from '@/executor/types'
57
import {
68
buildBlockExecutionError,
@@ -274,6 +276,65 @@ describe('classifyRecordedExecutionError', () => {
274276
})
275277
})
276278

279+
/** A recorded trace holding one failed block span, as a polled run reads it back. */
280+
const traceWithFailure = (failed: Partial<TraceSpan>): TraceSpan[] => [
281+
{
282+
id: 'workflow-execution',
283+
name: 'Workflow Execution',
284+
type: 'workflow',
285+
duration: 1,
286+
startTime: '2024-01-01T10:00:00.000Z',
287+
endTime: '2024-01-01T10:00:01.000Z',
288+
status: 'error',
289+
children: [
290+
{
291+
id: 'failed-span',
292+
name: 'Failed',
293+
type: 'function',
294+
duration: 1,
295+
startTime: '2024-01-01T10:00:00.000Z',
296+
endTime: '2024-01-01T10:00:01.000Z',
297+
status: 'error',
298+
...failed,
299+
},
300+
],
301+
},
302+
]
303+
const classifyFromTrace = (runError: string, failed: Partial<TraceSpan>) =>
304+
classifyRecordedExecutionError(
305+
runError,
306+
findFailedTraceBlock(traceWithFailure(failed), runError)
307+
)
308+
309+
it('attributes a loop failure, recorded without a name prefix, to the loop', () => {
310+
const runError = 'ForEach loop resolution failed: items is not an array'
311+
expect(
312+
classifyFromTrace(runError, {
313+
blockId: 'loop-1',
314+
name: 'Each lead',
315+
type: 'loop',
316+
output: { error: runError },
317+
})
318+
).toEqual({
319+
message: runError,
320+
code: 'BLOCK_EXECUTION_FAILED',
321+
blockId: 'loop-1',
322+
blockName: 'Each lead',
323+
blockType: 'loop',
324+
})
325+
})
326+
327+
it('does not attribute a run error to a failed block that did not record it', () => {
328+
expect(
329+
classifyFromTrace('Service unavailable', {
330+
blockId: 'block-1',
331+
name: 'Lookup',
332+
type: 'api',
333+
output: { error: 'upstream returned 500' },
334+
})
335+
).toEqual({ message: 'Service unavailable', code: 'EXECUTION_FAILED' })
336+
})
337+
277338
it('reports EXECUTION_FAILED when the trace names no block', () => {
278339
expect(classifyRecordedExecutionError('Lookup: upstream returned 500', null)).toEqual({
279340
message: 'Lookup: upstream returned 500',

‎apps/sim/executor/utils/errors.ts‎

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -376,11 +376,12 @@ export function classifyExecutionError(
376376

377377
/**
378378
* Classifies a run's persisted error message, attributing it to the failing
379-
* block its recorded trace names. A message headed by that block's name is the
380-
* block's own failure and is classified with the block attached, whatever the
381-
* name says. A message that does not name it stays unattributed when its text
382-
* classifies as run-level (a timeout, cancellation, usage limit, or invalid
383-
* input), and without a block this reports what the message alone supports.
379+
* block its recorded trace names as the owner of that message (see
380+
* `findFailedTraceBlock`). A message headed by that block's name is the block's
381+
* own failure and is classified with the block attached, whatever the name says.
382+
* A message that does not name it stays unattributed when its text classifies as
383+
* run-level (a timeout, cancellation, usage limit, or invalid input), and without
384+
* a block this reports what the message alone supports.
384385
*/
385386
export function classifyRecordedExecutionError(
386387
message: string,

‎apps/sim/lib/api/contracts/v2/workflows.ts‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1550,8 +1550,9 @@ export const v2ExecutionErrorSchema = z
15501550
* The synchronous execute and resume responses attribute the block at
15511551
* execution time. The polled run resource keeps only the error *string*, so
15521552
* it reads the block back from the run's recorded trace: the unhandled
1553-
* failure the stored error names, else the one that ended first, and none
1554-
* when concurrent failures cannot be told apart. When that trace is unavailable (not
1553+
* failure that recorded the stored error (named by it, or with the same
1554+
* text), the one that ended first among several, and none when they cannot
1555+
* be told apart or no failed block recorded it. When that trace is unavailable (not
15551556
* recorded, compacted, or withheld by the viewer's permission group) the
15561557
* three fields are absent and a block-level failure reports
15571558
* `EXECUTION_FAILED`. A failure its message classifies as run-level
@@ -1562,7 +1563,7 @@ export const v2ExecutionErrorSchema = z
15621563
.string()
15631564
.optional()
15641565
.describe(
1565-
'Identifier of the failing block, when attributable. Get workflow run reads it from the run trace, so it is absent there when the trace is unavailable or hidden by organization permission groups, and for a timeout, cancellation, usage-limit, or invalid-input failure.'
1566+
'Identifier of the failing block, when attributable. Get workflow run reads it from the run trace, so it is absent there when the trace is unavailable or hidden by organization permission groups, when no failed block in the trace recorded the run error, and for a timeout, cancellation, usage-limit, or invalid-input failure.'
15661567
),
15671568
blockName: z
15681569
.string()

‎apps/sim/lib/logs/execution/trace-spans/trace-spans.test.ts‎

Lines changed: 3 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1546,14 +1546,11 @@ describe('findFailedTraceBlock', () => {
15461546
expect(findFailedTraceBlock(concurrentFailures(), 'Agent B: timeout')?.blockId).toBe('b')
15471547
})
15481548

1549-
it('falls back to the failure that ended first when the error names no block', () => {
1550-
expect(findFailedTraceBlock(concurrentFailures(), 'upstream returned 500')?.blockId).toBe('a')
1551-
})
1552-
15531549
it('names no block when concurrent failures cannot be told apart', () => {
1550+
const output = { error: 'upstream returned 500' }
15541551
const spans = root([
1555-
span({ blockId: 'a', name: 'Agent A', status: 'error' }),
1556-
span({ blockId: 'b', name: 'Agent B', status: 'error' }),
1552+
span({ blockId: 'a', name: 'Agent A', status: 'error', output }),
1553+
span({ blockId: 'b', name: 'Agent B', status: 'error', output }),
15571554
])
15581555

15591556
expect(findFailedTraceBlock(spans, 'upstream returned 500')).toBeNull()

‎apps/sim/lib/logs/execution/trace-spans/trace-spans.ts‎

Lines changed: 22 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -231,6 +231,8 @@ export interface FailedTraceBlock {
231231
interface FailedSpanCandidate {
232232
block: FailedTraceBlock
233233
endedAt: number
234+
/** The block log's error, which the span builder merges into its output. */
235+
recordedError: string | undefined
234236
}
235237

236238
/** The innermost unhandled-error block spans: a container whose failure came from a block inside it yields that block. */
@@ -252,6 +254,10 @@ function collectFailedBlockSpans(nodes: unknown, into: FailedSpanCandidate[]): v
252254
blockType,
253255
},
254256
endedAt: typeof span.endTime === 'string' ? Date.parse(span.endTime) : Number.NaN,
257+
recordedError:
258+
isRecordLike(span.output) && typeof span.output.error === 'string'
259+
? span.output.error
260+
: undefined,
255261
})
256262
}
257263
}
@@ -260,11 +266,14 @@ function collectFailedBlockSpans(nodes: unknown, into: FailedSpanCandidate[]): v
260266
/**
261267
* The block a recorded run failed in, read back from its persisted trace spans.
262268
*
263-
* A run stores the error of the failure that settled first, while branches already
264-
* in flight may fail after it. So the span whose block name heads the stored error
265-
* (`Block name: message`, as a block failure is recorded) is the one named; failing
266-
* that, the failure that ended first; and when neither singles one out, none, so a
267-
* message is never paired with another block's identity.
269+
* Given the run's stored error, only a block that owns it is named: one whose name
270+
* heads it (`Block name: message`, as a block failure is recorded) or whose own
271+
* recorded error is exactly it, as a loop or parallel failure is recorded without a
272+
* prefix. A run stores the error of the failure that settled first, while branches
273+
* already in flight may fail after it, and a run can fail outside any block while
274+
* one is failing; so a failed block that does not own the error is never named, and
275+
* among several owners the one that ended first is, or none when they cannot be told
276+
* apart. Without a run error, the failure that ended first is named.
268277
*
269278
* Spans are walked as {@link hasUnhandledError} walks them, including under a block
270279
* that succeeded, and the same boundaries hold: a successful boundary block's
@@ -273,17 +282,18 @@ function collectFailedBlockSpans(nodes: unknown, into: FailedSpanCandidate[]): v
273282
* block that ran it, as it is at execution time, and a custom block's source workflow
274283
* is never named to its consumer.
275284
*
276-
* Reads stored, untyped spans structurally and never their input or output. Null when
277-
* the trace has no such span, including when it was not recorded, was compacted, or is
278-
* unavailable.
285+
* Reads stored, untyped spans structurally, and of their output only the recorded
286+
* error. Null when the trace has no such span, including when it was not recorded,
287+
* was compacted, or is unavailable.
279288
*/
280289
export function findFailedTraceBlock(spans: unknown, runError?: string): FailedTraceBlock | null {
281290
const candidates: FailedSpanCandidate[] = []
282291
collectFailedBlockSpans(spans, candidates)
283-
const named = runError
284-
? candidates.filter((span) => errorNamesBlock(runError, span.block.blockName))
285-
: []
286-
const pool = named.length > 0 ? named : candidates
292+
const pool = runError
293+
? candidates.filter(
294+
(span) => errorNamesBlock(runError, span.block.blockName) || span.recordedError === runError
295+
)
296+
: candidates
287297
if (pool.length <= 1) return pool[0]?.block ?? null
288298
if (!pool.every((span) => Number.isFinite(span.endedAt))) return null
289299
const [first, second] = pool.sort((a, b) => a.endedAt - b.endedAt)

0 commit comments

Comments
 (0)