Repository navigation
feat(framework): emit step boundaries instead of inferring them #663
Description
Outcome
Step boundaries are a fact the framework emits, rather than something inferred from a provider attribute that only one tool has and that it redacts.
Scope
Two measured problems share this one answer.
Only Claude Code marks steps, and badly. Its skill_activated event carries the real skill name — but only with OTEL_LOG_TOOL_DETAILS=1, which also logs Bash commands and tool inputs. And skill.name is sticky: once activated it rides every following datapoint, including subagents launched afterwards, so interleaved skills are attributed to whichever ran last. None of Codex, Cursor, Copilot or OpenCode emits any equivalent at all, yet #656 promises a per-step breakdown across tools.
The framework already knows. A skill starting and finishing is something the framework observes directly, on every tool, without asking a vendor and without a redaction flag.
- Includes: a step-boundary record written on skill start and skill end, correlatable with the run journal.
- Includes: nesting, since a skill can invoke another, and an unterminated step must be distinguishable from one still running.
- Includes: correlation keys that let a reader attach provider telemetry to the interval, per tool.
- Excludes: replacing
skill_activatedon Claude Code where it is available and cheaper. Both may coexist, and disagreement between them is a signal worth reading. - Excludes: any prompt or tool content.
What this unlocks
Per-step cost stops depending on OTEL_LOG_TOOL_DETAILS, which removes the privacy trade that currently forces collector-side redaction just to see a skill name. Interleaved skills become attributable. And the four remaining tools gain a breakdown they can never get from their own exports.
Done When
- A step boundary is recorded on every supported tool, including those with no vendor equivalent.
- Two interleaved skills produce two distinct attributions rather than one.
- A skill interrupted or crashed leaves a step distinguishable from one still running.
- Where both exist, framework boundaries and
skill_activatedagree; disagreement is surfaced, not hidden. - Per-step cost is obtainable on Claude Code without
OTEL_LOG_TOOL_DETAILS.
Completion Evidence
A task whose per-step cost is identical whether computed from framework boundaries or from the provider event, and a second one with interleaved skills where only the framework boundaries are correct.
Relations
| Field | Value |
|---|---|
| parent | #652 |
| related | #629, #653, #647 |
Activity
blafourcade commentedon Aug 16, 2026
Moved into milestone 14, 2026-08-16
Decision: the framework emits its own step boundaries in v1, rather than inferring them from a provider attribute.
Why this earns its place in a tight week
It does not add a sixth problem, it removes three.
OTEL_LOG_TOOL_DETAILS, which also logs Bash commands and tool inputsThe provider event stays useful where it exists: comparing our boundaries against
skill_activatedon Claude Code is a free correctness check, and a disagreement between the two is worth reading rather than hiding.Knock-on changes to sibling issues
skill_activated.OTEL_LOG_TOOL_DETAILS, so its refusal-until-redaction-active criterion becomes conditional rather than mandatory.Scope for v1
Claude Code only, like the rest of the milestone. The multi-tool value is real but arrives with #653.
blafourcade commentedon Aug 18, 2026
The mechanism this issue needs is already observable, and it was in our own recorded payloads.
This issue says "the framework already knows" a skill started, without saying how. It does: a
PostToolUsepayload carriesSo a step boundary is a tool call, not something to invent — and
plugins/aidd-telemetry/hooks/already listens on that event for task attachment. WhetherPreToolUsefires forSkillas well decides whether a step has an observable start or only an end; that is being measured now, along with whether cost can be split per step at all withoutOTEL_LOG_TOOL_DETAILS.The correlation key is
run_id, not a timestamp. This issue asks for "correlation keys that let a reader attach provider telemetry to the interval". Two sessions can work the same task at once — a background agent, a second worktree — so a step interval overlapping both cannot be resolved by time. Time is only ever used to split inside a session, never to choose one. Recorded in full on #649.What this changes if the measurement confirms it. Steps stop being something skills declare and become something the hook observes. That is the same correction already made twice in this layer: the run journal's task pointer was declared by the planning skills and is now observed from file writes, because a declaration can silently stop happening while everything still looks fine, and because it put a measurement concern inside skills whose concern is something else.
It would also make #649 smaller:
metadata.jsonkeeps its identity and its upward link to the backlog, and stops carrying a journal that something else records more reliably.blafourcade commentedon Aug 18, 2026
Measured on a real paid session ($0.61). The privacy trade disappears, and this issue gets a sharper reason to exist.
OTEL_LOG_TOOL_DETAILSis not needed for per-step costOrdered per-step cost is obtainable from the OTLP logs stream alone, with the flag unset:
skill_activatedevents carry timestamps that mark step starts.claude_code.api_requestrecords carry an individually-timestampedcost_usd.0.401955 + 0.054666 + 0.157919 = 0.61454, againsttotal_cost_usd: 0.61454from--output-format json.So the whole privacy argument — that seeing a skill's cost required also logging Bash commands and tool inputs — was based on needing the name from the export. It is not needed for the cost.
What the export still cannot give, and what this issue is for
The name. With the flag unset,
skill.nameandplugin.nameread"third-party"identically for two genuinely different skills, oncost.usage,token.usageandapi_requestalike. Andskill_activateduses a different placeholder again,"custom_skill"— so even a join keyed on the literal"third-party"misses that event type.The hook stream has it, un-redacted, on both events:
So
PreToolUsedoes fire forSkill. A step has an observable start and an observable end, with the realplugin:skillname, and no vendor event and no flag involved.The measurement that makes this issue necessary rather than nice
The boundary from
skill_activatedcarries a structural attribution error, not noise. Oneapi_requestlanded 59 ms before the next skill's dispatch — the turn that both finishes one step and decides to call the next. It was attributed to the earlier step, where it accounted for 47.7% of that step's entire cost.On short steps that is not a rounding concern, it is the majority of the number. A framework-emitted boundary is what removes it, because the framework knows which side of the turn it is on.
Two things this changes elsewhere
Build on logs, not metrics. Every
cost.usagedatapoint matched anapi_requestrecord float-for-float, and only because turns happened to be 3–7 s apart against a 1 s export tick. At the 60 s production default, several turns merge into one datapoint irrecoverably. The metrics stream is redundant here and strictly weaker under realistic settings. Also worth knowing: sums areaggregationTemporality: 1, delta and not cumulative, so no differencing is needed — andunit: "USD"lives only on the metric descriptor, never on a datapoint.query_sourceis not a join key. For the same underlying API calls it reads"main"on the metric and"sdk"on the log. It disagrees with itself across streams.Left undetermined, deliberately
invocation_triggerread"nested-skill"for the second skill. If skills nest rather than run in sequence, a flat partition is a modelling choice and not a fact. One session cannot say, and this issue's "nesting" scope item is exactly where it belongs.Also not run: whether the flag changes
skill.nameon these event types. The unset case answered the question without it, so a second paid session was not spent.blafourcade commentedon Aug 18, 2026
Nesting is not a blocker, and this issue should not treat it as one
I flagged
invocation_trigger: "nested-skill"as an open question, as though the design could not proceed until it was settled. That was wrong, and the maintainer's framing is the right one: people can invoke whatever they like; what matters is that we record what they did.Nesting is a question for the report, never for the trace.
Recording a start and an end per invocation settles it — and both are available, since
PreToolUseandPostToolUsewere both measured firing for theSkilltool with the realplugin:skillname. Nesting then becomes derivable rather than a modelling choice frozen at write time, exactly as a profiler separates self time from total time.So this issue's scope item about nesting stays, but it is a recording requirement — write both boundaries — not a question that has to be answered before building.
The corollary matters as much
A skill invoked outside the standard flow is a step like any other. Nobody is required to follow the path, and that is precisely what makes the measurement worth having: a task that went from specification to implementation with no review becomes visible, instead of assumed.
That reframes what "steps" are for. Not conformance to a flow — a record of what actually happened, whatever it was.
blafourcade commentedon Aug 18, 2026
Measured on all five tools. This issue's premise was wrong, and in our favour.
Paid probes on Codex, Copilot and Cursor; Claude Code already measured; OpenCode from documentation. Per-step measurement is portable. Four of five tools name the skill in a schema-guaranteed field, and all five give a distinguishable start and end.
Skilltool callPreToolUse/PostToolUsetool_input.skill=plugin:skillskillfunction toolpreToolUse/postToolUsetoolArgs: {"skill":"…"}ReadofSKILL.mdpreToolUse,beforeReadFile,postToolUsefile_pathendingskills/<name>/SKILL.mdskill({name})tooltool.execute.before/.afterinput.tool === "skill",output.argsSKILL.mdwith its own Bash callPreToolUse/PostToolUse,tool_name: "Bash"tool_input.command, free textA correction worth recording
I had ruled Codex out on its documentation saying
PreToolUseintercepts "currently Bash only". That is outdated. The current page states: "PreToolUse and PostToolUse can observe more than shell and MCP calls. Most local function tools use the same hook path… Hosted tools, such as WebSearch [do not]." Measured: asking Codex to use a skill produced aPreToolUse/PostToolUsepair carryingsed -n '1,120p' .agents/skills/probe-echo/SKILL.md.So the conclusion this issue rested on — that nothing observes a step outside Claude Code — is false.
What this issue becomes
Not "emit boundaries because nothing observes them". Rather: normalise five different observations into one step record, and emit our own only where observation is weak or absent.
SKILL.mdpath out of arbitrary shell text, and a differently phrased read (cat,head, a batched command) still fires the hook while making the identity harder to recover. This is the one tool where "no help from the model" is not strictly true, and where a framework-emitted boundary earns its place.hooks: { supported: false }, noHasHooks. That is feat(cli): OpenCode joins the chain through its plugin API, not through hooks #676, and it is a build-pipeline gap rather than a capability gap.What does not change
The step record stays the same everywhere, whatever the tool and whatever the kind of work: name, start, end,
run_id. Only where the name comes from differs, and that belongs in a per-tool extractor.Steps are written to the run record, which git ignores — not to
metadata.json, which is committed and would leave the working tree permanently dirty if a hook rewrote it on every skill invocation.blafourcade commentedon Aug 19, 2026
Où les étapes s'enregistrent, et pourquoi pas dans la tâche
Deux décisions à poser avant d'écrire quoi que ce soit ici.
Un fichier voisin, pas une onzième clé
La fiche de run a dix clés, verrouillées par un test. Elle est écrite au démarrage de session et relue à la fin. Les étapes, elles, arrivent sur
PostToolUse— des dizaines à des centaines de fois par session.Les mettre dans la fiche impose une lecture-modification-écriture du fichier entier à chaque appel d'outil. Coût quadratique, et surtout une fenêtre de corruption à chaque écriture : une session tuée au mauvais moment laisse un JSON tronqué, et on perd l'en-tête avec les étapes, donc le run entier.
Un fichier voisin en JSONL s'écrit en ajout pur : rien à relire, rien à réécrire, ordre naturel, et une ligne corrompue ne coûte qu'une étape.
La raison de fond compte plus que la performance : la fiche est un en-tête de taille bornée, fait pour être joint. Les étapes sont non bornées. Mettre du non borné dans une structure à forme fixe casse la propriété que le test des dix clés protège — ce test n'est pas un obstacle à contourner, il dit quelque chose de vrai.
L'enregistrement ne doit jamais exiger une tâche
Le
metadata.jsonpar tâche est le mauvais endroit, et pas pour une raison de forme : tout ce qui se passe hors flow n'y serait jamais enregistré. Le skill debug lancé à la volée, le prompt improvisé, la session qui ne suit aucun ticket. C'est exactement ce qu'on veut voir — c'est là que le process fuit.Le journal modélise déjà la bonne chose.
task_id: nulldanstasksveut dire « rattaché à aucune tâche » : ce n'est pas un trou, c'est une valeur.D'où la séparation :
null, elle est hors flowLe
metadata.jsonpar tâche devient un artefact de rapport — ce qu'une tâche a coûté — dérivé du journal, jamais la surface d'enregistrement.Deux conséquences directes :
Ce qui reste à mesurer, pas à supposer
Le hook lit le nom du skill sur
PostToolUse, donc on tient l'instant où il se termine. Savoir si la charge utile porte aussi l'instant de départ — donc si on obtient une durée par étape ou seulement une borne — demande une mesure. À faire avant d'arrêter le format d'une ligne.7 remaining items
blafourcade commentedon Aug 20, 2026
Narrowed by a measurement taken while planning #687: Claude Code's transcript already carries the skill per assistant message, as
attributionSkill, with the real name and no flag — measured across 40 transcripts, 2267 attributed messages, 25 distinct skills.On that one path the journal's step boundaries are the weaker source: the transcript's attribution is exact per message, where the journal's is a half-open interval closed by the next marker.
This does not undo what landed here. The journal remains the only route for the export path, where the vendor attribute reads
third-party, and for Codex, Cursor and Copilot, none of which was measured to have an equivalent. #687 records the per-path table and the two limits that constrain how the transcript field can be read.feat(cli): one AIDD switch the whole framework obeys, and Claude Code…
refactor(framework): the run journal records facts, not conclusions
feat(cli): the sink keeps the number that orders a session
feat(telemetry): the journal records which step was running, on four …
docs(telemetry): the contract and plan #663 was built from
feat(cli): one AIDD switch the whole framework obeys, and Claude Code…
refactor(framework): the run journal records facts, not conclusions
feat(cli): the sink keeps the number that orders a session
feat(telemetry): the journal records which step was running, on four …
docs(telemetry): the contract and plan #663 was built from
blafourcade commentedon Aug 28, 2026
Closed as completed, with three of five acceptance points unmet
Found by a backlog audit on 2026-08-28, verified against the code, which contradicts them in its own comments.
"Every supported tool opens a step." It does not.
plugins/aidd-telemetry/hooks/lib/tools/opencode.cjs:16declaresstepStart: null— OpenCode opens no step at all, so a session there attributes by interval or not at all."A skill interrupted or crashed leaves a step distinguishable from one still running." Nothing records a step ending.
grep -rn "step_end\|stepEnd\|step_stop"overcli/srcandpluginsis empty, andhooks/lib/record.cjs:183states the design outright: closing is the reader's derivation, from whichever boundary comes next. A crashed step and an open one are therefore the same observable state — which may well be the right call, since no tool measured so far reports when a skill's work finishes, but it is not what the box says.skill_activatedappears only inside a comment, so no disagreement between it and the journal is ever surfaced.Not reopening. What shipped is real and substantial — steps open on four of five tools, from the tool's own arguments, and the interval attribution works. The three points above are a genuine gap between the text and the code, and they matter because two of them are load-bearing elsewhere: OpenCode's
stepStart: nullis why an OpenCode session cannot report per-skill, and the absent step end is why every interval is closed by the next boundary rather than by the step's own completion.If the intent was to narrow the scope, the honest move is to amend the three points here and open a follow-up for OpenCode's step; leaving them as ticked-in-spirit means the next person reads this issue as proof of something the code says it does not do.
feat(framework): measurement that works on five tools, each proven by…