Repository navigation
fix(framework): a Copilot skill call opens no step, so its session journals as unattributed #701
Description
On the payload shape a real Copilot session actually sends, the journal records the session and never a step. Every record in it reads unattributed, which is the honest answer and not a useful one.
What is known
lib/step-starts.js reads Copilot's skill invocation as:
copilot: skillNameFromArgument({
toolField: "toolName", toolName: "skill",
argumentsField: "toolArgs", nameField: "skill",
})That is the canonical builder's spelling. The _vsCodeCompat builder — confirmed against @github/copilot@1.0.80 in #681 — sends Claude Code's spelling instead:
{ "hook_event_name": "PostToolUse", "session_id": "…", "timestamp": "…", "cwd": "…",
"tool_name": "Bash", "tool_input": { "command": "ls -a ." }, "tool_result": "…" }toolName is absent, so the reader returns null on every tool call and no step_start line is ever written.
What is not known, and must be measured first
The capture in #681 used a Bash tool. It establishes the field names for a tool call, and nothing about a skill call:
- What the compat builder puts in
tool_namewhen a skill is invoked. Claude Code sendsSkill; the canonical Copilot builder sendsskill. Neither can be assumed for the third. - Where the skill's name sits inside
tool_input. The canonical builder uses askillkey insidetoolArgs; whether the compat builder matches Claude Code'stool_input.skillis unverified.
Two field values, both guessable and neither known. Guessing them is exactly the mistake #681 was opened to correct, and it would fail the same way — silently, with a journal that looks healthy.
Done when
- A real Copilot session that invokes a skill is captured, and its
PostToolUsepayload held as a fixture. - The step reader recognises whichever spelling that payload carries, alongside the canonical one, and a test fails if either stops being recognised.
- A Copilot session running a skill produces a
step_startline naming it, and its records attribute to that step rather than readingunattributed.
Bounds
Copilot is outside the current milestone's tool coverage (epic #631 excludes the four remaining tools), so this does not block it. It matters for the milestone after, where per-step attribution is claimed on more than one tool.
Where it was found
While capturing the payload for #681. detectHost is fixed there; this is the next thing that reads the same payload and does not understand it.
Activity
blafourcade commentedon Aug 22, 2026
Captured, and neither value was derivable
One real
@github/copilot@1.0.80session, asked to invoke a skill by name. ThePostToolUsepayload, key set unmodified:{ "hook_event_name": "PostToolUse", "session_id": "…", "timestamp": "…", "cwd": "…", "tool_name": "skill", "tool_input": { "skill": "00-init" }, "tool_result": { … } }The two unknowns this ticket refused to guess:
tool_nameis"skill"— the canonical Copilot builder's lowercase spelling, not Claude Code's"Skill".tool_input.skill, andtool_inputarrives as an object — Claude Code's field name, but not the canonical Copilot builder's JSON-stringtoolArgs.So the compat builder mixes one host's tool-name spelling with another host's argument shape. Neither value follows from the other two captures, and picking either convention wholesale would have got one of the two wrong — silently, with a journal that looks healthy. That is exactly the failure #681 was opened to correct, and the reason this ticket said capture first.
What changed
step-starts.jsgained a small combinator that runs several argument-family readers in sequence and takes the first name found. Copilot's entry now runs both the canonical (toolName/toolArgs) and the compat (tool_name/tool_input) readers through it. Both shapes are recognised; neither replaced the other, because both are real captures.A tool call that is not a skill still opens nothing, on either shape — both readers gate on the tool name.
New fixture
copilot-compat-post-tool-use-skill.json, redacted the way its neighbours are. Two tests: a compat skill call leaves astep_startnaming00-init, and a compatBashcall opens none. The fixtures README's sentence saying the compat shape "never opens a step line" was itself written by #681 and is now false; corrected.What this does and does not buy
A Copilot session's records now attribute to the step that was running, at journal-interval strength, instead of reading
unattributed. That is attribution of which step ran.It is not a figure of what it cost. Copilot's own files carry
outputTokensonce at shutdown for the whole session and no per-request counter, so no amount can be placed inside a step however well the boundary is known. Closing that needs its opt-in OTLP export or a per-request field nobody has ever observed on disk — #697 and #653, not this.Gate: 324 plugin specs pass, 0 broken links. One of three budgeted sessions used.
blafourcade commentedon Aug 22, 2026
Done and awaiting review in #706, which closes this on merge.
The work is on
claude/aidd-telemetry-layer-e403uf, eleven commits, targetingnext. Gate at the time of push: 365 plugin specs, 1,931 CLI unit, 577 integration, 178 e2e,tscand biome clean, no broken markdown links.Read the eleven commits rather than the pull request's file count — the branch carries its own copy of the CLI migration that
nexthas since received, so the diff counts it twice. The range and the real figures are in the first comment on #706: 142 files, +10,518 −418.Every claim about a tool in this work rests on a session that was actually run. The negative results are kept in full in
aidd_docs/tasks/2026_08/2026_08_22_telemetry-every-tool/measurements.md, including the two probes that disagreed about OpenCode and the measurement that reconciled them.test(framework): the session #681 asked for, and what it settled
blafourcade commentedon Sep 2, 2026
Delivered by #706, squash-merged into
nextas 627408f.