Skip to content

feat(framework): extend the chain to the four remaining tools #653

Description

@blafourcade

Outcome

Each remaining tool joins the same chain, one at a time, with its own gate lifted and its coverage stated honestly.

Scope

Per tool: switch the export on, ship the hook, prove the identifier joins, and declare what is and is not reachable. Measured facts, not assumptions.

Tool Where tokens and the id meet Export switch Gate to lift Cost
Codex CLI codex.sse_event log events [otel] in config.toml; metrics_exporter defaults to statsig, not none feature flag + persisted hook trust none exported
GitHub Copilot invoke_agent / chat spans COPILOT_OTEL_ENABLED or an OTLP endpoint folder trust; user scope is not gated span attribute, currency undocumented
Cursor cursor.api.request logs team setting, Enterprise plan, beta workspace trust metric-only, so not joinable
OpenCode ai.streamText spans experimental.openTelemetry in opencode.json none found none exported
  • Includes: one issue's worth of work per tool, shippable independently.
  • Includes: the free verification method where it applies. Identifiers are minted client-side, so a provider pointed at a dead address still opens a session, fires the hook and emits telemetry — Codex and Copilot verify at zero cost. Cursor does not: it validates key, model and workspace trust before opening a session.
  • Excludes: Cursor's export installation. It is a team-level Enterprise setting; the framework can check it, never set it.
  • Excludes: any promise of cost for tools that export none. See the price table issue.

Two traps to carry into the work

Codex ships metrics to a third party by default. otel.metrics_exporter defaults to statsig. Enabling telemetry without setting that key sends data somewhere nobody chose.

OpenCode is loud. One trivial session produced 495 spans across 348 KB, since everything down to file reads is instrumented. Sampling is not optional there.

Done When

  • Per tool, the hook identifier and the export identifier are recorded side by side, from a real session.
  • Per tool, the diagnostic skill reports its own gate state rather than a generic verdict.
  • A tool whose export cannot be installed by the framework says so instead of reporting healthy.
  • The free verification runs in continuous integration for the tools where it applies.

Relations

Field Value
parent #652
depends_on #618
related #632

Activity

  1. added this to the milestone on Aug 14, 2026
  2. blafourcade commented on Aug 18, 2026

    @blafourcade
    ContributorAuthor

    The hook side, mapped across the five tools

    This issue's table covers the export side. Building the run journal produced the hook side, and it is a different shape. Recorded here so it is not re-derived: every line below is either from the tool's own documentation, fetched 2026-08-20, or measured on a real session.

    The framework stays Claude-oriented; the CLI adapts

    Not a preference — it is already how the code works. plugins/*/hooks/hooks.json is authored in Claude Code's shape, and cli/src/domain/formats/cursor-hooks.ts lowercases the first character of each event key while substitutePluginRoot rewrites ${CLAUDE_PLUGIN_ROOT}/. The CLI is a mechanical translator, and adding semantic per-tool knowledge to it would put the same concern in two places.

    Consequence, accepted deliberately: where a tool has a better event than the mechanical translation lands on, we do not take it. Cursor's afterFileEdit carries file_path directly, which is cleaner than postToolUse — and we still use postToolUse, because reaching for it would make the CLI clever.

    The three concepts, and what each tool calls them

    Meaning Claude Code Codex Cursor Copilot OpenCode
    session begins SessionStart SessionStart sessionStart sessionStart session.created
    a turn ends Stop Stop stop none session.idle
    a file was written PostToolUse PostToolUse postToolUse (afterFileEdit is better; not used) postToolUse file.edited

    The mechanical first-character transform gets Cursor all three for free.

    The event name travels in argv, not in the payload

    Measured across the recorded fixtures:

    claude-code   hook_event_name: "SessionStart"
    codex         hook_event_name: "SessionStart"
    copilot       hook_event_name: ABSENT
    cursor        hook_event_name: "sessionStart"
    

    Copilot sends no event name at all, so a payload-only dispatch cannot work there. The hook command therefore carries our own stable name as an argument — journal.js session-start|turn-end|file-written — which the CLI passes through untouched. Event keys stay each tool's; the argv is ours. Two vocabularies on purpose.

    Names describe intention, not mechanism: turn-end, not stop; file-written, not post-tool-use. Three of the five tools spell those mechanisms differently.

    Copilot has no turn-end event, and that is already handled

    Its events are sessionStart, sessionEnd, userPromptSubmitted, preToolUse, postToolUse, errorOccurred.

    ended_at was specified as "the last observed turn, never a session-end event" — because Codex grants a session-end handler one second at most and does not fire it for subagents. Making that literally true fixes Copilot for free: any event reaching the hook advances ended_at, so file-written becomes the de-facto turn signal wherever turn-end does not exist. No per-tool logic.

    Extracting the written file path is per-tool work

    This is the part that does not generalise, and the reason the journal ships a per-host extractor rather than one shape:

    Tool Where the path is
    Claude Code tool_input.file_path
    Cursor afterFileEdit.file_path — direct, but unused, see above
    Copilot postToolUse, shape to confirm
    Codex no path field at all — it is inside the apply_patch command string, and must be parsed out
    OpenCode file.edited

    OpenCode is not a hooks-in-JSON tool

    cli/.../tool-contracts.ts already says so: hooks: { supported: false }, // opencode has no HasHooks capability.

    It exposes a JavaScript plugin API (session.created, file.edited, tool.execute.after), not command hooks in a config file. So it is not a naming translation at all — it is a separate installation mode, and it deserves its own issue rather than a row in this one.

    One suspicion, probed and refuted

    flattenCopilotHooksShape copies the event key verbatim, so Copilot receives SessionStart in PascalCase while its documentation lists sessionStart. If Copilot rejected PascalCase, aidd-context's memory hook would be installed and silent there — the exact failure this layer exists to catch.

    Probed by wiring both spellings to the same event in one config. Both fired. No bug. Recorded so nobody re-opens it.

    Sources: Claude Code · Codex · Cursor · Copilot · OpenCode

  3. blafourcade commented on Aug 20, 2026

    @blafourcade
    ContributorAuthor

    Needs re-scoping twice over.

    #663 took the journal half. The run journal now serves Claude Code, Codex, Cursor and Copilot - each declaring how its session id and working directory are read. That part of "extend the chain to the four remaining tools" is delivered.

    #684 took most of the export half. The decision there is that cost comes from the files a tool already writes, not from its OTLP export. Codex and OpenCode are readable locally (#685), so extending their export buys nothing they do not already have.

    What honestly remains under this ticket:

    • Copilot's export, which is the only route to a complete token figure for it - its local file gives outputTokens per turn and nothing else. It needs the user to set COPILOT_OTEL_ENABLED, which AIDD deliberately does not do.
    • Cursor, which has no route at all: no local token count, and an export gated behind an Enterprise team setting. This ticket should say that plainly rather than carry it as pending work.

    Suggest re-titling around Copilot specifically, and moving Cursor's impossibility somewhere it is stated as a known limit rather than a backlog item.

  4. blafourcade commented on Aug 22, 2026

    @blafourcade
    ContributorAuthor

    This ticket and the work now running are two different routes, and only one of them is in flight

    A tool reaches a figure by two independent paths, and this ticket is about the second:

    • The journal and the local read — the tool's hooks run, they see a session id, a step boundary is written, and something the tool already wrote on disk carries token counts. No export, no endpoint, no configuration.
    • The export — the tool ships spans or logs to a collector, and the identifier in them joins the one the hook saw. That is the table above.

    Everything in flight on claude/aidd-telemetry-layer-e403uf is the first route. Plan: aidd_docs/tasks/2026_08/2026_08_22_telemetry-every-tool/. It matters here because the two routes share a prerequisite — a hook that fires and sees a session id — and that prerequisite is failing on three of the four tools for reasons this table does not mention.

    What the local route has established, per tool

    Tool Hook fires Sees its session id Journals a step Local figure
    Codex yes, once trusted (#699) yes, CODEX_THREAD_ID yes yes, reconciles
    Copilot yes, all three events yes, session_id in the compat payload (#681) no — a skill call opens no step (#701) no per-request input in its own files
    Cursor no plugin-scope hook fired at all, twice (#680) unknown no writes no token count anywhere
    OpenCode no — its runtime is JS modules, a declarative hooks.json means nothing to it (#676) unknown no readable, unjoinable

    Two of these bear directly on this ticket's gates:

    Codex's gate is not only the feature flag. It will not run a hook it has not been asked to trust, and says nothing when it declines — four consecutive sessions ran clean and journalled nothing. The table calls this "persisted hook trust"; what it does not say is that the failure is silent, which is what makes it expensive.

    Cursor's gate may be prior to trust. The table assumes a hook fires and only the export is in question. Two headless probes fired no plugin-scope hook at all, while an earlier probe fired five of seven events from a project-scope file, and nothing in Cursor's configuration names the plugins sitting in its own plugin directory. If plugin hooks never run there, neither route reaches Cursor, and the Enterprise export setting is the second obstacle rather than the first. A probe is running now to settle it.

    Not a proposal to change this ticket

    The export route stands as written, including its two traps — Codex's metrics_exporter defaulting to a third party nobody chose, and OpenCode's 495 spans for a trivial session. This is so that whoever picks it up knows which gates have already been measured, and which of them turned out to fail quietly.

  5. blafourcade commented on Aug 31, 2026

    @blafourcade
    ContributorAuthor

    Closing — the outcome arrived by the other route

    This issue describes extending the chain through each tool's own OTLP export configuration. That route was deleted: on the machine that built it, it produced 0 of 34 stored records, it was the only thing opening a port and the only thing sending an address off the machine, and receive/endpoint/endpoint clear are gone with it.

    The outcome it asked for is delivered anyway, by reading each tool's own files. All five tools carry a telemetryLocalRead declaration in cli/src/domain/tools/ai/ — claude.ts, codex.ts, copilot.ts, opencode.ts, cursor.ts (the last as unsupported, with the measured reason). Coverage and its absence are both reported per tool, each naming what it can and cannot supply.

    Its one remaining condition — that the identifier join hold over the export — is not merely unbuilt, it is unbuildable: there is no export. The join it protects is re-checked on every pull request over the route that does exist, by cli / Identifier join (Claude Code).

    Closed as delivered by another route rather than rewritten. The decision of record groups this issue with #655 and #662 as "describes a route that no longer exists"; that is imprecise for this one and worth correcting here — #653 describes no upload, and its result is already shipped.

  6. moved this from Ideation to Done in AIDD Roadmapon Aug 31, 2026
  7. added a commit that references this issue on Aug 31, 2026
    c10c222
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Fields

    Priority

    High

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions