Skip to content

feat(framework): an aidd-telemetry plugin whose hooks journal every session #620

Closed

Description

@alexsoyes

Outcome

Every session of a supported tool leaves a durable record tying it to a task, and that record never contains a measurement.

Scope

  • Includes: a new aidd-telemetry plugin, which carries the hooks. Installing the plugin installs the mechanism; the CLI wires nothing per tool.
  • Includes: one file per session at aidd_docs/runs/<run_id>__<vendor_id>.json, written on session start and refreshed on the last observed turn. AIDD_RUNS_DIR overrides the directory.
  • Includes: an explicit current-task pointer at .aidd/current-task, written by the planning and implementation skills. The hook reads it and never guesses.
  • Includes: a run_id minted by the framework, stored beside the vendor identifier and the name of the field it came from.
  • Includes: a stable project_id in every record, and the same value pushed into OTEL_RESOURCE_ATTRIBUTES by feat(cli): one command that turns the provider export on #646. Without it a sink mixes every repository on a machine with nothing to separate them, and it cannot be recovered after the fact.
  • Includes: Claude Code only. The hook identifies its host from the payload it receives and exits 0 on any tool it does not support.
  • Excludes: task identity and backlog links, which belong to the next milestone.
  • Excludes: any token, cost, model or duration. Those change mid-session and come from telemetry.
  • Excludes: backfilling sessions that predate the feature.

Why a dedicated plugin

aidd framework build copies every plugin's hooks/ into all five tool targets, and no exclusion mechanism exists (cli/src/application/use-cases/framework/strategies/tool-contracts.ts, hooksBundle). Putting telemetry hooks in an existing plugin would ship them to everyone with no way out. A separate plugin makes the opt-out trivial: do not install it.

It also matches docs/ARCHITECTURE.md — "Every capability lives in exactly one plugin, chosen by concern". Measurement is neither knowledge production, nor code transformation, nor version control.

The file

{
  "schema_version": 1,
  "run_id": "01J9X4M2K7QRVB",
  "project_id": "ai-driven-dev/framework",
  "tool": "claude-code",
  "vendor_id": "79041f53-35b0-4924-8855-e43e9de72431",
  "vendor_field": "session.id",
  "parent_run_id": null,
  "started_at": "2026-08-14T10:08:44Z",
  "ended_at": "2026-08-14T11:05:20Z",
  "tasks": [
    { "task_id": "2026_08_14_telemetry-v1", "from": "2026-08-14T10:08:44Z", "to": null }
  ]
}

A session can cover several tasks

task_id is a list of intervals, not a scalar. A session that plans task A then implements task B is otherwise attributed wholesale to one of them, and that is the common case rather than the edge.

"tasks": [
  { "task_id": "2026_08_15_alpha", "from": "...", "to": "..." },
  { "task_id": null,               "from": "...", "to": "..." }
]

An interval with a null task_id is out-of-flow work inside an otherwise attached session.

Three decisions, measured

runs/ is global, not inside the task folder. A per-task journal has nowhere to put out-of-flow work — the ten-minute debug with no folder, the exploration. One location handles both identically: attachment becomes a field, not a path. task_id null is a normal state, and reattachable afterwards since the file is ours. This is what makes "61% attached, 39% out of task" sayable instead of ignoring the second half.

vendor_field names the field, not just its kind. Cursor exposes three identifiers in one payload — session_id, conversation_id, generation_id — equal on a single-turn session with no guarantee beyond. The journal records which one it took.

ended_at refreshes on the last observed turn, never on a session-end event: Codex allows it one second, three at most, and does not fire it for subagents.

Open, to decide during implementation

Closed. Records live in aidd_docs/runs/ inside the repository, with .gitignore carrying aidd_docs/runs/* and the two marker files negated. The directory enters git so opt-in survives a clone; the records never do. Verified in a scratch repository: git add -A mid-session stages nothing and the tree stays clean.

They were briefly written outside the repository, at ${XDG_STATE_HOME:-~/.local/state}/aidd/runs/<owner__repo>/, and moved back. Ignoring the contents answers the dirty-tree problem completely, while a home directory is nobody’s to control: invisible, per-machine, unbacked-up, three platform branches, and an owner/repo → owner__repo flattening with its own collision. In-project, the opt-in marker is the store, so the repository root is the key and the platform stops mattering.

The pointer has three states nobody has handled

  • Two concurrent sessions in the same checkout share one .aidd/current-task. A background agent on another task silently re-points the foreground session.
  • .aidd/ is currently neither tracked nor gitignored in this repository. On a fresh clone or in CI the pointer is absent, so every session journals as out-of-flow and reads as a coverage failure.
  • Nothing validates the pointer against an existing task folder, so a stale value produces a confidently wrong chain.

The tracked-versus-ignored decision belongs here, with its consequence stated: committing runs/ puts who-worked-on-what-and-for-how-long into permanent git history, which is exactly what #652 says cannot ship without an organisational decision. Deciding it after the data exists is deciding it too late.

Done When

  • Two agents working the same task in two parallel worktrees produce two files and zero merge conflicts, verified by merging both branches.
  • A session producing no commit still appears in the journal.
  • A session with no current-task pointer produces a record with task_id null, never nothing.
  • The written JSON keys are exactly the ten documented ones, asserted as a whitelist.
  • The hook exits 0 in every failure mode and returns in under 200 ms at p95 over 100 invocations.
  • Installed on Codex, Cursor, Copilot or OpenCode, the hook writes nothing and exits 0.
  • On a public repository nothing is written until the repository has opted in.
  • A session that switches task mid-way produces two intervals, never one overwritten value.
  • Two concurrent sessions in the same checkout do not corrupt each other's attachment.
  • With no pointer present, the session journals out-of-flow and status reports it as such rather than as an error.
  • A record carries project_id, and two repositories on one machine are separable at read time.
  • scripts/__tests__/ carries a node:test file for the hook, and a runner is wired into lefthook.

Completion Evidence

A week of real work on this repository journaled, with the attached and out-of-flow shares, and the demonstration that no session was lost.

Relations

Field Value
parent #631
related #617, #629, #632, #646, #647

Design: aidd_docs/specs/2026_08/2026_08_13-work-tracking-linkage.md

Activity

  1. moved this from Ideation to Todo in AIDD Roadmapon Aug 11, 2026
  2. blafourcade commented on Aug 14, 2026

    @blafourcade
    Contributor

    Resserrage pour la v1, 2026-08-14

    Trois écarts avec la conception d'origine, tous issus de mesures. Détail dans aidd_docs/specs/2026_08/2026_08_13-work-tracking-linkage.md.

    1. runs/ est global, pas dans le dossier de tâche

    L'issue place sessions/<run_id>.json dans le dossier de tâche. Un journal par tâche ne sait pas où ranger le travail hors flux : le debug de dix minutes sans dossier, l'exploration, la question rapide. Ce travail existe, il coûte, et l'ignorer revient à annoncer une mesure complète en mesurant deux tiers.

    aidd_docs/runs/2026_08/
    ├── 01J9X4M2K7.json   task_id renseigné → rattaché
    └── 01J9XZP4T8.json   task_id nul       → hors flux, rattachable après coup

    Un emplacement unique traite les deux cas à l'identique : le rattachement devient un champ, pas un chemin. La propriété qui compte est préservée — un seul écrivain par fichier, donc conflit de fusion structurellement impossible. La liste des sessions d'une tâche devient une recherche sur task_id.

    Bénéfice direct : le tableau de bord peut afficher « 61 % du temps rattaché à une tâche, 39 % hors tâche », qui est le chiffre le plus honnête de l'ensemble.

    2. vendor_field doit nommer le champ, pas seulement l'espèce

    Mesuré : les noms diffèrent partout, et Cursor en expose trois dans un même payload — session_id, conversation_id, generation_id — égaux sur une session à un tour. Rien ne dit qu'ils le restent sur plusieurs. Le journal doit stocker conversation_id, le seul documenté comme stable, et enregistrer lequel il a pris.

    3. Sur parent_run_id, les outils ne se comportent pas pareil

    • Claude Code : un sous-agent n'a pas d'identifiant propre, il partage celui du parent et se distingue par query_source (main, subagent, agent:builtin:<Type>).
    • Cursor : « Subagents get their own conversation id ».

    Le champ reste nécessaire, mais il ne peut pas être rempli de la même façon partout, et il doit être alimenté par hook plutôt qu'espéré du fournisseur.

    Périmètre v1

    Claude Code seul. Les autres sont déclarés non couverts par status. La mécanique est identique ailleurs, seule la configuration d'export change — élargir avant d'avoir prouvé sur un multiplie les causes de panne.

    Ce qui sort de la v1

    metadata.json et les liens vers le backlog. Conçus dans la spec, mais un task_id dans le journal suffit pour produire un premier chiffre.

    Sur #585

    Même remarque que sur #617 : .aidd/config.yml n'existe dans aucun code, pas d'analyseur YAML dans la CLI. La v1 lit .aidd/telemetry.json.

  3. changed the issue type fromtoon Aug 14, 2026
  4. changed the title [-]feat(framework): per-session ledger in the task folder, keyed by an AIDD run id[/-] [+]feat(framework): journal des exécutions, une session un fichier[/+] on Aug 14, 2026
  5. changed the title [-]feat(framework): journal des exécutions, une session un fichier[/-] [+]feat(framework): an aidd-telemetry plugin whose hooks journal every session[/+] on Aug 14, 2026
  6. blafourcade commented on Aug 16, 2026

    @blafourcade
    Contributor

    Decision, 2026-08-16 — where the journal lives

    Default: outside the repository during the session, materialised into it on commit. Configurable.

    The hook writes to a local path while the session runs, and the record enters aidd_docs/runs/ only when a commit is made. Three consequences, all wanted:

    • The working tree stays clean. A file rewritten on every turn otherwise makes git status lie for the whole session and dirties every diff.
    • Only work that accompanies code enters permanent history. A session that produced nothing leaves nothing behind for anyone to read later.
    • The permanent-history exposure shrinks to what a commit already reveals, which defuses part of what chore(framework): support both anonymous and named measurement #660 has to decide rather than enlarging it.

    This reuses the mechanism #630 needs for the commit trailer — one marker written during the session, consumed at commit time. Building it once serves both.

    Configurable means the placement is a setting, not a constant: a project that wants everything in the repository, or nothing ever, must be able to say so. .aidd/telemetry.json carries it.

    Open, deliberately

    What else should be configurable has not been decided, and should not be guessed now. Candidates worth thinking through before v1 freezes the shape: retention, what a session that produces no commit leaves behind, whether out-of-flow work is journaled at all, and per-project versus per-machine defaults. This is a design conversation to have once the first figure exists and we know which knobs people actually reach for.

    Acceptance to add

    • With the default setting, a session in progress leaves the working tree unchanged.
    • A commit materialises the records of the session that produced it, and only those.
    • A session that never commits leaves nothing in the repository, and the diagnostic reports it as unattached rather than as an error.
    • The placement setting is honoured, including the two extremes.
  7. blafourcade commented on Aug 16, 2026

    @blafourcade
    Contributor

    Guard, 2026-08-16 — the journal never carries an identity

    The record holds a session number, a task, a tool, and timestamps. No author field, ever. Not an email, not an account identifier, not a git identity.

    This is a guard rather than an omission, because it is what makes deferring the anonymity policy free. The journal is materialised into git on commit; a name added there is permanent and unreachable by any later decision. The commit already names its author — that is git's doing, and adding it a second time buys nothing while closing a door.

    The person a record belongs to is resolved downstream, from the telemetry, through the stable label #647 produces.

    • The written keys are exactly the documented schema, asserted as a whitelist, and no key of it holds an identity.
  8. blafourcade commented on Aug 16, 2026

    @blafourcade
    Contributor

    Measured before planning, 2026-08-16

    Two of this issue's premises were unproven. Both were probed; one held, one was wrong.

    Installing the plugin does activate its hooks — the premise holds

    No plugin.json in this repository declares a hooks key, so "installing the plugin installs the mechanism, the CLI wires nothing per tool" might have been false: the bundled hook could have reached users only through aidd framework build copying hooksBundle, in which case "do not install it" is not the opt-out.

    Probed with the local marketplace under an isolated CLAUDE_CONFIG_DIR. After claude plugin install aidd-context@aidd-framework, one session filled the <aidd_project_memory> block. hooks/hooks.json is discovered by convention.

    The same probe answered a second question: the hook fired on a session that ended Not logged in. Verifying the journal costs nothing. That is the acceptance-test method for every criterion below.

    Host detection cannot use field names, and must not use environment

    Claude Code and Codex hand a SessionStart hook the same five keys — session_id, transcript_path, cwd, source, hook_event_name. Presence does not discriminate.

    Environment is worse than useless. A Codex session launched from inside a Claude Code session inherits CLAUDECODE, CLAUDE_CODE_SESSION_ID and CLAUDE_PID from its parent. Nesting is the normal case here, so env-based detection would attribute Codex runs to Claude Code.

    The discriminator is transcript_path, whose shape is tool-specific:

    Host Recognised by v1
    Cursor cursor_version in the payload exit 0
    Copilot sessionId, and no hook_event_name exit 0
    Codex transcript_path matching /sessions/<yyyy>/<mm>/<dd>/rollout- exit 0
    Claude Code transcript_path matching /projects/.*\.jsonl$ writes
    anything else — exit 0

    The last row is what makes it safe: unrecognised degrades to writing nothing, never to writing a wrong tool.

    The open decisions, closed

    The key set. The body carried three different ones — ten keys with a scalar task_id, then prose replacing it with tasks intervals, and nine in the spec. Pinned to ten, with tasks as the tenth and no scalar task_id. Body updated; the whitelist criterion now reads ten.

    Where a session writes. Outside the repository, at ${XDG_STATE_HOME:-~/.local/state}/aidd/runs/<project_id>/<run_id>.json. Stop fires every turn, so a tracked file would leave the working tree permanently dirty.

    Materialisation into aidd_docs/runs/ happens at commit, and the plugin cannot own it. Its hooks only see sessions; a commit can be made by a human with none running. Only git knows a commit happened, so the trigger is a git post-commit hook installed by the CLI gesture of #646. This is the one irreversible step — it is what puts who-worked-on-what-and-for-how-long into permanent history — so it is built last and confirmed before it is built.

    Opt-in is required unconditionally: the journal writes only when aidd_docs/runs/ exists as a directory. One existence check. No config format, no gh call, no repository-visibility detection, and the failure direction is off. A project opts in by committing the directory. This satisfies the public-repository criterion by removing the condition rather than implementing it.

    project_id is derived, never stored: git remote get-url origin as owner/repo, falling back to the repository root's basename. #646 pushes the same value into OTEL_RESOURCE_ATTRIBUTES and derives it by the same rule, so there is one rule and no second writer.

    Concurrent sessions sharing .aidd/current-task: last-writer-wins, plus an interval boundary. The interval model already absorbs a re-point. Recording the mis-attribution is honest; preventing it would need a mechanism the schema does not have. .aidd/ gets a .gitignore line — it is today neither tracked nor ignored, and aidd clean nukes it.

    parent_run_id is written and always null in v1. A Claude Code subagent shares its parent's session id and differs only by query_source, a telemetry attribute no hook ever sees.

    The 200 ms budget is asserted on in-process work, not on process spawn, which is flaky under CI load. Spawn latency stays a manual smoke.

    Order and file paths: aidd_docs/plans/2026_08_14-telemetry-v1/phase-1-run-journal.md.

  9. blafourcade commented on Aug 16, 2026

    @blafourcade
    Contributor

    Four checks on the plan, three of which changed it.

    vendor_field names the export-side attribute, not the hook field. The body's example shows session.id while the spec's prose said "the name of the field at the tool", which reads as session_id. Those are different strings, and #629's join queries telemetry — handed the hook's field name, a reader has nothing to look up by. Pinned to the export side: session.id on Claude Code, conversation.id on Codex, gen_ai.conversation.id on Copilot, cursor.conversation.id on Cursor. The hook-side name needs no storage: it has already given its value in vendor_id.

    The Codex path shape is the tool's, not the probe's. The /sessions/<yyyy>/<mm>/<dd>/rollout- segment was recorded under a probe CODEX_HOME, so the regex could have been anchored on something the probe contributed. Checked against the default home: ~/.codex/sessions/2026/04/24/rollout-<iso>-<uuid>.jsonl. It holds. Both hosts end in .jsonl and separate on /projects/ versus /sessions/.

    Gitignoring .aidd/ does not re-open the coverage failure this issue flagged. The pointer is deliberately ephemeral — it answers "what is being worked on right now", it is written by the planning and implementation skills, and a fresh clone legitimately has no answer until one runs. aidd clean wiping it mid-work costs one interval boundary; the next skill invocation rewrites it. The requirement this places on #617 is that no pointer is not a fault: it must be distinguished from pointer stale and from hook silent, and only those two are.

    The gate is not the destination, and status has to say so. Opt-in is aidd_docs/runs/ existing; writes go outside the repository until the materialisation step ships. So the directory sits empty in git in between. Two consequences, both now in the plan: a one-line README beside its .gitkeep stating what committing the directory turns on, so it reads as an intention rather than an accident; and #617 reporting that state as on, not yet materialised — reporting "on" hides a missing half, reporting "not wired" claims a failure that is not one.

  10. self-assigned this
    on Aug 17, 2026
  11. 8 remaining items

  12. blafourcade commented on Aug 19, 2026

    @blafourcade
    Contributor

    Le journal en ajout pur est livré (572b86fa), revu sur trois axes, findings corrigés. Un jeton d'accès partait en clair dans project_remote — la spécification que j'avais écrite était fautive, pas l'implémentation ; l'identifiant est caviardé et un test échoue sans lui. Reste ouvert ici : rien.

  13. moved this from In review to Done in AIDD Roadmapon Aug 19, 2026
  14. blafourcade commented on Aug 28, 2026

    @blafourcade
    Contributor

    One acceptance point is genuinely unbuilt, and this issue is the only place that says so

    Found by a backlog audit on 2026-08-28.

    This is the one closed issue in these milestones that ticked its boxes honestly — eleven ticked, one left blank. That blank one is still blank in the code:

    with no pointer present, the session journals out-of-flow and status reports it as such

    Work belonging to no task is journalled, but nothing reports it as its own thing. Corroborated independently from the other end: #720 — still Ideation — carries the reporting half of the same gap, "work belonging to no task appears as its own row with an explicit label".

    Also unsettled rather than unmet: the hook p95 under 200 ms over 100 invocations has not been re-measured since. Not a defect, just a number nobody can currently quote.

    Not reopening; #720 covers the reporting half. Noting it because this issue's honesty is what made the gap findable at all — eight of the nine other closed issues here have every box unticked regardless of what shipped, so tick state carries no information and this one is the exception.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Fields

Priority

High

Projects

Relationships

None yet

Development

No branches or pull requests

Issue actions