Skip to content

feat(cli): one command that turns the provider export on #646

Closed

Description

@blafourcade

As a developer installing AIDD telemetry
I want one command that turns the provider's export on and records my consent
So that the measurement has something to measure, and nothing was switched on behind my back

Acceptance

  • aidd telemetry enable writes the env block into the resolved settings scope, and states which file it wrote before writing it.
  • It writes .aidd/telemetry.json with documented keys: at minimum enabled, endpoint, and the date and identity of the consent.
  • It refuses to run on a public repository unless --yes is passed, and says why.
  • aidd telemetry disable removes the entries it added and nothing else, verified by diffing the settings file before and after a full enable/disable cycle.
  • Running enable twice leaves the settings file unchanged the second time.
  • It never overwrites an unrelated key in the settings file it edits.
  • A tests/e2e/*.e2e.test.ts journey covers enable, re-enable and disable.
  • aidd clean preserves .aidd/telemetry.json, or the command documents that cleaning resets consent.

Why this cannot be the plugin

The plugin carries the hooks and the skills. It cannot carry this. Claude Code's plugin documentation is explicit: a plugin's settings.json supports only the agent and subagentStatusLine keys, and "unknown keys are silently ignored". A plugin declaring an env block would be dropped without an error — the worst possible failure for a measurement layer.

Environment variables are also read at process start, before any hook runs, so a hook writing them would only take effect one session later. Turning on an export that may ship data to a remote endpoint deserves an explicit gesture, not an invisible mutation.

This leaves the CLI with exactly one irreducible job.

What it writes

At minimum, per the measurement campaign: CLAUDE_CODE_ENABLE_TELEMETRY, OTEL_METRICS_EXPORTER, OTEL_LOGS_EXPORTER, OTEL_EXPORTER_OTLP_PROTOCOL, OTEL_EXPORTER_OTLP_ENDPOINT, and a short OTEL_METRIC_EXPORT_INTERVAL since the default is 60 s with no documented flush on exit.

OTEL_LOG_TOOL_DETAILS is a separate, explicit decision. It is what makes skill_activated carry the real skill name instead of a placeholder, so per-step cost depends on it — but it is not selective: it also logs Bash commands and tool inputs. The command must surface that trade rather than set it silently, and the answer is collector-side redaction (see the sink issue).

Out of scope

  • Installing hooks. The plugin carries them.
  • Any diagnostic or reporting. Those are skills.
  • The four other tools.

Relations

Field Value
parent #631
related #620, #617

Activity

  1. moved this from Ideation to Todo in AIDD Roadmapon Aug 14, 2026
  2. blafourcade commented on Aug 15, 2026

    @blafourcade
    ContributorAuthor

    Two additions from the conformance review, 2026-08-15

    Stamp a project identity into the export. Nothing in the design carries a stable repository or project identifier, and the sink writes to one path per machine — so two repositories land in the same store with nothing to separate them. This cannot be recovered afterwards: telemetry that shipped without it is telemetry that can never be attributed to a project.

    The gesture is small and belongs here, since this is the only component that writes resource attributes:

    OTEL_RESOURCE_ATTRIBUTES=aidd.project_id=<stable id>
    

    Measured: Claude Code attaches these values "on every metric datapoint and event record", so a project identifier posted this way reaches both grains without any further work. #620 carries the same value in the run record.

    Do not set OTEL_LOG_TOOL_DETAILS blind. That flag is what makes step names readable, and it is not selective — it also logs Bash commands and tool inputs. Setting it without a filter in front turns a measurement layer into an exfiltration path.

    Acceptance to add: the command refuses to set it unless the sink reports its redaction allowlist active, and says so rather than failing silently. The allowlist itself is owned by #647.

  3. blafourcade commented on Aug 16, 2026

    @blafourcade
    ContributorAuthor

    Relaxed by #663 moving into v1, 2026-08-16

    Step boundaries are now emitted by the framework itself in this milestone, so per-step cost no longer depends on OTEL_LOG_TOOL_DETAILS.

    Consequence for this command: do not set that flag by default. It stops being necessary for the headline feature, and it remains what it always was — a switch that also logs Bash commands and tool inputs.

    The earlier criterion "refuses to set it unless the sink reports redaction active" becomes the rule for anyone who explicitly asks for it, rather than a gate on the normal path. The normal path simply never sets it.

    Everything else stands: the env block, .aidd/telemetry.json, the aidd.project_id resource attribute, enable and disable.

  4. blafourcade commented on Aug 16, 2026

    @blafourcade
    ContributorAuthor

    Shared contract with #620, pinned there: project_id is derived, never stored. git remote get-url origin as owner/repo, falling back to the repository root's basename.

    Both sides derive it by that rule — the hook for the journal, this ticket for OTEL_RESOURCE_ATTRIBUTES. Neither reads it from a file, so there is one rule and no second writer. Without it a sink mixes every repository on a machine with nothing to separate them, and it cannot be recovered after the fact.

  5. blafourcade commented on Aug 18, 2026

    @blafourcade
    ContributorAuthor

    Decided: the sharing choice is asked at init, with its consequence, and written to the repository.

    The .gitignore block is the switch — no second mechanism:

    aidd_docs/runs/*
    !aidd_docs/runs/.gitkeep
    !aidd_docs/runs/README.md
    

    Removing the first line commits the records; keeping it does not. Visible in a diff, already understood by everyone, and a project that never answers inherits the private default.

    Asked at init because that is the only moment someone is already thinking about telemetry — asking later means asking someone who has forgotten. Written to the repository rather than to a machine-local config, so it binds the project rather than whoever ran the command.

    The consequence must be printed at the moment of choosing, not discovered afterwards. With the private answer, records live exactly as long as the checkout, and four routine actions destroy them: git clean -xdf (-x removes ignored files), deleting a merged worktree, a fresh clone or new machine, and any ephemeral CI checkout. Nothing is lost in flight — the record is rewritten every turn, so no state waits to be flushed — but the history is unrecoverable, because no one else ever had a copy.

    One measured caveat for whoever implements the committed answer: a tracked record is reported modified by git on every turn, whether it is rewritten or appended to. Verified both ways. So choosing to commit needs a mutable-versus-immutable split that does not exist yet; #620 records the measurement and the reason to build it only when a project first asks.

  6. self-assigned this
    on Aug 18, 2026
  7. moved this from Todo to In Progress in AIDD Roadmapon Aug 18, 2026
  8. blafourcade commented on Aug 18, 2026

    @blafourcade
    ContributorAuthor

    Planned: aidd_docs/tasks/2026_08/2026_08_20_telemetry-export-enable/

    Four phases: the variable set, the surgical write and its exact inverse, the command and its scope, the journeys.

    The premise was verified before planning. This issue rests entirely on Claude Code reading an env block from settings.json. It does — and the documentation uses OTEL variables as its own example. Settings resolve highest-first: managed, command line, .claude/settings.local.json (not git-tracked), .claude/settings.json (tracked), ~/.claude/settings.json.

    That precedence turned out to be the plan's central fact: the scope choice is a sharing choice. One file affects only the person who ran the command; another turns telemetry on for everyone who clones.

    Three corrections to this issue, made in the plan

    Two different consents are conflated here. Turning the provider's export on is a per-developer decision about data leaving a process. Whether run records get committed is a per-project decision, already settled and already living in .gitignore. Same word, different questions, and they must not share a mechanism.

    .aidd/telemetry.json should not exist. The settings file the command writes is the record; a second file restates it and the two drift. It also removes the question this issue had to ask — whether aidd clean preserves consent — because nothing is stored twice, and a directory clean deletes cannot hold the only copy of a decision. "Is telemetry on" then has one honest answer: read the settings file. That is #617's job, and it must read the real file rather than a record of intent, or it reports what someone meant instead of what is true.

    "Refuses to run on a public repository unless --yes" guards the wrong thing. The export goes to an endpoint the user names; a private repository is not safer than a public one, and detecting visibility needs a network call that can fail open. The real hazard is writing the tracked scope, which enables telemetry for everyone who clones. Guarding that is a path check: no network, cannot fail open. Same shape as the fix that removed repository-visibility detection from #620.

    Two things added that this issue did not list

    OTEL_RESOURCE_ATTRIBUTES carrying aidd.project_id. Without it a sink receiving several repositories from one machine has nothing to separate them by, and it cannot be repaired after the fact. Derived by the same rule as the journal — git remote get-url origin as owner/repo, falling back to the root's basename — on both sides, stored on neither.

    OTEL_LOGS_EXPORTER, not only metrics. Measured: third-party plugin skill names are replaced with third-party on metric attributes, so every AIDD skill collapses into one bucket there. Only the skill_activated event carries the real name. A metrics-only export answers "what did this session cost" and can never answer "what did the specification step cost".

    OTEL_LOG_TOOL_DETAILS stays off, and the command says so

    It is what makes skill_activated carry the real skill name, so per-step cost appears to depend on it. But it is not selective: it also logs Bash commands, MCP tool names and tool inputs. No setting buys the skill name without the command line.

    #663 is the answer — the framework emitting its own step boundaries removes the need for the flag rather than trading privacy for it. Until it lands, per-step cost is unavailable, and the plan requires enable to print that rather than leave a user to discover the flag's full effect later.

    Also carried in

    The plan requires e2e child processes to be built without GIT_* in their environment. Not a precaution: the journal's own tests shipped with that bug and it surfaced when a commit ran them through lefthook — git exports GIT_DIR inside a hook, so git init in a temp directory operated on the real repository. It failed loudly there; a test that only read would have been silently wrong.

  9. blafourcade commented on Aug 18, 2026

    @blafourcade
    ContributorAuthor

    Replanned: the earlier plan answered the smaller half

    It described how to write an env block into a Claude Code settings file and called that the feature. Two things were wrong with that, and the second matters more.

    It was Claude-shaped, and the framework is not

    Tool Where the export is turned on Who can
    Claude Code env in settings.json the CLI
    Codex [otel] in config.toml the CLI — and it must set metrics_exporter, which defaults to statsig, so enabling without that key ships metrics to a third party nobody chose
    OpenCode experimental.openTelemetry in opencode.json the CLI
    Copilot COPILOT_OTEL_ENABLED, an environment variable no file to write; the CLI can only instruct
    Cursor a team setting, Enterprise plan, beta nobody. We can check it, never set it

    "One command turns the export on" is true for three, partial for one, false for the fifth. The plan now says which is which, and requires the command to report a tool it cannot enable as exactly that — never as enabled.

    It had no AIDD-level switch, and that is the actual guarantee

    A tool may be exporting telemetry for reasons that have nothing to do with this framework: an organisation's collector, an unrelated setting, a default nobody chose. If AIDD's components keyed off "is the provider exporting", then telemetry enabled for one purpose would silently enrol a project in another.

    So there is now one switch AIDD owns, aidd_docs/telemetry.json, committed so it survives a clone and binds the project. Every component reads it — the journal hook first, since it is the one already shipping — checked at the point of use and never cached, so turning it off stops mattering immediately rather than next session. Absent, unreadable or unparseable all mean off: a project that never decided has not consented.

    It is deliberately not in .aidd/. That is the CLI's install manifest, machine-local, and aidd clean removes it — a project-level decision cannot live where a routine cleanup erases it.

    The acceptance criterion this exists for: AIDD off, provider exporting, journal writes nothing.

    Command or skill — decided by what each can do

    A skill runs in a session and is the right place to read state and explain it; that is #617. It is the wrong place to write configuration that must be exactly reversible. A command runs outside a session, which is also the only place that can set an environment variable a session will read, since those resolve at process start.

    So the CLI writes, the skill explains, and neither reimplements the other.

    One thing the earlier plan was about to duplicate

    .aidd/manifest.json already records mergeFiles — which file, which section, which entries, per tool — and clean-use-case.ts already removes exactly those via removeEntriesFromJson. Enabling an export is one more merge entry in machinery that exists and is already exercised by aidd clean. The earlier phase 2 was rewriting it, which would have left the repository with two ways to edit the same file and only one of them undoable.

    One asymmetry worth keeping

    The switch is project-scoped and committed; the per-tool configuration defaults to personal (.claude/settings.local.json). Consenting to be measured is the project's call; sending data from your machine is yours.

    Plan: aidd_docs/tasks/2026_08/2026_08_20_telemetry-export-enable/.

  10. blafourcade commented on Aug 18, 2026

    @blafourcade
    ContributorAuthor

    Two corrections from the cost measurement, both narrowing what this command writes.

    OTEL_LOG_TOOL_DETAILS stays unset, and now for a stronger reason than a trade-off. Per-step cost turned out to be reachable without it — skill_activated timestamps bound the steps, api_request records carry a per-request cost_usd, and the partition reconciles exactly to the vendor total. The flag was only ever needed for the skill name, which the hook stream already has un-redacted. So there is no privacy trade to surface: there is simply no reason to set it.

    OTEL_LOGS_EXPORTER is the one that matters; metrics are optional. The plan already set both, for the right reason. The measurement sharpens it: the metrics stream is redundant with the logs stream and strictly weaker, because at the 60 s default several turns merge into a single datapoint irrecoverably. Keep exporting metrics if a consumer wants session totals cheaply, but nothing downstream may depend on them.

    OTEL_METRIC_EXPORT_INTERVAL therefore matters less than the plan assumed — it protects only the metrics stream, which nothing critical reads. Worth keeping short anyway, and worth not treating as load-bearing.

  11. blafourcade commented on Aug 18, 2026

    @blafourcade
    ContributorAuthor

    Four decisions taken before implementation

    Recording them here because each one closed a question the plan left open, and two of them change files outside this ticket's obvious scope.

    1. .aidd/ is ignored today, and a negation alone will not fix it.

    .gitignore:37 reads .aidd/, which excludes the directory. Git does not descend into an excluded directory, so !.aidd/config.json would never have matched. The rule becomes .aidd/* plus the negation — the same shape as the aidd_docs/runs/* block a few lines below. Proven with git check-ignore -v, not with "the file exists".

    2. The config file is the only authoritative switch.

    Phase 1 left open whether aidd_docs/runs/ existing stays a second condition. It does not. Two conditions that can disagree is the bug shape the plan itself names. The directory is demoted to a location and is created on demand; the file answers the permission question alone.

    3. aidd clean deletes what the manifest recorded, then removes .aidd/ only if empty.

    CleanUseCase.execute ends with a blanket deleteDirectory of .aidd/, which would eat a committed config.json. Rather than special-casing one filename, clean now removes what it recorded writing and leaves anything else alone — the file-level analogue of this plan's own standing rule, "never touch a key we did not add". Three e2e cases assert .aidd/ is gone after a clean; that contract changes deliberately, and the tests change with it rather than being quietly relaxed.

    4. The Claude Code writer takes the endpoint, it does not read it.

    Phase 2 lands before phase 3, and phase 2 forbids an endpoint default. So the writer is a pure function of (endpoint, projectId, scope); phase 3's use-case is what reads .aidd/config.json and calls it. Otherwise phase 2 has to invent a config reader that phase 3 immediately replaces.

    One acceptance criterion under verification

    Phase 2 asks that aidd.project_id be asserted against the journal's own function rather than a copied literal. That means TypeScript in cli/ importing zero-dependency CommonJS from plugins/aidd-telemetry/hooks/lib/repo.js. If that import does not resolve cleanly, the fallback is a shared fixture of remote URL -> expected project_id pairs exercised by both suites — never a copied literal, which is the thing the criterion exists to forbid. Which of the two shipped will be stated when the work lands.

  12. blafourcade commented on Aug 18, 2026

    @blafourcade
    ContributorAuthor

    Implemented — four phases, reviewed, findings fixed

    All four phases land together: the switch, Claude Code's export block, aidd telemetry on|off, and the end-to-end journeys. 2250 CLI tests and 128 hook tests pass; biome, tsc and knip are clean. Nothing is committed yet.

    What a review pass caught that the phases did not

    Three axes ran over the finished diff — clean code, the acceptance tables, and whether the change belongs. Five findings, all fixed.

    The GIT_DIR criterion was proven vacuously. Phase 4 asks that the suite pass with GIT_DIR exported, "proving it under a git hook". It did pass — but both harnesses strip GIT_* before the hook or the binary is ever spawned, so the hook's own defence was never exercised. Deleting that defence would have failed no test in either suite.

    The replacement hands the hook the poisoned environment git itself exports. Reverting the fix now produces the real symptom:

    actual:   'acme/elsewhere'
    expected: 'acme/here'
    

    A session recorded against the wrong repository. That is what the criterion was for, and it took a test that actually lets the variable through to see it. The same defect existed in the CLI's GitAdapter.getRemoteUrl, which is the other half of the project_id join — both sides now strip, and each has a test that fails without it.

    Only Claude was thought about per tool. Claude's status was gated on the manifest; Codex, OpenCode, Copilot and Cursor reported unconditionally. A Claude-only project was told "Codex: not yet supported, tracked in #653" — a roadmap item aimed at nobody. All five are now gated the same way.

    A TypeScript declaration was shipping to every user project. Phase 2 put a .d.ts beside the hook so the CLI's tests could import it, and the plugin build copies hooks/ recursively into every user project. docs/ARCHITECTURE.md already forbids exactly this: "A plugin never contains its own tests." The first attempt filtered .d.ts out of the build — patching around the misplacement rather than fixing it. The typing now lives in cli/tests/, and the filter came back out.

    Two smaller ones: a declaration narrower than the runtime it described (dissolved by the move), and an e2e test named for a gate the code does not have.

    Two decisions from the earlier comment, revised by contact with the code

    git check-ignore -v cannot prove a negation. Exit status is 0 for any matched pattern, negations included — verified on git 2.50.1. git ls-files --others --exclude-standard and git status --ignored carry the proof instead.

    Enabling the export needs an endpoint, and there is no default. aidd telemetry on --endpoint <url> writes it into the switch and later runs reuse it; neither flag nor stored value is a hard error that writes nothing. Not even localhost, per phase 2.

    Two limits worth stating rather than burying

    "Byte-identical" holds for a canonically-formatted settings file. removeEntriesFromJson rewrites the whole file through JSON.stringify(x, null, 2), so a file with different indentation round-trips semantically but not byte-for-byte. Pre-existing behaviour of the merge machinery this plan directed reusing, not a regression — but the criterion's wording claims more than the code guarantees.

    --scope user records its undo path relative to the project root. Move the project and aidd telemetry off may not find ~/.claude/settings.json to clean. The command says so when you use that scope, and warns by name when an entry no longer resolves.

    Full report at aidd_docs/tasks/2026_08/2026_08_20_telemetry-export-enable/review.md.

  13. blafourcade commented on Aug 18, 2026

    @blafourcade
    ContributorAuthor

    Testing it end to end found a critical bug the 2250 tests could not

    The request was to prove the whole chain on a real installation rather than trust the suite. That turned up something no acceptance criterion covered.

    The journal hook was dead on every real installation.

    aidd plugin install copies a plugin's hooks/ tree, and plugin-content-translator.ts kept only each file's basename:

    const filename = file.relativePath.split("/").at(-1) ?? "";

    So hooks/lib/repo.js shipped as hooks/repo.js, while journal.js still asked for ./lib/repo.js. Running the installed hook:

    Error: Cannot find module './lib/host.js'
    exit=1
    

    Every session on every installed project. Nothing recorded, ever.

    Why 2250 tests missed it. All of them execute the hook from the source tree, where hooks/lib/ exists. Installation is the only place the layout changes, and nothing ran the hook after installing it. The unit tests were right about the code and wrong about the product.

    aidd-telemetry is the only plugin with a subdirectory under hooks/, which is why nothing else broke.

    The fix keeps a hook's own directories, and comes with two guards: a unit test on the path shape, and an e2e that installs the plugin and executes the installed copy. Reverting the fix reproduces the original error in that e2e.

    The chain, proven on a real installation

    Real project, real git remote, real binary, plugin installed through aidd plugin install:

    Step Result
    Hook with no .aidd/config.json exit 0, nothing written
    aidd telemetry on --endpoint … switch written, env block in .claude/settings.local.json, four other tools reported not installed
    Hook with the switch on a run record, project_id: aidd-lab/telemetry-proof derived from the project's own remote
    aidd telemetry off switch to enabled: false with the endpoint preserved, the env entries removed
    Hook with the switch off exit 0, no second record

    The record, written by the installed hook:

    {
      "schema_version": 1,
      "run_id": "01M0AT5C89VKKYP5B519RVFYYD",
      "project_id": "aidd-lab/telemetry-proof",
      "tool": "claude-code",
      "vendor_id": "9f1c2d34-aaaa-4bbb-8ccc-000000000001",
      "vendor_field": "session.id",
      "parent_run_id": null,
      "started_at": "2026-08-18T16:09:38Z",
      "ended_at": "2026-08-18T16:09:38Z",
      "tasks": [{ "task_id": null, "from": "2026-08-18T16:09:38Z", "to": null }]
    }

    2252 tests pass. What is still unproven is the last hop: that the env block actually makes Claude Code export, and that the cost lands somewhere readable. That needs a real paid session against a listening collector, and it is what #647 is for.

  14. moved this from In Progress to Done in AIDD Roadmapon Aug 20, 2026
  15. closed this as completedby moving to Done in AIDD Roadmapon Aug 20, 2026
  16. blafourcade commented on Aug 20, 2026

    @blafourcade
    ContributorAuthor

    What shipped, for the record

    Per-tool activation is a declaration in the domain, not a branch in the use-case. Five tools, five different declarations, and the command reports all five states rather than skipping the four it cannot configure:

    Tool Declares What the command actually does
    Claude Code settings-file writes the OTLP env block, merge-tracked so off removes exactly those keys
    Copilot environment-variable writes nothing; prints the variable to export, since AIDD does not set env vars on someone's behalf
    Cursor external writes nothing; an Enterprise team setting nobody here can turn on
    Codex planned writes nothing, tracked in #653
    OpenCode planned writes nothing, tracked in #653

    Three decisions that were argued and could be reversed by accident:

    • No default endpoint, not even localhost. An absent endpoint is a caller error; nothing is written.
    • OTEL_METRIC_EXPORT_INTERVAL is 10 s, not the 60 s default, so a session shorter than a minute still flushes its counters.
    • OTEL_LOG_TOOL_DETAILS is never set. It would log every Bash command line, MCP tool name and tool input. The install notice says so out loud rather than leaving a user to find out.
  17. blafourcade commented on Aug 28, 2026

    @blafourcade
    ContributorAuthor

    Closed as completed, but this issue no longer describes what shipped

    Found by a backlog audit on 2026-08-28, verified against the code rather than the text.

    All eight acceptance boxes name aidd telemetry enable / disable. That command does not exist:

    $ aidd telemetry enable
    error: unknown command 'enable'
    

    The shipped surface is on / off, and the divergence is not only spelling. on deliberately stopped arming exports (commit 38de97d8): it writes the project switch and git-ignores the journal, nothing else. Arming a tool to export moved to aidd telemetry endpoint <url>, which nothing else turns on. So this issue's own title — "one command that turns the provider export on" — is now inverted: the command it describes was split in two on purpose, so that allowing measurement locally never implies sending anything anywhere.

    Two more of its statements no longer hold:

    • .aidd/telemetry.json is referenced nowhere. The switch lives in .aidd/config.json under telemetry.enabled, read strictly (=== true, so a half-written config reads as off).
    • Consent is never recorded. grep -rn "consent" cli/src finds one comment and no stored date or identity.

    Not reopening. The outcome — a person can turn measurement on with one command — is delivered, and better than described. Leaving the text standing is the problem: anyone reading this issue to learn the interface learns a command that errors.

    Worth knowing for whoever amends it: the tick state of a box carries no information in this repository. Eight of nine closed issues in these milestones have every box unticked, including ones whose work is genuinely complete. Unticked boxes here are not evidence of anything, which is why this was verified from the code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Fields

Priority

High

Projects

Relationships

None yet

Development

No branches or pull requests

Issue actions