Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# TokenEconomizer

A plugin for Claude Code, Codex, the Copilot CLI and Pi (experimentally opencode, Cursor, Grok Build,
Hermes Agent, the Antigravity CLI and Factory Droid) that reads finished sessions and changes what
Hermes Agent, the Antigravity CLI, Factory Droid and ZCode) that reads finished sessions and changes what
later sessions in the same repository pay for: micro-skills, tool pruning, and a loop guard.
See `README.md` for the overview and the method, `docs/PLUGIN.md` for the design and the verified
agent behavior, and `docs/ARCHITECTURE.md` for how the package fits together.
Expand All @@ -12,7 +12,7 @@ agent behavior, and `docs/ARCHITECTURE.md` for how the package fits together.
- `toolprune/` — unused-tool evidence and the deny-list proposal; Codex capability packages
- `loopguard/` — the per-call failure-loop guard and its per-repository envelope
- `trace/` — each agent's session-log format (Claude Code, Codex, Copilot CLI, Pi, opencode,
Cursor, Grok Build, Hermes Agent, Antigravity CLI, Factory Droid)
Cursor, Grok Build, Hermes Agent, Antigravity CLI, Factory Droid, ZCode)
- `plugin/` — the runtime: hook dispatch, per-repository state, settings, the `te` CLI
- `config.py`, `ledger.py`, `pricing.py`, `domain.py`, `stage.py` — shared by all three
- `plugin/` — the installable plugin directory: manifests per agent, hook files, launchers
Expand All @@ -21,7 +21,8 @@ agent behavior, and `docs/ARCHITECTURE.md` for how the package fits together.
skills (`codex-skills/`, Codex having no plugin slash commands), the Pi extension, the opencode
plugin (`opencode/`, one file whose default export serves 1.x via `server` and 2.x via
`setup`), the Hermes plugin, and the Droid manifest (`plugin/.factory-plugin/`, with
`plugin/hooks/droid-hooks.json`); `lib/` links the package
`plugin/hooks/droid-hooks.json`), and the ZCode hook entries (`plugin/hooks/zcode-hooks.json`, added
to a repository's `zcode.json`); `lib/` links the package
- `experiments/` — benchmark-only: `te_bench/` (adapters, arms, facade, benchmark-only
controllers), `te_harness/` (Docker runners), `scripts/`, `workloads/`, its own `tests/`;
its `README.md` is the run book
Expand Down
13 changes: 11 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ mechanisms:

TokenEconomizer is a plugin for **Claude Code**, **Codex**, the **GitHub Copilot CLI** and **Pi**,
with experimental support for **opencode**, **Cursor Agent**, **Grok Build**, **Hermes Agent**,
the **Antigravity CLI** and **Factory Droid**. Each italic
the **Antigravity CLI**, **Factory Droid** and **ZCode**. Each italic
letter above is a setting, shown with its default ([Settings](#settings)), and each mechanism can be turned on or
off. The plugin announces each action in one line, and if any part of it fails, your session
carries on as if it were not installed. Its model calls (the fact proposer, and the judge with the
Expand Down Expand Up @@ -185,6 +185,14 @@ micro-skills' paid proposer and the loop guard's judge stay off until `droid_sid
(when on, they may use only Droid's read-only `Read` tool).
See **[`docs/DROID.md`](docs/DROID.md)** for what works and how to import earlier sessions.

**ZCode** (experimental; micro-skills, the loop guard and session cost, not tool pruning)

ZCode has no plugin format this repository builds. Add the entries of `plugin/hooks/zcode-hooks.json`
to the `hooks.events` of the repository's `zcode.json`, then trust them (`zcode hooks trust
review`). The fold reads ZCode's SQLite store, not the hook's own transcript. Its `-p` side call
stays off until `zcode_sidecall` is on. See **[`docs/ZCODE.md`](docs/ZCODE.md)**. Kimi Code is
not supported yet; **[`docs/KIMI.md`](docs/KIMI.md)** says what is missing.

**Cursor Agent** (experimental)

```sh
Expand Down Expand Up @@ -380,6 +388,7 @@ are these settings:
| *r* | `loopguard_nudges` | 1 | redirects per turn before the guard only tells you |
| *b* | `loopguard.backstop_mult` | 2.5 | how many times its usual length a churning turn may run before it is redirected without the judge |
| | `droid_sidecall` | off | on Droid, allow the plugin's own model calls, which run `droid exec` |
| | `zcode_sidecall` | off | on ZCode, allow the plugin's own model calls, which run `zcode -p` |

Change them with `config set`, shown here in Claude Code's form (other agents type it as in
[Everyday commands](#everyday-commands)):
Expand Down Expand Up @@ -501,7 +510,7 @@ agent's own limits; afterwards, `te status` counts the redirects and "still stuc
| [`tokeneconomizer/plugin`](tokeneconomizer/plugin/) | The runtime the hooks and the `te` command run |
| [`plugin`](plugin/) | The installable plugin: manifests, hook files, launchers, slash commands, the Pi extension, the opencode and Hermes plugins, the Droid bundle (`.factory-plugin/`) |
| [`experiments`](experiments/) | The harness, launchers and task sets behind the paper's results |
| [`docs`](docs/) | `PLUGIN.md` (design, verified agent behavior), `ARCHITECTURE.md` (how the package fits together) and `DROID.md` (the Factory Droid adapter) |
| [`docs`](docs/) | `PLUGIN.md` (design, verified agent behavior), `ARCHITECTURE.md` (how the package fits together), `ZCODE.md` and `KIMI.md` (the ZCode adapter; why Kimi Code has none) and `DROID.md` (the Factory Droid adapter) |
| [`tests`](tests/) | Offline tests for the package and the plugin |

The package is standard-library only. Run the tests with `python -m pytest tests` (add
Expand Down
39 changes: 39 additions & 0 deletions docs/KIMI.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# TokenEconomizer on Kimi Code

Kimi Code 2.x is not supported. It is observe-only: there is no adapter, no reader and no hook
entry, and `tests/test_kimi.py` pins that. This note says why and what is needed.

## What is known (Kimi Code 2.1.1)

- The binary is `~/.kimi-code/bin/kimi`. The Python `kimi_cli` in `~/.kimi/` is the older 1.x.
- A session is `~/.kimi-code/sessions/wd_<name>_<hash>/session_<uuid>/agents/main/wire.jsonl`, one
JSON record per line with a `type` key. The types seen are `metadata`, `config.update` and
`tools.set_active_tools`. No record of a prompt, a tool call or a tool result has been seen.
- Hooks are `[[hooks]]` in `~/.kimi-code/config.toml` only. Events include `SessionStart`,
`UserPromptSubmit`, `PreToolUse`, `PostToolUse`, `Stop` and `SessionEnd`. Exit 2 blocks on
`UserPromptSubmit`, `PreToolUse` and `Stop`; other failures let the call go on. There is no
`updatedInput`.
- Context: the docs say stdout of an exit-0 hook "may be appended to context". There is no
`additionalContext` field. This has not been tried.
- Side call: `kimi -p <prompt>` runs under the `auto` permission policy and has no flag that
removes tools, so a side call could act on text a repository controls.

## What is missing, and why no reader was written

A reader written from the three record types above would have to guess where a prompt and a tool
result are, and would pass its own tests while counting the wrong things. Two pieces of evidence
come first. Each has a skipped test in `tests/test_kimi.py` that becomes a real one.

1. **A multi-turn `wire.jsonl`.** In one Kimi Code session, send two prompts; make one of them
run a shell command and read a file. Then list the record `type` and the key names of every
line of that session's `agents/main/wire.jsonl` (types and key names only, no text) and save
them as `tests/fixtures/kimi_wire_types.json`. A one-line `jq` over the file is enough:
`jq -c '[.type, (keys)]' wire.jsonl`.
2. **An injection test.** In a scratch directory, add a `UserPromptSubmit` `[[hooks]]` entry to
`~/.kimi-code/config.toml` whose command prints a made-up sentence and exits 0. Ask Kimi Code
to repeat the sentence the hook printed. Record whether it can.

If the sample has the three records and the injection works, the adapter is the ZCode one with
a `wire.jsonl` reader (`trace/kimi.py`) instead of the SQLite one, and plain-stdout context. If
the injection fails, Kimi stays observe-only with micro-skills' block and the loop guard's judge
off: nothing can be injected, and its side call cannot be restricted.
32 changes: 32 additions & 0 deletions docs/PLUGIN.md
Original file line number Diff line number Diff line change
Expand Up @@ -720,6 +720,38 @@ was not.)
call also writes a session log of its own, so side calls stay off until `droid_sidecall` is
on (or `TE_DROID_SIDECALL=1`).

**ZCode 0.17.1 -- adapter built 2026-11-12, read from its code, not run live:**

* **Offline, tested here:** the session reader over a synthetic copy of ZCode's SQLite store (its
JSON keys were read off a real store, keys only); prompt counting; usage; host detection; the
`zcode -p` side call's command; and the hooks end to end through `hook.main`, with a payload
built as ZCode's own code builds it. **Not done:** a live ZCode session with the hook file
(`docs/ZCODE.md`, "Verify it"), and a real `zcode -p` call (what it prints, whether
`--disallowed-tools` keeps out every tool).
* The hook contract is Claude-shaped: `session_id`, `tool_name`, `tool_input`, `tool_use_id`,
`tool_response`, `hook_event_name`, beside ZCode's own camelCase fields (`hookEventName`,
`sessionId`, `toolName`, `turnId`, `agentName`, `mode`). Events: `SessionStart`,
`UserPromptSubmit`, `PreToolUse`, `PostToolUse`, `PostToolUseFailure`, `Stop`, and no
`SessionEnd`. Its output schema takes `hookSpecificOutput.additionalContext` (the event's own
name in `hookEventName`), a top-level `additionalContext` and `systemMessage`; the contexts of
SessionStart, UserPromptSubmit, PostToolUse, PostToolUseFailure and Stop are added to the
conversation as a `hook_context` entry, cut at 24,000 characters. `systemMessage` is used only to
explain a block, and `continue: false` is a block request on every event but Stop.
* `transcript_path` and `transcriptPath` are one temporary file, `zcode-hook-*/transcript.jsonl`,
with the event's prompt or answer, deleted after the hook. That name is how `hosts.detect` tells
ZCode from Claude Code and from Grok (which also sends `hookEventName`). The log is the SQLite
store, `<ZCODE_STORAGE_DIR or ~/.zcode>/cli/db/db.sqlite`.
* ZCode's tool names are Claude Code's. `model_usage.input_tokens` counts the cached reads (total
= input + output on every row of a real store, counted by aggregate queries only;
`cache_creation_input_tokens` was 0 on all of them); the ledger's input does not, so the reader subtracts them.
* **Side call.** `zcode -p <text> --mode plan --cwd <empty dir> --disallowed-tools <names>`; `-p`
has no `--model`, and defaults to `yolo`. A deny list cannot be inverted, so the call is off
until `zcode_sidecall` is on (or `TE_ZCODE_SIDECALL=1`).

**Kimi Code 2.1.1 -- not supported** (`docs/KIMI.md`): the `wire.jsonl` record that carries a
prompt or a tool call has not been seen, and whether an exit-0 hook's stdout reaches the model has
not been tried.

## 4. State and privacy

All state is local, under `$TOKENECONOMIZER_HOME` (default `~/.tokeneconomizer`), one directory
Expand Down
78 changes: 78 additions & 0 deletions docs/ZCODE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
# TokenEconomizer on ZCode

ZCode (Z.ai) support is experimental. It was written against ZCode `0.17.1` on macOS from its
hook code, its command-line help and the schema of its session store. It has not yet been run
under a live ZCode session with this plugin's hook file: the "Verify it" steps below do that.

## What works, and what does not

| Mechanism | ZCode | Why |
|---|---|---|
| Micro-skills (repository facts at session start) | yes, unverified live | `SessionStart` and `UserPromptSubmit` take `additionalContext`; ZCode adds it to the conversation as a `hook_context` entry |
| Session cost / effective tokens | yes | `model_usage` in the store; the cached part is taken out of `input_tokens`, which counts it |
| Loop guard (redirect a failing loop) | partially | `PostToolUse` takes `additionalContext`; the judge needs `zcode -p`, which is off by default |
| History import (earlier sessions) | no | the store holds them, but no importer reads it yet |
| Tool pruning | no | ZCode has no per-tool setting a hook can write |

## How a session is read

ZCode keeps every session in one SQLite file, `<ZCODE_STORAGE_DIR or ~/.zcode>/cli/db/db.sqlite`
(tables `session`, `message`, `part`, `tool_usage`, `model_usage`). The hook's own
`transcript_path` is no log: ZCode writes the one prompt or answer of the event to a temporary
file and deletes it when the hook returns. The fold therefore opens the store read-only,
copies one session's tool calls, exit codes and token usage into a file of the plugin's
(`<state>/zcode-sessions/<session id>.json`), and reads that. The copy has no words: a prompt is
counted and not kept, and what the model said or reasoned is not read.

ZCode has no `SessionEnd`. A session is folded by the next session start in its repository once
it has been quiet, as on the Antigravity CLI.

## How ZCode is told from Claude Code

ZCode's payload has `session_id`, `tool_name`, `tool_input` and the other Claude names, so
nothing in them tells it apart. Grok Build also sends the camelCase `hookEventName`. What only
ZCode sends is the transcript: both `transcript_path` and `transcriptPath` hold
`<tmpdir>/zcode-hook-<random>/transcript.jsonl`, and `detect` takes a payload for ZCode only when
both are that, and equal. The environment is not a mark: `ZCODE_SESSION_ID` is set for ZCode's
hooks and shell, and an agent started from that shell inherits it, so it only chooses the host of
a `te` command run in a shell (`default_host`), after every other agent's variable. The hook file
passes `--host zcode`, so detection is the fallback.

## Install

ZCode reads workspace hooks from `zcode.json` at the root of a repository (its schema is
`{"hooks": {"enabled": true, "events": {...}}}`) and runs a workspace hook only after the person
trusts it (`zcode hooks trust review`). `plugin/hooks/zcode-hooks.json` holds the five entries;
add them to the `events` of the repository's `zcode.json`, after any entry that guards a tool,
and point the command at the clone if it is not in `~/.tokeneconomizer/src`.

## Verify it

1. Start `zcode` in a repository where the entries are trusted. Run
`~/.tokeneconomizer/src/plugin/scripts/te --host zcode status`: a session is listed after its
first hook.
2. After a session that ran a shell command, run `te --host zcode fold` (or start another session
in the same repository after a quiet period). `te --host zcode status` shows the folded
session with its prompts and tokens.
3. To see that a learned fact reaches the model, ask ZCode to repeat what its session start added.

## Side calls

`te` asks the model for facts and for a verdict on a stuck turn through `zcode -p`, in a throwaway
directory, in `plan` mode (its `-p` default is `yolo`), with the acting tools removed by
`--disallowed-tools`. That option is a deny list, and a tool an MCP server adds is not on it;
the call also writes a session into the store and reports no usage. So it is off until
`te --host zcode config set zcode_sidecall=on` (or `TE_ZCODE_SIDECALL=1`). `zcode -p` has no
`--model` flag, so there is no `zcode_paid_model`: the call uses the model in ZCode's settings.
What `zcode -p` prints has not been checked against a real model: the plugin takes its stdout.

## Known limits

- Not run end to end under ZCode; the reader is tested on a synthetic store whose JSON keys were
read off a real one.
- ZCode's `hook_context` entries after a resumed session were not traced, so `keeps_start_context`
keeps its default (the context is not added again on resume).
- A subagent's own hook calls carry its session id; its rows are counted in the parent's usage,
and a session forked from another is a session of its own.
- `model_usage.cache_creation_input_tokens` was 0 in every row seen, so cache writes are not
priced.
62 changes: 62 additions & 0 deletions plugin/hooks/zcode-hooks.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
{
"hooks": {
"enabled": true,
"events": {
"SessionStart": [
{
"hooks": [
{
"type": "command",
"command": "\"$HOME/.tokeneconomizer/src/plugin/scripts/te-hook\" --host zcode",
"timeout": 15
}
]
}
],
"UserPromptSubmit": [
{
"hooks": [
{
"type": "command",
"command": "\"$HOME/.tokeneconomizer/src/plugin/scripts/te-hook\" --host zcode",
"timeout": 20
}
]
}
],
"PostToolUse": [
{
"hooks": [
{
"type": "command",
"command": "\"$HOME/.tokeneconomizer/src/plugin/scripts/te-hook\" --host zcode",
"timeout": 60
}
]
}
],
"PostToolUseFailure": [
{
"hooks": [
{
"type": "command",
"command": "\"$HOME/.tokeneconomizer/src/plugin/scripts/te-hook\" --host zcode",
"timeout": 60
}
]
}
],
"Stop": [
{
"hooks": [
{
"type": "command",
"command": "\"$HOME/.tokeneconomizer/src/plugin/scripts/te-hook\" --host zcode",
"timeout": 10
}
]
}
]
}
}
}
2 changes: 1 addition & 1 deletion tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -58,5 +58,5 @@ def _no_model_calls(request, monkeypatch):
if request.node.get_closest_marker("real_sidecalls"):
return
for name in ("claude_call", "codex_call", "copilot_call", "droid_call", "grok_call",
"hermes_call", "pi_call"):
"hermes_call", "pi_call", "zcode_call"):
monkeypatch.setattr(sidecall, name, lambda *a, **k: ("", None))
Loading