Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,18 @@

All notable changes to cc-autodream. Format loosely follows Keep a Changelog.

## 2026-10-29

### Changed
- **The engine that writes the summary now reviews every harness's sessions in L1 by default.** Which harness holds a transcript and which engine reads it used to be one choice: each adapter's sessions went to its own engine. A run now has one reviewer, `AUTODREAM_L2_ENGINE` (else the first enabled adapter), and it reads every source's sessions unless `AUTODREAM_L1_ENGINE` (every source) or `AUTODREAM_L1_ENGINE_<NAME>` (one source) picks another. The source's own adapter still enumerates, linearizes and adds its triage notes; only the model call, its warmup and the network probe move. A host with one harness behaves exactly as before. **Upgrade note:** a host with both `claude` and `omp` enabled and `AUTODREAM_L2_ENGINE=omp` moves every Claude Code session from Haiku on the claude CLI to omp's L1 model on this upgrade. Set `AUTODREAM_L1_ENGINE_CLAUDE=claude` to keep them where they were.
- **`AUTODREAM_L1_MODEL_<NAME>` is keyed by the engine that makes the call, not by the harness that holds the session.** A model id belongs to an engine, so `AUTODREAM_L1_MODEL_OMP` is the model of every L1 call on the omp engine, whichever harness the session came from, and never reaches the claude CLI. A host whose harnesses each review their own sessions sees no change. One that moves a harness to another engine pins that engine's model, the same keying `AUTODREAM_L2_MODEL_<NAME>` already had.
- **A session with no usable L1 engine is retried once a later run gives it one.** Its error record counted as done, which held while the only cause was an adapter missing on disk. An engine choice naming no accepted adapter, a missing model and a failed login check are config, so the record now carries `no_engine: true`, and a later run of the date (a catch-up or `AUTODREAM_FORCE=1`) that resolves an engine and a model for the session's source drops it and triages the session.

### Added
- **`AUTODREAM_SESSION_ROOTS_<NAME>`** lists the session roots of one adapter other than claude, colon-separated, in place of its manifest's defaults, for a harness that keeps one store per profile.
- **Who each engine runs as, and which model answered each L1 call.** Each engine the run uses reports its login once through a new adapter verb, `whoami` (claude: `claude auth status --json`, no model call), logged and written to run-stats as `login_<engine>`. The claude L1 worker runs with `--output-format json`, and the new `served` verb reads the models that answered, the CLI's cost estimate at API prices and a verdict, `match` or `mismatch`, that knows the CLI's naming (`haiku` answers as `claude-haiku-5-5`). Every call goes to `findings/<date>/l1-served.tsv`. run-stats gains `l1_engine_<source>`, `l1_served_models`, `l1_served_mismatch`, `l1_served_unmeasured` (calls whose engine reports no model, such as omp), `l1_cost_usd_estimate` and `l1_cost_unmeasured` (calls left out of that sum, so an all-omp night does not read as free).
- **Both are record-only unless the host asks.** `AUTODREAM_REQUIRE_LOGIN` (space-separated `provider=`, `auth=`, `plan=`, `account=` pairs) hands an L1 engine whose login does not match no transcript, through the no-engine record above, and stops the run before it starts when the L2 engine does not match. `AUTODREAM_L1_REQUIRE_SERVED=1` discards the findings of any L1 call whose verdict is not `match`, an engine that cannot report one included. The verdict is stored beside every accepted answer (`<answer>.served`), and before L1 the run drops cached findings and chunk answers from earlier runs that no stored verdict vouches for and triages them again (`l1_cache_dropped`).

## 2026-10-27

### Fixed
Expand Down
5 changes: 3 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,8 +108,9 @@ plus env `CLAUDE_CODE_DISABLE_CLAUDE_MDS=1 DISABLE_TELEMETRY=1 DISABLE_ERROR_REP

- **The source map is held in the environment, not read from the findings dir.** `AUTODREAM_SOURCE_MAP` (hash and adapter per session) and `AUTODREAM_L1_MODELS` (model per adapter) are built before the first model call. An L1 worker holds the Write tool and could rewrite `sessions-source.txt`, and the source decides which binary the next worker executes. `test_l1_engine_cannot_be_redirected_by_a_worker` has a worker do exactly that and checks the next session still runs on the real engine; reading the file instead fails it.
- **Haiku 5.5, effort and a replaced system prompt (2026-10-07).** The claude manifest's `l1_model` is `claude-haiku-5-5` (verified: `--model claude-haiku-5-5` and the `haiku` alias both answer as it; an id the CLI does not know falls back silently, so re-check `modelUsage` in `--output-format json` when changing it). `l1-argv` and `warmup-argv` add `--effort` from `AUTODREAM_L1_EFFORT_CLAUDE`, then `AUTODREAM_L1_EFFORT`, then `high`; `off` omits it and an unknown level exits 2. They also pass `--system-prompt` instead of `--append-system-prompt`, which drops Claude Code's default prompt: one-word call 6.9k -> 5.0k input tokens, and a real triage run on a 250 KB slice cost $0.159 against $0.215 with the same number of findings. L2 is untouched.
- **The model** resolves `AUTODREAM_L1_MODEL_<ADAPTER>`, then `AUTODREAM_L1_MODEL`, then the manifest's `l1_model` (`adapter_l1_model`). A model id belongs to one engine, which is why a host with two engines pins them per adapter.
- **No engine is a deterministic error.** A session whose adapter is not executable, or resolves no model, gets a findings record with `error: no L1 engine for this session`, left in place and counted by `l1_findings_with_error`. Retrying cannot change the answer, and the run still reports.
- **The engine, then the model.** A source's L1 engine is `AUTODREAM_L1_ENGINE_<SOURCE>`, then `AUTODREAM_L1_ENGINE`, then the run's reviewer (the L2 engine) (`adapter_l1_engine`). The model is keyed by that engine, not by the source: `AUTODREAM_L1_MODEL_<ENGINE>`, then `AUTODREAM_L1_MODEL`, then the engine's manifest `l1_model` (`adapter_l1_model`). A model id belongs to one engine, so a source-keyed pin sent omp sessions to the claude CLI with `--model deepseek/...` once a claude reviewer read them (PR #165 review). The engine choice is per source and the model pin per engine, the same as `AUTODREAM_L2_MODEL_<NAME>`.
- **What answered, and whether it counts.** Every L1 call's stdout goes through the engine adapter's `served` verb with the requested model: it returns the models that answered, the cost estimate and a verdict, `match` or `mismatch`, judged by the adapter because only it knows its CLI's naming (claude: `haiku` answers as `claude-haiku-5-5`). An engine with no `served` verb (omp) is `unmeasured`. The verdict is the sixth column of `l1-served.tsv` (run-stats `l1_served_mismatch`, `l1_served_unmeasured`) and is stored beside every accepted answer, findings JSON or chunk answer, as `<answer>.served`. With `AUTODREAM_L1_REQUIRE_SERVED=1` a call whose verdict is not `match` loses its answer, and before L1 `drop_unacceptable_l1_cache` holds the cache to the same rule: a findings JSON or chunk answer from an earlier run with no verdict or a non-`match` one is dropped and triaged again (`l1_cache_dropped`). A record with `error` or `skipped` and `findings: []` is kept, because the runner writes those itself and the post-L1 passes add fields to them, so their shape alone cannot tell them from a worker's. The empty array is the test, not the key: a worker can write either key, but every record the runner writes has no findings, so one with findings is model output and is held to its verdict.
- **No engine is an error for this run's config, not forever.** A session whose engine is not executable, resolves no model, names no accepted adapter or fails `AUTODREAM_REQUIRE_LOGIN` gets a findings record with `no_engine: true` and `error: no L1 engine for this session`, counted by `l1_findings_with_error` and skipped by the retry rounds, since retrying in the same run cannot change the answer, and the run still reports. Every one of those causes is config, which can change between runs, so `drop_unacceptable_l1_cache` drops the record in a later run (a catch-up or `AUTODREAM_FORCE=1`) once that run resolves an engine and a model for the session's source, read from `AUTODREAM_SOURCE_MAP` and never from the record. Records written before the marker are recognised by their error text. Either way the record must have `findings: []`: one with findings is model output, and with `AUTODREAM_L1_REQUIRE_SERVED=1` it is held to its verdict like any other.

## Self-pollution (the eating-its-own-tail bug)

Expand Down
12 changes: 8 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,9 +78,11 @@ cd ~/git/autodream
```

`--adapters` lists the harnesses to read, and `--l2-engine` names the one whose CLI runs the
aggregation pass. Pin a model per harness in `config` with `AUTODREAM_L1_MODEL_<NAME>` and
`AUTODREAM_L2_MODEL_<NAME>` (for example `AUTODREAM_L1_MODEL_OMP`). A host with only one harness
installed needs none of this.
aggregation pass. That engine also reviews every harness's sessions in L1 unless `config` says
otherwise: `AUTODREAM_L1_ENGINE_<NAME>=<name>` keeps one harness on its own engine (for example
`AUTODREAM_L1_ENGINE_CLAUDE=claude`). Pin a model per engine with `AUTODREAM_L1_MODEL_<NAME>` and
`AUTODREAM_L2_MODEL_<NAME>` (for example `AUTODREAM_L1_MODEL_OMP`, the model of every L1 call the
omp engine makes). A host with only one harness installed needs none of this.

This symlinks `bin/*.sh`, `prompts/*.md` and the `adapters/` tree into `~/.claude/autodream/`, creates
`~/.claude/dreams/`, and on macOS installs and bootstraps the nightly launchd
Expand Down Expand Up @@ -223,7 +225,9 @@ Commands, fallbacks and how to test it are in [`mods/autodream-band/README.md`](
the changelog window, the review popup dedup) and the OMP session reader is `adapters/omp/`.
To move an existing OMP install: unload its launchd jobs, run
`./install.sh --adapters claude,omp --l2-engine omp`, and carry its L1 model over as
`AUTODREAM_L1_MODEL_OMP`. Do not copy its `SESSION_ROOTS` line: the omp adapter finds
`AUTODREAM_L1_MODEL_OMP`. With omp as the L2 engine it reviews the Claude Code sessions too, on
that model; add `AUTODREAM_L1_ENGINE_CLAUDE=claude` to keep them on Claude Code's own engine and
model. Do not copy its `SESSION_ROOTS` line: the omp adapter finds
`~/.omp/agent/sessions` itself, and a stray root would make the Claude adapter parse OMP files.
The full runbook, with rollback, is in `docs/plans/2026-10-03-omp-adapter.md`.

Expand Down
39 changes: 39 additions & 0 deletions adapters/claude/adapter.sh
Original file line number Diff line number Diff line change
Expand Up @@ -170,11 +170,15 @@ case "$cmd" in
# adapter: tests/adapter-claude.sh pins it against that literal text, so a change here
# that alters what the nightly runs has to say so. NO shell expansion of the paths in
# the system prompt, hence the escaped dollar sign in the text.
# --output-format json makes stdout one small result object that names the models that
# answered (read back by `served`). Not stream-json: --verbose would echo every Read result,
# the whole transcript, onto stdout, and a failed worker copies its stdout into the .err.
effort=$(l1_effort) || exit 2
set -- "$1" ${effort:+--effort "$effort"}
model=$1; shift
printf '%s\0' "${CLAUDE_BIN:-$HOME/.local/bin/claude}" \
--print \
--output-format json \
--permission-mode bypassPermissions \
--model "$model" \
"$@" \
Expand Down Expand Up @@ -207,6 +211,41 @@ case "$cmd" in
--system-prompt 'Reply with the single word ok and exit.'
;;

served) # $1=stdout of one L1 call $2=the model it asked for
# -> "<model>[,<model>...]<TAB><estimated cost USD><TAB><match|mismatch>", exit 1 without a result
# The models come from the result object's modelUsage, which lists every model that answered
# any part of the call. The cost is the CLI's estimate at API prices, not what a subscription
# bills; it is kept because it compares models on real numbers.
# The verdict is match when every model that answered is the one asked for, judged here because
# only this adapter knows the CLI's naming: modelUsage carries canonical ids (claude-haiku-5-5)
# while a host may ask by alias (haiku). A bare family name matches any id of that family
# (claude-<name>-...), a full id only itself, and a context suffix such as [1m] is ignored on
# both sides. An alias the family rule cannot read (default, opusplan) is a mismatch: the
# runner then cannot vouch for the model, which is what AUTODREAM_L1_REQUIRE_SERVED asks.
[ "$#" -ge 1 ] && [ -r "$1" ] || exit 1
line=$(jq -r --arg want "${2:-}" 'select(type == "object" and .type == "result")
| ((.modelUsage // {}) | keys | sort) as $m
| ($want | sub("\\[[^\\]]*\\]$"; "")) as $w
| [($m | join(",")), ((.total_cost_usd // "") | tostring),
(if ($m | length) > 0 and all($m[]; sub("\\[[^\\]]*\\]$"; "") as $s
| $s == $w or (($w | test("^[a-z]+$")) and ($s | startswith("claude-" + $w + "-"))))
then "match" else "mismatch" end)]
| @tsv' "$1" 2>/dev/null | head -1)
case "$line" in ""|$'\t'*) exit 1 ;; esac
printf '%s\n' "$line"
;;

whoami) # -> one JSON object {provider, auth, plan, account}, each "unknown" when not reported
# Who this engine's calls run as, read from the login they will use, with no model call:
# `claude auth status --json` reports the API provider (firstParty is Anthropic's own API),
# the sign-in method, the subscription plan and the account email.
out=$("${CLAUDE_BIN:-$HOME/.local/bin/claude}" auth status --json </dev/null 2>/dev/null) || out=""
[ -n "$out" ] || out="{}"
printf '%s' "$out" | jq -c '{provider: (.apiProvider // "unknown"), auth: (.authMethod // "unknown"),
plan: (.subscriptionType // "unknown"), account: (.email // "unknown")} | map_values(tostring)' 2>/dev/null \
|| printf '{"provider":"unknown","auth":"unknown","plan":"unknown","account":"unknown"}\n'
;;

l1-env) # -> KEY=VALUE lines the engine needs in its environment
# Lean-query env: keep subscription auth, strip per-call bloat (no CLAUDE.md auto-load,
# no telemetry or error reporting). See CLAUDE.md, "How claude is invoked".
Expand Down
44 changes: 35 additions & 9 deletions bin/adapters.sh
Original file line number Diff line number Diff line change
Expand Up @@ -196,21 +196,47 @@ adapter_run() { # $1=name $2=subcommand [args...]
"$root/$name/adapter.sh" "$@"
}

# The model one adapter's L1 workers run, resolved in a fixed order so a host can pin one
# harness without touching another:
# AUTODREAM_L1_MODEL_<NAME> that adapter only (the name upper-cased, dashes to underscores)
# AUTODREAM_L1_MODEL every adapter
# the manifest's l1_model the adapter's own default
# A model id belongs to one engine (neuralwatt/glm-5.3-flash means nothing to claude), which
# is why the per-adapter form exists: the generic one is an override for hosts with one engine.
adapter_l1_model() { # $1=name -> model on stdout, exit 1 when none resolves
# The adapter whose engine reviews one source's sessions in L1. Which harness holds a transcript
# and which engine reads it are separate choices: the run has one reviewer, the engine that writes
# the summary, unless the host picks another, resolved in a fixed order:
# AUTODREAM_L1_ENGINE_<NAME> that source only (the name upper-cased, dashes to underscores)
# AUTODREAM_L1_ENGINE every source
# $2 the run's reviewer: the L2 engine (AUTODREAM_L2_ENGINE, else the
# first enabled adapter), passed by the runner
# the source itself only when no reviewer is passed
# Only the model call moves. The source's own adapter still enumerates, normalizes, describes and
# adds its triage notes. A choice that names no accepted adapter resolves nothing, so the caller
# leaves those sessions untriaged instead of falling back to some other engine.
adapter_l1_engine() { # $1=source $2=the run's reviewer -> engine adapter on stdout, exit 1 when none resolves
local name="$1" up var v
_adapter_name_safe "$name" || return 1
up=$(printf '%s' "$name" | tr 'a-z-' 'A-Z_')
var="AUTODREAM_L1_ENGINE_$up"
v="${!var:-${AUTODREAM_L1_ENGINE:-${2:-$name}}}"
if [ "$v" != "$name" ]; then
_adapter_name_safe "$v" && _adapter_ok "$v" || return 1
fi
printf '%s' "$v"
}

# The model one source's L1 calls run. A model id belongs to one engine (neuralwatt/glm-5.3-flash
# means nothing to claude), so the engine is resolved first (adapter_l1_engine) and the model is
# keyed by it, never by the source, in a fixed order so a host can pin one engine without touching
# another:
# AUTODREAM_L1_MODEL_<ENGINE> that engine only, whichever source's sessions it reads (the name
# upper-cased, dashes to underscores), like AUTODREAM_L2_MODEL_<NAME>
# AUTODREAM_L1_MODEL every engine: an override for hosts with one engine
# the manifest's l1_model that engine's default
# With the reviewer on claude, AUTODREAM_L1_MODEL_OMP therefore does not reach omp sessions, and
# an omp model id never reaches the claude CLI.
adapter_l1_model() { # $1=source $2=the run's reviewer -> model on stdout, exit 1 when none resolves
local engine up var v
engine=$(adapter_l1_engine "$1" "${2:-}") || return 1
up=$(printf '%s' "$engine" | tr 'a-z-' 'A-Z_')
var="AUTODREAM_L1_MODEL_$up"
v="${!var:-}"
[ -n "$v" ] || v="${AUTODREAM_L1_MODEL:-}"
[ -n "$v" ] || v=$(adapter_manifest_get "$name" '.l1_model' 2>/dev/null) || v=""
[ -n "$v" ] || v=$(adapter_manifest_get "$engine" '.l1_model' 2>/dev/null) || v=""
[ -n "$v" ] || return 1
printf '%s' "$v"
}
Expand Down
Loading
Loading