Skip to content

skill shorten loses its oracle work when --max-usd stops it #630

Closed

Description

@antonwolfy

skill shorten is wrapped by stops_at_cap (dokimi/dokimi/commands/skill.py:166), which turns SpendCapReached into SystemExit("...; nothing was written") (commands/novelty_run.py:37). analyze, atoms and probe keep a partial result at the cap; shorten does not.

Repro (mocked client, offline): save an atoms file for 30 spans whose statements only partly overlap their spans, then run skill shorten --artifact rows.md --atoms atoms.json --model o --out short.md --max-usd 0.05 with a client whose cap is already spent. Output: EXIT: spend cap reached: $0.05 of $0.05 after 0 calls (next call: o). Raise --max-usd, or measure less (a cheaper --tier, a lower --max-atoms); nothing was written, and the directory holds atoms.json and rows.md only.

What shorten pays for is retention-oracle verdicts. Options: keep the verdicts already obtained (--cache-dir already stores them; say so in the cap message), or score the candidates with what was adjudicated and mark the table partial. cmd_novelty_shorten is dokimi/dokimi/commands/novelty.py:451.

Found while fixing #624. The fix's PR is #633.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions