CHANGELOG.md

Changelog

Unreleased

• Eval-gated fallbacks no longer fake progress. When [progress] require_eval_progress = true, a fallback marker such as LOCAL-UPGRADE resets the stall window so the next iteration can test the new work, but it does not mark the current iteration as progressed unless eval emits the configured metric marker.
• Progress can now require eval improvement. [progress] require_eval_progress = true makes LoopForge ignore act/learn progress markers unless [eval] emits a configured marker such as EVAL-UPGRADE. This prevents long-running quality loops from treating artifact refreshes or self-reported edits as real improvement when the score stayed flat. L016 now flags require_eval_progress without both an eval command and markers.
• A self-reported progress marker no longer counts when the independent verify fails. progressed now requires the marker AND verified is not False, so an agent that prints LOCAL-UPGRADE every round can no longer reset the stall counter and defeat the convergence brake without actually changing anything. Loops with no verify step are unaffected. Guards the exact failure where a progress marker was logged while the underlying edit silently no-op'd.
• No-progress can be observational instead of terminal. stop_after_fallback_no_progress now hands back only when no-progress is explicitly listed in [handback].on; otherwise the runner records no-progress, runs the fallback after each configured stall window, and continues to the hard iteration cap. The fallback counter resets after a no-change fallback when handback is not enabled, so a dry source search does not run on every later iteration. This supports research loops that must keep cycling through worst-item improvement plus fresh-idea discovery without wasting a human handback on every dry well.
• run now executes [learn] as a first-class pre-act stage. Loops can refresh sources, mine reading lists, or generate candidate work before each action instead of waiting until the no-progress fallback fires. Learn output is written to the JSONL trace, can emit progress markers, and repeated learn failures hand back learn-failed-twice.
• New rule L018 (learn command required). A declared [learn] block with no command is now a major lint finding.
• Progress fallback is now real convergence control. Progress markers are explicit line-start status tokens, so NO-LOCAL-UPGRADE no longer accidentally counts as LOCAL-UPGRADE. A new stop_after_fallback_no_progress option lets an improvement loop hand back no-progress after its fallback also finds nothing new, instead of repeating the same discovery step until the hard iteration cap. NEEDS-HUMAN handback now respects [handback].on, so loops can treat it as an advisory status while continuing until their configured convergence brake fires.
• New example examples/convergent-improve/ — the reference pattern for an open-ended "improve X" loop that satisfies L015: a [progress] block (markers + no_progress_limit + a discovery fallback_command) plus goal-reached/no-progress handback. The shape that converges-or-discovers instead of burning budget on an asymptote or manufacturing progress. Guarded by a new tests/test_examples.py so shipped examples stay lint-clean.
• run now executes [eval] as a first-class stage after verify. Eval output is written to the JSONL trace, eval failures stop the loop after repeated failures, and eval output can emit progress markers such as LOCAL-UPGRADE. This closes a dogfood gap where [eval] existed in the parser/rules but the runner ignored it, so loops could "verify" without measuring whether the actual improvement metric moved.
• New rule L017 (eval command required). A declared [eval] block with no command is now a major lint finding.
• New rule L015 (convergence brake). An open-ended "improve X" loop (goal mentions improve / raise / optimize / toward 10-10 / better) must declare a stop-when-progress-stalls signal — a goal-reached / no-progress / all-gaps-resource-bound handback condition, a [convergence] block, or an [eval] metric gate. Without it the loop spends its whole budget chasing an asymptote, and an agent with no way to declare "done" tends to manufacture progress (inflate a score, fabricate evidence) rather than stop. major, since the budget cap is still the ultimate hard stop. Maintenance loops with a concrete goal are exempt.
• Require a named handback owner so autonomous output always returns to an accountable person or team; include that owner in dry-run plans and notifications.
• Add an optional JSONL execution trace with complete act/verify commands, outputs, timestamps, worktree, outcome, and accountable owner for harness-level evaluation and recovery.
• Runner trace now records command durations. learn, act, verify, eval, and fallback trace entries include duration_seconds, making slow improvement loops easier to profile and optimize instead of guessing which stage burns the run budget.
• Runner commands now receive loop state in env vars. LOOPFORGE_ITERATION, LOOPFORGE_NO_PROGRESS_ROUNDS, LOOPFORGE_NO_PROGRESS_LIMIT, and LOOPFORGE_FALLBACK_DUE_NEXT let mature loops run cheap probes during stalled rounds and reserve expensive work for changed artifacts or fallback windows.

0.5.3

• eval no longer lets one bad row silently corrupt a metric. Two scoring bugs:
  • A prob outside [0,1] (e.g. a confidence logged as 75 for 0.75) added a ~(75-1)² term that wrecked the mean Brier score. Out-of-range probabilities are now skipped, so calibration reflects only valid rows.
  • Decimal odds <= 1.0 would book a winning bet as flat/negative P&L (stake*(odds-1) <= 0). Such rows are now treated as invalid odds and skipped from P&L. Both found by routing a correctness review of evaluate() through modelbroker (review → claude).

0.5.2

• run accepts a directory like lint does — loopforge run mydir/ resolves the single loop.toml inside instead of crashing with "Is a directory". Found by dogfooding (lint a dir, then run it). Multiple loops in one dir → a clear "name the one to run" error.

0.5.1

• eval gate fix — --min-accuracy no longer fails when no predictions are resolved yet. A scheduled loop whose outcomes resolve later would previously fail its gate every tick (no data read as failure); now the gate passes until there is real accuracy evidence to judge. CLI prints a note so the pass is explicit.

0.5.0

• loopforge eval — score recorded predictions against real outcomes (the deepest verify: predicted vs ground truth). Reads a CSV (id,predicted,actual,prob,stake,odds,result) and reports accuracy, calibration (Brier), and betting P&L/ROI. --min-accuracy makes it a verify gate, so a loop that keeps predicting badly fails itself. Domain-agnostic (bets, stock calls, anything). Pair it with a resolver that fetches the latest data + loopforge schedule to auto-validate on a cadence. See examples/eval-soccer/.

0.4.0

• loopforge schedule — turn a loop's declarative [trigger].cron into a real cron entry, so a scheduled loop actually fires. schedule install <loop.toml> (with --dry-run), schedule list, schedule remove <name>. Entries are tagged # loopforge:<name> and managed idempotently. A hard no-clobber guard refuses to write if the existing crontab can't be read, so it never wipes your other jobs. (macOS: cron needs Full Disk Access to touch protected dirs.)

0.3.0

• Real worktree isolation in the runner. When [isolation] mode = "worktree" and the loop is in a git repo, run now executes act/verify in an isolated git worktree (left in place for you to review and merge), while the memory ledger is written back to the original repo so the record survives. Previously isolation was linted but ignored by the runner — a describe-vs-do gap, now closed. The worktree path is reported in the run output.
• reviewer_command now receives the act output. A second-model review can finally see the work it's reviewing (it was being run with no input).
• Memory ledger fixed. Each iteration now records a distinct action summary (what act did) and outcome — previously both columns held the same string.
• New rule L013 — referenced files exist. Flags skills / memory / prompt_file paths in loop.toml that don't exist on disk (only when linting a real file). Catches the typo'd path the runner would otherwise load as "(MISSING)".
• 63 tests (was 56).

0.2.0

• L002 now requires a hard stop. A goal (until) no longer counts as a brake on its own — a goal that's never met loops forever. The runaway check is satisfied only by a positive max_iterations or a [budget] time/token/cost ceiling. A non-int or zero max_iterations no longer counts either (it now agrees with the runner).
• New rule L011 — trigger is actually wired up. Catches a schedule trigger with no cron, an event trigger with no source, an until-goal trigger with no until, and typo'd trigger types (e.g. "scheduled") that would silently never fire.
• 56 tests (was 50).

0.1.0

First release.

• lint — grade any loop A–F against the six blocks of Loop Engineering (trigger, isolation, skills, act, verify, memory) plus a cost brake and a human brake. 11 deterministic rules (L001–L012); only the no-brake runaway is critical.
• init — scaffold a complete, lint-clean loop (loop.toml + skills + memory + prompt).
• run — execute a loop (wake → act → verify → record → decide). Enforces iteration and wall-clock limits, appends every step to the memory ledger, and refuses to start a loop that has no brake.
• Zero runtime dependencies, Python ≥ 3.11. human and json output; --score, --select, --fail-at.