Afterforge

Turn agent mistakes into lasting regression checks.

Afterforge is a local-first workbench for Python projects: inspect a failed agent tool run, review the correction, freeze a meaningful pytest regression, and retain evidence from real executions against committed code. A passing old run is not verification of today's code. Unknowns, failed checks and unreviewed proposals stay visible.

Version 0.5.1 adds bounded notification-shaped correction abstention while preserving surrounding user speech and immutable prior evidence. Shape is not authenticated origin, and selected corrections remain pending hypotheses. The stable locator identifies the supported version and exact source/distribution checksums and CI; an unlisted candidate is not certified merely by its version. Earlier 0.5.0 validation does not certify this revision. See validation.

This is bounded, reviewed regression retention—not autonomous learning, a model replay engine or an OS sandbox. An optional fixed verification reminder is experimental and opt-in; code receipts do not measure model efficacy.

Install into Hermes

Requires Linux, Git, Python 3.11+ and uv. The public repository is ussyverse/afterforge, renamed in place without changing repository identity or visibility. No GitHub credentials are needed merely to read the public source. Dependency installation needs network initially; normal local analysis does not need model API keys or personal logs.

Once a supported 0.5.1 release is certified, its stable locator will be at:

https://raw.githubusercontent.com/ussyverse/afterforge/release-metadata/stable.json

The locator must name version 0.5.1, the full source and distribution commits, SHA-256 file checksums, and successful application/native CI links for that exact source. An absent locator or one for an older version is not a verified 0.5.1 installation. plugin-release is only a candidate branch. Do not install a moving branch or development SHA as a supported release.

After inspecting the locator, distribution and stock scanner warnings, substitute its full distribution_commit:

hermes plugins install ussyverse/afterforge --ref FULL_DISTRIBUTION_COMMIT --enable
hermes afterforge setup
hermes afterforge doctor
export AFTERFORGE_SOURCE_ID=local-hermes
hermes afterforge scan
hermes afterforge serve

Dependencies not installed by Hermes. pyproject.toml pins three components as direct Git URLs (triage, petrichor, correction-aware-learning), which are not published on PyPI. hermes plugins install does not install direct-URL requirements; nothing works until you run the setup step, which fetches those three repositories from GitHub into the plugin's own private uv environment from the committed lock (network access to github.com required):

hermes afterforge setup   # or, from a source checkout: uv sync --locked --no-dev

What fixlab_scan copies. Scanning reads Hermes' state.db (WAL-consistent, read-only) and copies failed tool results—up to 64 KiB of tool output per result, exit codes, session/message identifiers, timestamps and the two preceding messages (first 2000 characters each) as context—into the plugin's own SQLite store under profile-local plugin data. That copy is permission-restricted, not encrypted, and persists until you delete the data directory; it is a second location holding conversation-derived content. See privacy.

Recipe review is operator-only. fixlab_build_regression always registers a recipe as unreviewed; a model cannot mark it reviewed. Inspect the recipe and its test file yourself, then run hermes afterforge review-recipe RECIPE_ID --approve-digest DIGEST (or standalone afterforge review-recipe). fixlab_verify_regression refuses recipes without that operator review. The managed backend subprocess starts from a minimal allowlisted environment and does not inherit provider API keys.

Use /afterforge status, /afterforge scan and /afterforge review in a Hermes session. Launch Hermes with the same explicit AFTERFORGE_SOURCE_ID used by the browser/native scans and standalone --source-id; choose a different value for an independent store. See the guided queue/draft/retained-check workflow. hermes fixlab, /fixlab and the standalone agent-fix-lab command remain aliases for this migration release. The plugin ID remains agent-fix-lab: do not install a second plugin under the new brand.

Setup uses the committed uv lock and a private staged environment, records transitive hashes/resolved packages, and switches readiness only after doctor. It never installs dependencies into Hermes's environment. Failed upgrades retain the last working runtime. See native installation/host matrix, release protocol and migration.

Five-minute synthetic demo from source

This development demo does not install into Hermes, read personal history, call a model or claim reconstruction of a real incident. The first dependency download may take longer than five minutes.

git clone https://github.com/ussyverse/afterforge.git
cd afterforge
uv sync --locked
uv run afterforge --help
DEMO_ROOT=$(mktemp -d)
uv run afterforge --home "$DEMO_ROOT/lab" demo
uv run afterforge --home "$DEMO_ROOT/lab" serve

Open http://127.0.0.1:8765/guided and select the synthetic case. Inspect the generated draft, both miniature source revisions, assertion, frozen examples and unknowns. Declare code review and deterministic input completeness only after inspection; authorize the exact draft, then run faulty/corrected archives. The source workbench remains at / for expert JSON/configuration evidence. Three consecutive fresh packaged guided servers exercised scan, correction review, draft authorization, red/green, retained checks and a later reviewed commit despite failing live edits. Separate packaged expert/intervention workflows and a portable installed-CLI round trip also passed; see the validation ledger.

In a second shell, using the same DEMO_ROOT, review and execute the fixture:

# Substitute the demo's draft ID and digest only after inspecting it:
uv run afterforge --home "$DEMO_ROOT/lab" authorize-draft DRAFT_ID --approve-digest DRAFT_DIGEST --reviewed --declared-inputs
# Substitute the returned reviewed recipe ID, not the case or draft ID:
uv run afterforge --home "$DEMO_ROOT/lab" run RECIPE_ID
uv run afterforge --home "$DEMO_ROOT/lab" compare RECIPE_ID

Expected acceptance is faulty assertion failure for the reviewed reason, corrected pass, and compatible regression inputs—not just two exit codes. The commands produce actual local receipts. See recipes and portable bundles for input declarations, limits and fresh-import review.

Use your own evidence

Keep the data home outside the checkout. HERMES_HOME is already the active profile's resolved home: do not append profiles/default. Choose an explicit stable private source ID for that history store; reuse it across its snapshots, not across independent stores merely named state.db.

export AGENT_FIX_LAB_HOME="$HOME/.local/share/agent-fix-lab"
uv run afterforge import-hermes --source "$HERMES_HOME/state.db" --source-id local-hermes --limit 100 --dry-run
uv run afterforge import-hermes --source "$HERMES_HOME/state.db" --source-id local-hermes --limit 100
uv run afterforge list --status fail
uv run afterforge show CASE_ID
uv run afterforge annotate CASE_ID --text 'Proposed correction, not a historical quote' --expected 'The assertion that must hold'

Imports are bounded and resumable with the returned next_after_id passed as --after-id. --after is inclusive Unix seconds; --before is exclusive. --cohort dogfood separates build activity from a frozen historical cohort. Import is WAL-consistent and read-only; unsupported source schemas are rejected, not migrated. The newer inspected host has schema 30, not 26; final support requires its adapter and real-host gates.

Correction discovery and review are distinct from annotation. Use correction-scan --help, corrections --status pending and review-correction --help; declare agent reviews as --reviewer agent, never as a human review. Role/chronology are observations, not authenticated user intent. Native/browser/CLI profile-source mapping and resumable correction linking are being integrated and must pass the shared-workflow gate before release.

Freeze, verify, retain

Recipe v2 declared-v1 review binds deterministic regression input dependencies. Freeze external UTF-8 fixture bytes separately from subject revisions; typed pytest parameter values and collection identities must agree. Same test count or custom test IDs cannot conceal changed values. Intentional subject-data changes need path-specific rationale; they must not change the regression itself. Unsupported ambient inputs mean equivalence is inconclusive. Legacy recipes and results remain immutable and do not gain new claims on rerun.

Tests execute from committed archives, not live worktree edits. Review the code and use separate OS/container isolation for hostile code. No transcript command is replayed. Portable bundle v2 contains only explicitly selected reviewed source, assertions, frozen inputs, sanitized metadata and checksums; it is not a transcript export or a code signature.

For a later commit, use afterforge retained-plan RECIPE_ID --target-revision NEW_COMMIT, inspect the returned plan and unchanged regression, then afterforge retained-check PLAN_ID --approve-digest PLAN_DIGEST --reviewed. This creates a new binding and receipt; do not edit the historical recipe or reuse its approval. Ignored/live files are not executed or certified. The legacy current-check-plan / verify-current path requires a clean checkout at its already reviewed corrected commit. Final later-commit browser/native acceptance remains open. A receipt never certifies external services, model learning or reminder efficacy.

Remove or upgrade

Inspect a newly certified full distribution SHA and reinstall with the stock installer's replacement option, then rerun setup. A pinned hermes plugins update agent-fix-lab must not silently advance the pin. Inspect changed reminder implementation and reapprove explicitly; old approvals do not migrate automatically.

hermes afterforge uninstall-runtime --confirm
hermes plugins disable agent-fix-lab
hermes plugins remove agent-fix-lab

Run runtime removal before removing source if environment cleanup is desired. Evidence in profile-local plugin-data is retained. The older standalone skill/environment is not deleted or automatically merged; legacy skill instructions explain owned removal.

Privacy, evidence and development

See the aggregate retrieval-behavior findings for model-specific premature-stopping experiments, an inactive task-scoped reminder, and new-case limitations. These are exploratory behavioral results, not efficacy claims for the shipped hook or deployment approval.

Data is permission-restricted, not encrypted. Never publish private histories, session IDs, local paths, credentials or raw outputs. The loopback web service has mutation tokens, origin/host checks and text-only history rendering; it is not a multi-user hosted service. Read privacy before import, execution or export. The stock scanner and its exact-warning gate remain enabled and unchanged in scope.

The historical evaluation cohort and its weak holdout are documented in dataset methodology; they are not a new 0.5.0 benchmark. The three included history-derived examples are synthetic reductions, not recovered historical source or proof of repair of original incidents. Component pins remain unchanged; see sources and attribution.

uv run ruff check __init__.py hermes_plugin src tests scripts examples
uv run ruff format --check __init__.py hermes_plugin src tests scripts examples
uv run pytest -q
uv build

See SPEC, architecture, data model, contributing, CHANGELOG and the current handoff. Unfinished release gates are explicit; an intermediate test total or candidate branch is not final certification.