Skip to content

VERIFICATION.md

Latest commit

 

History

History
274 lines (262 loc) · 36.4 KB

VERIFICATION.md

File metadata and controls

274 lines (262 loc) · 36.4 KB
Raw

Verification Record

Status date: 2026-07-04

Passed

  • Latest production gate:

    • go test ./... -count=1
    • go vet ./...
    • sh scripts/secret-scan.sh
    • sh scripts/smoke.sh
    • sh scripts/dogfood.sh
    • VERSION=0.1.1 sh scripts/release-local.sh --dist dist
    • sh scripts/production-local-gate.sh --workspace . --dist dist --output-dir .omo/evidence/ci-production-local-gate
    • task ci

  • Latest strict gate:

    • gofumpt -l cmd internal
    • golangci-lint run ./...
    • nilaway ./...
    • sh scripts/secret-scan.sh
    • sh -n over shell scripts when ShellCheck is unavailable.
    • sh scripts/strict-checks.sh

  • Latest production-readiness aggregate gate:

    • sh scripts/production-readiness.sh --dist dist --output-dir .omo/evidence/production-readiness-final
    • Result: local production checks pass; public production readiness remains blocked until public release and HTTP provider evidence are complete. The comparison check now passes from .omo/evidence/external-agent-production-core-29-final-result-retry-r1/summary.json.
    • The packet writes launch-checklist.md with the exact remaining public-production commands and records its SHA-256 plus action count in summary.json.

  • Latest production status command:

    • ceo-packet production-status --workspace . --format text
    • Result: reports Local ready: true, Public ready: false, External setup required: true, five blocked public checks, the launch checklist hash, finalizer action counts, declared-evidence match counts, setup checklist hash, and the latest finalizer next action from saved evidence.

  • Latest production finalizer dry-run:

    • sh scripts/production-finalize.sh --dry-run --output-dir .omo/evidence/production-finalize-dry-run-r1 --dist dist
    • Result: writes index.md, summary.json, and commands.sh for the guarded final release/provider/comparison/readiness sequence without publishing, tagging, uploading, or saving secret values.

  • Latest guarded production finalizer:

    • sh scripts/production-finalize.sh --output-dir .omo/evidence/production-finalize-setup-hash-r1 --dist dist
    • Result: blocked as expected on public release readiness plus OpenAI/OpenRouter/Moonshot provider keys; competitor smoke passed; all-agent comparison evidence is clean; summary records publish_actions_performed: false, secret_value_saved: false, finalizer setup checklist hash metadata, and declared-evidence fingerprints for the remaining actions.

  • Latest production finalizer CLI dry-run:

    • go run ./cmd/ceo-packet production-finalize --workspace . --dry-run --output-dir .omo/evidence/production-finalize-cli-dry-run-r1 --dist dist
    • Result: CLI wrapper runs the guarded finalizer and writes a planned evidence packet without publishing or saving secrets.

  • Latest replay-safe production finalizer dry-run:

    • sh scripts/production-finalize.sh --dry-run --output-dir .omo/evidence/production-finalize-dry-run-r2 --evidence-root '.omo/evidence/root with spaces' --dist dist
    • Result: generated commands.sh shell-quotes paths with spaces and still writes planned evidence without publishing or saving secrets.

  • Latest semantic competitor-smoke finalizer proof:

    • sh scripts/production-finalize.sh --output-dir .omo/evidence/production-finalize-smoke-semantic-r2 --dist dist --skip-release-readiness --skip-provider-proofs --skip-production-readiness
    • Result: command exits blocked because competitor-smoke/summary.json has setup_blocked: 1; the finalizer records competitor-smoke-command: pass and competitor-smoke: blocked.

  • Latest finalizer next-actions proof:

    • sh scripts/production-finalize.sh --output-dir .omo/evidence/production-finalize-next-actions-r1 --dist dist --skip-release-readiness --skip-provider-proofs --skip-production-readiness
    • Result: writes next-actions.md with the exact competitor setup and all-agent comparison follow-up commands, and records next_actions.required_action_count.

  • Latest production status next-action proof:

    • go run ./cmd/ceo-packet production-status --workspace . --format text
    • Result: reports the launch checklist plus latest complete finalizer next-actions.md, prints Finalizer actions JSON: .omo/evidence/production-finalize-setup-hash-r1/next-actions.json, summarizes finalizer action states, runnable/blocked command counts, and declared-evidence match counts from that JSON, prints Finalizer setup actions: .omo/evidence/production-finalize-setup-hash-r1/setup-actions.md, surfaces the latest release bootstrap handoff when present, rechecks launch/finalizer setup checklist hashes live, ignores skipped production-readiness packets plus skipped or planned finalizer packets, and sets Next action to open .omo/evidence/production-finalize-setup-hash-r1/next-actions.md.

  • Latest production actions command proof:

    • go run ./cmd/ceo-packet production-actions --workspace . --format text
    • go run ./cmd/ceo-packet production-actions --workspace . --format text --action-id provider-openai
    • go run ./cmd/ceo-packet production-actions --workspace . --format text --action-kind release_proof
    • go run ./cmd/ceo-packet production-actions --workspace . --format text --action-kind provider_proof
    • go run ./cmd/ceo-packet production-actions --workspace . --format text --action-state missing_env
    • go run ./cmd/ceo-packet production-actions --workspace . --format text --action-state empty_env
    • go run ./cmd/ceo-packet production-actions --workspace . --format text --action-state setup_blocked
    • go run ./cmd/ceo-packet production-actions --workspace . --format text --env-ready-only
    • go run ./cmd/ceo-packet production-actions --workspace . --format text --ready-only
    • go run ./cmd/ceo-packet production-actions --workspace . --format text --next
    • go run ./cmd/ceo-packet production-actions --workspace . --format text --action-kind competitor_setup
    • go run ./cmd/ceo-packet production-actions --workspace . --format text --action-kind final_readiness
    • go run ./cmd/ceo-packet production-actions --workspace . --action-id provider-openai --commands-only
    • Result: prints five structured finalizer actions from .omo/evidence/production-finalize-setup-hash-r1/next-actions.json, including release, provider, and final readiness actions; filtered mode narrows that list by exact action id, release, provider-proof, action state, env-ready, ready-now, first next action, competitor setup, or final-readiness queues without printing secret values, generated shell completions include production action state/kind/provider values, and invalid action states are rejected. Output separates Env ready from Ready now, every report includes action_state_counts, runnable_command_count, blocked_command_count, and declared-evidence match/mismatch totals, every action includes action_state plus action_reason, blank provider key env vars are reported as empty_env, release/competitor setup blockers are reported as setup_blocked, release setup markdown is exposed as structured setup_action_items, provider setup checklists are exposed as structured checklist_items, and dependency-blocked actions are not counted as immediately runnable. Release and provider setup output include direct setup checklist paths; provider output also shows blocked reason and model. Provider command files include no-secret missing/empty env guards before rerunning paid proof. Competitor setup and all-agent comparison are no longer remaining actions after passing saved evidence. Downstream actions print Waiting on dependency lists, text output includes shell-quoted Command: lines, finalizer-declared evidence files include SHA-256 and size metadata, current reports show declared fingerprint matches, and --commands-only prints setup items plus blocker reasons as secret-safe comments while commenting out blocked command lines.

  • Latest complete finalizer next-actions proof:

    • sh scripts/production-finalize.sh --output-dir .omo/evidence/production-finalize-setup-hash-r1 --dist dist
    • Result: exits blocked as expected, records no skipped steps, and writes five repo-relative remaining actions covering release, providers, and final readiness in next-actions.md; next-actions.json includes structured action ids, kinds, commands, provider names, required env vars, evidence paths, and declared evidence fingerprints. Release readiness writes setup-actions.md plus safe setup-commands.sh; competitor smoke covers all six configured competitors and passes for Codex CLI, Claude Code, Aider, OpenCode, Goose, and Pi; all-agent comparison passes from existing clean evidence; root setup-actions.md consolidates release, provider, and final rerun work.

  • Latest source-checkout finalizer runner proof:

    • sh scripts/production-finalize.sh --dry-run --output-dir .omo/evidence/production-finalize-source-runner-r1 --dist dist
    • sh scripts/release-readiness.sh --dist dist --output-dir .omo/evidence/release-readiness-source-runner-r1
    • Result: generated setup actions use go run ./cmd/ceo-packet ... when ceo-packet is not installed on PATH; production status/actions still ignore the planned dry-run packet and use .omo/evidence/production-finalize-setup-hash-r1/next-actions.json as the authoritative blocker queue.

  • Latest release workflow guard:

    • go test ./internal/cli -run Test_ReleaseWorkflow_publishesGitHubReleaseAssets -count=1
    • Result: verifies the tag-triggered GitHub release workflow has write permission, derives the version from GITHUB_REF_NAME, runs local release plus verification, creates the GitHub Release, and attaches archives, checksums, and the manifest.

  • Latest release bootstrap handoff proof:

    • go test ./internal/cli -run Test_ReleaseBootstrapScript -count=1
    • Result: verifies release-bootstrap writes release-handoff.md with required public assets, checksums, operator boundaries, and post-publish verification commands while avoiding tag/push/upload commands.

  • Latest local production gate:

    • sh scripts/production-local-gate.sh --workspace . --output-dir .omo/evidence/security-production-final
    • Result: pass. Local production readiness is true; public blockers remain recorded as evidence. The gate also requires the production action queue while public blockers remain, writes production-actions.json, production-status.json, and production-actions.commands.sh, validates action row, state-count, runnable, and blocked command counts, requires action_state and action_reason on every action, enforces release setup no-publish/no-secret policy fields and setup-action file content, enforces provider setup no-secret policy fields plus setup artifact hashes and command-file content, rejects declared-evidence mismatches, requires finalizer setup checklist hash/count metadata, fails launch/finalizer checklist fingerprint drift, and confirms blocked commands are commented with reasons in the command script.

  • Latest live external-agent comparison:

    • go run ./cmd/ceo-eval --local-agent-benchmark --local-agents ceo_harness,codex_cli,opencode,pi --local-agent-benchmark-task production-core --local-agent-benchmark-repeat 1 --tasks evals/tasks --output-dir .omo/evidence/external-agent-production-core-25-r1 --timeout-seconds 240 --ceo-benchmark-mode model-command --ceo-benchmark-model-command-json '["sh","/Users/jakedom/Documents/Codex/2026-06-30/new-chat/work/ceo-harness/scripts/benchmark-model-command.sh"]'
    • Result: 100 runs / 99 pass / 0 partial / 0 fail / 1 timed out / 0 skipped / 1 incomplete evidence.
    • Cod Code result: 25 pass / 0 partial / 0 fail / 0 timed out / 0 incomplete evidence.
    • Agent totals: Codex CLI 25 pass; OpenCode 25 pass; Pi 24 pass and 1 timed out.

  • Latest current-suite external-agent comparison:

    • go run ./cmd/ceo-eval --local-agent-benchmark --local-agents ceo_harness,codex_cli,opencode,pi --local-agent-benchmark-task production-core --local-agent-benchmark-repeat 1 --local-agent-benchmark-concurrency 4 --local-agent-benchmark-timeout-retries 1 --local-agent-benchmark-result-retries 1 --ceo-binary ./bin/ceo-packet --tasks evals/tasks --output-dir .omo/evidence/external-agent-production-core-29-final-result-retry-r1 --timeout-seconds 240 --ceo-benchmark-mode model-command --ceo-benchmark-model-command-json '["sh","/Users/jakedom/Documents/Codex/2026-06-30/new-chat/work/ceo-harness/scripts/benchmark-model-command.sh"]'
    • Result: 116 runs / 116 pass / 0 partial / 0 fail / 0 timed out / 0 skipped / 0 incomplete evidence.
    • One OpenCode partial on refactor-check-fix-prompt was retried once and then passed; the prior attempt is preserved in prior_attempts.

  • Focused path-safety rubric repair:

    • go run ./cmd/ceo-eval --local-agent-benchmark --local-agents ceo_harness,codex_cli,opencode,pi --local-agent-benchmark-task safety-policy-path-escape --local-agent-benchmark-repeat 1 --local-agent-benchmark-concurrency 4 --ceo-binary ./bin/ceo-packet --tasks evals/tasks --output-dir .omo/evidence/external-agent-path-escape-rubric-r1 --timeout-seconds 240 --ceo-benchmark-mode model-command --ceo-benchmark-model-command-json '["sh","/Users/jakedom/Documents/Codex/2026-06-30/new-chat/work/ceo-harness/scripts/benchmark-model-command.sh"]'
    • Result: 4 runs / 4 pass / 0 partial / 0 fail / 0 timed out / 0 incomplete evidence.

  • Latest timeout-retry comparison proof:

    • go run ./cmd/ceo-eval --local-agent-benchmark --local-agents ceo_harness,codex_cli,opencode,pi --local-agent-benchmark-task docs-product-status-weak-spots,safety-policy-observe-no-write,multi-file-lean-context-autonomy,multi-file-secret-safe-provider-proof,report-quality-evidence-summary --local-agent-benchmark-repeat 1 --local-agent-benchmark-concurrency 4 --local-agent-benchmark-timeout-retries 1 --ceo-binary ./bin/ceo-packet --tasks evals/tasks --output-dir .omo/evidence/external-agent-timeout-retry-r1 --timeout-seconds 240 --ceo-benchmark-mode model-command --ceo-benchmark-model-command-json '["sh","/Users/jakedom/Documents/Codex/2026-06-30/new-chat/work/ceo-harness/scripts/benchmark-model-command.sh"]'
    • Result: 20 planned runs / 15 pass / 0 partial / 0 fail / 5 timed out / 5 incomplete evidence.
    • Cod Code, Codex CLI, and Pi each passed 5/5; OpenCode exhausted both attempts on all five focused tasks.

  • Latest prompt-discipline comparison proof:

    • go run ./cmd/ceo-eval --local-agent-benchmark --local-agents codex_cli,opencode,pi --local-agent-benchmark-task docs-product-status-weak-spots,safety-policy-observe-no-write,multi-file-lean-context-autonomy,multi-file-secret-safe-provider-proof,report-quality-evidence-summary --local-agent-benchmark-repeat 1 --local-agent-benchmark-concurrency 3 --tasks evals/tasks --output-dir .omo/evidence/external-agent-timeout-prompt-discipline-r1 --timeout-seconds 300
    • Result: 15 planned runs / 10 pass / 0 partial / 0 fail / 5 timed out / 5 incomplete evidence. Codex CLI and Pi each passed 5/5; OpenCode timed out 5/5.

  • Latest per-agent timeout override proof:

    • go run ./cmd/ceo-eval --local-agent-benchmark --local-agents opencode --local-agent-benchmark-task docs-product-status-weak-spots --local-agent-benchmark-agent-timeouts opencode=600 --tasks evals/tasks --output-dir .omo/evidence/opencode-agent-timeout-r1 --timeout-seconds 240
    • Result: OpenCode timed out at 600s with no required file changes; agent_timeouts.opencode is recorded in summary.json.

  • Latest OpenCode provider-error visibility proof:

    • go run ./cmd/ceo-eval --local-agent-benchmark --local-agents opencode --local-agent-benchmark-task docs-product-status-weak-spots --tasks evals/tasks --output-dir .omo/evidence/opencode-setup-blocked-r1 --timeout-seconds 25
    • Result: OpenCode records setup_blocked: 1, timed_out: 0, and incomplete_evidence: 0; stderr.log shows Token Plan usage limit reached for minimax-coding-plan/MiniMax-M3.

  • Latest competitor smoke setup preflight:

    • go run ./cmd/ceo-eval --comparison-smoke --competitors evals/competitors.json --output-dir .omo/evidence/competitor-smoke-after-installs-r1 --timeout-seconds 25
    • Result: six configured competitors passed smoke: Codex CLI 0.142.4, Claude Code 2.1.201, Aider 0.86.2, OpenCode 1.17.13, Goose 1.41.0, and Pi 0.80.3. This is still a version/dry-run smoke, not the full head-to-head task comparison.

  • Latest comparison report decision smoke:

    • go run ./cmd/ceo-eval --local-agent-benchmark --local-agents ceo_harness --local-agent-benchmark-task docs-roadmap-cli-first --output-dir .omo/evidence/comparison-report-decision-smoke-r1 ...
    • Result: comparison report includes Overall comparison: pass, Cod Code result: clean, and External blockers: none.

  • Latest focused external-agent comparison for the newest multi-file task:

    • go run ./cmd/ceo-eval --local-agent-benchmark --local-agents ceo_harness,codex_cli,opencode,pi --local-agent-benchmark-task multi-file-operator-safety-flow --local-agent-benchmark-repeat 1 --local-agent-benchmark-concurrency 4 --ceo-binary ./bin/ceo-packet --tasks evals/tasks --output-dir .omo/evidence/external-agent-operator-safety-flow-r1 --timeout-seconds 240 --ceo-benchmark-mode model-command --ceo-benchmark-model-command-json '["sh","/Users/jakedom/Documents/Codex/2026-06-30/new-chat/work/ceo-harness/scripts/benchmark-model-command.sh"]'
    • Result: 4 runs / 4 pass / 0 partial / 0 fail / 0 timed out / 0 skipped / 0 incomplete evidence.

  • Latest market-parity-core CEO comparison:

    • go run ./cmd/ceo-eval --local-agent-benchmark --local-agents ceo_harness --local-agent-benchmark-task market-parity-core --local-agent-benchmark-repeat 1 --ceo-binary ./bin/ceo-packet --tasks evals/tasks --output-dir .omo/evidence/market-parity-core-ceo-r2 --timeout-seconds 180 --ceo-benchmark-mode model-command --ceo-benchmark-model-command-json '["sh","/Users/jakedom/Documents/Codex/2026-06-30/new-chat/work/ceo-harness/scripts/benchmark-model-command.sh"]'
    • Result: 10 tasks / 10 pass / 0 partial / 0 fail / 0 timed out / 0 skipped / 0 incomplete evidence.

  • Latest expanded production-core CEO comparison:

    • go run ./cmd/ceo-packet gauntlet --suite production-core --agents ceo_harness --ceo-binary ./bin/ceo-packet --tasks evals/tasks --output-dir .omo/evidence/production-core-29-ceo-r1 --timeout-seconds 180 --concurrency 4 --ceo-benchmark-mode model-command --ceo-benchmark-model-command-json '["sh","/Users/jakedom/Documents/Codex/2026-06-30/new-chat/work/ceo-harness/scripts/benchmark-model-command.sh"]'
    • Result: 29 tasks / 29 pass / 0 partial / 0 fail / 0 timed out / 0 incomplete evidence.

  • Latest concurrent production-core CEO comparison:

    • go run ./cmd/ceo-packet gauntlet --suite production-core --agents ceo_harness --ceo-binary ./bin/ceo-packet --tasks evals/tasks --output-dir .omo/evidence/production-core-25-ceo-concurrency-r1 --timeout-seconds 120 --concurrency 4 --ceo-benchmark-mode model-command --ceo-benchmark-model-command-json '["sh","/Users/jakedom/Documents/Codex/2026-06-30/new-chat/work/ceo-harness/scripts/benchmark-model-command.sh"]'
    • Result: 25 tasks / concurrency 4 / 25 pass / 0 partial / 0 fail / 0 timed out / 0 incomplete evidence.

  • Latest cross-language CEO comparison:

    • go run ./cmd/ceo-packet gauntlet --suite cross-language-core --agents ceo_harness --ceo-binary ./bin/ceo-packet --tasks evals/tasks --output-dir .omo/evidence/cross-language-core-ceo-r1 --timeout-seconds 120 --concurrency 2 --ceo-benchmark-mode model-command --ceo-benchmark-model-command-json '["sh","/Users/jakedom/Documents/Codex/2026-06-30/new-chat/work/ceo-harness/scripts/benchmark-model-command.sh"]'
    • Result: 2 tasks / concurrency 2 / 2 pass / 0 partial / 0 fail / 0 timed out / 0 incomplete evidence.

  • Latest focused multi-file task proof:

    • go run ./cmd/ceo-eval --local-agent-benchmark --local-agents ceo_harness --local-agent-benchmark-task multi-file-provider-fallback-reporting --local-agent-benchmark-repeat 1 --tasks evals/tasks --output-dir .omo/evidence/multi-file-provider-fallback-ceo-r2 --timeout-seconds 120 --ceo-benchmark-mode model-command --ceo-benchmark-model-command-json '["sh","/Users/jakedom/Documents/Codex/2026-06-30/new-chat/work/ceo-harness/scripts/benchmark-model-command.sh"]'
    • Result: 1 run / 1 pass / 9 scored checks / 0 incomplete evidence.

  • Latest larger multi-file task proof:

    • go run ./cmd/ceo-eval --local-agent-benchmark --local-agents ceo_harness --local-agent-benchmark-task multi-file-operator-safety-flow --local-agent-benchmark-repeat 1 --tasks evals/tasks --output-dir .omo/evidence/multi-file-operator-safety-flow-ceo-r1 --timeout-seconds 120 --ceo-benchmark-mode model-command --ceo-benchmark-model-command-json '["sh","/Users/jakedom/Documents/Codex/2026-06-30/new-chat/work/ceo-harness/scripts/benchmark-model-command.sh"]'
    • Result: 1 run / 1 pass / 13 scored checks / 0 incomplete evidence.

  • Latest full benchmark fixture scoring:

    • go run ./cmd/ceo-eval --benchmark-fixtures --tasks evals/tasks --output-dir .omo/evidence/benchmark-fixtures-31-r1
    • Result: 31 tasks / 31 pass / 0 partial / 0 fail / 0 skipped.

  • Latest repeated real Kimi provider proof:

    • go run ./cmd/ceo-eval --local-agent-benchmark --local-agents ceo_harness --local-agent-benchmark-task safety-policy-path-escape --local-agent-benchmark-repeat 3 --ceo-binary ./bin/ceo-packet --tasks evals/tasks --output-dir .omo/evidence/provider-kimi-path-safety-repeat-r7 --timeout-seconds 600 --ceo-benchmark-mode model-command --ceo-benchmark-model-command-json '["sh","/Users/jakedom/Documents/Codex/2026-06-30/new-chat/work/ceo-harness/scripts/kimi-model-command.sh"]'
    • Result: 3 runs / 3 pass / 18 scored checks / 0 partial / 0 fail / 0 timed out / 0 incomplete evidence.

  • Latest real Kimi provider JS app-code proof:

    • go run ./cmd/ceo-eval --local-agent-benchmark --local-agents ceo_harness --local-agent-benchmark-task cross-language-js-state-reducer --local-agent-benchmark-repeat 1 --ceo-binary ./bin/ceo-packet --tasks evals/tasks --output-dir .omo/evidence/provider-kimi-js-state-reducer-r2 --timeout-seconds 600 --ceo-benchmark-mode model-command --ceo-benchmark-model-command-json '["sh","/Users/jakedom/Documents/Codex/2026-06-30/new-chat/work/ceo-harness/scripts/kimi-model-command.sh"]'
    • Result: 1 run / 1 pass / 6 scored checks / 0 incomplete evidence. Kimi changed frontend/state.js, created .omo/evidence/cross-language-js-state-reducer.md, and passed node frontend/state.test.js.

  • Latest first-class Kimi provider proof gate:

    • sh scripts/provider-proof.sh --provider kimi --output-dir .omo/evidence/provider-proof-kimi-r2
    • Result: pass. JS reducer scored 6/6 and Python retry policy scored 7/7; both changed required source files, created required evidence artifacts, and passed their task commands through Kimi-backed Cod Code.

  • Latest first-class Codex provider proof gate:

    • sh scripts/provider-proof.sh --provider codex --output-dir .omo/evidence/provider-proof-codex-r1
    • Result: pass. JS reducer scored 6/6 and Python retry policy scored 7/7; both changed required source files, created required evidence artifacts, and passed their task commands through Codex-backed Cod Code.

  • Latest HTTP provider proof setup gate:

    • sh scripts/provider-setup-preflight.sh --output-dir .omo/evidence/provider-setup-preflight
    • Result: writes secret-safe provider readiness evidence before paid HTTP provider proofs; missing or empty env vars are setup blockers, not benchmark failures.
    • sh scripts/provider-proof.sh --provider openrouter --output-dir .omo/evidence/provider-proof-openrouter
    • Result: blocked setup because OPENROUTER_API_KEY is missing. Evidence writes blocked.md, summary.json, env.template, commands.sh, setup-checklist.md, and index.md; summary.json records setup checklist count, SHA-256 fingerprints for the setup artifacts, and command_script_secret_policy: no_secret_assignment. This is not scored as a benchmark failure.

  • Latest real-repo dogfood:

    • sh scripts/dogfood-real.sh --repo ceo-harness-repeat:/Users/jakedom/Documents/Codex/2026-06-30/new-chat/work/ceo-harness --repeat 3 --timeout-ms 250 --output-dir .omo/evidence/dogfood-real-repeat-self-r1
    • Result: 3 live passes / 0 fails, including expected timeout failure evidence in each run.

  • Latest copied-workspace dogfood:

    • sh scripts/dogfood-real.sh --copy-workspace --repo ceo-harness-copy:/Users/jakedom/Documents/Codex/2026-06-30/new-chat/work/ceo-harness --timeout-ms 250 --output-dir .omo/evidence/dogfood-real-copy-self-r1
    • Result: pass; all five scenarios ran against workspace-mode=copied, with source and workspace paths recorded separately.

  • Latest independent copied-workspace dogfood:

    • sh scripts/dogfood-real.sh --copy-workspace --repo chemcheck:/Users/jakedom/Documents/chemcheck-main --repo axis-health:'/Users/jakedom/Documents/Axis health' --timeout-ms 250 --output-dir .omo/evidence/dogfood-real-independent-r1
    • Result: ChemCheck pass and Axis Health pass; each ran doctor, plan-only, observe, patch-preview, and timeout-guard scenarios in copied workspaces.

  • Latest expanded independent copied-workspace dogfood:

    • sh scripts/dogfood-real.sh --copy-workspace --repo clicky:/Users/jakedom/Documents/clicky-main --repo dps:/Users/jakedom/Documents/DPS-internal-coms-main --repo janus:/Users/jakedom/Documents/janus-code --repo radian:'/Users/jakedom/Documents/Radian notes app ' --timeout-ms 250 --output-dir .omo/evidence/dogfood-real-independent-r2
    • Result: Clicky pass, DPS Internal Comms pass, Janus pass, and Radian pass; each ran doctor, plan-only, observe, patch-preview, and timeout-guard scenarios in copied workspaces.

  • Latest task-specific copied-workspace dogfood:

    • sh scripts/dogfood-real.sh --copy-workspace --repo chemcheck:/Users/jakedom/Documents/chemcheck-main --repo axis-health:'/Users/jakedom/Documents/Axis health' --task 'Plan a repo-specific onboarding docs cleanup and inspect the safest first patch without writing source files' --timeout-ms 250 --output-dir .omo/evidence/dogfood-real-task-specific-r1
    • Result: ChemCheck pass and Axis Health pass; both saved the custom task text, copied workspace path, git status evidence, plan-only output, observe-mode output, patch-preview digest, and expected timeout failure evidence.

  • Latest copied-workspace write-probe dogfood:

    • sh scripts/dogfood-real.sh --copy-workspace --write-probe --repo chemcheck:/Users/jakedom/Documents/chemcheck-main --repo axis-health:'/Users/jakedom/Documents/Axis health' --task 'Apply and prove a copied-workspace write probe without touching source checkouts' --timeout-ms 250 --output-dir .omo/evidence/dogfood-real-write-probe-r1
    • Result: ChemCheck pass and Axis Health pass; both previewed, approved, and applied ceo-dogfood-write-probe.txt inside copied workspaces, saved after-state git status, and left the source checkouts without the marker file.

  • Latest copied-workspace feature-edit dogfood:

    • sh scripts/dogfood-real.sh --copy-workspace --feature-edit-probe --repo chemcheck:/Users/jakedom/Documents/chemcheck-main --repo axis-health:'/Users/jakedom/Documents/Axis health' --task 'Add a copied-workspace onboarding note that proves approved feature edits stay isolated' --timeout-ms 250 --output-dir .omo/evidence/dogfood-real-feature-edit-r2
    • Result: ChemCheck pass and Axis Health pass; both previewed, approved, and applied a repo-specific ceo-dogfood-feature.md note inside copied workspaces, saved the final feature file plus after-state git status, and left source checkouts without the marker file.

  • Latest copied-workspace app-code dogfood:

    • sh scripts/dogfood-real.sh --copy-workspace --app-code-probe --repo chemcheck:/Users/jakedom/Documents/chemcheck-main --repo axis-health:'/Users/jakedom/Documents/Axis health' --task 'Add a copied-workspace source module proving approved app-code edits stay isolated' --timeout-ms 250 --output-dir .omo/evidence/dogfood-real-app-code-r1
    • Result: ChemCheck pass and Axis Health pass; both previewed, approved, and applied src/ceoDogfoodProbe.mjs inside copied workspaces, saved the final source file plus after-state git status, and left source checkouts without the marker file.

  • Latest copied-workspace integrated app-code dogfood:

    • sh scripts/dogfood-real.sh --copy-workspace --integrated-app-code-probe --repo chemcheck:/Users/jakedom/Documents/chemcheck-main --repo axis-health:'/Users/jakedom/Documents/Axis health' --task 'Wire a copied-workspace app-code marker into an existing source file without touching source checkouts' --timeout-ms 250 --output-dir .omo/evidence/dogfood-real-integrated-app-code-r1
    • Result: ChemCheck pass and Axis Health pass; both previewed, approved, and applied ceoDogfoodIntegratedProbe to copied src/App.jsx, saved the target path plus modified source file and after-state git status, and left source checkouts without the marker.

  • Latest copied-workspace multi-file app-code dogfood:

    • sh scripts/dogfood-real.sh --copy-workspace --multi-file-app-code-probe --repo chemcheck:/Users/jakedom/Documents/chemcheck-main --repo axis-health:'/Users/jakedom/Documents/Axis health' --task 'Wire a copied-workspace app-code marker across two existing source files without touching source checkouts' --timeout-ms 250 --output-dir .omo/evidence/dogfood-real-multi-file-app-code-r1
    • Result: ChemCheck pass and Axis Health pass; both previewed, approved, and applied ceoDogfoodMultiFileProbe* markers to copied src/App.jsx and src/main.jsx, saved target paths plus modified source files and after-state git status, and left source checkouts without the marker.

  • Broadened copied-workspace multi-file app-code dogfood:

    • sh scripts/dogfood-real.sh --copy-workspace --multi-file-app-code-probe --repo janus:/Users/jakedom/Documents/janus-code --task 'Wire a copied-workspace app-code marker across two existing source files without touching the Janus source checkout' --timeout-ms 250 --output-dir .omo/evidence/dogfood-real-multi-file-janus-r1
    • Result: Janus pass; previewed, approved, and applied ceoDogfoodMultiFileProbe* markers to copied src/cli/args.ts and src/cli/base64-payload-byte-count.ts, saved target paths plus modified source files and after-state git status, and left the Janus source checkout without the marker.

  • Broader app-shaped copied-workspace dogfood:

    • sh scripts/dogfood-real.sh --copy-workspace --multi-file-app-code-probe --repo family-os:/Users/jakedom/Documents/family-os --repo pools:/Users/jakedom/Documents/jakedompools--main --repo dps-frontend:/Users/jakedom/Documents/DPS-internal-coms-main/frontend --repo janus-mobile:/Users/jakedom/Documents/janus-code/mobile --task 'Wire a copied-workspace app-code marker across two existing source files in broader app repos without touching source checkouts' --timeout-ms 250 --output-dir .omo/evidence/dogfood-real-broader-apps-r1
    • Result: Family OS, Pools, DPS frontend, and Janus Mobile pass; each previewed, approved, and applied two source-file markers inside copied workspaces, saved target paths plus modified source files and after-state git status, and left all source checkouts without the marker.

  • Nightly eval task:

    • make eval-nightly
    • /Users/jakedom/go/bin/task eval:nightly
    • Result: both passed locally; each ran 27/27 deterministic fixture scoring, 2/2 cross-language CEO gauntlet, and 2-pass real-repo dogfood under .omo/evidence/nightly/.

  • Endurance eval task:

    • sh scripts/endurance.sh --iterations 3 --output-dir .omo/evidence/endurance-local-r1
    • Result: 3 iterations / 3 pass / 0 fail / elapsed 8 seconds, with per-iteration command logs and summary rows.

  • Longer endurance eval task:

    • sh scripts/endurance.sh --iterations 10 --output-dir .omo/evidence/endurance-local-r2
    • Result: 10 iterations / 10 pass / 0 fail / elapsed 30 seconds, with each iteration running build, 28-task fixture scoring, cross-language gauntlet, and real-repo dogfood.

  • Extended endurance eval task:

    • sh scripts/endurance.sh --iterations 30 --output-dir .omo/evidence/endurance-local-r3
    • Result: 30 iterations / 30 pass / 0 fail / elapsed 102 seconds, with each iteration writing a run summary.

  • Focused additions test:

    • go test ./internal/cli -run 'Test_Run_(start|inbox|provider_wizard|init_demo_repo|tui|write_policy|init_config_uses_external_adapter|prints_help)' -count=1

  • Focused rollback test:

    • go test ./internal/workspace -run Test_Workspace_RollbackReplaceText -count=1
    • go test ./internal/cli -run 'Test_Run_rollback_report|Test_HelperProcess_cli_model_create_file_patch' -count=1

  • GitHub Actions CI workflow:

    • go test ./... -count=1
    • go vet ./...
    • sh scripts/secret-scan.sh
    • sh scripts/smoke.sh
    • sh scripts/production-local-gate.sh --workspace . --dist dist --output-dir .omo/evidence/ci-production-local-gate

  • make ci

    • gofmt -w ./cmd ./internal
    • go test ./... -count=1
    • go vet ./...
    • sh scripts/smoke.sh
    • sh scripts/dogfood.sh
    • go build ... ./cmd/ceo-packet

  • go test -race -shuffle=on -count=1 ./...

  • VERSION=0.1.1 sh scripts/release-local.sh --dist dist

  • sh scripts/verify-release.sh --dist dist

    • Release workflow also runs sh scripts/release-readiness.sh --dist dist --output-dir .omo/evidence/release-readiness-ci after GitHub release assets are published and verified.

  • sh scripts/release-bootstrap.sh --dist dist --output-dir .omo/evidence/release-bootstrap writes a blocked/pass public-release bootstrap packet without publishing anything; summary.json records the checklist item count and SHA-256 fingerprints for the bootstrap files.

  • sh scripts/release-preflight.sh dist blocks public release claims when remote URL, public release URL, Homebrew URL, and signature or checksum-only notes are missing.

  • sh scripts/release-homebrew-formula.sh --dist dist --repo-url <repo-url> --homebrew-archive-base-url <archive-base-url> updates dist/homebrew/ceo-packet.rb to the real remote archive URL and checksum without publishing a tap.

  • sh scripts/release-signatures.sh --dist dist --private-key <key.pem> writes detached .sig files for every archive; RELEASE_SIGNING_PUBLIC_KEY=<public.pem> sh scripts/verify-release.sh dist verifies those signatures.

  • sh scripts/release-readiness.sh --dist dist --output-dir .omo/evidence/release-readiness-final writes a blocked/pass public-release evidence packet without publishing anything, plus setup-actions.md when public release blockers remain; summary.json records the setup action count, SHA-256 fingerprint, no_publish_no_secret_assignment setup policy, and false publish/secret flags.

  • sh scripts/production-readiness.sh --dist dist --output-dir .omo/evidence/production-readiness-final writes one release/provider/eval/security/endurance/comparison readiness packet without publishing, pushing, tagging, uploading, or calling paid providers.

  • With --skip-release-readiness, scripts/production-readiness.sh uses the newest saved release-readiness-* packet by file timestamp instead of pinning stale release-readiness-r1 evidence.

  • shasum -a 256 -c checksums.txt from dist/

  • Temporary install QA via scripts/install-local.sh

  • Local markdown link check from Trust Surface

  • Shared output binary QA:

    • ceo-packet --help
    • ceo-packet --demo --format text
    • ceo-packet --doctor with bundled example model, CEO, and research adapters
    • outputs/ceo-packet --version

  • New operator binary QA:

    • bin/ceo-packet --start <temp> --format text
    • bin/ceo-packet --workspace <temp> --provider-wizard openai --http-model gpt-5 --format text
    • bin/ceo-packet --init-demo-repo <temp> --format text
    • bin/ceo-packet --workspace <demo> --write-policy dry-run --replace app.txt old new --format text Patch demo app
    • bin/ceo-packet --workspace <demo> --replace app.txt old new --format text Patch demo app
    • bin/ceo-packet --workspace <demo> --inbox
    • bin/ceo-packet --workspace <demo> --tui
    • bin/ceo-packet --workspace <demo> --write-policy preview --replace app.txt old new --format json Patch demo app
    • bin/ceo-packet --workspace <demo> --write-policy approved-write --approve-preview <preview_digest> --replace app.txt old new --format json Patch demo app
    • bin/ceo-packet --workspace <demo> --write-policy approved-write --replace app.txt old new --format json Patch demo app failed as expected without --approve-preview <preview_digest>

Release Artifacts Verified

  • dist/ceo-packet_0.1.1_darwin_arm64.tar.gz
  • dist/ceo-packet_0.1.1_linux_amd64.tar.gz
  • dist/ceo-packet_0.1.1_linux_arm64.tar.gz
  • dist/checksums.txt
  • dist/release-manifest.json

Tooling Available Locally

These optional strict tools are installed under the local Go bin and passed during the latest gate:

  • gofumpt
  • golangci-lint
  • nilaway
  • task

Tooling Not Available Locally

  • shellcheck

ShellCheck is still optional for a source install. scripts/strict-checks.sh now runs sh -n over shell scripts when ShellCheck is unavailable, so shell syntax is still checked.