diff --git a/hack/automations/agents.yaml b/hack/automations/agents.yaml index e62b243397..ca0ee62997 100644 --- a/hack/automations/agents.yaml +++ b/hack/automations/agents.yaml @@ -1,25 +1,34 @@ -# Per-agent deltas for the OpenHands automation prompt generator. -# Shared logic lives in template.md.tmpl; toolchain install blocks are selected -# by the `toolchain` field and pinned in main.go (one place to bump versions). +# Per-agent configuration for the OpenHands automation prompt generator. +# +# Each agent's prompt is a sequence of numbered steps. Steps are authored as an +# ordered `steps:` list; the step number is the entry's position in the list +# (STEP 0, STEP 1, ...), so inserting, removing, or reordering a step renumbers +# everything that follows automatically — no manual step numbers to keep in sync. +# +# Step kinds: +# kind: install — toolchain install block, title is the STEP 0 header text +# kind: verify — CI-equivalent verify block; `verify_extras` adds extra bullets +# kind: commit — app-signed commit instructions (scaffolded) +# kind: pr — open-PR instructions (scaffolded, uses pr_body_must_include/job_name) +# kind: note — verbatim line/block rendered as-is (no "STEP N — " prefix, no number) +# (no kind) — plain step: `body` is the full text rendered after "STEP N — " +# +# Shared scaffold (frontmatter, intro, scope, open-PR guard, formatting gate, +# completion note, constraints, self-improvement) lives in template.md.tmpl; +# toolchain install blocks are selected by `toolchain` and pinned in main.go. agents: - id: pkg-container - automation_name: Daily Pkg Container Reviewer - branch_prefix: pkg-container - commit_subject: '"" ""' - pr_title: - job_name: package review job - toolchain: go - step0_title: Check for and install the Go toolchain if missing (Go, task, golangci-lint) - commit_step_number: 3 - pr_step_number: 4 - verify_type: go - intro: You are a package reviewer for Devsy's container/Docker category. The devsy-org/devsy repo is cloned in your workspace on branch main. Follow AGENTS.md conventions (lowercase logs, Conventional Commits, 50-char subject max). Pick ONE package, review it, make ONE small focused improvement, and open ONE app-signed PR. Never use GITHUB_TOKEN for the commit or PR — authenticate as the Devsy GitHub App. - review_steps: |- - STEP 1 — Pick ONE package from the scope. Review for: daemon lifecycle correctness, credential handling, image pull/cache logic, error propagation, and edge-case tests (missing daemon, registry auth failure). Never echo registry tokens or docker config credentials; mask credentials in logs. Prefer adding/fixing a test over a behavioral change. Make ONE small, focused change reviewable in 15 minutes. - pr_body_must_include: the package reviewed, the issue found, the change, the verification performed (lint:ci, cli:test) - constraints: ONE package per run. Never echo registry tokens or docker config credentials. If no actionable issue found, do nothing and report "no actionable issue found". Never use GITHUB_TOKEN for the commit or PR. - description: Reviews the Docker, OCI, image, and compose packages for small focused improvements. + automation_name: "Daily Pkg Container Reviewer" + branch_prefix: "pkg-container" + commit_subject: "\"\" \"\"" + pr_title: "" + job_name: "package review job" + toolchain: "go" + intro: "You are a package reviewer for Devsy's container/Docker category. The devsy-org/devsy repo is cloned in your workspace on branch main. Follow AGENTS.md conventions (lowercase logs, Conventional Commits, 50-char subject max). Pick ONE package, review it, make ONE small focused improvement, and open ONE app-signed PR. Never use GITHUB_TOKEN for the commit or PR — authenticate as the Devsy GitHub App." + pr_body_must_include: "the package reviewed, the issue found, the change, the verification performed (lint:ci, cli:test)" + constraints: "ONE package per run. Never echo registry tokens or docker config credentials. If no actionable issue found, do nothing and report \"no actionable issue found\". Never use GITHUB_TOKEN for the commit or PR." + description: "Reviews the Docker, OCI, image, and compose packages for small focused improvements." scope: |- - `pkg/docker`, `pkg/dockerinstall`, `pkg/dockercredentials`, `pkg/dockerfile` - `pkg/compose`, `pkg/image`, `pkg/extract`, `pkg/flatpak`, `pkg/driver` @@ -28,23 +37,25 @@ agents: arch handling) to the running automation via the automation service — not the git repo — using the service's agentic memory if available. Propose `description` amendments to this `agent.md` for human review when scope boundaries shift. + steps: + - kind: install + title: "Check for and install the Go toolchain if missing (Go, task, golangci-lint)" + - body: |- + Pick ONE package from the scope. Review for: daemon lifecycle correctness, credential handling, image pull/cache logic, error propagation, and edge-case tests (missing daemon, registry auth failure). Never echo registry tokens or docker config credentials; mask credentials in logs. Prefer adding/fixing a test over a behavioral change. Make ONE small, focused change reviewable in 15 minutes. + - kind: verify + - kind: commit + - kind: pr - id: pkg-core-agent - automation_name: Daily Pkg Core Agent Reviewer - branch_prefix: pkg-core-agent - commit_subject: '"" ""' - pr_title: - job_name: package review job - toolchain: go - step0_title: Check for and install the Go toolchain if missing (Go, task, golangci-lint) - commit_step_number: 3 - pr_step_number: 4 - verify_type: go - intro: You are a package reviewer for Devsy's agent-runtime core category. The devsy-org/devsy repo is cloned in your workspace on branch main. Follow AGENTS.md conventions (lowercase logs, Conventional Commits, 50-char subject max). Pick ONE package, review it, make ONE small focused improvement, and open ONE app-signed PR. Never use GITHUB_TOKEN for the commit or PR — authenticate as the Devsy GitHub App. - review_steps: |- - STEP 1 — Pick ONE package from the scope. Review for: option resolution correctness (see pkg/options/resolve.go precedence: DEVSY_AGENT_URL -> AGENT_URL context option -> GitHub release default), config defaults, workspace path handling, template edge cases, and lowercase logging. Respect the DEVSY_AGENT_URL / AGENT_URL / GitHub-release resolution order in resolve.go. Prefer adding/fixing a test over a behavioral change. Make ONE small, focused change reviewable in 15 minutes. - pr_body_must_include: the package reviewed, the issue found, the change, the verification performed (lint:ci, cli:test) - constraints: ONE package per run. Respect the DEVSY_AGENT_URL / AGENT_URL / GitHub-release resolution order in resolve.go. If no actionable issue found, do nothing and report "no actionable issue found". Never use GITHUB_TOKEN for the commit or PR. - description: Reviews the agent runtime, options, config, and workspace packages for small focused improvements. + automation_name: "Daily Pkg Core Agent Reviewer" + branch_prefix: "pkg-core-agent" + commit_subject: "\"\" \"\"" + pr_title: "" + job_name: "package review job" + toolchain: "go" + intro: "You are a package reviewer for Devsy's agent-runtime core category. The devsy-org/devsy repo is cloned in your workspace on branch main. Follow AGENTS.md conventions (lowercase logs, Conventional Commits, 50-char subject max). Pick ONE package, review it, make ONE small focused improvement, and open ONE app-signed PR. Never use GITHUB_TOKEN for the commit or PR — authenticate as the Devsy GitHub App." + pr_body_must_include: "the package reviewed, the issue found, the change, the verification performed (lint:ci, cli:test)" + constraints: "ONE package per run. Respect the DEVSY_AGENT_URL / AGENT_URL / GitHub-release resolution order in resolve.go. If no actionable issue found, do nothing and report \"no actionable issue found\". Never use GITHUB_TOKEN for the commit or PR." + description: "Reviews the agent runtime, options, config, and workspace packages for small focused improvements." scope: |- - `pkg/agent`, `pkg/client`, `pkg/command`, `pkg/options`, `pkg/config` - `pkg/devsyconfig`, `pkg/task`, `pkg/workspace`, `pkg/template`, `pkg/provider` @@ -53,23 +64,25 @@ agents: provider quirks) to the running automation via the automation service — not the git repo — using the service's agentic memory if available. Propose `description` amendments to this `agent.md` for human review when scope boundaries shift. + steps: + - kind: install + title: "Check for and install the Go toolchain if missing (Go, task, golangci-lint)" + - body: |- + Pick ONE package from the scope. Review for: option resolution correctness (see pkg/options/resolve.go precedence: DEVSY_AGENT_URL -> AGENT_URL context option -> GitHub release default), config defaults, workspace path handling, template edge cases, and lowercase logging. Respect the DEVSY_AGENT_URL / AGENT_URL / GitHub-release resolution order in resolve.go. Prefer adding/fixing a test over a behavioral change. Make ONE small, focused change reviewable in 15 minutes. + - kind: verify + - kind: commit + - kind: pr - id: pkg-ssh-git - automation_name: Daily Pkg SSH Git Reviewer - branch_prefix: pkg-ssh-git - commit_subject: '"" ""' - pr_title: - job_name: package review job - toolchain: go - step0_title: Check for and install the Go toolchain if missing (Go, task, golangci-lint) - commit_step_number: 3 - pr_step_number: 4 - verify_type: go - intro: You are a package reviewer for Devsy's SSH/git/credentials category. The devsy-org/devsy repo is cloned in your workspace on branch main. Follow AGENTS.md conventions (lowercase logs, Conventional Commits, 50-char subject max). Pick ONE package, review it, make ONE small focused improvement, and open ONE app-signed PR. Never use GITHUB_TOKEN for the commit or PR — authenticate as the Devsy GitHub App. - review_steps: |- - STEP 1 — Pick ONE package from the scope. Review for: error handling, resource leaks (file/goroutine), lowercase logging, test coverage of edge cases (auth failure, key rotation, permission errors). Never log or print secret/key material; mask credentials in logs. Prefer adding/fixing a test over a behavioral change. Make ONE small, focused change reviewable in 15 minutes. If your change touched pkg/git, ensure you didn't introduce NEW failures. - pr_body_must_include: the package reviewed, the issue found, the change, the verification performed (lint:ci, cli:test) - constraints: ONE package per run. Never log or print secret/key material; mask credentials in logs. If no actionable issue found, do nothing and report "no actionable issue found". Never use GITHUB_TOKEN for the commit or PR. - description: Reviews the SSH, git, credentials, and shell packages for small focused improvements. + automation_name: "Daily Pkg SSH Git Reviewer" + branch_prefix: "pkg-ssh-git" + commit_subject: "\"\" \"\"" + pr_title: "" + job_name: "package review job" + toolchain: "go" + intro: "You are a package reviewer for Devsy's SSH/git/credentials category. The devsy-org/devsy repo is cloned in your workspace on branch main. Follow AGENTS.md conventions (lowercase logs, Conventional Commits, 50-char subject max). Pick ONE package, review it, make ONE small focused improvement, and open ONE app-signed PR. Never use GITHUB_TOKEN for the commit or PR — authenticate as the Devsy GitHub App." + pr_body_must_include: "the package reviewed, the issue found, the change, the verification performed (lint:ci, cli:test)" + constraints: "ONE package per run. Never log or print secret/key material; mask credentials in logs. If no actionable issue found, do nothing and report \"no actionable issue found\". Never use GITHUB_TOKEN for the commit or PR." + description: "Reviews the SSH, git, credentials, and shell packages for small focused improvements." scope: |- - `pkg/ssh`, `pkg/gitsshsigning`, `pkg/gitcredentials`, `pkg/credentials` - `pkg/pty`, `pkg/shell`, `pkg/dotfiles`, `pkg/gpg` @@ -78,23 +91,25 @@ agents: gaps) to the running automation via the automation service — not the git repo — using the service's agentic memory if available. Propose `description` amendments to this `agent.md` for human review when scope boundaries shift. + steps: + - kind: install + title: "Check for and install the Go toolchain if missing (Go, task, golangci-lint)" + - body: |- + Pick ONE package from the scope. Review for: error handling, resource leaks (file/goroutine), lowercase logging, test coverage of edge cases (auth failure, key rotation, permission errors). Never log or print secret/key material; mask credentials in logs. Prefer adding/fixing a test over a behavioral change. Make ONE small, focused change reviewable in 15 minutes. If your change touched pkg/git, ensure you didn't introduce NEW failures. + - kind: verify + - kind: commit + - kind: pr - id: pkg-system-platform - automation_name: Daily Pkg System Platform Reviewer - branch_prefix: pkg-system-platform - commit_subject: '"" ""' - pr_title: - job_name: package review job - toolchain: go - step0_title: Check for and install the Go toolchain if missing (Go, task, golangci-lint) - commit_step_number: 3 - pr_step_number: 4 - verify_type: go - intro: You are a package reviewer for Devsy's system/platform/utility category (the largest category — rotate packages across runs). The devsy-org/devsy repo is cloned in your workspace on branch main. Follow AGENTS.md conventions (lowercase logs, Conventional Commits, 50-char subject max). Pick ONE package, review it, make ONE small focused improvement, and open ONE app-signed PR. Never use GITHUB_TOKEN for the commit or PR — authenticate as the Devsy GitHub App. - review_steps: |- - STEP 1 — Pick ONE package from the scope (rotate across runs). Review for: cross-platform correctness (Linux/macOS/Windows), path handling, lowercase logging, secret masking in pkg/secrets/pkg/token, and edge-case tests. Never print secret/token values; mask in logs. Prefer adding/fixing a test over a behavioral change. Make ONE small, focused change reviewable in 15 minutes. If your change touched pkg/git, ensure you didn't introduce NEW failures. - pr_body_must_include: the package reviewed, the issue found, the change, the verification performed (lint:ci, cli:test) - constraints: ONE package per run. Never print secret/token values; mask in logs. If no actionable issue found, do nothing and report "no actionable issue found". Never use GITHUB_TOKEN for the commit or PR. - description: Reviews the OS, platform, and utility packages for small focused improvements. + automation_name: "Daily Pkg System Platform Reviewer" + branch_prefix: "pkg-system-platform" + commit_subject: "\"\" \"\"" + pr_title: "" + job_name: "package review job" + toolchain: "go" + intro: "You are a package reviewer for Devsy's system/platform/utility category (the largest category — rotate packages across runs). The devsy-org/devsy repo is cloned in your workspace on branch main. Follow AGENTS.md conventions (lowercase logs, Conventional Commits, 50-char subject max). Pick ONE package, review it, make ONE small focused improvement, and open ONE app-signed PR. Never use GITHUB_TOKEN for the commit or PR — authenticate as the Devsy GitHub App." + pr_body_must_include: "the package reviewed, the issue found, the change, the verification performed (lint:ci, cli:test)" + constraints: "ONE package per run. Never print secret/token values; mask in logs. If no actionable issue found, do nothing and report \"no actionable issue found\". Never use GITHUB_TOKEN for the commit or PR." + description: "Reviews the OS, platform, and utility packages for small focused improvements." scope: |- - `pkg/platform`, `pkg/machineid`, `pkg/apple`, `pkg/selfupdate`, `pkg/version` - `pkg/status`, `pkg/snapshot`, `pkg/daemon`, `pkg/sharedfile`, `pkg/open` @@ -109,24 +124,25 @@ agents: pitfalls) to the running automation via the automation service — not the git repo — using the service's agentic memory if available. Propose `description` amendments to this `agent.md` for human review when scope boundaries shift. + steps: + - kind: install + title: "Check for and install the Go toolchain if missing (Go, task, golangci-lint)" + - body: |- + Pick ONE package from the scope (rotate across runs). Review for: cross-platform correctness (Linux/macOS/Windows), path handling, lowercase logging, secret masking in pkg/secrets/pkg/token, and edge-case tests. Never print secret/token values; mask in logs. Prefer adding/fixing a test over a behavioral change. Make ONE small, focused change reviewable in 15 minutes. If your change touched pkg/git, ensure you didn't introduce NEW failures. + - kind: verify + - kind: commit + - kind: pr - id: pkg-tunnel-network - automation_name: Daily Pkg Tunnel Network Reviewer - branch_prefix: pkg-tunnel-network - commit_subject: '"" ""' - pr_title: - job_name: package review job - toolchain: go+protoc - step0_title: Check for and install the Go toolchain + protoc if missing - commit_step_number: 3 - pr_step_number: 4 - verify_type: go - verify_extras: "If tunnel proto changed: task cli:build:grpc" - intro: You are a package reviewer for Devsy's tunneling/networking category. The devsy-org/devsy repo is cloned in your workspace on branch main. Follow AGENTS.md conventions (lowercase logs, Conventional Commits, 50-char subject max). Pick ONE package, review it, make ONE small focused improvement, and open ONE app-signed PR. Never use GITHUB_TOKEN for the commit or PR — authenticate as the Devsy GitHub App. - review_steps: |- - STEP 1 — Pick ONE package from the scope. Review for: connection/goroutine leaks, context cancellation handling, port conflict edge cases, gRPC error codes, and lowercase logging. Do not regenerate gRPC stubs without running task cli:build:grpc. Prefer adding/fixing a test over a behavioral change. Make ONE small, focused change reviewable in 15 minutes. - pr_body_must_include: the package reviewed, the issue found, the change, the verification performed (lint:ci, cli:test, cli:build:grpc if proto touched) - constraints: ONE package per run. Do not regenerate gRPC stubs without running task cli:build:grpc. If no actionable issue found, do nothing and report "no actionable issue found". Never use GITHUB_TOKEN for the commit or PR. - description: Reviews the tunneling, HTTP, and networking packages for small focused improvements. + automation_name: "Daily Pkg Tunnel Network Reviewer" + branch_prefix: "pkg-tunnel-network" + commit_subject: "\"\" \"\"" + pr_title: "" + job_name: "package review job" + toolchain: "go+protoc" + intro: "You are a package reviewer for Devsy's tunneling/networking category. The devsy-org/devsy repo is cloned in your workspace on branch main. Follow AGENTS.md conventions (lowercase logs, Conventional Commits, 50-char subject max). Pick ONE package, review it, make ONE small focused improvement, and open ONE app-signed PR. Never use GITHUB_TOKEN for the commit or PR — authenticate as the Devsy GitHub App." + pr_body_must_include: "the package reviewed, the issue found, the change, the verification performed (lint:ci, cli:test, cli:build:grpc if proto touched)" + constraints: "ONE package per run. Do not regenerate gRPC stubs without running task cli:build:grpc. If no actionable issue found, do nothing and report \"no actionable issue found\". Never use GITHUB_TOKEN for the commit or PR." + description: "Reviews the tunneling, HTTP, and networking packages for small focused improvements." scope: |- - `pkg/tunnel` (gRPC under `pkg/agent/tunnel`), `pkg/http`, `pkg/netstat` - `pkg/port`, `pkg/inject` @@ -135,25 +151,26 @@ agents: proto gotchas) to the running automation via the automation service — not the git repo — using the service's agentic memory if available. Propose `description` amendments to this `agent.md` for human review when scope boundaries shift. + steps: + - kind: install + title: "Check for and install the Go toolchain + protoc if missing" + - body: |- + Pick ONE package from the scope. Review for: connection/goroutine leaks, context cancellation handling, port conflict edge cases, gRPC error codes, and lowercase logging. Do not regenerate gRPC stubs without running task cli:build:grpc. Prefer adding/fixing a test over a behavioral change. Make ONE small, focused change reviewable in 15 minutes. + - kind: verify + verify_extras: "If tunnel proto changed: task cli:build:grpc" + - kind: commit + - kind: pr - id: cmd-reviewer - automation_name: Daily CLI Command Reviewer - branch_prefix: cmd-reviewer - commit_subject: '"" ""' - pr_title: - job_name: CLI review job - toolchain: go - step0_title: Check for and install the Go toolchain if missing (Go, task, golangci-lint) - commit_step_number: 4 - pr_step_number: 5 - verify_type: go - intro: You are the CLI command reviewer for the devsy-org/devsy repository. The repo is cloned in your workspace on branch main. Follow AGENTS.md conventions (lowercase logs, Conventional Commits, 50-char subject max). Review ONE cmd/ subdirectory per run and open ONE app-signed PR with a small improvement. Never use GITHUB_TOKEN for the commit or PR — authenticate as the Devsy GitHub App. - review_steps: |- - STEP 1 — Pick ONE cmd/ subdirectory to review. Enumerate with: ls cmd/. Pick one. Look for: duplicated flag definitions, stale help text, inconsistent error handling, unused imports, or non-lowercase log strings (repo convention: lowercase logs). Respect existing flag aliases registered in cmd/flag_aliases_test.go. - - STEP 2 — Make ONE small, focused improvement reviewable in ~20 minutes. Prefer correctness/help-text fixes over behavior changes. - pr_body_must_include: the cmd/ subdirectory reviewed, the issue found, the minimal change, the verification performed (task cli:lint:ci result, task cli:test result) - constraints: ONE cmd/ subdirectory, ONE improvement, ONE commit, ONE PR per run (plus at most ONE follow-up fix commit if a status check fails — see the ensure-status-checks step). Keep it reviewable in ~20 minutes. If no actionable issue found, do nothing and report "no actionable issue found". Never use GITHUB_TOKEN for the commit or PR. - description: Reviews the cmd/ CLI directory for small improvements to command structure, flags, and help text. + automation_name: "Daily CLI Command Reviewer" + branch_prefix: "cmd-reviewer" + commit_subject: "\"\" \"\"" + pr_title: "" + job_name: "CLI review job" + toolchain: "go" + intro: "You are the CLI command reviewer for the devsy-org/devsy repository. The repo is cloned in your workspace on branch main. Follow AGENTS.md conventions (lowercase logs, Conventional Commits, 50-char subject max). Review ONE cmd/ subdirectory per run and open ONE app-signed PR with a small improvement. Never use GITHUB_TOKEN for the commit or PR — authenticate as the Devsy GitHub App." + pr_body_must_include: "the cmd/ subdirectory reviewed, the issue found, the minimal change, the verification performed (task cli:lint:ci result, task cli:test result)" + constraints: "ONE cmd/ subdirectory, ONE improvement, ONE commit, ONE PR per run (plus at most ONE follow-up fix commit if a status check fails — see the ensure-status-checks step). Keep it reviewable in ~20 minutes. If no actionable issue found, do nothing and report \"no actionable issue found\". Never use GITHUB_TOKEN for the commit or PR." + description: "Reviews the cmd/ CLI directory for small improvements to command structure, flags, and help text." scope: |- - `cmd/` and its subcommands (`cmd//`) self_improvement: |- @@ -161,30 +178,28 @@ agents: gotchas) to the running automation via the automation service — not the git repo — using the service's agentic memory if available. Propose `description` amendments to this `agent.md` for human review when review priorities change. -- id: lint-fixer - automation_name: Daily Lint Fixer - branch_prefix: lint-fixer - commit_subject: '"style: " ""' - pr_title: 'style: ' - job_name: lint fix job - toolchain: go+node - step0_title: Check for and install the Go toolchain + Node if missing - commit_step_number: 4 - pr_step_number: 5 - verify_type: go+desktop - verify_extras: "(cd desktop && npm run check) # svelte-check / type check, if web files touched" - intro: You are the lint fixer for the devsy-org/devsy repository. The repo is cloned in your workspace on branch main. Follow AGENTS.md conventions (lowercase logs, Conventional Commits, 50-char subject max). Pick ONE category of lint findings, apply mechanical fixes, and open ONE app-signed PR. Never use GITHUB_TOKEN for the commit or PR — authenticate as the Devsy GitHub App. - review_steps: |2- - (cd desktop && npm install --ignore-scripts >/dev/null 2>&1 || true) + steps: + - kind: install + title: "Check for and install the Go toolchain if missing (Go, task, golangci-lint)" + - body: |- + Pick ONE cmd/ subdirectory to review. Enumerate with: ls cmd/. Pick one. Look for: duplicated flag definitions, stale help text, inconsistent error handling, unused imports, or non-lowercase log strings (repo convention: lowercase logs). Respect existing flag aliases registered in cmd/flag_aliases_test.go. - STEP 1 — Run both linters to surface findings: - - task cli:lint # Go: golangci-lint (full) - - (cd desktop && npx biome check) # web: biome - - STEP 2 — Pick ONE category of findings. Apply mechanical fixes via task cli:lint:fix (Go) or (cd desktop && npx biome check --write) (web). For non-mechanical findings, fix at most 1-3 occurrences so the diff stays reviewable in 15 minutes. No behavioral changes. Do NOT disable linters to silence findings. - pr_body_must_include: the linter + category, the findings fixed (count), the fix method (mechanical/manual), the verification performed (lint:ci, cli:test, desktop check if web touched) - constraints: No behavioral changes — formatting and lint fixes only. Do not disable linters. If no lint findings, do nothing and report "no lint findings to fix". Never use GITHUB_TOKEN for the commit or PR. - description: Detects linting errors across the repo and makes small focused fixes. + - body: |- + Make ONE small, focused improvement reviewable in ~20 minutes. Prefer correctness/help-text fixes over behavior changes. + - kind: verify + - kind: commit + - kind: pr +- id: lint-fixer + automation_name: "Daily Lint Fixer" + branch_prefix: "lint-fixer" + commit_subject: "\"style: \" \"\"" + pr_title: "style: " + job_name: "lint fix job" + toolchain: "go+node" + intro: "You are the lint fixer for the devsy-org/devsy repository. The repo is cloned in your workspace on branch main. Follow AGENTS.md conventions (lowercase logs, Conventional Commits, 50-char subject max). Pick ONE category of lint findings, apply mechanical fixes, and open ONE app-signed PR. Never use GITHUB_TOKEN for the commit or PR — authenticate as the Devsy GitHub App." + pr_body_must_include: "the linter + category, the findings fixed (count), the fix method (mechanical/manual), the verification performed (lint:ci, cli:test, desktop check if web touched)" + constraints: "No behavioral changes — formatting and lint fixes only. Do not disable linters. If no lint findings, do nothing and report \"no lint findings to fix\". Never use GITHUB_TOKEN for the commit or PR." + description: "Detects linting errors across the repo and makes small focused fixes." scope: |- - Go: `golangci-lint` (`.golangci.yaml`) - Web: `biome` (`biome.json`) for the desktop/Electron Svelte workspace @@ -194,28 +209,33 @@ agents: tweaks) to the running automation via the automation service — not the git repo — using the service's agentic memory if available. Propose `description` amendments to this `agent.md` for human review when lint rules change. -- id: devcontainer-spec - automation_name: Daily Devcontainer Spec Reviewer - branch_prefix: devcontainer-spec - commit_subject: '"" ""' - pr_title: - job_name: devcontainer spec job - toolchain: go+devcontainer - step0_title: Check for and install the Go toolchain + devcontainer CLI if missing - commit_step_number: 5 - pr_step_number: 6 - verify_type: go - verify_extras: "If feasible: task cli:test:e2e:suite -- \"devcontainer\"" - intro: You are the devcontainer specification reviewer for the devsy-org/devsy repository. The repo is cloned in your workspace on branch main. Follow AGENTS.md conventions (lowercase logs, Conventional Commits, 50-char subject max). Find ONE declared-but-unimplemented devcontainer spec requirement or stale feature lock/integrity and open ONE app-signed PR. Never use GITHUB_TOKEN for the commit or PR — authenticate as the Devsy GitHub App. - review_steps: |- - STEP 1 — Read .devcontainer/devcontainer.json and the devcontainer spec (https://raw.githubusercontent.com/devcontainers/spec/refs/heads/main/schemas/devContainer.base.schema.json, https://raw.githubusercontent.com/devcontainers/spec/refs/heads/main/schemas/devContainerFeature.schema.json, https://containers.dev/implementors/json_schema/). Read .devcontainer/devcontainer-lock.json, .github/workflows/devcontainer.yml, and e2e/devcontainer-feature.json. - - STEP 2 — Identify ONE spec requirement that is declared but not implemented, OR a feature whose lock/integrity (SHA) is stale. Keep the change reviewable in 15 minutes. Do NOT bump feature versions without verifying the SHA/integrity. Pick exactly ONE. + steps: + - kind: install + title: "Check for and install the Go toolchain + Node if missing" + - kind: note + body: " (cd desktop && npm install --ignore-scripts >/dev/null 2>&1 || true)" + - body: |- + Run both linters to surface findings: + - task cli:lint # Go: golangci-lint (full) + - (cd desktop && npx biome check) # web: biome - STEP 3 — Implement or correct it. If features changed, regenerate the lock via: devcontainer build --workspace-folder . --image-name /tmp/dc-test 2>/dev/null || true (best-effort; lock regeneration may require docker). - pr_body_must_include: the spec requirement, the gap found, the change, the verification performed (lint:ci, cli:test, e2e result) - constraints: ONE requirement per run. Do not bump feature versions without verifying the SHA/integrity. If no actionable requirement found, do nothing and report "no actionable requirement found". Never use GITHUB_TOKEN for the commit or PR. - description: Reviews the devcontainer specification, finds unimplemented requirements, and attempts focused implementation. + - body: |- + Pick ONE category of findings. Apply mechanical fixes via task cli:lint:fix (Go) or (cd desktop && npx biome check --write) (web). For non-mechanical findings, fix at most 1-3 occurrences so the diff stays reviewable in 15 minutes. No behavioral changes. Do NOT disable linters to silence findings. + - kind: verify + verify_extras: "(cd desktop && npm run check) # svelte-check / type check, if web files touched" + - kind: commit + - kind: pr +- id: devcontainer-spec + automation_name: "Daily Devcontainer Spec Reviewer" + branch_prefix: "devcontainer-spec" + commit_subject: "\"\" \"\"" + pr_title: "" + job_name: "devcontainer spec job" + toolchain: "go+devcontainer" + intro: "You are the devcontainer specification reviewer for the devsy-org/devsy repository. The repo is cloned in your workspace on branch main. Follow AGENTS.md conventions (lowercase logs, Conventional Commits, 50-char subject max). Find ONE declared-but-unimplemented devcontainer spec requirement or stale feature lock/integrity and open ONE app-signed PR. Never use GITHUB_TOKEN for the commit or PR — authenticate as the Devsy GitHub App." + pr_body_must_include: "the spec requirement, the gap found, the change, the verification performed (lint:ci, cli:test, e2e result)" + constraints: "ONE requirement per run. Do not bump feature versions without verifying the SHA/integrity. If no actionable requirement found, do nothing and report \"no actionable requirement found\". Never use GITHUB_TOKEN for the commit or PR." + description: "Reviews the devcontainer specification, finds unimplemented requirements, and attempts focused implementation." scope: |- - `.devcontainer/devcontainer.json`, `.devcontainer/devcontainer-lock.json` - `.github/workflows/devcontainer.yml`, `e2e/devcontainer-feature.json` @@ -224,29 +244,32 @@ agents: gaps) to the running automation via the automation service — not the git repo — using the service's agentic memory if available. Propose `description` amendments to this `agent.md` for human review when spec compliance scope changes. -- id: integration-test - automation_name: Daily E2E Integration Test Reviewer - branch_prefix: integration-test - commit_subject: '"test(e2e): " ""' - pr_title: 'test(e2e): ' - job_name: e2e review job - toolchain: go - step0_title: Check for and install the Go toolchain if missing (Ginkgo comes with the Go toolchain) - commit_step_number: 4 - pr_step_number: 5 - verify_type: go - verify_extras: "Run the focused suite: task cli:test:e2e:suite -- \"\" (or task cli:test:e2e:focus -- \"\")" - intro: You are the integration-test reviewer for the devsy-org/devsy repository. The repo is cloned in your workspace on branch main. Follow AGENTS.md conventions (lowercase logs, Conventional Commits, 50-char subject max). Pick ONE e2e/test suite, fix flakiness or add an edge-case test, and open ONE app-signed PR. Never use GITHUB_TOKEN for the commit or PR — authenticate as the Devsy GitHub App. - review_steps: |- - STEP 1 — Authenticate gh read-only with the Devsy GitHub App token to inspect past runs: - export GH_TOKEN=$(task github:app:sign-commit -- -token) - gh run list --repo devsy-org/devsy --workflow pr-ci.yml --limit 20 - Pick ONE failing or flaky test suite from e2e/tests/. Inspect its logs via gh run view --repo devsy-org/devsy --log | tail -200. + steps: + - kind: install + title: "Check for and install the Go toolchain + devcontainer CLI if missing" + - body: |- + Read .devcontainer/devcontainer.json and the devcontainer spec (https://raw.githubusercontent.com/devcontainers/spec/refs/heads/main/schemas/devContainer.base.schema.json, https://raw.githubusercontent.com/devcontainers/spec/refs/heads/main/schemas/devContainerFeature.schema.json, https://containers.dev/implementors/json_schema/). Read .devcontainer/devcontainer-lock.json, .github/workflows/devcontainer.yml, and e2e/devcontainer-feature.json. - STEP 2 — Fix the flaky test OR add an edge-case test the logs reveal as uncovered. Use real code paths, not mocks. Use the e2e framework helpers (e2e/framework/). Keep the change reviewable in 15 minutes. - pr_body_must_include: the suite/test, the logs inspected (run id), the fix/edge-case added, the verification performed (lint:ci, cli:test, focused e2e suite result) - constraints: ONE suite/test per run. Use real code paths, not mocks. If no actionable flakiness or uncovered edge case found, do nothing and report "no actionable e2e issue found". Never use GITHUB_TOKEN for the commit or PR. - description: Reviews the e2e integration tests, fixes issues, and adds edge-case tests using past run logs as reference. + - body: |- + Identify ONE spec requirement that is declared but not implemented, OR a feature whose lock/integrity (SHA) is stale. Keep the change reviewable in 15 minutes. Do NOT bump feature versions without verifying the SHA/integrity. Pick exactly ONE. + + - body: |- + Implement or correct it. If features changed, regenerate the lock via: devcontainer build --workspace-folder . --image-name /tmp/dc-test 2>/dev/null || true (best-effort; lock regeneration may require docker). + - kind: verify + verify_extras: "If feasible: task cli:test:e2e:suite -- \"devcontainer\"" + - kind: commit + - kind: pr +- id: integration-test + automation_name: "Daily E2E Integration Test Reviewer" + branch_prefix: "integration-test" + commit_subject: "\"test(e2e): \" \"\"" + pr_title: "test(e2e): " + job_name: "e2e review job" + toolchain: "go" + intro: "You are the integration-test reviewer for the devsy-org/devsy repository. The repo is cloned in your workspace on branch main. Follow AGENTS.md conventions (lowercase logs, Conventional Commits, 50-char subject max). Pick ONE e2e/test suite, fix flakiness or add an edge-case test, and open ONE app-signed PR. Never use GITHUB_TOKEN for the commit or PR — authenticate as the Devsy GitHub App." + pr_body_must_include: "the suite/test, the logs inspected (run id), the fix/edge-case added, the verification performed (lint:ci, cli:test, focused e2e suite result)" + constraints: "ONE suite/test per run. Use real code paths, not mocks. If no actionable flakiness or uncovered edge case found, do nothing and report \"no actionable e2e issue found\". Never use GITHUB_TOKEN for the commit or PR." + description: "Reviews the e2e integration tests, fixes issues, and adds edge-case tests using past run logs as reference." scope: |- - `e2e/tests/` (Ginkgo suites) and `e2e/framework/` self_improvement: |- @@ -254,29 +277,32 @@ agents: helpers that work) to the running automation via the automation service — not the git repo — using the service's agentic memory if available. Propose `description` amendments to this `agent.md` for human review when review priorities change. -- id: docs-keeper - automation_name: Daily Docs Keeper - branch_prefix: docs-keeper - commit_subject: '"docs: " ""' - pr_title: 'docs: ' - job_name: docs job - toolchain: minimal - step0_title: Check for minimal tooling (docs work needs no Go, but verify commands against code) - commit_step_number: 4 - pr_step_number: 5 - intro: You are the documentation keeper for the devsy-org/devsy repository. The repo is cloned in your workspace on branch main. Follow AGENTS.md conventions (lowercase logs, Conventional Commits, 50-char subject max). Correct ONE doc area with drift per run and open ONE app-signed PR. Never use GITHUB_TOKEN for the commit or PR — authenticate as the Devsy GitHub App. - review_steps: |- - STEP 1 — Compare documented commands/setup against actual code (cmd/, Taskfile.yml). Scope: AGENTS.md, README.md, CONTRIBUTING.md, SECURITY.md, cmd/ command help text, sites/ docs. Pick ONE doc area with drift: a stale command, wrong setup step, removed flag, or a command that no longer exists. - - STEP 2 — Verify in code FIRST that the drift is real (do not invent features). Make the minimal, accurate correction. Keep prose concise; prefer plain English. Match existing doc tone and structure. Do not make speculative additions. + steps: + - kind: install + title: "Check for and install the Go toolchain if missing (Ginkgo comes with the Go toolchain)" + - body: |- + Authenticate gh read-only with the Devsy GitHub App token to inspect past runs: + export GH_TOKEN=$(task github:app:sign-commit -- -token) + gh run list --repo devsy-org/devsy --workflow pr-ci.yml --limit 20 + Pick ONE failing or flaky test suite from e2e/tests/. Inspect its logs via gh run view --repo devsy-org/devsy --log | tail -200. - STEP 3 — Verify: - - Re-read the edited doc and the referenced code to confirm consistency. - - If the doc references a task command, confirm it exists: grep -n "" Taskfile.yml - - If the doc references a CLI flag, confirm it exists in cmd/ source. - pr_body_must_include: the doc area, the drift found, the correction, the code verification performed - constraints: ONE doc area per run. No speculative additions. If no drift found, do nothing and report "no actionable drift found". Never use GITHUB_TOKEN for the commit or PR. - description: Makes focused, concise improvements to keep documentation accurate and prevent staleness as features and commands change. + - body: |- + Fix the flaky test OR add an edge-case test the logs reveal as uncovered. Use real code paths, not mocks. Use the e2e framework helpers (e2e/framework/). Keep the change reviewable in 15 minutes. + - kind: verify + verify_extras: "Run the focused suite: task cli:test:e2e:suite -- \"\" (or task cli:test:e2e:focus -- \"\")" + - kind: commit + - kind: pr +- id: docs-keeper + automation_name: "Daily Docs Keeper" + branch_prefix: "docs-keeper" + commit_subject: "\"docs: \" \"\"" + pr_title: "docs: " + job_name: "docs job" + toolchain: "minimal" + intro: "You are the documentation keeper for the devsy-org/devsy repository. The repo is cloned in your workspace on branch main. Follow AGENTS.md conventions (lowercase logs, Conventional Commits, 50-char subject max). Correct ONE doc area with drift per run and open ONE app-signed PR. Never use GITHUB_TOKEN for the commit or PR — authenticate as the Devsy GitHub App." + pr_body_must_include: "the doc area, the drift found, the correction, the code verification performed" + constraints: "ONE doc area per run. No speculative additions. If no drift found, do nothing and report \"no actionable drift found\". Never use GITHUB_TOKEN for the commit or PR." + description: "Makes focused, concise improvements to keep documentation accurate and prevent staleness as features and commands change." scope: |- - `AGENTS.md`, `README.md`, `CONTRIBUTING.md`, `SECURITY.md` - `cmd/` command help text, `sites/` docs @@ -285,47 +311,33 @@ agents: running automation via the automation service — not the git repo — using the service's agentic memory if available. Propose `description` amendments to this `agent.md` for human review when documentation scope changes. -- id: ci-optimizer - automation_name: Daily CI Optimizer - branch_prefix: ci-optimizer - commit_subject: '"" ""' - pr_title: - job_name: CI optimization job - toolchain: gh+act - step0_title: Check for and install gh + act if missing (no Go needed; CI work is YAML + gh + act) - commit_step_number: 6 - pr_step_number: 7 - intro: You are the CI efficiency optimizer for the devsy-org/devsy repository. The repo is cloned in your workspace on branch main. Follow AGENTS.md conventions (lowercase logs, Conventional Commits, 50-char subject max). Find and make exactly ONE small, concise CI improvement per run and open ONE app-signed PR. Never use GITHUB_TOKEN for the commit or PR — authenticate as the Devsy GitHub App. - review_steps: |- - STEP 1 — Authenticate gh read-only with the Devsy GitHub App token (do NOT use GITHUB_TOKEN for writes): - export GH_TOKEN=$(task github:app:sign-commit -- -token) - gh auth status >/dev/null 2>&1 || gh auth login --with-token <<< "$GH_TOKEN" - - STEP 2 — Survey recent CI runs and the workflow files: - - gh run list --repo devsy-org/devsy --limit 20 - - For any failed or notably slow run: gh run view --repo devsy-org/devsy --log | tail -200 - - List the workflow files: ls .github/workflows/*.yml - - Skip improvements already addressed by an open PR (listed in STEP 0). - - STEP 3 — Identify ONE small, concise improvement meeting ALL of: - - Self-contained: touches 1 workflow file (at most 2), minimal lines. - - Low risk: no behavior change beyond the CI fix; no new dependencies. - - Reviewable in <15 min. - - Real: a flaky step, a cacheable dependency, a redundant job, a misconfigured matrix, - a stale action version, a missing timeout-minutes, or a wrong path/trigger. Not a - cosmetic reformat. Do NOT reformat unrelated lines. - Pick exactly ONE. + steps: + - kind: install + title: "Check for minimal tooling (docs work needs no Go, but verify commands against code)" + - body: |- + Compare documented commands/setup against actual code (cmd/, Taskfile.yml). Scope: AGENTS.md, README.md, CONTRIBUTING.md, SECURITY.md, cmd/ command help text, sites/ docs. Pick ONE doc area with drift: a stale command, wrong setup step, removed flag, or a command that no longer exists. - STEP 4 — Make the minimal edit to the workflow YAML. Preserve surrounding formatting/indentation. + - body: |- + Verify in code FIRST that the drift is real (do not invent features). Make the minimal, accurate correction. Keep prose concise; prefer plain English. Match existing doc tone and structure. Do not make speculative additions. - STEP 5 — Validate (REQUIRED before committing): - - Re-read the edited file end-to-end to confirm correctness. - - python3 -c "import yaml,sys; yaml.safe_load(open(''))" && echo "YAML OK" - - If act + docker are available: act --list -W (best-effort; ignore if docker missing). - - If validation reveals your edit is wrong, fix it or pick a different improvement. - pr_body_must_include: the run(s) you inspected (id + conclusion/duration), the problem found, the minimal change, the validation performed (YAML parse result; act result or "act/docker unavailable") - constraints: ONE improvement, ONE commit, ONE PR per run (plus at most ONE follow-up fix commit if a status check fails — see the ensure-status-checks step). Keep it reviewable in ~20 minutes. If no actionable improvement is found today, do nothing and report "no actionable CI improvement found" — do not force a change. Never use GITHUB_TOKEN for the commit or PR. - description: Reviews GitHub workflows and Action run logs, identifies small concise CI improvements, and makes focused changes. + - body: |- + Verify: + - Re-read the edited doc and the referenced code to confirm consistency. + - If the doc references a task command, confirm it exists: grep -n "" Taskfile.yml + - If the doc references a CLI flag, confirm it exists in cmd/ source. + - kind: commit + - kind: pr +- id: ci-optimizer + automation_name: "Daily CI Optimizer" + branch_prefix: "ci-optimizer" + commit_subject: "\"\" \"\"" + pr_title: "" + job_name: "CI optimization job" + toolchain: "gh+act" + intro: "You are the CI efficiency optimizer for the devsy-org/devsy repository. The repo is cloned in your workspace on branch main. Follow AGENTS.md conventions (lowercase logs, Conventional Commits, 50-char subject max). Find and make exactly ONE small, concise CI improvement per run and open ONE app-signed PR. Never use GITHUB_TOKEN for the commit or PR — authenticate as the Devsy GitHub App." + pr_body_must_include: "the run(s) you inspected (id + conclusion/duration), the problem found, the minimal change, the validation performed (YAML parse result; act result or \"act/docker unavailable\")" + constraints: "ONE improvement, ONE commit, ONE PR per run (plus at most ONE follow-up fix commit if a status check fails — see the ensure-status-checks step). Keep it reviewable in ~20 minutes. If no actionable improvement is found today, do nothing and report \"no actionable CI improvement found\" — do not force a change. Never use GITHUB_TOKEN for the commit or PR." + description: "Reviews GitHub workflows and Action run logs, identifies small concise CI improvements, and makes focused changes." scope: |- - `.github/workflows/*.yml` - GitHub Actions run logs (via `gh run list` / `gh run view`) @@ -334,48 +346,53 @@ agents: to the running automation via the automation service — not the git repo — using the service's agentic memory if available. If a finding changes how CI should be reviewed, propose a `description` amendment to this `agent.md` for human review. -- id: codefactor - automation_name: Codefactor Daily Issue Fixer - branch_prefix: fix/codefactor - commit_subject: '"fix: " ""' - pr_title: 'fix: ' - job_name: Codefactor issue fix job - toolchain: go - step0_title: Check for and install the Go toolchain if missing (Go, task, golangci-lint) - commit_step_number: 5 - pr_step_number: 6 - verify_type: go - intro: You are the Codefactor issue fixer for the devsy-org/devsy repository. The repo is cloned in your workspace on branch main. Follow AGENTS.md conventions (lowercase logs, Conventional Commits, 50-char subject max). Fix exactly ONE Codefactor issue per run and open ONE app-signed PR. Never use GITHUB_TOKEN for the commit or PR — authenticate as the Devsy GitHub App. - review_steps: |- - STEP 1 — Fetch ALL Codefactor issues (deterministic, paginated, no auth): - curl -s "https://www.codefactor.io/repository/github/devsy-org/devsy/issues?page=1" -o /tmp/cf1.html - python3 - <<'PY' - import json, urllib.request - def parse(path): - h=open(path,encoding='utf-8').read() - s=h.find("app.value('model',") - js=h[s+len("app.value('model',"):].lstrip() - return json.JSONDecoder().raw_decode(js)[0] - d=parse('/tmp/cf1.html'); pages=d['Filter'].get('PageCount',1) - issues=list(d['Issues']['List']) - for p in range(2, pages+1): - urllib.request.urlretrieve(f"https://www.codefactor.io/repository/github/devsy-org/devsy/issues?page={p}", f"/tmp/cf{p}.html") - issues += parse(f"/tmp/cf{p}.html")['Issues']['List'] - seen=set(); uniq=[] - for it in issues: - if it['Key'] not in seen: seen.add(it['Key']); uniq.append(it) - print("TOTAL", len(uniq)) - for it in uniq: - loc=it.get('Locations',[{}])[0] - print(f"{it['Key']} | {it.get('FilePath')} L{loc.get('StartLine')}-{loc.get('EndLine')} | {it.get('Category')} {it.get('Severity')} | {it.get('Name')}") - PY + steps: + - kind: install + title: "Check for and install gh + act if missing (no Go needed; CI work is YAML + gh + act)" + - body: |- + Authenticate gh read-only with the Devsy GitHub App token (do NOT use GITHUB_TOKEN for writes): + export GH_TOKEN=$(task github:app:sign-commit -- -token) + gh auth status >/dev/null 2>&1 || gh auth login --with-token <<< "$GH_TOKEN" + + - body: |- + Survey recent CI runs and the workflow files: + - gh run list --repo devsy-org/devsy --limit 20 + - For any failed or notably slow run: gh run view --repo devsy-org/devsy --log | tail -200 + - List the workflow files: ls .github/workflows/*.yml + - Skip improvements already addressed by an open PR (listed in STEP 0). + + - body: |- + Identify ONE small, concise improvement meeting ALL of: + - Self-contained: touches 1 workflow file (at most 2), minimal lines. + - Low risk: no behavior change beyond the CI fix; no new dependencies. + - Reviewable in <15 min. + - Real: a flaky step, a cacheable dependency, a redundant job, a misconfigured matrix, + a stale action version, a missing timeout-minutes, or a wrong path/trigger. Not a + cosmetic reformat. Do NOT reformat unrelated lines. + Pick exactly ONE. - STEP 2 — Pick ONE issue (any category: Maintainability, Duplication, Complexity). Prefer the most contained change reviewable in ~20 minutes. Pick exactly ONE. (Open PRs were already listed in STEP 0 — do not pick an issue addressed by one of them.) + - body: |- + Make the minimal edit to the workflow YAML. Preserve surrounding formatting/indentation. - STEP 3 — Fix it: read FilePath around StartLine..EndLine, make the minimal correct change. Do NOT disable linters or add workaround comments. If your change touched pkg/git, ensure you didn't introduce NEW failures. - pr_body_must_include: the Codefactor issue Key, file:line, the Codefactor message, the fix summary, and that task cli:lint:ci passed (0 new issues) + task cli:test status. Add label "codefactor" if it exists. - constraints: ONE issue, ONE commit, ONE PR per run (plus at most ONE follow-up fix commit if a status check fails — see the ensure-status-checks step). Keep it reviewable in ~20 minutes. If no fixable issue today, do nothing and report "no actionable issue found". Never use GITHUB_TOKEN for commit/PR. - description: Fixes one Codefactor issue per run by making a minimal, lint-clean change and opening an app-signed PR. + - body: |- + Validate (REQUIRED before committing): + - Re-read the edited file end-to-end to confirm correctness. + - python3 -c "import yaml,sys; yaml.safe_load(open(''))" && echo "YAML OK" + - If act + docker are available: act --list -W (best-effort; ignore if docker missing). + - If validation reveals your edit is wrong, fix it or pick a different improvement. + - kind: commit + - kind: pr +- id: codefactor + automation_name: "Codefactor Daily Issue Fixer" + branch_prefix: "fix/codefactor" + commit_subject: "\"fix: \" \"\"" + pr_title: "fix: " + job_name: "Codefactor issue fix job" + toolchain: "go" + intro: "You are the Codefactor issue fixer for the devsy-org/devsy repository. The repo is cloned in your workspace on branch main. Follow AGENTS.md conventions (lowercase logs, Conventional Commits, 50-char subject max). Fix exactly ONE Codefactor issue per run and open ONE app-signed PR. Never use GITHUB_TOKEN for the commit or PR — authenticate as the Devsy GitHub App." + pr_body_must_include: "the Codefactor issue Key, file:line, the Codefactor message, the fix summary, and that task cli:lint:ci passed (0 new issues) + task cli:test status. Add label \"codefactor\" if it exists." + constraints: "ONE issue, ONE commit, ONE PR per run (plus at most ONE follow-up fix commit if a status check fails — see the ensure-status-checks step). Keep it reviewable in ~20 minutes. If no fixable issue today, do nothing and report \"no actionable issue found\". Never use GITHUB_TOKEN for commit/PR." + description: "Fixes one Codefactor issue per run by making a minimal, lint-clean change and opening an app-signed PR." scope: |- - Codefactor issues for `devsy-org/devsy` (https://www.codefactor.io/repository/github/devsy-org/devsy) - Any Go source file flagged by Codefactor (Maintability, Duplication, Complexity) @@ -385,38 +402,52 @@ agents: via the automation service — not the git repo — using the service's agentic memory if available. Propose a `description` amendment to this `agent.md` for human review when issue-selection priorities change. -- id: ui-polish - automation_name: Daily Desktop UI Polish - branch_prefix: ui-polish - commit_subject: '"fix(ui): " ""' - pr_title: 'fix(ui): ' - job_name: UI polish job - toolchain: go+node - step0_title: Check for and install the Go + Node toolchain if missing - commit_step_number: 6 - pr_step_number: 7 - intro: You are a senior frontend engineer doing a daily UI polish pass on the devsy desktop app (an Electron + Svelte + TypeScript app). The repo is cloned in your workspace on branch main. Follow AGENTS.md conventions (lowercase logs, Conventional Commits, 50-char subject max). Find and make ONE small, self-contained UI improvement per run and open ONE app-signed PR. Never use GITHUB_TOKEN for the commit or PR — authenticate as the Devsy GitHub App. - review_steps: |- - STEP 1 — Survey the UI. The UI lives in desktop/src/renderer/src/ (App.svelte, app.css, main.ts, pages/, lib/components/, lib/hooks/, lib/stores/, lib/utils/, lib/types/). Do NOT analyze or modify unrelated Go code, the main process (desktop/src/main), or backend packages. - - STEP 2 — Identify a shortlist of candidate issues, each meeting ALL constraints: - - Self-contained: touches 1-3 files, ideally one component or one CSS rule. - - Low risk: no routing/store/architecture changes, no new dependencies, no behavioral changes to data flows or IPC. - - Reviewable in <15 min: a reviewer can read the diff and verify it quickly. - - Real: an actual accessibility, correctness, or small UX problem — not a style nitpick or invented refactor. Prefer: a11y issues (missing aria-labels, non-semantic buttons, focus handling), visible text/contrast/copy issues, broken/missing keyboard handlers, an obvious layout glitch in CSS, a small missing loading/empty/error state, or a dangling/hardcoded string. - - STEP 3 — Pick exactly ONE candidate — the best balance of user impact and reviewability. Make the minimal edit needed. Keep the diff small and idiomatic to existing style. Never touch package.json, package-lock.json, or any dependency manifest. + steps: + - kind: install + title: "Check for and install the Go toolchain if missing (Go, task, golangci-lint)" + - body: |- + Fetch ALL Codefactor issues (deterministic, paginated, no auth): + curl -s "https://www.codefactor.io/repository/github/devsy-org/devsy/issues?page=1" -o /tmp/cf1.html + python3 - <<'PY' + import json, urllib.request + def parse(path): + h=open(path,encoding='utf-8').read() + s=h.find("app.value('model',") + js=h[s+len("app.value('model',"):].lstrip() + return json.JSONDecoder().raw_decode(js)[0] + d=parse('/tmp/cf1.html'); pages=d['Filter'].get('PageCount',1) + issues=list(d['Issues']['List']) + for p in range(2, pages+1): + urllib.request.urlretrieve(f"https://www.codefactor.io/repository/github/devsy-org/devsy/issues?page={p}", f"/tmp/cf{p}.html") + issues += parse(f"/tmp/cf{p}.html")['Issues']['List'] + seen=set(); uniq=[] + for it in issues: + if it['Key'] not in seen: seen.add(it['Key']); uniq.append(it) + print("TOTAL", len(uniq)) + for it in uniq: + loc=it.get('Locations',[{}])[0] + print(f"{it['Key']} | {it.get('FilePath')} L{loc.get('StartLine')}-{loc.get('EndLine')} | {it.get('Category')} {it.get('Severity')} | {it.get('Name')}") + PY - STEP 4 — Verify (REQUIRED before committing): - - cd desktop && npm install --ignore-scripts - - cd desktop && npm run check # svelte-check / type check - - cd desktop && npx vitest run # unit tests - If a check fails because of your change, fix it. If a check fails for a pre-existing/unrelated reason, note it in the PR body and proceed. + - body: |- + Pick ONE issue (any category: Maintainability, Duplication, Complexity). Prefer the most contained change reviewable in ~20 minutes. Pick exactly ONE. (Open PRs were already listed in STEP 0 — do not pick an issue addressed by one of them.) - STEP 5 — Write the PR body to /tmp/pr_body.md. It MUST contain: Problem (1-3 sentences, with the file/component), Change (what you changed, minimal), Why it's safe / reviewable in <15 min, Verification (the exact checks you ran and their results). - pr_body_must_include: Problem, Change, Why it's safe/reviewable in <15 min, Verification (the exact checks run and their results) - constraints: Make exactly ONE improvement per run. Do not bundle multiple fixes. Never change logic outside the UI layer, and never touch package.json, package-lock.json, or any dependency manifest. Use a Conventional Commit message, type "fix" or "style". If no qualifying issue, do NOT force a change — report it instead. Never use GITHUB_TOKEN for the commit or PR. - description: Makes one small, self-contained UI polish improvement to the desktop app per run and opens an app-signed PR. + - body: |- + Fix it: read FilePath around StartLine..EndLine, make the minimal correct change. Do NOT disable linters or add workaround comments. If your change touched pkg/git, ensure you didn't introduce NEW failures. + - kind: verify + - kind: commit + - kind: pr +- id: ui-polish + automation_name: "Daily Desktop UI Polish" + branch_prefix: "ui-polish" + commit_subject: "\"fix(ui): \" \"\"" + pr_title: "fix(ui): " + job_name: "UI polish job" + toolchain: "go+node" + intro: "You are a senior frontend engineer doing a daily UI polish pass on the devsy desktop app (an Electron + Svelte + TypeScript app). The repo is cloned in your workspace on branch main. Follow AGENTS.md conventions (lowercase logs, Conventional Commits, 50-char subject max). Find and make ONE small, self-contained UI improvement per run and open ONE app-signed PR. Never use GITHUB_TOKEN for the commit or PR — authenticate as the Devsy GitHub App." + pr_body_must_include: "Problem, Change, Why it's safe/reviewable in <15 min, Verification (the exact checks run and their results)" + constraints: "Make exactly ONE improvement per run. Do not bundle multiple fixes. Never change logic outside the UI layer, and never touch package.json, package-lock.json, or any dependency manifest. Use a Conventional Commit message, type \"fix\" or \"style\". If no qualifying issue, do NOT force a change — report it instead. Never use GITHUB_TOKEN for the commit or PR." + description: "Makes one small, self-contained UI polish improvement to the desktop app per run and opens an app-signed PR." scope: |- - `desktop/src/renderer/src/` — App.svelte, app.css, pages/, lib/components/, lib/hooks/, lib/stores/, lib/utils/, lib/types/ @@ -427,54 +458,44 @@ agents: fragility, svelte-check gotchas) to the running automation via the automation service — not the git repo — using the service's agentic memory if available. Propose a `description` amendment to this `agent.md` for human review when polish priorities change. -- id: agent-analytics - automation_name: Daily Agent Analytics - branch_prefix: agent-analytics - commit_subject: '"" ""' - pr_title: - job_name: agent analytics job - toolchain: go+python - step0_title: Check for and install the Go toolchain + uv if missing - commit_step_number: 6 - pr_step_number: 7 - intro: You are the analytics engineer for the devsy-org/devsy agent fleet. The repo is cloned in your workspace on branch main. Follow AGENTS.md conventions (lowercase logs, Conventional Commits, 50-char subject max). Review the previous day's agent runs, reduce the errors and failures agents encounter by applying deterministic methods that lower cognitive load, and open ONE app-signed PR. Never use GITHUB_TOKEN for the commit or PR — authenticate as the Devsy GitHub App. - review_steps: |- - STEP 1 — Gather yesterday's run data deterministically. The repo ships hack/analytics/analyze_runs.py, a self-contained uv script (PEP 723 inline deps: typer, numpy, scikit-learn, pandas, seaborn, matplotlib, prettytable). Run it ONLY with uv run (never pip/venv) and yesterday's date window: - uv run hack/analytics/analyze_runs.py --since yesterday --until today \ - --out-dir dist/analytics - It authenticates with OPENHANDS_API_KEY (read-only), lists app-conversations created in the window, pulls each conversation's events, and writes: - - dist/analytics/runs.csv (one row per run: agent, status, duration, event/tool counts, error count) - - dist/analytics/failure_clusters.csv (sklearn KMeans clusters of failure signatures) - - dist/analytics/top_errors.csv (ranked recurring error signatures) - - dist/analytics/daily_report.md (human-readable summary with recommendations) - - dist/analytics/failure_heatmap.png (seaborn visualization of failure mode x agent) - If OPENHANDS_API_KEY is unavailable OR no runs exist for the window, fall back to the bundled synthetic sample so the pipeline still produces outputs: add `--sample`. Treat the API as the source of truth when available; the sample is only for offline development. - The analyzer also zips the five outputs and uploads them to https://temp.sh/upload (3-day expiry, returns a plain-text share URL), then writes a `## pr-gate` verdict at the end of daily_report.md: ACTIONABLE (top failure signature recurs across >=2 runs at >=10% share of failed runs) or NOT-ACTIONABLE. Set ANALYTICS_NO_UPLOAD=1 to skip the upload (e.g. in an offline sandbox). If temp.sh is unreachable or rejects the upload the upload is recorded as skipped with the reason; the pipeline still completes. + steps: + - kind: install + title: "Check for and install the Go + Node toolchain if missing" + - body: |- + Survey the UI. The UI lives in desktop/src/renderer/src/ (App.svelte, app.css, main.ts, pages/, lib/components/, lib/hooks/, lib/stores/, lib/utils/, lib/types/). Do NOT analyze or modify unrelated Go code, the main process (desktop/src/main), or backend packages. - STEP 2 — Analyze the daily_report.md and top_errors.csv. Identify ONE high-impact, deterministic intervention that reduces the cognitive load agents spend recovering from errors, so reasoning budget stays on the primary task. Prefer, in order of reviewability: - - A script/automation fix in hack/analytics/ or .agents/ that codifies a recurring recovery (e.g. a missing-tool check, a PR-duplicate guard, a deterministic pre-flight). - - A prompt (agent.md) clarification that removes an ambiguity the data shows agents repeatedly stumble on (stale assertion, wrong resolution order, missing masking rule). Prompt edits must be regenerated through hack/automations/agents.yaml — never hand-edit a generated agent.md. - - A small code fix in pkg/ or cmd/ that eliminates a recurring failure mode the data pinpoints. - Pick exactly ONE. The intervention must be justified by a number from the analytics output (cluster size, error count, recurrence rate). Quote the metric in the PR body. + - body: |- + Identify a shortlist of candidate issues, each meeting ALL constraints: + - Self-contained: touches 1-3 files, ideally one component or one CSS rule. + - Low risk: no routing/store/architecture changes, no new dependencies, no behavioral changes to data flows or IPC. + - Reviewable in <15 min: a reviewer can read the diff and verify it quickly. + - Real: an actual accessibility, correctness, or small UX problem — not a style nitpick or invented refactor. Prefer: a11y issues (missing aria-labels, non-semantic buttons, focus handling), visible text/contrast/copy issues, broken/missing keyboard handlers, an obvious layout glitch in CSS, a small missing loading/empty/error state, or a dangling/hardcoded string. - STEP 3 — Implement the ONE intervention. Keep it minimal and reviewable in ~20 minutes. If it touches a generated agent prompt, edit hack/automations/agents.yaml and regenerate: `go run ./hack/automations` then confirm `go run ./hack/automations -check` is clean. + - body: |- + Pick exactly ONE candidate — the best balance of user impact and reviewability. Make the minimal edit needed. Keep the diff small and idiomatic to existing style. Never touch package.json, package-lock.json, or any dependency manifest. - STEP 4 — Verify (CRITICAL — use CI-equivalent lint): - - mkdir -p dist - - git fetch --quiet origin main - - task cli:lint:ci # Must be 0 new issues. - - task cli:test - - uv run hack/analytics/analyze_runs.py --sample --out-dir /tmp/analytics-verify # smoke test the pipeline (sets ANALYTICS_NO_UPLOAD=1 to skip the live upload in CI) - - uvx --from mypy mypy --strict --ignore-missing-imports --disable-error-code untyped-decorator hack/analytics/analyze_runs.py # static typing (must be 0 issues) - - uvx --from radon radon cc -s hack/analytics/analyze_runs.py | grep -oE '\([0-9]+\)' | tr -d '()' | sort -n -r | head -1 # cyclomatic complexity; must be < 10 - KNOWN PRE-EXISTING FAILURE: pkg/git tests (TestRepoClone*) fail on origin/main already — a stale assertion, NOT caused by your change. If ONLY pkg/git fails and your change did not touch pkg/git, proceed. If your change introduces any NEW lint issue or test failure, fix the root cause or pick a different intervention. + - body: |- + Verify (REQUIRED before committing): + - cd desktop && npm install --ignore-scripts + - cd desktop && npm run check # svelte-check / type check + - cd desktop && npx vitest run # unit tests + If a check fails because of your change, fix it. If a check fails for a pre-existing/unrelated reason, note it in the PR body and proceed. - STEP 5 — PR-GATE DECISION (read the `## pr-gate` verdict in daily_report.md): - - If the verdict is NOT-ACTIONABLE, DO NOT create a PR. Report "no actionable failure mode found — pr-gate NOT-ACTIONABLE" and stop. A run that opens a PR against a NOT-ACTIONABLE verdict is a FAILED run. - - If the verdict is ACTIONABLE, write the PR body to /tmp/pr_body.md. It MUST contain: the analytics window (since/until dates), the pr-gate verdict, the headline metric(s) from daily_report.md that motivated the change (e.g. "cluster_0: 7 runs, signature: missing-tool 'golangci-lint'"), the ONE intervention, the verification performed (lint:ci, cli:test, uv smoke run, mypy, radon complexity), the temp.sh upload link (or the skip reason if temp.sh was unreachable/rejected), and attach/reference the visualization (failure_heatmap.png path or PR-attached image). State the expected reduction in failure recurrence. - pr_body_must_include: the analytics window, the pr-gate verdict, the headline metric(s) from daily_report.md, the ONE intervention, the verification performed (lint:ci, cli:test, uv smoke run, mypy, radon complexity), the temp.sh upload link or skip reason, the visualization reference - constraints: ONE intervention per run, justified by a metric from the analytics output. Do not invent metrics. Run the script only via `uv run` (never pip/venv). If OPENHANDS_API_KEY is unavailable, use --sample and say so. ONLY create a PR when the pr-gate verdict in daily_report.md is ACTIONABLE; if NOT-ACTIONABLE, do nothing and report "no actionable failure mode found — pr-gate NOT-ACTIONABLE". Never use GITHUB_TOKEN for the commit or PR. - description: Reviews the previous day's agent-fleet runs with Python data-science tooling, clusters failure modes, and ships one deterministic intervention (script, prompt, or code fix) that lowers agent cognitive load and reduces recurring errors. + - body: |- + Write the PR body to /tmp/pr_body.md. It MUST contain: Problem (1-3 sentences, with the file/component), Change (what you changed, minimal), Why it's safe / reviewable in <15 min, Verification (the exact checks you ran and their results). + - kind: commit + - kind: pr +- id: agent-analytics + automation_name: "Daily Agent Analytics" + branch_prefix: "agent-analytics" + commit_subject: "\"\" \"\"" + pr_title: "" + job_name: "agent analytics job" + toolchain: "go+python" + intro: "You are the analytics engineer for the devsy-org/devsy agent fleet. The repo is cloned in your workspace on branch main. Follow AGENTS.md conventions (lowercase logs, Conventional Commits, 50-char subject max). Review the previous day's agent runs, reduce the errors and failures agents encounter by applying deterministic methods that lower cognitive load, and open ONE app-signed PR. Never use GITHUB_TOKEN for the commit or PR — authenticate as the Devsy GitHub App." + pr_body_must_include: "the analytics window, the pr-gate verdict, the headline metric(s) from daily_report.md, the ONE intervention, the verification performed (lint:ci, cli:test, uv smoke run, mypy, radon complexity), the temp.sh upload link or skip reason, the visualization reference" + constraints: "ONE intervention per run, justified by a metric from the analytics output. Do not invent metrics. Run the script only via `uv run` (never pip/venv). If OPENHANDS_API_KEY is unavailable, use --sample and say so. ONLY create a PR when the pr-gate verdict in daily_report.md is ACTIONABLE; if NOT-ACTIONABLE, do nothing and report \"no actionable failure mode found — pr-gate NOT-ACTIONABLE\". Never use GITHUB_TOKEN for the commit or PR." + description: "Reviews the previous day's agent-fleet runs with Python data-science tooling, clusters failure modes, and ships one deterministic intervention (script, prompt, or code fix) that lowers agent cognitive load and reduces recurring errors." scope: |- - `hack/analytics/` — the deterministic run-analysis pipeline (analyze_runs.py + outputs) - `.agents/agents/`, `.agents/tasks/`, `hack/automations/agents.yaml` — agent prompts/schedules @@ -486,3 +507,46 @@ agents: automation service — not the git repo — using the service's agentic memory if available. Propose a `description` amendment to this `agent.md` for human review when the analysis method or intervention priorities change. + steps: + - kind: install + title: "Check for and install the Go toolchain + uv if missing" + - body: |- + Gather yesterday's run data deterministically. The repo ships hack/analytics/analyze_runs.py, a self-contained uv script (PEP 723 inline deps: typer, numpy, scikit-learn, pandas, seaborn, matplotlib, prettytable). Run it ONLY with uv run (never pip/venv) and yesterday's date window: + uv run hack/analytics/analyze_runs.py --since yesterday --until today \ + --out-dir dist/analytics + It authenticates with OPENHANDS_API_KEY (read-only), lists app-conversations created in the window, pulls each conversation's events, and writes: + - dist/analytics/runs.csv (one row per run: agent, status, duration, event/tool counts, error count) + - dist/analytics/failure_clusters.csv (sklearn KMeans clusters of failure signatures) + - dist/analytics/top_errors.csv (ranked recurring error signatures) + - dist/analytics/daily_report.md (human-readable summary with recommendations) + - dist/analytics/failure_heatmap.png (seaborn visualization of failure mode x agent) + If OPENHANDS_API_KEY is unavailable OR no runs exist for the window, fall back to the bundled synthetic sample so the pipeline still produces outputs: add `--sample`. Treat the API as the source of truth when available; the sample is only for offline development. + The analyzer also zips the five outputs and uploads them to https://temp.sh/upload (3-day expiry, returns a plain-text share URL), then writes a `## pr-gate` verdict at the end of daily_report.md: ACTIONABLE (top failure signature recurs across >=2 runs at >=10% share of failed runs) or NOT-ACTIONABLE. Set ANALYTICS_NO_UPLOAD=1 to skip the upload (e.g. in an offline sandbox). If temp.sh is unreachable or rejects the upload the upload is recorded as skipped with the reason; the pipeline still completes. + + - body: |- + Analyze the daily_report.md and top_errors.csv. Identify ONE high-impact, deterministic intervention that reduces the cognitive load agents spend recovering from errors, so reasoning budget stays on the primary task. Prefer, in order of reviewability: + - A script/automation fix in hack/analytics/ or .agents/ that codifies a recurring recovery (e.g. a missing-tool check, a PR-duplicate guard, a deterministic pre-flight). + - A prompt (agent.md) clarification that removes an ambiguity the data shows agents repeatedly stumble on (stale assertion, wrong resolution order, missing masking rule). Prompt edits must be regenerated through hack/automations/agents.yaml — never hand-edit a generated agent.md. + - A small code fix in pkg/ or cmd/ that eliminates a recurring failure mode the data pinpoints. + Pick exactly ONE. The intervention must be justified by a number from the analytics output (cluster size, error count, recurrence rate). Quote the metric in the PR body. + + - body: |- + Implement the ONE intervention. Keep it minimal and reviewable in ~20 minutes. If it touches a generated agent prompt, edit hack/automations/agents.yaml and regenerate: `go run ./hack/automations` then confirm `go run ./hack/automations -check` is clean. + + - body: |- + Verify (CRITICAL — use CI-equivalent lint): + - mkdir -p dist + - git fetch --quiet origin main + - task cli:lint:ci # Must be 0 new issues. + - task cli:test + - uv run hack/analytics/analyze_runs.py --sample --out-dir /tmp/analytics-verify # smoke test the pipeline (sets ANALYTICS_NO_UPLOAD=1 to skip the live upload in CI) + - uvx --from mypy mypy --strict --ignore-missing-imports --disable-error-code untyped-decorator hack/analytics/analyze_runs.py # static typing (must be 0 issues) + - uvx --from radon radon cc -s hack/analytics/analyze_runs.py | grep -oE '\([0-9]+\)' | tr -d '()' | sort -n -r | head -1 # cyclomatic complexity; must be < 10 + KNOWN PRE-EXISTING FAILURE: pkg/git tests (TestRepoClone*) fail on origin/main already — a stale assertion, NOT caused by your change. If ONLY pkg/git fails and your change did not touch pkg/git, proceed. If your change introduces any NEW lint issue or test failure, fix the root cause or pick a different intervention. + + - body: |- + PR-GATE DECISION (read the `## pr-gate` verdict in daily_report.md): + - If the verdict is NOT-ACTIONABLE, DO NOT create a PR. Report "no actionable failure mode found — pr-gate NOT-ACTIONABLE" and stop. A run that opens a PR against a NOT-ACTIONABLE verdict is a FAILED run. + - If the verdict is ACTIONABLE, write the PR body to /tmp/pr_body.md. It MUST contain: the analytics window (since/until dates), the pr-gate verdict, the headline metric(s) from daily_report.md that motivated the change (e.g. "cluster_0: 7 runs, signature: missing-tool 'golangci-lint'"), the ONE intervention, the verification performed (lint:ci, cli:test, uv smoke run, mypy, radon complexity), the temp.sh upload link (or the skip reason if temp.sh was unreachable/rejected), and attach/reference the visualization (failure_heatmap.png path or PR-attached image). State the expected reduction in failure recurrence. + - kind: commit + - kind: pr diff --git a/hack/automations/main.go b/hack/automations/main.go index c3b46ac3c7..4c471953dd 100644 --- a/hack/automations/main.go +++ b/hack/automations/main.go @@ -16,6 +16,17 @@ import ( //go:embed template.md.tmpl var templateText string +// step kinds. kindInstall/kindCommit/kindPR are required per agent; kindVerify +// selects the scaffolded verify block; kindNote is an unnumbered preamble; an +// empty kind is a plain (verbatim) numbered step. +const ( + kindInstall = "install" + kindVerify = "verify" + kindCommit = "commit" + kindPR = "pr" + kindNote = "note" +) + const ( goVersion = "1.26.3" golangciVersion = "2.12.2" @@ -38,29 +49,39 @@ type agent struct { PRTitle string `yaml:"pr_title"` JobName string `yaml:"job_name"` Toolchain string `yaml:"toolchain"` - Step0Title string `yaml:"step0_title"` - CommitStepNumber int `yaml:"commit_step_number"` - PRStepNumber int `yaml:"pr_step_number"` Intro string `yaml:"intro"` - ReviewSteps string `yaml:"review_steps"` - VerifyType string `yaml:"verify_type"` - VerifyExtras string `yaml:"verify_extras"` PRBodyMustInclude string `yaml:"pr_body_must_include"` Constraints string `yaml:"constraints"` Description string `yaml:"description"` Scope string `yaml:"scope"` SelfImprovement string `yaml:"self_improvement"` + Steps []step `yaml:"steps"` +} + +// step is one numbered entry in an agent's prompt. Kind selects a scaffolded +// body generated from shared logic; a plain step (empty Kind) renders its Title +// and optional Body verbatim. The step number is the entry's position in Steps, +// so reordering or inserting a step never requires manual renumbering. +type step struct { + Kind string `yaml:"kind"` // install, verify, commit, pr, or "" (plain) + Title string `yaml:"title"` // plain-step title (full first line, incl. trailing punctuation) + Body string `yaml:"body"` // plain-step body (lines after the title; optional) + VerifyExtras string `yaml:"verify_extras"` // kind=verify: extra verify bullets } type config struct { Agents []agent `yaml:"agents"` } +// renderView carries the pre-rendered, auto-numbered step blocks plus the agent +// fields the template still references directly. Numbering is baked into each +// block string, so the template only arranges the scaffold around them. type renderView struct { Agent agent - Step0Body string - VerifyBlock string - PRTitleJSON string + InstallStep string + ReviewSteps string + CommitStep string + PRStep string } func main() { @@ -133,14 +154,28 @@ func validateAgent(a agent) { if a.SelfImprovement == "" { fail("agent %s missing self_improvement", a.ID) } + if len(a.Steps) == 0 { + fail("agent %s has no steps", a.ID) + } + kinds := map[string]bool{} + for _, s := range a.Steps { + kinds[s.Kind] = true + } + for _, required := range []string{kindInstall, kindCommit, kindPR} { + if !kinds[required] { + fail("agent %s steps missing a %q step", a.ID, required) + } + } } func renderAgent(tmpl *template.Template, a agent) string { + blocks := renderSteps(a) view := renderView{ Agent: a, - Step0Body: installBlock(a.Toolchain), - VerifyBlock: verifyBlock(a), - PRTitleJSON: escapeJSONString(a.PRTitle), + InstallStep: blocks.install, + ReviewSteps: blocks.review, + CommitStep: blocks.commit, + PRStep: blocks.pr, } var buf bytes.Buffer if err := tmpl.Execute(&buf, view); err != nil { @@ -212,25 +247,158 @@ const knownGitFailure = "KNOWN PRE-EXISTING FAILURE: pkg/git tests (TestRepoClon "If your change introduces any NEW lint issue or test failure, " + "fix the root cause or pick a different improvement." -func verifyBlock(a agent) string { - if a.VerifyType == "" { - return "" +// renderSteps turns an agent's ordered Steps into the pre-numbered block strings +// the template arranges. The step number is assigned in list order but only to +// numbered steps — a kind=note step (a verbatim preamble) renders as-is and does +// not consume a number, so it never throws off the commit/pr numbering. +// install/commit/pr are scaffolded (generated bodies); every other numbered step +// renders verbatim from its Body (this includes kind=verify, whose body comes from +// verifyBody, and plain review/verify-inline steps authored in YAML). +func renderSteps(a agent) stepBlocks { + idx := stepKindIndices(a.Steps) + numbered := numberedStepMap(a.Steps) + + blocks := stepBlocks{ + install: renderStep(a.Steps[idx.install], numbered[idx.install], a), + commit: renderStep(a.Steps[idx.commit], numbered[idx.commit], a), + pr: renderStep(a.Steps[idx.pr], numbered[idx.pr], a), + } + blocks.review = joinReviewSteps(a, idx, numbered) + return blocks +} + +// stepIndices holds the list positions of the install/commit/pr steps. +type stepIndices struct { + install int + commit int + pr int +} + +// stepKindIndices returns the list positions of the install/commit/pr steps. +func stepKindIndices(steps []step) stepIndices { + idx := stepIndices{commit: -1, pr: -1} + for i, s := range steps { + switch s.Kind { + case kindInstall: + idx.install = i + case kindCommit: + idx.commit = i + case kindPR: + idx.pr = i + } } - stepNum := a.CommitStepNumber - 1 + return idx +} + +// numberedStepMap assigns sequential STEP numbers to every non-note step. Notes +// render verbatim and do not consume a number, so they never shift commit/pr. +func numberedStepMap(steps []step) map[int]int { + numbered := make(map[int]int) + num := -1 + for i, s := range steps { + if s.Kind == kindNote { + continue + } + num++ + numbered[i] = num + } + return numbered +} + +func joinReviewSteps(a agent, idx stepIndices, numbered map[int]int) string { + skip := map[int]bool{idx.install: true, idx.commit: true, idx.pr: true} + var parts []string + for i, s := range a.Steps { + if skip[i] { + continue + } + parts = append(parts, renderStep(s, numbered[i], a)) + } + return strings.Join(parts, "\n\n") +} + +// stepBlocks holds the four pre-rendered, auto-numbered step regions the template +// arranges around its static scaffold. +type stepBlocks struct { + install string + review string + commit string + pr string +} + +// renderStep emits one block. Scaffolded kinds generate a fixed title and body; +// kind=note renders its Body verbatim (no "STEP N — " prefix); a plain step +// renders "STEP N — " followed by its Body verbatim. +func renderStep(s step, num int, a agent) string { + switch s.Kind { + case kindInstall: + return fmt.Sprintf("STEP %d — %s:\n set -e\n%s", num, s.Title, installBlock(a.Toolchain)) + case kindVerify: + return fmt.Sprintf( + "STEP %d — Verify (CRITICAL — use CI-equivalent lint):\n%s", + num, + verifyBody(s), + ) + case kindCommit: + return fmt.Sprintf( + "STEP %d — Commit via the Devsy GitHub App (signed, verified).\n%s", + num, + commitBody(a), + ) + case kindPR: + return fmt.Sprintf( + "STEP %d — Open the PR as the app (no GITHUB_TOKEN):\n%s", + num, + prBody(a), + ) + case kindNote: + return s.Body + default: + return fmt.Sprintf("STEP %d — %s", num, s.Body) + } +} + +func verifyBody(s step) string { lines := []string{ - fmt.Sprintf("STEP %d — Verify (CRITICAL — use CI-equivalent lint):", stepNum), " - mkdir -p dist", " - git fetch --quiet origin main", " - task cli:lint:ci # Must be 0 new issues.", " - task cli:test", } - if a.VerifyExtras != "" { - lines = append(lines, " - "+a.VerifyExtras) + if s.VerifyExtras != "" { + lines = append(lines, " - "+s.VerifyExtras) } lines = append(lines, " "+knownGitFailure) return joinLines(lines...) } +// commitBody is the shared commit-instruction body (the part after the STEP line). +func commitBody(a agent) string { + return joinLines( + "The repo ships a Go tool (hack/sign_commit) that authenticates as the app installation", + "and creates the commit through GitHub's GraphQL API, so GitHub signs it (committer:", + "web-flow, verified). It auto-detects all working-tree changes (staged, unstaged, and", + "untracked files) and auto-creates the remote branch from origin/main if needed.", + "Do NOT run `git commit` locally — it produces an unsigned commit that fails the signature", + "check.", + " - git fetch --quiet origin main", + " - git checkout -b "+a.BranchPrefix+"/ origin/main", + " - task github:app:sign-commit -- -m "+a.CommitSubject, + " - Confirm output: verified=true.", + ) +} + +// prBody is the shared PR-instruction body (the part after the STEP line). +func prBody(a agent) string { + return joinLines( + " - Write the PR body to /tmp/pr_body.md. It MUST include: "+a.PRBodyMustInclude+ + ", a line \"This PR was created by an AI agent as part of an automated daily "+a.JobName+".\"", + " - task github:app:sign-commit -- -pr-only -title "+escapeJSONString(a.PRTitle)+ + " -b \"$(cat /tmp/pr_body.md)\"", + " - Report the PR URL from the output. A run that does not produce a PR URL is a FAILED run.", + ) +} + func goInstallBlock() string { golangciExtract := "tar -C /tmp -xzf /tmp/gcl.tgz && sudo mv /tmp/golangci-lint-" + golangciVersion + "-linux-amd64/golangci-lint /usr/local/bin/ && " + diff --git a/hack/automations/template.md.tmpl b/hack/automations/template.md.tmpl index 39ab05dadc..bd2d34d647 100644 --- a/hack/automations/template.md.tmpl +++ b/hack/automations/template.md.tmpl @@ -13,9 +13,7 @@ enabled: true Runtime secrets: DEVSY_GITHUB_APP_PRIVATE_KEY, DEVSY_GITHUB_APP_COMMIT_USER. App id: use ${DEVSY_GITHUB_APP_ID:-}. -STEP 0 — {{.Agent.Step0Title}}: - set -e -{{.Step0Body}} +{{.InstallStep}} Before selecting your task, check for open PRs to avoid creating a duplicate. Run: TOKEN=$(task github:app:sign-commit -- -token) @@ -25,10 +23,8 @@ work on, skip it and pick a different one. Do not open a PR for a fix that is al flight. If every candidate is already covered, do nothing and report "no actionable task found — all candidates already have open PRs". -{{.Agent.ReviewSteps}} -{{if .VerifyBlock}} -{{.VerifyBlock}} -{{end}} +{{.ReviewSteps}} + FORMATTING GATE (CRITICAL — run BEFORE committing, must pass): Write self-documenting code: clear names, small functions, obvious structure. Avoid wordy comments — prefer no comments unless the code expresses something @@ -42,22 +38,9 @@ FORMATTING GATE (CRITICAL — run BEFORE committing, must pass): Common formatting failures: gci (import ordering), gofumpt (struct/function spacing), golines (line length > 120). Running task cli:format first auto-fixes most of these. -STEP {{.Agent.CommitStepNumber}} — Commit via the Devsy GitHub App (signed, verified). -The repo ships a Go tool (hack/sign_commit) that authenticates as the app installation -and creates the commit through GitHub's GraphQL API, so GitHub signs it (committer: -web-flow, verified). It auto-detects all working-tree changes (staged, unstaged, and -untracked files) and auto-creates the remote branch from origin/main if needed. -Do NOT run `git commit` locally — it produces an unsigned commit that fails the signature -check. - - git fetch --quiet origin main - - git checkout -b {{.Agent.BranchPrefix}}/ origin/main - - task github:app:sign-commit -- -m {{.Agent.CommitSubject}} - - Confirm output: verified=true. +{{.CommitStep}} -STEP {{.Agent.PRStepNumber}} — Open the PR as the app (no GITHUB_TOKEN): - - Write the PR body to /tmp/pr_body.md. It MUST include: {{.Agent.PRBodyMustInclude}}, a line "This PR was created by an AI agent as part of an automated daily {{.Agent.JobName}}." - - task github:app:sign-commit -- -pr-only -title {{.PRTitleJSON}} -b "$(cat /tmp/pr_body.md)" - - Report the PR URL from the output. A run that does not produce a PR URL is a FAILED run. +{{.PRStep}} The PR is the final step. Do NOT wait for or poll CI status checks after opening the PR. The task is complete once the commit is app-signed (verified=true), the code is ready,