Skip to content

feat: review Alex's PR submissions with OpenCode - #1150

Open
alexsoyes wants to merge 39 commits into
nextfrom
feat/opencode-review-after-submit
Open

alexsoyes wants to merge 39 commits into
nextfrom
feat/opencode-review-after-submit

Conversation

@alexsoyes

@alexsoyes alexsoyes commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

🎯 What & why

Review a PR when Alex submits a review. Reuse checkpoints for unchanged files while answering new feedback, including when the commit has not changed.

🛠️ How it works

  • A trusted scheduled workflow reviews the submitted commit. The reviewer can inspect its Git diff and the PR discussion; a separate workflow token publishes the result.
  • npx installs the project's recorded AIDD CLI version and available plugins in an isolated directory. If the manifest is absent, it uses latest and reports that the plugin inventory is unavailable. Other plugin sources and version differences are reported.
  • Bot comments hold per-file fingerprints. An interrupted publication can require another model run, while completed submissions are deduplicated.
  • /opencode receives PR reviews, inline comments and the general discussion. It replies in the original thread when applicable.

🧪 How to verify

  • make check
  • bash scripts/__tests__/pr-review/test-review.sh and bash scripts/__tests__/pr-review/test-setup-project.sh
  • Build both images, then run bash scripts/__tests__/pr-review/test-runner.sh.
  • An OrbStack run used the real OpenCode model against a simulated PR and GitHub API; it identified a seeded authorization regression. Real npx setup runs with CLI 5.1.4 and 5.6.0 installed the requested skills directly. The final image smoke passed; its remote setup could not be repeated after the anonymous GitHub API quota was exhausted.

⚠️ Heads-up

  • The new schedule is inactive until the workflow is on main, the dedicated runner group exists and AIDD_REVIEW_ENABLED_AT is set. Stop the existing server timer during cutover to avoid duplicate reviews.
  • Central has a registered dedicated rootless runner with network isolation. Its group currently authorizes no workflows and its OAuth volume is empty. Live OAuth, representative concurrent load and GitHub publication remain unverified.
  • Binary and submodule changes are reported as incomplete; unavailable commits or failed model runs do not create a successful checkpoint.
  • The reviewer reports its own coverage; this is not an independent proof that every listed file was fully examined.

✅ I certify

  • I DO CERTIFY I READ EACH LINE OF THE PULL REQUEST BECAUSE I AM A SOFTWARE ENGINEER, NOT A AI PUPPY.

@alexsoyes

Copy link
Copy Markdown
Contributor Author

Test manuel de la revue OpenCode

Test lancé localement avec OpenCode 2.2.5, le provider OpenAI, gpt-6.1-terra#medium et la skill 05-review, sur le diff du commit 39dc9959393305b849dc3ca61c4f77432c7dfc05 par rapport à f0df90fc14599817ece2739ad56f0a71ba6a2b93. Le vérificateur du prototype a accepté le rapport ci-dessous. Ce commentaire est publié manuellement : le déclenchement GitHub automatique n’a pas encore été testé.

Note : la mention « non commitées » vient de proof.md, écrit avant le commit ; ces fichiers sont maintenant commités dans cette PR.


Revue : Prototype OpenCode reviews after submission

Round 1

  • Verdict : changes-requested — l’authentification du modèle n’est pas configurée pour le runner.
  • Périmètre : les deux fichiers de diff fournis et la description de la PR.
  • Axes : code et pertinence ; fonctionnel appliqué à l’objectif décrit dans la PR.
  • Modifications non commitées : indiquées dans proof.md ; non vérifiées indépendamment.

Critères

  • Sélectionner les fichiers nouveaux ou modifiés depuis la dernière revue enregistrée — scripts/opencode-review-state.mjs:52.
  • Demander une revue avec le skill et vérifier sa couverture avant publication — .github/workflows/opencode-review.yml:143, scripts/opencode-review-ci.mjs:124.
  • Produire automatiquement une revue sur le runner GitHub — aucune authentification OpenAI n’est transmise à l’étape de revue.

Hypothèses

  • Le skill de confiance est présent sur la branche par défaut — vérifié avant l’appel au modèle dans .github/workflows/opencode-review.yml:98.
  • Les patches GitHub sont complets et lisibles — contrôlés dans scripts/opencode-review-state.mjs:21 ; les patches absents ou incomplets arrêtent le traitement.

Constat

  • 🟡 Authentification du modèle absente — .github/workflows/opencode-review.yml:147. L’étape configure OPENCODE_PURE, mais ne fournit ni clé API ni session OpenAI au runner hébergé. aidd_docs/tasks/2026_11/2026_11_05_opencode_pr_review/proof.md:67 confirme que l’authentification locale n’y est pas disponible. Les revues nécessitant le modèle échoueront tant que ce prérequis n’est pas rempli. Correction : configurer une méthode d’authentification adaptée au runner, puis vérifier le parcours GitHub complet avant activation.

Limites

  • Lecture seule : aucun test ni workflow exécuté.
  • La description ne fournit pas de tableau formel de critères ; les critères ci-dessus dérivent de son objectif.
  • Le comportement réel du runner et la publication GitHub restent à vérifier lors de l’activation.

Fichiers examinés

  • .github/workflows/opencode-review.yml
  • aidd_docs/tasks/2026_11/2026_11_05_opencode_pr_review/proof.md
  • scripts/__tests__/opencode-review-ci.test.js
  • scripts/__tests__/opencode-review-discover.test.js
  • scripts/__tests__/opencode-review-state.test.js
  • scripts/opencode-review-ci.mjs
  • scripts/opencode-review-discover.mjs
  • scripts/opencode-review-state.mjs

Verdict : changes-requested

@alexsoyes alexsoyes changed the title feat(ci): prototype OpenCode reviews after submission feat: review Alex's PR submissions with OpenCode Nov 5, 2026

@alexsoyes alexsoyes left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test du déclenchement automatique de la revue OpenCode.

@alexsoyes

Copy link
Copy Markdown
Contributor Author

Revue OpenCode — 1/1

Commit examiné : 57e11fd98e963da5d1f54f7af2aa3ac3c102fdab ; la PR peut avoir évolué depuis.

Lot 1/1

Verdict: changes requested

  • Major — File mode changes can be omitted from the review. scripts/opencode-review/ci.mjs:201 invokes Git recovery only for files whose API patches are skipped. For a file with both content and executable-bit changes, the supplied GitHub patch can pass validation while omitting the mode change; the model receives only the status and patch (ci.mjs:228–229). The metadata-only recovery path in scripts/opencode-review/git.mjs:86–92 does not cover that case. Include old and new modes in the review input for content-changing files as well.

Runtime verification was not performed; the supplied context says the timer-to-comment path remains untested.

Verdict des segments de ce commentaire : changes-requested

@alexsoyes alexsoyes left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test de non-répétition après une nouvelle soumission, sans changement.

@alexsoyes alexsoyes left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test du correctif des modes Git et de la revue incrémentale.

@alexsoyes

Copy link
Copy Markdown
Contributor Author

Revue OpenCode — 1/1

Commit examiné : 90f23eed5e397f1d181670483620547dc93f61d6 ; la PR peut avoir évolué depuis.

Lot 1/1

Verdict: changes requested

  • Major — Split files are marked reviewed without a whole-file review. scripts/opencode-review/ci.mjs:227-260 divides a large patch into batches and runs each batch in a separate model session. Lines 269–270 then treat the file as reviewed once every segment has been individually confirmed. No session necessarily sees changes in different batches together, so cross-hunk defects can be missed while the file is checkpointed as complete. Add a whole-file synthesis pass, or preserve enough shared context across batches to review their interactions before checkpointing the file.

Runtime behavior was not independently verified; this was a read-only review.

Verdict des segments de ce commentaire : changes-requested

@alexsoyes alexsoyes left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test de la revue automatique par fichier et de la reprise incrémentale.

@alexsoyes

Copy link
Copy Markdown
Contributor Author

Revue OpenCode — 1/1

Commit examiné : 8a56aba42c7cc147bb72a9e541417fa9250fa354 ; la PR peut avoir évolué depuis.

Lot 1/1

Verdict: approve. No actionable defects found in the supplied files.

The review was limited to the supplied diffs and PR context; runtime checks were not run.

Verdict des fichiers de ce commentaire : approve

@alexsoyes alexsoyes left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test de la revue OpenCode simplifiée et des fichiers déjà examinés.

@alexsoyes

Copy link
Copy Markdown
Contributor Author

Revue OpenCode

Commit examiné : 91305061b9b8978df6731a7c954b59fe51660a7b ; la PR peut avoir évolué depuis.

Lot 1

Verdict: changes requested

  • Major — Large PR descriptions can prevent every file from being reviewed. scripts/opencode-review/ci.mjs:200-209 includes the entire PR title and body in the 120,000-byte session budget. If that context alone exceeds the budget, even a tiny diff is marked “Diff trop volumineux pour une seule session,” no model call is made, and the omissions are checkpointed. Bound or truncate the supporting context separately, disclose the truncation, and reserve space for at least one diff.

Limit: This was a read-only review; runtime behavior was not independently verified.

Verdict de ce lot : changes-requested

@alexsoyes alexsoyes left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Vérification finale de la revue simplifiée après limitation du contexte.

@alexsoyes

Copy link
Copy Markdown
Contributor Author

Revue OpenCode

Commit examiné : 5b4535cb9afbf428020ac7474164a692ab9ab7db ; la PR peut avoir évolué depuis.

Lot 1

Verdict: approve

No actionable defects found in the supplied files.

Limit: This was a read-only review; tests were not run.

Verdict de ce lot : approve

@aidd-bot

aidd-bot Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Test report for an earlier commit; it does not review the current PR head.

OpenCode review

Reviewed commit: 5b4535cb9afbf428020ac7474164a692ab9ab7db.

Review: changes needed

  • Major — The described trusted workflow is absent. The PR context says a workflow on main runs on an isolated runner, but this snapshot adds only the server-side systemd timer and installer (scripts/opencode-review/aidd-opencode-review.timer:5, scripts/opencode-review/install-server.sh:40). No review workflow exists under .github/workflows/. This also contradicts the stated cutover from the old server timer. Add the trusted workflow and isolated runner integration, or correct the PR’s deployment design and cutover instructions.
  • Major — Unavailable commits are retried without a PR-visible result. scripts/opencode-review/git.mjs:38–50 throws when fetch or commit verification fails; ci.mjs:188 and ci.mjs:296–299 only log and back off. The PR description promises unavailable commits are reported as not reviewed, but reviewers receive no comment. Publish an explicit incomplete-review notice for a confirmed unavailable commit without checkpointing its files.
  • Major — A pending draft can be stranded after dismissal. discover.mjs:33–38 selects a current submitted review before loading the draft. If the review backing a saved draft is dismissed and there is no other eligible submission, discovery skips the PR, so ci.mjs:168–176 never gets the opportunity to discard the draft. Check for saved drafts independently of selection and reconcile their review status.

Limits: Static review only; repository code and checks were not executed. No plan or acceptance criteria were supplied, so the functional axis was not applied.

Fully examined targets:

  • aidd_docs/tasks/2026_11/2026_11_05_opencode_pr_review/proof.md
  • scripts/__tests__/opencode-review-ci.test.js
  • scripts/__tests__/opencode-review-discover.test.js
  • scripts/__tests__/opencode-review-git.test.js
  • scripts/__tests__/opencode-review-snapshot.test.js
  • scripts/__tests__/opencode-review-state.test.js
  • scripts/opencode-review/aidd-opencode-review.service
  • scripts/opencode-review/aidd-opencode-review.timer
  • scripts/opencode-review/ci.mjs
  • scripts/opencode-review/discover.mjs
  • scripts/opencode-review/git.mjs
  • scripts/opencode-review/install-server.sh
  • scripts/opencode-review/policy.json
  • scripts/opencode-review/prompt.txt
  • scripts/opencode-review/review.sh
  • scripts/opencode-review/snapshot.py
  • scripts/opencode-review/state.mjs

Targeted files not fully reviewed: None.

@aidd-bot

aidd-bot Bot commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

OpenCode smoke test passed.


No file changed. · run

@ai-driven-dev ai-driven-dev deleted a comment from aidd-bot Bot Nov 5, 2026
@alexsoyes
alexsoyes marked this pull request as ready for review November 5, 2026 13:58
@alexsoyes
alexsoyes requested a review from a team as a code owner November 5, 2026 13:58

@alexsoyes alexsoyes left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

acknowledge all of it, plan a new refactoring.

Comment thread .github/workflows/opencode-review.yml Outdated
REVIEW_MODEL_RUNNER: /opt/aidd/run-review
AIDD_REVIEW_ENABLED_AT: ${{ vars.AIDD_REVIEW_ENABLED_AT }}
run: |
set -euo pipefail

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

call a outside script please

name: Run OpenCode
# 8973343 is @alexsoyes (Alex), the repository owner.
if: github.event.comment.user.id == 8973343 && startsWith(github.event.comment.body, '/opencode')
runs-on: ubuntu-slim

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why? Really necessary? Slim is preferable

Comment thread .github/workflows/opencode.yml Outdated
env:
GH_TOKEN: ${{ github.token }}
run: |
set -euo pipefail

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

outside script please

Comment thread .github/workflows/opencode.yml Outdated
{
"$schema": "https://opencode.ai/config.json",
"skills": [
"/trusted/.claude/skills",

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do I need this? to use the framework installed on the project?

remember we want to use the framework only if installed from the project (please read .aidd/*.json)

if so we can install it using npx with the version number and all loaded plugins, sounds fair? simplicity always wins i am ok reinstallating like this.

Comment thread .github/workflows/opencode.yml Outdated
"skills": [
"/trusted/.claude/skills",
"/trusted/.agents/skills",
"/trusted/.opencode/skills"

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And since the whole PR is about opencode i would focus on it

Comment thread docker/aidd-opencode/README.md Outdated
@@ -0,0 +1,35 @@
# AIDD OpenCode image

Build the image from this directory:

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do not duplication info

Comment thread docker/aidd-opencode/review-prompt.txt Outdated
@@ -0,0 +1,7 @@
Launch the `aidd-orchestrator-reviewer` subagent with this task:

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

never name agent, launch a "reviewer" agent is enough.

Review the supplied code and Answer in the PR directly?

Simplicity wins

Comment thread scripts/pr-review/model.sh Outdated
#!/usr/bin/env bash
set -euo pipefail

# Run only under the dedicated reviewer UID, with the checkout mounted read-only.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WTF is that?

Top comments on EVERY sh scripts please to explain it.

Comment thread scripts/pr-review/test-conversation.sh Outdated
#!/usr/bin/env bash
set -euo pipefail

root=$(cd "$(dirname "$0")/../.." && pwd -P)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wtf is that

Comment thread scripts/pr-review/test-model.sh Outdated
@@ -0,0 +1,94 @@
#!/usr/bin/env bash
set -euo pipefail
root=$(cd "$(dirname "$0")/../.." && pwd -P)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

omg stop testing those this does not NEED to be commited once we prooved the success locally

@alexsoyes

Copy link
Copy Markdown
Contributor Author

Revue OpenCode

Commit examiné : 20da4d009c8b60657dc9cfa7fcd35eee171f19b5 ; la PR peut avoir évolué depuis.

La revue est partielle ; les fichiers restants seront réessayés.

Lot 1

Review: OpenCode PR reviews

Verdict: approve. No actionable defects were identified in the supplied changes.

The review was read-only; runtime behavior and the documented production cutover were not independently verified.

Verdict de ce lot : approve

@alexsoyes

Copy link
Copy Markdown
Contributor Author

Revue OpenCode

Commit examiné : 20da4d009c8b60657dc9cfa7fcd35eee171f19b5 ; la PR peut avoir évolué depuis.

Lot 1

Verdict: approve — no actionable defects found in scripts/pr-review/test-review.sh.

The supplied test file was reviewed in full against the PR context. The tests were not run.

Verdict de ce lot : approve

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants