feat(tasks,tools): consumer-supplied rubrics for assess_brief + repo-specific merge gates#1131
Merged
Merged
Conversation
MoltNet-Diary: 1ece9904-b651-469a-b4fb-f05e4e90cc39 Task-Group: assess-rubric-decoupling Task-Family: refactor Task-Completes: true
…judgment PR comment MoltNet-Diary: 02fd7113-1290-45c9-a7e8-9d2d2e274b68 Task-Group: assess-rubric-decoupling Task-Completes: true
Contributor
|
Contributor
🚨 Dependency Audit — Vulnerabilities foundFull report |
Contributor
|
getlarge
approved these changes
May 12, 2026
MoltNet-Diary: 2ec7f541-9f90-45c4-b712-eaa96f9d9b59 Task-Group: assess-rubric-decoupling Task-Completes: true
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two-commit PR validated end-to-end against a live
assess_briefrun on PR #1120.Decouple rubric content from the
@moltnet/taskslibrary — the lib hadPR_COMPLEXITY_V1hardcoded, which forced our PR-review opinions onto every external consumer. Lib now ships only theRubricTypeBox schema; rubric content lives where it should — in the consumer's repo.Author two starter rubrics for this repo —
rubrics/pr-complexity-v1.json(the moved llm-scored content, byte-equivalent) andrubrics/pr-merge-gates.json(six boolean auto-merge gates tuned to this repo's risk model).What changed
libs/tasks/libs/tasks/src/rubrics/pr-complexity.tsand itsindex.tsre-export.RubricTypeBox value (still inrubric.ts) is now the lib's only contribution to the assess flow — it's the contract, not the content.tools/src/tasks/assess-pr.ts--rubric <path>(no default — rubrics are repo-specific).Rubricschema withValue.Check, with concrete error messages fromValue.Errors.--rubricrelative to the repo root (viagit rev-parse --show-toplevel), not CWD —rubrics/pr-merge-gates.jsonworks the same from any subdirectory or worktree.rubrics/at repo rootpr-complexity-v1.json— the moved llm-scored content. Five criteria, weights sum to 1, byte-for-byte equivalent to the deleted Go-side const.pr-merge-gates.json— six boolean auto-merge gates tuned to this repo:gate_line_countgate_no_protobuf.protofiles touchedgate_no_infrainfra/, Docker, GitHub workflows/actions,fly*,nx.json, or release-please configgate_no_authlibs/auth/,libs/crypto-service/, or path matchingauth|crypto|signing|jwt|token|ory|ketogate_no_audit_logginglibs/diary-service/,apps/rest-api/src/routes/diary-entries*/signing*, themoltnet_create_entrycustom tool, or diary signing inlibs/crypto-service/gate_no_agent_runtime_rulesAGENTS.md/ nestedCLAUDE.md,runtime-instructor.ts,.claude/rules/,.claude/skills/legreffier/,.agents/skills/legreffier/Deliberately not gated:
libs/database/drizzle/) — schema-without-migration is caught by e2e (the stack won't start). The gate would have been belt-and-suspenders.packages/*— release-please ships on a manual trigger, not on merge; reviewers see the release-please PR separately.pr-complexity-v1).The rubric's preamble also instructs the agent to post a markdown checklist as a PR comment via
gh pr comment <num>from the guest VM's bash tool (nomoltnet_host_execinvolved —ghis on PATH with a validGH_TOKENfrom the GitHub App).Verified end-to-end
Task
a408075d-fb7b-4f17-b79b-dae2ee6be855judged PR #1120 with the originalpr-merge-gates.json(the v1.0 6-gate version before tuning). 6/6 PASS, composite 1.0, ~3.5 min wall, ~22k input + ~4.8k output tokens. Per-criterion evidence cited specific commit hashes, file paths, andgit show --numstatline counts. The runtime didn't know about merge gates as a concept — they're just rubric content the caller supplied.A follow-up assess run against this PR (with the tuned rubric — added
gate_no_agent_runtime_rules, removedgate_no_migrations, line cap 300, and the PR-comment instruction) is the natural next test.Test plan
pnpm -w run typecheckclean.pnpm --filter @moltnet/tasks exec vitest run— 98 tests pass.--dry-runagainst the merge-gates rubric prints the correct synthesizedAssessBriefInput.gate_no_agent_runtime_rulesexists and the PR-comment-after-submit instruction fires (lands a comment on this PR viagh pr comment).Diary
1ece9904-b651-469a-b4fb-f05e4e90cc39— rubric decoupling commit.02fd7113-1290-45c9-a7e8-9d2d2e274b68— tuned merge gates + PR comment instruction.Cross-references
assess_briefwork; complements feat(cli): publish JSON Schema for moltnet.json #1120 (the producer PR being judged).host_execin daemon mode — partly subsumed:git pushnow inhostExec.autoApproveper main;gh pr commentcould be added similarly).moltnet task attempts <id>subcommand #1130 (moltnet task attempts <id>CLI subcommand — currently the assess output is only reachable via SDK).