Skip to content

Roll the spend-limited reviewer out to the remaining enrolled callers #130

Closed

Description

@loganrenz

#128 shipped composer-2.5, four per-repo spend limits and the review-deep escape hatch; #129 moved this repo's own self-caller onto it. Nothing changes for the other enrolled callers until each moves its pin — they are still reviewing with grok-4.6 at effort xhigh and no caps at all.

A pin-only bump is unsafe here

Each caller's concurrency group mirrors the callable's job if:, and review-deep is now in both. A caller that bumps the pin without adding review-deep to its own group expression buckets a deep review as other, where the next stray event cancels the live run before the job condition is evaluated — so cancel_previous never runs and the paid Cursor agent behind it keeps burning. Both edits go in one commit.

The correct expression is in the copy-paste template beside the job if: in .github/workflows/cursor-review.yml. The change per caller:

-contains(fromJSON('["review-now","review-p0","review-p1"]'), github.event.label.name)
+contains(fromJSON('["review-now","review-p0","review-p1","review-deep"]'), github.event.label.name)

plus the pin to 0a3aa8bfc (or later main).

Per caller

  • Bump the pin and the group expression in the same commit
  • Create the review-deep label (it currently exists only in this repo)
  • Set daily-cap if 12/day is wrong for that repo — it is deliberately per-repo, because some need more oversight than others

Also still open

  • composer-2.5 has not yet completed an end-to-end review. Its launch shape is verified live (POST /v1/agents with {"id":"composer-2.5","params":[{"id":"fast","value":"false"}]} and no effort key → 201), and the model id is confirmed against GET /v1/models, but no review has run start-to-finish on it: this repo has been over its own new daily cap since the pin landed. Watch the first review after the 24h window clears.
  • review-deep has never been exercised end-to-end. Its three legs are each verified independently (the label is in the job if: at the pinned SHA, the gate bypass is unit-tested, grok-4.6/xhigh launches in production) but their composition is not. Not worth a paid grok-xhigh run purely to test; worth watching the first genuine use.
  • Six stale cursor-review-* branches on the remote from earlier sessions (budget-routing, callable, launch-shape, path-prefix, self-enroll, self-pin) — not mine to prune.

Context: the driver was 99 review agents launched against 2 implementation lanes in one evening. Unit economics are ~0.131% of the monthly pool per review, so the plan affords roughly 25 reviews/day; 139 ran on 2026-09-19.

Refs #128, refs #129.

Activity

  1. added
    enhancementNew feature or request
    P1-highReal defect on a load-bearing path; next session
    on Sep 20, 2026
  2. loganrenz commented on Sep 20, 2026

    @loganrenz
    ContributorAuthor

    Caller audit — narduk-libs is the one urgent case, the other eight are routine

    Evidence relayed from the portals program (two of its lanes), then verified here against the repos and the Cursor API rather than taken on trust.

    repo pin synchronize draft guard caps
    narduk-libs 852e0f0c (#120) YES NO none
    agent-infrastructure, fleet, lakestat-us, narduk-data, operator-portal, riverstatus, runners, stonx 246013d4 (#125) no yes none
    workflows a3557de7 (#128) no yes all four

    narduk-libs is on the oldest callable and it is the expensive one on every axis at once. At 852e0f0c the job if: is bare ${{ inputs.enabled }} — no draft guard, no no-ai-review guard, no label gating — and its fast input defaults to true. Combined with synchronize in its triggers, that repo reviews every push, including pushes to drafts, at grok-4.6 / effort xhigh / fast true.

    fast is a real price multiplier on grok, not just on composer (verified, cursor.com/docs/models-and-pricing):

    input cache read output
    grok-4.6 $2 $0.50 $6
    grok-4.6 fast $4 $1 $12
    composer-2.5 fast (its own default) $3 $0.50 $15
    composer-2.5 fast: false (#128 default) $0.50 $0.20 $2.50

    grok fast is exactly 2× standard grok, and 5–8× the new default depending on the axis — cache read, which dominates an agentic review, is 5×. So a narduk-libs review costs roughly six times what the same review costs at the #128 default, and it runs on every push rather than once per head.

    This was not in the #128 cost model, which assumed fast=false estate-wide. It does not change the measured baseline — that came from Cursor's own agent ledger, which counts launched agents at whatever price — but it does mean the spend was more concentrated in one repo than the reviews-per-PR multipliers suggested.

    Priority

    1. narduk-libs first. Biggest single lever in the estate, and one PR. Caveat worth flagging before it moves: dropping synchronize removes push-review from a repo with active lanes, and on 852e0f0c its review-now is inert — so those lanes currently have no working re-request path. Moving to a3557de7 gives them a working review-now and review-deep in exchange. Land the pin, the group expression, and the label together.
    2. The eight on 246013d4 are routine: correct triggers already, but still grok-4.6/xhigh and no caps at all.

    Two related wastes, verified

    • buoys is enrolled nowhere and burning quota anyway. #258 ("ci: enroll the Cursor Cloud pull-request reviewer") has been OPEN and BLOCKED since 2026-09-18T23:15Z, and all of that repo's reviewer activity has gone to it while merged buoys PRs got reviews: []. Either finish the enrolment or close it; right now it is paying for coverage it does not have.
    • A repo with no CURSOR_CLOUD_AGENTS_API_KEY exits green in ~8s having reviewed nothing (reported for lakestat-us and narduk-data). Those runs cost no Cursor quota — a skipped run never launches an agent — but they make a check-context-based coverage metric read as covered when it is not. Worth a follow-up so "green" is not mistaken for "reviewed".

    Evidence from the portals master orchestrator session; pins, triggers, input defaults and prices re-verified here before recording.

  3. loganrenz commented on Sep 20, 2026

    @loganrenz
    ContributorAuthor

    Progress: narduk-libs done, buoys blocked upstream

    narduk-libs — merged (#618, 911a9fcc). The estate's single biggest Cursor line item is closed.

    The measured number, from the new cap's own first run there: "27 review(s) in the last 24h, the cap is 12." The #128 analysis put the entire estate's affordable rate at ~25 reviews/day. One repo was running 27/day at ~6× the new default unit cost. That is the concrete shape of "13% of usage in an evening".

    All four parts landed together — pin → a3557de7, synchronize dropped, labeled added (its absence is why review-now was inert there: no label could reach the job), and a concurrency group verified as an exact string match against the canonical template. review-deep label created.

    buoys — change complete, blocked on a dependency (#258).

    Two corrections to my earlier comment on that PR:

    1. I said merging main was the whole fix. It was not. The fresh run is still red, and the cause is current rather than stale.
    2. The real cause is a dependency typecheck failure, not anything in buoys: @narduk-enterprises/narduk-analytics@1.21.7 ships .ts that does not typecheck in a consumer's context (requireMutationBody's result collapses to {}). narduk-libs' own package lane passes because it typechecks inside that workspace. Filed as narduk-libs#621.

    buoys main pins the same 1.21.7, so main is latently red — its last run predates this and has not re-run since 2026-09-19T23:12. Worth knowing before anyone opens a buoys PR expecting green.

    The enrolment change itself is correct and ready: a3557de7, no synchronize, labeled added, group an exact template match, and the full class label set created (only no-ai-review existed). Landing it as originally written would have enrolled buoys on the same 852e0f0 every-push grok-fast config narduk-libs just came off.

    Remaining

    Eight callers on 246013d4 — agent-infrastructure, fleet, lakestat-us, narduk-data, operator-portal, riverstatus, runners, stonx. Correct triggers already, but still grok-4.6/xhigh and no caps. Each needs the same four-part change plus a review-deep label.

    composer-2.5 still has not completed an end-to-end review — ironically prevented by the caps themselves, since every repo tried so far was over its window. The first PR in narduk-libs once its 24h drains (no more push-review feeding it) is the one to watch.

  4. loganrenz commented on Oct 8, 2026

    @loganrenz
    ContributorAuthor

    Done: every caller in the 2026-09-20 audit moved to the spend-limited callable (narduk-libs#618, agent-infrastructure#1604, fleet#473, lakestat-us#143, narduk-data#496, operator-portal#398, riverstatus#211, runners#167, stonx#809, all merged 2026-09-20). The Actions reviewer was then retired estate-wide in #166 (2026-09-28) in favor of the runners PR review bot, so no caller is left on the uncapped config.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1-highReal defect on a load-bearing path; next sessionarea:foundationDomain: foundationenhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions