Skip to content

Add reusable model-backed code review tool#4

Merged
mabry1985 merged 3 commits into
mainfrom
feature/model-code-review-tool
May 21, 2026
Merged

Add reusable model-backed code review tool#4
mabry1985 merged 3 commits into
mainfrom
feature/model-code-review-tool

Conversation

@mabry1985
Copy link
Copy Markdown
Contributor

@mabry1985 mabry1985 commented May 21, 2026

Summary

  • add a reusable review-code CLI for bounded model-backed code review across protoLabs repos
  • map feature specs, run strict JSON review through the OpenAI-compatible gateway, persist findings, preserve triage state, and emit Markdown reports
  • add tests, CI smoke coverage, and an ESLint flat config so the existing lint script works

Validation

  • node --check bin/rewrite-release-notes.mjs
  • node --check bin/build-updater-manifest.mjs
  • node --check bin/review-code.mjs
  • node --check lib/code-review.mjs
  • npm test
  • npm run smoke
  • npm run lint
  • live gateway smoke: protolabs/fast reviewed one feature and generated /tmp/release-tools-review-live/report.md

Notes

  • Minimax aliases can be used through --model / CODE_REVIEW_MODEL once the active gateway key is allowed to access them.
  • This is intentionally report-only: no automatic patching or code edits.

Summary by CodeRabbit

  • New Features

    • Added a review-code command supporting code review initialization, feature mapping, automated review execution, status tracking, and report generation.
  • Chores

    • Updated CI pipeline to validate and test the new review tool.
    • Added linting configuration for JavaScript modules.
    • Updated ignore rules.
  • Tests

    • Added comprehensive test suite for the review functionality.

Review Change Stack

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 21, 2026

Warning

Rate limit exceeded

@mabry1985 has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 3 minutes and 26 seconds before requesting another review.

You’ve run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: e4c76582-314a-4331-a76a-8033ae37acd6

📥 Commits

Reviewing files that changed from the base of the PR and between e211154 and 14a901b.

📒 Files selected for processing (4)
  • bin/review-code.mjs
  • lib/code-review.mjs
  • package.json
  • test/code-review.test.mjs

Walkthrough

This PR introduces a complete reusable code-review tool with a core library (lib/code-review.mjs), CLI entrypoint (bin/review-code.mjs), and test suite. The tool maps repository features by glob rules, invokes an OpenAI-compatible LLM with bounded context, validates and normalizes findings with deterministic IDs, preserves triage metadata across runs, and generates Markdown reports.

Changes

Code Review Tool

Layer / File(s) Summary
Configuration constants and error handling
lib/code-review.mjs
Review schema version, state directory defaults, model and API configuration constants, context limits, and CodeReviewError class for consistent error handling.
State initialization and feature spec loading
lib/code-review.mjs
initReviewState creates state directory and persists config.json with loaded feature specs and project.json with repo metadata; feature spec loading with field normalization and validation.
Repository scanning and feature mapping
lib/code-review.mjs
mapReviewFeatures scans repo and selects owned/context/test files via glob rules; walkRepo traverses repo with symlink-safe handling and exclusion directories; glob-to-regexp conversion and entrypoint detection.
LLM integration and finding validation
lib/code-review.mjs
callStructuredReview makes OpenAI-compatible API calls with JSON-schema response formatting; normalizeFindings validates finding schema and location; validateFindingLocation allowlists file paths and bounds-checks line numbers; preserveTriage merges existing triage fields; review output schema and system prompt.
Core review orchestration and prompt generation
lib/code-review.mjs
reviewMappedFeatures runs feature-by-feature reviews, persists findings with run metadata, and returns summary counts; reviewPrompt assembles bounded file excerpts with context budgeting and omission markers.
Status tracking and report generation
lib/code-review.mjs
reviewStatus computes finding counts by status and severity; loadReviewFindings loads and sorts findings; buildReviewReport generates Markdown reports; utilities for finding ID generation, severity ranking, path resolution, JSON persistence, and timestamp formatting.
CLI entrypoint and command dispatch
bin/review-code.mjs
CLI script with --help mode reading its own source; custom argument parser for positional commands and --flag options; dispatch to init, map, run, status, report with flag validation, environment variable checks for API keys, and error handling with exit codes.
Package configuration, linting, and CI integration
package.json, eslint.config.js, .github/workflows/ci.yml, .gitignore
Package.json adds review-code binary and updates smoke and test scripts; ESLint flat config with @eslint/js recommended rules and Node environment; CI workflow syntax checks and test execution for the review tool; .gitignore ignores .release-tools-review/ state directory.
Comprehensive test suite
test/code-review.test.mjs
Test cases covering feature mapping with symlink allowlisting, finding persistence and triage field preservation, finding validation and rejection on out-of-scope locations, context budgeting with omission markers, CLI help output; FakeClient stub and createRepo helper with fixture files and config.

🎯 4 (Complex) | ⏱️ ~60 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main changeset—adding a new reusable, model-backed code review tool with CLI, library, tests, and CI integration.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feature/model-code-review-tool

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 7

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@bin/review-code.mjs`:
- Around line 166-168: The resultPath function only checks for POSIX-style
absolute paths with startsWith('/') which breaks on Windows; update resultPath
(function name) to use Node's path module (imported from 'node:path') and
replace the manual startsWith/replace logic with path.isAbsolute to detect
absolute stateDir and path.join (or path.resolve) to construct the final path so
both POSIX and Windows absolute/relative --state-dir values are handled
correctly.

In `@lib/code-review.mjs`:
- Around line 814-816: The timestampId function contains a no-op
.replace('T','T'); remove that redundant replace call from timestampId (or if
the intent was to substitute the 'T' with another character such as a space or
underscore, change the second argument accordingly) so the function returns the
intended ISO-based id without an unnecessary no-op; update the implementation in
the timestampId function accordingly.
- Around line 403-426: The fetch call that posts to
`${baseUrl}/chat/completions` can hang because it has no timeout; wrap the
request with an AbortController: create an AbortController, pass its signal into
the existing fetch call, start a setTimeout (e.g., 30s or configurable) that
calls controller.abort(), and clear that timeout once a response is received;
ensure the abort is caught and rethrown as a CodeReviewError (or mapped to the
existing error-handling path) so callers get a clear timeout message. Reference
the existing fetch invocation (variable res) and the CodeReviewError thrown on
non-ok responses when adding the AbortController/signal and timeout logic.

In `@test/code-review.test.mjs`:
- Around line 42-101: The test "review run persists findings, preserves triage,
and reports status" creates a temporary repo via createRepo() but never removes
it; update the test to clean up the temp repo (use fs.rmSync or fs.promises.rm)
in a finally block around the async test body or register cleanup with the Node
test runner (e.g., test.afterEach/test.teardown) so the directory created by
createRepo() is removed after the test; modify the test function (the test(...)
block) to capture the repo variable and ensure deletion even on failure.
- Line 167: The test uses import.meta.dirname (seen in the cwd:
path.resolve(import.meta.dirname, '..') call) which requires Node >=20.11.0;
either update package.json engines to "node": ">=20.11.0" to guarantee
compatibility, or refactor the test to avoid import.meta.dirname by deriving the
directory from import.meta.url (use fileURLToPath(import.meta.url) and
path.dirname) and then pass that into path.resolve — update the code referencing
import.meta.dirname accordingly.
- Around line 18-40: The test leaves temporary resources around: clean up the
external temp file and the repo directory created by createRepo() by wrapping
the setup in a try/finally (or using after/teardown) and in the finally call
fs.unlinkSync(outside) if the outside file exists and remove the repo directory
recursively (e.g., fs.rmSync(repo, { recursive: true, force: true }) or a
project teardown helper); ensure these cleanup calls run even if assertions fail
so mapReviewFeatures and the assertions around feature (feature_id 'app') still
execute but resources are always removed afterwards.
- Around line 183-214: createRepo currently creates temp repos but never removes
them; change createRepo to return an object like { repo, cleanup } where cleanup
is a function that removes the temp directory (e.g., using fs.rmSync(repo, {
recursive: true, force: true }) or equivalent), and update tests that call
createRepo to destructure the returned cleanup and call it in a finally block
(or afterEach) to ensure temp directories are removed; reference the createRepo
function and tests that call it when making the changes.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: c63a1221-b11f-47b4-8685-cfeb81f6354e

📥 Commits

Reviewing files that changed from the base of the PR and between fac2445 and e211154.

⛔ Files ignored due to path filters (2)
  • README.md is excluded by !*.md
  • package-lock.json is excluded by !**/package-lock.json
📒 Files selected for processing (7)
  • .github/workflows/ci.yml
  • .gitignore
  • bin/review-code.mjs
  • eslint.config.js
  • lib/code-review.mjs
  • package.json
  • test/code-review.test.mjs

Comment thread bin/review-code.mjs Outdated
Comment thread lib/code-review.mjs Outdated
Comment thread lib/code-review.mjs
Comment thread test/code-review.test.mjs
Comment thread test/code-review.test.mjs
Comment thread test/code-review.test.mjs Outdated
Comment thread test/code-review.test.mjs
@mabry1985 mabry1985 merged commit aab2539 into main May 21, 2026
2 checks passed
@mabry1985 mabry1985 deleted the feature/model-code-review-tool branch May 21, 2026 10:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant