Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/review.yml
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ on:
description: "Gatehouse release image tag. Bump per release."
type: string
required: false
default: "quay.io/crunchtools/gatehouse:0.2.0"
default: "quay.io/crunchtools/gatehouse:0.3.0"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

HIGH (Test Coverage): The workflow now uses Gatehouse image 0.3.0, which, according to the commit message, introduces a 'structured BEFORE/AFTER diff view' instead of a raw unified diff. This represents a significant change in core functionality (diff parsing and structuring, likely in diffview.py). There is no evidence in the provided diff that corresponding unit or integration tests have been added or updated to cover this new, complex logic, including various diff formats and edge cases.

Suggestion: Ensure comprehensive unit tests are implemented for the new 'structured BEFORE/AFTER diff view' functionality. These tests should cover: correct parsing of various unified diff formats into the structured view, handling of edge cases (e.g., empty files, only additions/deletions, renames, binary files, large diffs), and verification that the structured output correctly represents changes for agents. If diffview.py was modified, existing tests should be updated to reflect the new behavior.

Evidence:

-        default: "quay.io/crunchtools/gatehouse:0.2.0"
+        default: "quay.io/crunchtools/gatehouse:0.3.0"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

HIGH (Documentation): The default image tag for the Gatehouse review workflow has been updated to 0.3.0, which introduces a significant behavioral change: agents now receive a structured BEFORE/AFTER diff view instead of a raw unified diff. This change in core functionality likely invalidates existing user-facing documentation (e.g., README, usage guides) that describes how Gatehouse processes diffs.

Suggestion: Update the project's main documentation (e.g., README.md, USAGE.md) to reflect that Gatehouse now provides a structured BEFORE/AFTER diff view to agents, rather than a raw unified diff. Explain the benefits or implications of this new structured view for users.

Evidence:

-        default: "quay.io/crunchtools/gatehouse:0.2.0"
+        default: "quay.io/crunchtools/gatehouse:0.3.0"

secrets:
GEMINI_API_KEY:
required: true
Expand Down
10 changes: 10 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,16 @@ gatehouse --advisory

All 8 agents run concurrently. Findings below 80% confidence are filtered out.

## How Agents See Changes

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

HIGH (Test Coverage): The provided diff only updates the workflow version and documentation for a new "structured BEFORE/AFTER diff view" feature (v0.3.0). The actual implementation of this feature, which involves complex diff parsing and structuring logic (e.g., handling unparseable diffs), is not included in the diff. This prevents assessing its test coverage for new or modified functions, complex logic branches (e.g., parsing failures, edge cases), and changed behavior.

Suggestion: Provide the code changes for the "structured BEFORE/AFTER diff view" feature (likely in diffview.py) to allow for a comprehensive test coverage review. Specifically, tests should cover successful parsing of various diff structures, edge cases for hunk parsing, and the described fallback to raw unified format for unparseable diffs.

Evidence:

Since v0.3.0, agents do not receive raw unified diffs. Each hunk is rendered
as a structured view: a BEFORE block (the old code, removed lines marked
`[-]`) and an AFTER block (the new code with real file line numbers, added
lines marked `[+]`), plus instructions to judge the direction of a change.
Protections *added* by a change are treated as fixes, not findings;
protections *removed* by a change are flagged. Diffs that cannot be parsed
fall back to the raw unified format.


Since v0.3.0, agents do not receive raw unified diffs. Each hunk is rendered
as a structured view: a BEFORE block (the old code, removed lines marked
`[-]`) and an AFTER block (the new code with real file line numbers, added
lines marked `[+]`), plus instructions to judge the direction of a change.
Protections *added* by a change are treated as fixes, not findings;
protections *removed* by a change are flagged. Diffs that cannot be parsed
fall back to the raw unified format.

## Exit Codes

| Code | Meaning |
Expand Down
Loading