Skip to content

chore(forge): store conformance fixtures compactly - #300

Merged
tonibergholm-codento merged 3 commits into
mainfrom
chore-compact-fixtures
Oct 23, 2026
Merged

tonibergholm-codento merged 3 commits into
mainfrom
chore-compact-fixtures

Conversation

@tonibergholm-codento

@tonibergholm-codento tonibergholm-codento commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

 fixtures/<version>/X-scenario.json
-{
-  "scenario": "A",
-  "exchanges": [
-    {
-      "kind": "http",
-      ...
+{"scenario":"A","title":"meta","error":null,"exchanges":[
+{"kind":"http","method":"GET","path":"/api/v1/version",...},
+{"kind":"http",...}
+]}

run.py record writes this format through one helper (dump_fixture); the existing 48 fixture files were converted with the same helper.

Evidence

  • Before: 3.78 MB across fixtures/1.22.6 and fixtures/1.26.2.
    After: 2.34 MB (0.62x). Most of what remains is real payload (diffs, base64, git output), so "roughly half" was optimistic.
  • Every converted file reloads to a structure equal to the original, key order included (asserted per file during conversion).
  • run.py diff still reports 334 differing exchanges between versions; show output unchanged; forge-check.sh still 2/2 scenarios ok; 0 failed; cargo test -p scl-forge --test route_table passes.
  • Not run: a fresh record against Docker Gitea. The writer is the same helper the conversion used.

Merge Danger

Door: two-way.

Blast Radius: fixtures and the recorder's writer only; one large mechanical diff (48 files), reviewable by exchange line.

Summary by CodeRabbit

  • Tests
    • Expanded conformance coverage for repository privacy settings, visibility updates, and access responses.
    • Existing recorded scenarios and expected results remain unchanged.
  • Chores
    • Compact fixture formatting reduced the combined fixture size from 3.8 MB to 2.3 MB.

Header on one line, one exchange per line: 3.8 MB -> 2.3 MB across both
versions, still one reviewable line per exchange. Contents are unchanged
(every file reloads to an identical structure); record writes the same format.
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-23T08:13:32.061226Z 36d9950 New commits
🔒 Security Review ✅ Completed 2026-10-23T08:14:52.006537Z 36d9950 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@coderabbitai

coderabbitai Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Important

Review skipped

We couldn't safely recover the incremental review. No full review was started, and the last reviewed checkpoint was preserved. Retry later, or explicitly request a full review by commenting @coderabbitai full review.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 95ecff20-f92f-40e6-8b3e-64ceeacf7c74
📥 Commits

Reviewing files that changed from the base of the PR and between a7c8cd0 and 9fc38be.

⛔ Files ignored due to path filters: 1
  • crates/forge/conformance/__pycache__/run.cpython-311.pyc is excluded by !**/*.pyc
📒 Files selected for processing: 50
  • crates/forge/conformance/FINDINGS.md
  • crates/forge/conformance/fixtures/1.22.6/A-meta.json
  • crates/forge/conformance/fixtures/1.22.6/B-users-orgs-tokens.json
  • crates/forge/conformance/fixtures/1.22.6/C-repos.json
  • crates/forge/conformance/fixtures/1.22.6/D-empty-repo.json
  • crates/forge/conformance/fixtures/1.22.6/E-history-reads.json
  • crates/forge/conformance/fixtures/1.22.6/F-pulls.json
  • crates/forge/conformance/fixtures/1.22.6/G-merge.json
  • crates/forge/conformance/fixtures/1.22.6/H-statuses.json
  • crates/forge/conformance/fixtures/1.22.6/I-branch-protection.json
  • crates/forge/conformance/fixtures/1.22.6/J-smart-http.json
  • crates/forge/conformance/fixtures/1.22.6/K-visibility-probing.json
  • crates/forge/conformance/fixtures/1.22.6/L-force-private.json
  • crates/forge/conformance/fixtures/1.22.6/M-names.json
  • crates/forge/conformance/fixtures/1.22.6/N-default-branch.json
  • crates/forge/conformance/fixtures/1.22.6/O-pr-head-lifecycle.json
  • crates/forge/conformance/fixtures/1.22.6/P-pagination.json
  • crates/forge/conformance/fixtures/1.22.6/Q-protection-bypass.json
  • crates/forge/conformance/fixtures/1.22.6/R-token-scope-edges.json
  • crates/forge/conformance/fixtures/1.22.6/S-ref-pushes.json
  • crates/forge/conformance/fixtures/1.22.6/T-merge-outcomes.json
  • crates/forge/conformance/fixtures/1.22.6/U-status-check-edges.json
  • crates/forge/conformance/fixtures/1.22.6/V-smart-http-bytes.json
  • crates/forge/conformance/fixtures/1.22.6/W-protected-contents.json
  • crates/forge/conformance/fixtures/1.22.6/X-auth-forms.json
  • crates/forge/conformance/fixtures/1.26.2/A-meta.json
  • crates/forge/conformance/fixtures/1.26.2/B-users-orgs-tokens.json
  • crates/forge/conformance/fixtures/1.26.2/C-repos.json
  • crates/forge/conformance/fixtures/1.26.2/D-empty-repo.json
  • crates/forge/conformance/fixtures/1.26.2/E-history-reads.json
  • crates/forge/conformance/fixtures/1.26.2/F-pulls.json
  • crates/forge/conformance/fixtures/1.26.2/G-merge.json
  • crates/forge/conformance/fixtures/1.26.2/H-statuses.json
  • crates/forge/conformance/fixtures/1.26.2/I-branch-protection.json
  • crates/forge/conformance/fixtures/1.26.2/J-smart-http.json
  • crates/forge/conformance/fixtures/1.26.2/K-visibility-probing.json
  • crates/forge/conformance/fixtures/1.26.2/L-force-private.json
  • crates/forge/conformance/fixtures/1.26.2/M-names.json
  • crates/forge/conformance/fixtures/1.26.2/N-default-branch.json
  • crates/forge/conformance/fixtures/1.26.2/O-pr-head-lifecycle.json
  • crates/forge/conformance/fixtures/1.26.2/P-pagination.json
  • crates/forge/conformance/fixtures/1.26.2/Q-protection-bypass.json
  • crates/forge/conformance/fixtures/1.26.2/R-token-scope-edges.json
  • crates/forge/conformance/fixtures/1.26.2/S-ref-pushes.json
  • crates/forge/conformance/fixtures/1.26.2/T-merge-outcomes.json
  • crates/forge/conformance/fixtures/1.26.2/U-status-check-edges.json
  • crates/forge/conformance/fixtures/1.26.2/V-smart-http-bytes.json
  • crates/forge/conformance/fixtures/1.26.2/W-protected-contents.json
  • crates/forge/conformance/fixtures/1.26.2/X-auth-forms.json
  • crates/forge/conformance/run.py

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 6 remain after this review.

📝 Walkthrough

The conformance recorder now writes compact JSON, and the fixtures for versions 1.22.6 and 1.26.2 use that format. The force-private fixtures add repository privacy and access cases. The fixture-size note reports 2.3 MB across both versions.

Changes: Conformance fixture format and coverage

Layer / File(s) Summary
Compact fixture recording and corpus
crates/forge/conformance/run.py, crates/forge/conformance/fixtures/1.22.6/*.json, crates/forge/conformance/fixtures/1.26.2/*.json, crates/forge/conformance/FINDINGS.md
The recorder writes compact JSON with each exchange on its own line. The fixtures use compact JSON; unchanged fixture values are retained. The size note reports 2.3 MB across both versions, down from 3.8 MB.
Repository privacy fixture cases
crates/forge/conformance/fixtures/1.22.6/L-force-private.json, crates/forge/conformance/fixtures/1.26.2/L-force-private.json
The fixtures add repository creation, privacy update, and access exchanges. Recorded responses include private repositories after requests with private:false, privacy field changes after PATCH requests, and 403 or 404 access responses.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related issues

Possibly related PRs

Merge Risk: ⚪ Minimal · up to 9fc38

The fixture compaction preserves recorded expectations. No change needs to be resolved before merging beyond normal checks.

Architecture risk: 🔵 Low · up to 9fc38

The change affects 1 system.

Changed systems: crates

Architecture concerns
No architecture-level concerns identified.

Review details

Systems and components

  • observed — crates (service) was modified; 29 changed files map to changed impact.

Before / after behavior

  • observed — Modified behavior in crates/forge/conformance/FINDINGS.md: The fixture note changes the reported combined size from 3.5 MB to 2.3 MB, states that this is down from 3.8 MB, and attributes most remaining size to real payload; it replaces the estimate that compact JSON would roughly halve the size.
  • observed — Modified behavior in crates/forge/conformance/fixtures/1.22.6/A-meta.json: The fixture was compacted into single-line JSON; scenario metadata and all exchange methods, paths, authentication values, statuses, headers, and bodies remain unchanged.
  • observed — Modified behavior in crates/forge/conformance/fixtures/1.22.6/B-users-orgs-tokens.json: The fixture’s JSON was compacted from multi-line formatting to one line per exchange; the recorded data and expectations are unchanged.
  • observed — Modified behavior in crates/forge/conformance/fixtures/1.22.6/D-empty-repo.json: The fixture is reformatted as compact JSON; the scenario, repository response, and all subsequent request/response values are unchanged.
Pre-merge checks: Passed 5
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly and concisely describes the main change: storing Forge conformance fixtures in compact JSON format.
Docstring Coverage Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 3 functions across 1 files. (28 skipped: 2…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 3 functions across 1 files. (28 skipped: 28 unsupported.)

✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review


P2 Badge Remove the generated Python bytecode

This is a generated CPython 3.11 cache rather than a source fixture; it embeds the developer's absolute source path, is interpreter-specific, and future executions can replace it with opaque binary diffs. Remove this file and ignore __pycache__//*.pyc so the compact-fixture change does not add a 216 KB machine-generated artifact.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +917 to +920
h = J.dumps(head, separators=(",", ":"))[:-1]
f.write(h + (',' if head else '') + '"exchanges":[\n')
f.write(",\n".join(J.dumps(e, separators=(",", ":")) for e in obj["exchanges"]))
f.write("\n]}\n")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Add a regression test for compact fixture serialization

This introduces a hand-built JSON serialization path without an automated test that round-trips its output or verifies the promised header/exchange line layout, including an empty exchange list. Add a focused test so malformed or semantically changed recordings are caught before replacing the conformance fixtures; the repository explicitly requires every new behavior to ship with a test. CLAUDE.mdL90-L92

Useful? React with 👍 / 👎.

…lftest

The conversion left __pycache__/run.cpython-311.pyc in the previous commit. run.py
selftest checks that dump_fixture round-trips awkward values and that every stored
fixture is in canonical compact form; forge-check.sh (CI) runs it.

@tonibergholm tonibergholm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved: required checks green on the final head; Codex code and security reviews found nothing; bot findings (stray bytecode, writer regression test) fixed.

@tonibergholm-codento
tonibergholm-codento merged commit 2aaee3b into main Oct 23, 2026
10 checks passed
@tonibergholm-codento
tonibergholm-codento deleted the chore-compact-fixtures branch October 23, 2026 08:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants