[napetrov]

Adds a "Patterns that hide a defect" checklist to REVIEW.md: the reviewer-judgement items of SlopCodeBench's 45-item LLM-judge rubric (configs/rubrics/llm_judge.jsonl, https://github.com/SprocketLab/slop-code-bench) that apply to dokimi, merged into 11 bullets. Items an existing gate covers, and items that do not fit a sync Python CLI, are left out.

Severity rule in the new section: must-fix when the pattern changes a number, verdict or exit code (matching "What a defect looks like here"), otherwise a nit.

Mapping

Gate = already enforced in CI. Mech = a cheap mechanical check is possible (follow-up below). Review = added to REVIEW.md. Skip = does not fit dokimi.

#	Rubric item	Disposition
1	narration_comments	Partly gate: check_comment_blocks.py (≤2 lines). Review bullet for 1-2 line narration. Mech F3
2	echo_docstrings	Review. Mech F4
3	entry_exit_logging	Skip: rare in dokimi; ruff LOG/G cover logger misuse
4	class_prefix_in_members	Skip: naming style, outside review scope (REVIEW.md: no style nits ruff leaves out)
5	delegation_only_class	Review ("only forwards calls")
6	lonely_interface	Review ("structure without a second user"). Mech F5
7	one_subclass_hierarchy	Review (same bullet). Mech F5
8	single_entry_registry	Review (same bullet)
9	factory_returns_constant	Review (same bullet)
10	strategy_never_varies	Review (same bullet)
11	interface_per_class	Review (same bullet). Mech F5
12	cache_for_literals	Skip: ruff B019 covers the harmful lru_cache case; the rest is perf style
13	parallel_small_collection	Skip: perf, not a correctness defect here
14	lazy_always_accessed	Skip: perf/style
15	redundant_null_check	Partly gate: mypy warn_unreachable flags is None on non-Optional. Review for the rest
16	default_on_required_field	Review (same bullet)
17	silent_truncation	Review ("lost data or failure")
18	silent_resource_creation	Review ("missing directory or file created"). Mech F6
19	unrequested_features	Review ("code nobody asked for"); AGENTS.md "Before opening a PR" scope line too
20	handling_excluded_cases	Review (same bullet)
21	scaffold_remnants	Review (same bullet)
22	stub_implementations	Review (same bullet); coverage gate (80% changed lines) catches unexercised stubs. Mech F2
23	unnecessary_monadic_wrappers	Review (None-on-never-None bullet)
24	unnecessary_type_wrappers	Review ("wrapper class with one ... ")
25	nested_async_flows	Skip: dokimi is sync apart from a few call sites; ruff ASYNC covers misuse
26	inline_state_machines	Gate-adjacent: complexity ≤10 in check_growth.py limits it; otherwise skip
27	boolean_success_flags	Review ("reported as False, None...")
28	sentinel_values_as_errors	Review (same bullet)
29	error_message_destruction	Review ("rewritten to a generic message"); ruff E722 gates a bare except; open #729 adds BLE001/S110/S112 for blind and silent except
30	retry_loops_hiding_failures	Review ("lost data or failure": retry)
31	lenient_parsing	Review (same bullet); records are also gated by schema_compat and test_records_guard.py
32	type_coercion_hiding_mismatches	Gate: mypy (check_untyped_defs, no_implicit_optional, strict_equality)
33	defensive_cloning_hiding_mutation	Review ("defensive copy on a value ... never mutated")
34	handrolled_standard_operations	Review ("a loop that pathlib... already does"); ruff PIE/PLW cover a few
35	log_level_hiding	Skip: CLI prints results; no production log pipeline
36	correlation_id_absence	Skip: no distributed flows; run IDs already key records
37	implicit_default_parameters	Review ("a default on a parameter that switches behaviour")
38	silent_fallbacks	Review ("lost data or failure": fallback chain)
39	auto_correction_without_warning	Review (same bullet: clamp)
40	null_coalescing_abuse	Review (or default bullet). Mech F7
41	stringly_typed_logic	Review ("bare string ... Enum or Literal")
42	ignoring_return_values	Review ("return value that matters left unused"); ruff PLW1510 (subprocess check=) is deferred in pyproject. Mech F1
43	validation_mismatch	Covered by existing "What a defect looks like here" (doc/schema says something the code does not do)
44	validation_theater	Review ("validation result nothing acts on"); audit_test_quality.py gates the test-side version (asserts nothing, echoes input)
45	domain_blind_normalization	Covered by existing defect list (wrong number/verdict); not added

Follow-ups (cheap mechanical checks, not implemented here)

• F1 Un-defer ruff PLW1510 (subprocess.run without check=), so each call says whether a failure is checked. Covers feat(repo): repo compare, treatment arms against a baseline arm paired by task #42.
• F2 Grep gate on added lines for raise NotImplementedError, pass # TODO, return None # placeholder outside abstract methods. Covers refactor(gate): merge soft_gate and hard_gate into one score_gate #22, fix(ci-local): match the intel/dokimi remote by the URL git fetches from #21.
• F3 Extend check_comment_blocks.py to flag added comments starting Step N:, First,, Now we, Next,. Covers Make dokimi public #1.
• F4 Flag a one-line docstring whose words are only the function name's tokens (split on _). Covers Make dokimi public #2.
• F5 AST scan: an ABC/Protocol subclass with exactly one concrete subclass in dokimi/. Covers docs: generate the documentation site and publish it to GitHub Pages #6, fix(novelty): stop failed judge/oracle replies from reading as verdicts #7, fix(cli): offer only the judge providers the Judge accepts #11.
• F6 List added mkdir(parents=True, exist_ok=True) / os.makedirs(..., exist_ok=True) sites on read paths for review. Covers ci: run CodeQL from a workflow so PRs from forks are scanned #18 (noisy; likely advisory only).
• F7 mypy already runs warn_unreachable and strict_equality, which flag an is None branch on a non-Optional type; it does not flag x or default. A small AST check for or on a parameter typed non-Optional, non-bool would cover docs: open PRs from a fork, based on main; ci-local finds the intel/dokimi remote #16, fix(novelty): rows that are not atoms, verdicts that are not booleans, and self-conflicts no longer read as results #40.

Testing

Docs-only change to REVIEW.md; no code path changes. make ci-local static job (license header, docs refs) is the relevant check.

🤖 Generated with Claude Code

[Copilot]

🟢 Approval recommended

No active review issues block the documentation-only guidance update.

No open issues 🎉

What changed in this PR

0 new findings; nothing to fix.

This documentation-only PR expands dokimi’s review guidance with 11 checklist items adapted from SlopCodeBench’s rubric, focusing on patterns that can hide defects.

Changes:

• Adds checks for hidden failures, lost data, unnecessary abstractions, and misleading documentation.
• Defines these patterns as must-fix when they change a number, verdict, or exit code; otherwise, as nits.
• Leaves runtime behavior and CI checks unchanged.

File	Description
REVIEW.md	Adds the “Patterns that hide a defect” checklist and its severity rule.

_{Review effort: Balanced}

[exai-bot]

✅ OpenCodeReview: Review complete: 0 finding(s) across 1 selected item(s).

CI at 880bab3: 21 passing, 3 skipped.

Review details

• Commit reviewed: 880bab3 against merge base 60044a8
• Coverage: 1 files in 1 groups
• Engine: OpenCodeReview v1.13.5-exai.4.8f73eada, us.openai.gpt-6.1-sol
• Usage: 8k tokens (0 from cache), 3s
• Tool calls: 0
• REVIEW.md: 1 file

[vchamarthi]

Pushed 1 commit to review-slop-rubric (fb22c56..d4e9ea6).

• The last two bullets of the checklist are exempt from the nit rule, so they no longer clash with the no-style-nits line.
• A finding that matches both the checklist and a listed defect keeps the listed defect's must-fix severity.
  Run: the full suite with main merged in, 19789 passed.
  Static checks fails on main as well: the committed docs/reference/cli-interface.json is over the 500 KB limit of the pre-commit large-file hook. fix(cli): the CLI snapshot writes one line per parameter #737 fixes it.

[xaleryb]

• Ran on the head merged with current main: test_docs_code_refs.py and test_check_comment_blocks.py (81 passed); check_docs_review_triggers.py reports no affected docs; checked the PR body's lint and mypy claims against dokimi/pyproject.toml.
• No findings in the REVIEW.md change; the severity rule in d4e9ea6 is consistent with the existing "no style nits" line.
• Static checks was red at d4e9ea6 only because docs/reference/cli-interface.json exceeded the 500 KB limit on the old base; the branch is now updated with main (fix(cli): the CLI snapshot writes one line per parameter #737, 292,874 bytes). Approving once that run is green.

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.73%. Comparing base (3486531) to head (880bab3).
⚠️ Report is 1 commits behind head on main.
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #731   +/-   ##
=======================================
  Coverage   95.73%   95.73%           
=======================================
  Files         404      405    +1     
  Lines       54139    54211   +72     
  Branches     9317     9325    +8     
=======================================
+ Hits        51829    51899   +70     
- Misses       1424     1427    +3     
+ Partials      886      885    -1     

see 2 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[xaleryb]

Re-checked on the head merged with main: the docs tests pass (81), the docs-review trigger reports no affected page, and the REVIEW.md rules are consistent with the existing ones. Static checks is green after the update with main; the PR's own diff is unchanged.