[antonwolfy]

Summary

Follow-up to the reviews of #590, #595, #472, #706 and #637, which left these points open. Each test below was a mutant that passed the whole suite on main.

• feat(cli): every --json leaf emits cli_output.v1 #590 (B3.4, review): the abbreviation check of _json_requested always true, and _LeafParser.error ignoring the --json flag, both passed. No shipped leaf has --json next to another --js* option (catalog check has --jsonl and no --json), so the ambiguity is pinned on a probe leaf and on catalog check. repo validate . --bogus is reported by the root parser, not the leaf, so the flag gate is pinned with repo validate and repo validate . --cache-dir; the --bogus case was already pinned.
• feat(cli): skill routing and catalog routing replace novelty routing (#94 M3) #595 (B1.1b, review): picks_flag in _conflict_hint_for flipped (if not all_vs_all) passed, because no test ran the no-holder branch.
• novelty: program checks on probes (identifiers veto) + grading-ladder design note #472 (B4.1 to B4.3, review): dropping but or while from the clause split of _negated_before, dropping avoid\w* or without from _NEGATED, and four mutants of checks_summary passed.
• Two more survivors found by the same sweep: ; in _ESCAPE_RE and _INNER_ESCAPE_RE of bundle_paths.py. Only the first is pinned here; the second swaps \; inside double quotes, where a shell keeps the backslash, so it is tracked as a defect instead of pinned. A third, len(nodes) > 0 for _has_a_node in needs.py, is already killed by test_the_need_graph_caches_only_a_graph_with_a_node ({"nodes": [{"text": ""}]}), so nothing changes there.
• fix(repo): a pinned source reads a .. after a gitlink directory as a checkout does #706 (thread 1, thread 2): on main, a gitlink at third_party/mod with no .gitmodules and a README claim mod/ is unresolved, the same as when it is declared. The ok_basename flip the first thread describes was fixed before fix(repo): a pinned source reads a .. after a gitlink directory as a checkout does #706 merged, and test_a_bare_submodule_name_is_unresolved_like_its_full_path pins both cases. Breaking the gitlink part of Scan.submodules fails test_a_submodule_without_a_gitmodules_entry_is_still_a_directory. The contract did not say which paths count as submodules.
• fix(eval): eval score reference refuses an unparseable --answers file (#605) #637 (comment): eval score reference --answers "" reads Path(""), which is ., and says Error: --answers : Is a directory, exit 2, with nothing after the flag.

Changes

• tests/test_cli_output.py: a probe leaf with --json and --jsonl reads --jso as no --json request and prints no document; catalog check --jso is no request either; repo validate and repo validate . --cache-dir without --json print nothing on stdout, the message on stderr, exit 2.
• tests/test_novelty_routing_gaps.py: two skills named review, one in review/ and one in review/review/, are held by no single directory. catalog routing says no --keep-copies-under picks one of its copies and never names --against; skill routing says no --against or --keep-copies-under picks one of its copies.
• tests/test_novelty_checks.py: "Do not use perf script but perf report --stdin" and "Never guess flags while you run perf report --stdin" are vetoed, with controls (or, when) that stay negated; "Avoid", "Avoiding" and "without" before perf report --stdin make no claim, and the same sentences with a neutral word are vetoed; one checks_summary call over seven runs asserts all six fields. The veto tests sit here, next to the other identifiers-veto tests, because tests/test_reference_checks.py has none.
• tests/test_catalog_path_missing.py: a missing assets/a\;b.svg is named assets/a;b.svg whole when bare.
• dokimi/docs/repo-evaluation-contract.md: a pinned snapshot's submodules are its .gitmodules paths and every gitlink, declared or not; a bare name matching only a submodule (mod/) is unresolved.
• dokimi/dokimi/utils.py: read_json_input refuses an empty path first, as InputError --answers "": no file given (cannot read "": no file given without a flag). Exit stays 2. dokimi/dokimi/questions/artifact.py: load_questions hands the raw path to read_json_input before it makes a Path, so --questions "" is refused by name too (--questions "": no file given). tests/test_load_or_fail.py covers both forms, eval score reference --answers "" through cli.main, and load_questions("", flag="--questions").
• Changelog fragment (Fixed), for the read_json_input change.

Risk

Only read_json_input and the way load_questions calls it change behaviour: an empty path, which was read as . and failed with Is a directory, is now refused by name; every caller still exits 2 with an InputError. A non-empty path is read unchanged. The other commits are tests and one doc sentence.

Testing

On a Linux review host, at 58c79ee (on main b4bcfa4):

• make ci-local: 31 passed, 2 failed, 14 skipped (the skips are the jobs that do not run on a PR, the other matrix legs, conda-forge, semgrep, fuzz). 20418 tests passed, 19 skipped; changed-line coverage 100% (2 lines). The 2 failures are the host: Install gitleaks (the download returned HTTP 504 four times) and Scan commits (no gitleaks binary).
• eval score reference --answers "": on main Error: --answers : Is a directory; here Error: --answers "": no file given, exit 2.
• Mutants, each failing in a copy of the tree: _json_requested abbreviation test always true (2 tests fail); _LeafParser.error without and self._json_argv (3); if not all_vs_all for picks_flag (2); but dropped (1); while dropped (1); avoid\w* dropped (2); without dropped (1); checks_summary overruled ==, decided without not in vetoed, judge_agreed !=, checked_runs 0 (1 test, fails for each); ; dropped from _ESCAPE_RE (1); the empty-path refusal removed (3). ; dropped from _INNER_ESCAPE_RE is not pinned: inside double quotes a shell keeps the backslash of \;, so the code's current name there is wrong and no test fixes it. The _ESCAPE_RE mutant and _has_a_node with len(nodes) > 0 were run against the whole suite first: the escape mutant passed it, the _has_a_node one failed test_the_need_graph_caches_only_a_graph_with_a_node.
• fix(repo): a pinned source reads a .. after a gitlink directory as a checkout does #706: with the gitlink part of Scan.submodules removed, test_a_submodule_without_a_gitmodules_entry_is_still_a_directory fails; with the bare-name submodule filter removed, both cases of test_a_bare_submodule_name_is_unresolved_like_its_full_path fail. On the code before fix(repo): a pinned source reads a .. after a gitlink directory as a checkout does #706 those tests pass, as the L0 verdicts were meant not to change.
• Merged together with fix/run-wide-credentials, fix/url-extraction-delimited on main b4bcfa4: pytest -n auto 20429 passed, 19 skipped, mypy clean (420 source files), ruff check clean.
• Not run: the other Python versions (3.11, 3.12, 3.14), a real runner, and gitleaks (the download failed).

Revert

A plain revert restores the old behaviour: an empty path reads as . again, and the tests and the doc sentence go. Nothing is stored.

Checklist

• Tests added/updated and pytest passes
• ruff, mypy, and schema checks pass
• Docs updated (or N/A) — see the docs-drift checklist in CONTRIBUTING.md
• Self-reviewed per "Before opening a PR" in AGENTS.md
• No ad-hoc run artifacts committed
• New files carry the SPDX license header (or are exempt in .github/.licenserc.yaml)

[exai-bot]

✅ OpenCodeReview: Review complete: 0 finding(s) across 9 selected item(s).

CI at ee2bfb9: 26 passing, 3 skipped.

Review details

• Commit reviewed: ee2bfb9 against merge base b4bcfa4
• Coverage: 9 files in 1 groups
• Engine: OpenCodeReview v1.13.5-exai.4.8f73eada, us.openai.gpt-6.1-sol
• Usage: 187k tokens (129k from cache), 31s
• Tool calls: 29 (21 file_read, 6 code_search, 1 ci_status, 1 local_check_results)
• Empty JSON input handling and evaluation contract regression coverage: 9 files

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.73%. Comparing base (b4bcfa4) to head (ee2bfb9).
⚠️ Report is 8 commits behind head on main.
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #753      +/-   ##
==========================================
+ Coverage   95.71%   95.73%   +0.01%     
==========================================
  Files         414      415       +1     
  Lines       55310    55365      +55     
  Branches     9509     9517       +8     
==========================================
+ Hits        52941    53003      +62     
+ Misses       1466     1461       -5     
+ Partials      903      901       -2     

Files with missing lines	Coverage Δ
dokimi/dokimi/questions/artifact.py	97.58% <100.00%> (ø)
dokimi/dokimi/utils.py	95.03% <100.00%> (+0.02%)	⬆️

... and 7 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[vchamarthi]

Pushed 1 commit to test/leftovers-2026-11-14 (58c79ee..ee2bfb9).

• load_questions passes the raw path to read_json_input before Path(path).
• Removed test_a_missing_escaped_semicolon_inside_double_quotes_is_named_whole.
  Run: the full suite with main merged in, 20463 passed.

Seen in files outside this PR, not changed here:

• dokimi/dokimi/external/bundle_paths.py:80: _INNER_ESCAPE_RE swaps \; inside double quotes, so cp "assets/a\;b.svg" /tmp/out names assets/a;b.svg; expected assets/a\;b.svg.

The PR description needs these corrections:

• "tests/test_catalog_path_missing.py: a missing assets/a\;b.svg is named assets/a;b.svg whole, bare and inside double quotes." should read: tests/test_catalog_path_missing.py: a missing assets/a\;b.svg is named assets/a;b.svg whole when bare.

[antonwolfy]

Pushed 1 commit to test/leftovers-2026-11-14 (58c79ee..ee2bfb9).

@vchamarthi (#753 (comment)): No code change. The description line now reads: "tests/test_catalog_path_missing.py: a missing assets/a\;b.svg is named assets/a;b.svg whole when bare." I also corrected two other lines that ee2bfb9 made untrue: the mutant list no longer claims a test for ; dropped from _INNER_ESCAPE_RE (I ran that mutant on a copy of the head: test_catalog_path_missing.py passes, 90 passed), and Changes/Risk now name the load_questions change.

The bundle_paths.py:80 report holds: _words('cp "assets/a\\;b.svg" /tmp/out') returns assets/a;b.svg, and the same for \, inside double quotes (assets/a,b.svg); a shell keeps the backslash there. It is outside this PR, so it is tracked in #773.

[antonwolfy]

Ready for another look at ee2bfb9; CI is green.

Follow-up issues opened:

• bundle_paths._words swaps a backslash escape inside double quotes, which a shell keeps literal #773: bundle_paths._words swaps a backslash escape inside double quotes, which a shell keeps literal