Repository navigation
ci(policy): read git path listings with -z so quoted names reach the gates - #1474
Conversation
…gates Without -z, git quotes path names that contain a byte above 0x80 under the default core.quotePath, a control character, a double quote or a backslash. The nox runner dropped such names from the hygiene selection, tools/policy/common.changed_paths handed git's quoted text to the repository policy and JSON artifact checks, and the schema publication base listing dropped them. List with -z and split on NUL in all three, as check_requirement_governance.py and verification_plan.py already do. changed_paths now runs one listing for its three comparisons.
Without -z, git quotes a newline in a name as it does any other control character. The test now covers such a name next to the tab, non-ASCII and double-quote names; dev's three helpers fail on it alone.
…ter entry Each open entry in the divergence register now names its issue: #1525 to #1557 for the production changes, and #1521 for the smaller corrections. The entries that open pull requests now address move to "Addressed by pull requests" with their issues: GH-7 (#1491, #1490), NS-1 (#1476, #1475), NS-2 (#1480, #1479), MF-2 (#1483, #1482), CL-2 (#1474, #1473), ST-2 (#1472, #1559) and NS-7 and NS-8 (#1488, #1560 and #1561). ST-1 names #1558, GH-3 records that #1415 merged, and "What remains" names #1522 for the unverified and unexamined boundaries.
|
Recommendation: do not merge as is. Reviewed head
Required action:
Coverage and outstanding work: #1473's currently tested names are handled and -z is the correct Git interface, but text-mode decoding violates the stated verbatim-name invariant. #1538 is separately tracked pre-existing test setup behavior, outside this fix. Verification limits: Confirmed Python subprocess text-mode normalization with an isolated byte-output probe; no contributor code or repository mutations executed. |
…mes survive Text mode turned a carriage return or CRLF in a name into LF before the NUL split, so the runner dropped the name and the policy helpers returned a path that does not exist. tools.policy.common.git_paths now runs each listing with -z, reads bytes and decodes every name as strict UTF-8, like the repository's other -z listings. The runner, changed_paths and the requirement gate's merge-time diff use it; the schema publication listing reads bytes the same way. The regression test drives real repositories with CR, CRLF, LF, tab, quote, non-ASCII and space-edged names through the hygiene selection, the 500-line cap, the base schema publication manifest and the merge listing, and checks that a name that is not UTF-8 stops each listing.
…e locator The Cambridge mirror's dist directory now holds Isabelle2026 only, so the locked Isabelle2025-2_linux.tar.gz answers 404 there and canonical / generic-tool-local-inputs fails exact verification, which skips the checks, integration and proof jobs. The official release locator serves the same bytes; the fetch still verifies the unchanged lock size and SHA-256.
Dev's source at ecd7b20, where #1458 merged after #1444, matches no admitted capture, so the evidence checkers and the tests that load the current capture fail on every branch built on it. This branch changes no package source. collect_research_evidence.py ran both pipelines on it: 38 of 42 specification stages passed (4 preregistered non-executions) and all 20 formal observations matched their expected outcomes.
|
Updated to
|
… module SEM-204 reached semantics-expansion only through the shared ^SEM- pattern, whose ownership roots exclude raes_runtime/control_plane_workflows.py, so requirement governance for the #1462 compensation fix reported an ownership mismatch (#1510). Add a workflow-compensation-semantics phase listed before semantics-expansion, with the same reference-implementations prerequisite. It keeps the shared semantic roots SEM-204 already reached, without the paths named there for SEM-218 and SEM-233, and adds the compensation module and the carried #1474 locator doc. The shared phase's roots and the other #1510 requirements are unchanged. SEM-204's record gains the IMPLEMENTS and TESTS links governance requires. The new policy test pins the route, its bounds and SEM-223's unchanged phase, and runs SEM-204 governance over the delivery files.
…ce runs on this branch DSL-120 reached sdl-language only through the shared ^DSL- pattern, whose ownership roots exclude the episode-policy fixtures, the participant-control guide, the semantic profile and the assurance record that #307 amends, so forced requirement governance reported ownership mismatches and the branch carried no UID (#1510). Add a participant-episode-policy-language phase listed before sdl-language, with sdl-language's prerequisites (none). It keeps the shared SDL roots DSL-120 already reached, without the paths named there for other issues and requirements, and adds the files #307 changes and the carried #1474 locator doc. The shared phase and the other #1510 requirements are unchanged. docs/governance/requirement-scopes/307.json binds every delivery file to DSL-120, so the canonical resolver selects DSL-120 for this issue branch and CI evaluates requirement governance instead of skipping it. The DSL-120 record gains the IMPLEMENTS and TESTS links governance requires. The new policy test pins the route and its bounds, keeps DSL-121 on the shared phase, and evaluates the issue scope over its bound files.
|
This branch conflicts with dev again after the Quality refactor merges, in |
Summary
Four helpers feed git path listings to the gates:
runner._git_lines,common.changed_paths,check_schema_publication._git_pathsand the merge-time diff incheck_requirement_governance.requirement_changed_paths. On dev the first three lack-z, so git quotes names with non-ASCII bytes, control characters or double quotes, and the gates drop or mangle them (#1473). All four read text mode, which turns a carriage return or CRLF in a name into LF, so even with-zthe runner drops it.Each listing now runs with
-z, reads bytes, splits on NUL and decodes every name as strict UTF-8, so names reach the gates exactly as stored.Encoding policy: none is stated; its bytes-mode
-zlistings decode strict UTF-8 (check_identity_cutover,check_tooling_artifact_policy,verification_plan), and #1473's Chosen Architecture has a non-UTF-8 name raiseUnicodeDecodeError. Notos.fsdecode: a surrogate-escaped name can still raise later, wherever a gate prints or encodes it.Requirement UIDs
Related Issues
Closes #1473
ADR Impact
Changes
common.py:git_paths(new) runsgit <command> -z <options>and reads bytes.changed_paths, the runner's_git_pathsandrequirement_changed_pathsuse it.run_gitstays text forrev-parseandmerge-base.check_schema_publication.py:_git_pathsdecodesls-tree -zbytes the same way (it imports nothing fromtools).generic-tool-local-inputsgot a404and skippedchecks,integrationandproof.1db12336selectsofficial-isabelle-release, the lock's other same-byte locator, in the workflow andoperations.md.ecd7b205matches no capture, failing the evidence tests in every shard.collect_research_evidence.pyran both pipelines on this branch (38 of 42 spec stages passed, 4 preregistered; 20 of 20 formal observations as expected), recorded ascapture-c31b568f(spec) andcapture-602eb23f(formal).ef9c8de4mergesorigin/devecd7b205(no conflicts).git_paths, both_git_paths68dc0474's filesdev."nox -s verify-fast-feedback [...],nox -s lintandmake policy."Test Plan
Repositories hold names with CR, CRLF, LF, tab, double quote, non-ASCII letter, and leading or trailing space; every listing returns them exactly:
changed_paths,_changed_paths, the hygiene selection, and the 500-line cap rejecting each 501-line module;_tracked_repo_paths,--all-files,ls-tree, and_load_base_manifestloading every record;requirement_changed_paths, staged and unstaged;UnicodeDecodeError.On
9716654e: 10 passed. One change at a time: text-modegit_pathsfails 6,runner.py4,check_schema_publication.py1,check_requirement_governance.py2.Caller modules (36): 1045 passed,
-m integration47. Evidence checkers and the 6 evidence modules (554,-m integration25),verify-fast-feedback -- --base-rev origin/dev,lint,make policyanddocs-local: passed.CI on
9716654e: run 40189123506, 32 checks passed; Sonar gate OK.Ground Control Checks
make policypassed.Traceability
tools/{policy/common,nox_support/runner,check_schema_publication,check_requirement_governance}.py,.github/workflows/canonical-verification.ymlimplementations/python/tests/test_issue_1473_nul_path_listings.pyChecklist
cidoes not cut a release.