Skip to content

ci: enforce distribution quality gate#11

Merged
wesleysimplicio merged 7 commits into
masterfrom
codex/issue-10-ci-audit
Jul 14, 2026
Merged

ci: enforce distribution quality gate#11
wesleysimplicio merged 7 commits into
masterfrom
codex/issue-10-ci-audit

Conversation

@wesleysimplicio

Copy link
Copy Markdown
Owner

Summary

  • add a mandatory PR/push quality workflow with lint, unit, integration, E2E, security, benchmark, flaky-repeat, and aggregate quality-gate jobs
  • enforce Python/Node coverage and distribution consistency, publish JUnit/coverage/failure artifacts, and document skips, exceptions, duration, and benchmark policy
  • align canonical distribution metadata to the current 3.5.2 release and add dry-run installer/package contracts

Measured local evidence

  • Python: 25 passed; 96% overall coverage; 97% critical verifier coverage
  • Node unit: 12 passed; 100% lines/functions and 90.91% branches
  • E2E: 3 passed
  • strict distribution: 0 errors, 0 warnings
  • security/skip policy: passed
  • benchmark: 16,522 bytes <= 18,174; 15.74 ms <= 125 ms
  • flaky checks: Python and Node stable across 3/3 runs
  • npm pack dry-run 3/3; Python sdist+wheel built
  • shell, PowerShell, Node, Python, JSON, YAML syntax and git diff checks passed

Tracking

Related: #10. Remaining acceptance stays tracked until this PR is merged, Actions is enabled, the live workflow is green, and the master ruleset rejects a non-green/up-to-date violation.

@wesleysimplicio

Copy link
Copy Markdown
Owner Author

Reopening immediately after enabling repository Actions so the new pull_request quality workflow receives a live event.

Simplicio, Wesley (ext) and others added 6 commits July 14, 2026 07:43
…world release design

Resolves the release.yml conflict in favor of this branch's workflow_dispatch-
only, provenance-script-driven publisher (scripts/verify_release_provenance.py):
per repo-owner decision, releases are no longer supposed to auto-trigger on
push, so master's push-triggered direct-cp release.yml is discarded.

Resolves scripts/verify_distribution_consistency.py by keeping both audit
functions and wiring them into the same run_audit() aggregate:
- release_workflow_errors() (this branch): validates release.yml's exact
  closed-world step contract.
- check_target_triplet_consistency() (master, issue #5): validates that
  distribution/targets.json is the single naming source of truth across
  release.yml, the installers and the update manifest. Adjusted its
  release.yml assertion from a per-asset "cp asset dist/asset" text match to
  a generic "publishes dist/*" check, since the kept closed-world workflow
  stages assets via the provenance script rather than literal per-asset copy
  steps.

Merged the docstring and imports (kept this branch's sys/ElementTree/asdict
for --junit/--json plus master's issue #5 target-triplet note).

Also fixes fallout from the merge:
- tests/test_distribution_consistency.py and tests/unit/conftest.py fixtures
  now include a distribution/targets.json + signed manifest artifact so the
  newly-wired target-triplet check doesn't add spurious findings.
- tests/test_verify_distribution_consistency.py and
  tests/unit/test_verify_distribution_consistency.py (both added by master)
  assumed the older main()/ROOT-mutation/rel() API; updated to the kept
  argparse-based main(argv)/run_audit(root) API and current message wording.
- scripts/bench_verify_distribution_consistency.py called vdc.main() with no
  args, which now parses the caller's own sys.argv; passes argv=[] instead.
- added tests/__init__.py and tests/unit/__init__.py: two same-named test
  modules (tests/test_verify_distribution_consistency.py and
  tests/unit/test_verify_distribution_consistency.py, both pre-existing on
  master) collided under pytest's default import mode when collecting
  tests/ as a whole; this is unrelated to the release-workflow conflict but
  blocked verifying the full suite.

Verified: python -m pytest tests/test_distribution_consistency.py
tests/test_release_provenance.py -v -> 37 passed. Full tests/ -> 81 passed,
7 pre-existing failures in tests/test_quality_helpers.py (untouched by this
merge) caused by a subprocess/stdio handle quirk in this sandbox, unrelated
to the merge.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@wesleysimplicio

Copy link
Copy Markdown
Owner Author

Merge conflict with master resolved

Merged origin/master (up to #15) into this branch and pushed the result (fc10ec1). gh pr view 11 --json mergeable now reports MERGEABLE.

What won and why (per repo owner's decision)

  • .github/workflows/release.yml: kept this branch's workflow_dispatch-only, provenance-script-driven design (scripts/verify_release_provenance.py: state → plan → download → verify-staged → metadata → publish, fail_on_unmatched_files: true, overwrite_files: false). Master's push-triggered, direct-cp-staging release.yml was discarded — releases are no longer supposed to auto-trigger on push.
  • scripts/verify_distribution_consistency.py: kept both validators and wired them into the same run_audit() aggregate:
    • release_workflow_errors() (this branch) — validates release.yml's exact closed-world step contract.
    • check_target_triplet_consistency() (master, issue [P0][Distribution] Instalador único, assets canônicos e update seguro do ecossistema #5) — validates distribution/targets.json as the single naming source of truth across release.yml, both installers, and the update manifest.
    • Merged docstrings/imports (kept sys/ElementTree/asdict for --junit/--json plus master's issue [P0][Distribution] Instalador único, assets canônicos e update seguro do ecossistema #5 target-triplet note).
    • One substantive adjustment: check_target_triplet_consistency's release.yml assertion moved from a per-asset "cp asset dist/asset" text match to a generic "publishes dist/*" check, because the kept closed-world workflow stages assets via the provenance script rather than literal per-asset copy steps in the workflow file itself.

Test fallout from the merge (fixed)

  • tests/test_distribution_consistency.py and tests/unit/conftest.py fixtures didn't include distribution/targets.json / a fully signed manifest artifact, so the newly-wired target-triplet check added spurious findings — fixtures updated.
  • tests/test_verify_distribution_consistency.py and tests/unit/test_verify_distribution_consistency.py (both added by master) assumed the older main()/global-ROOT-mutation/rel() API; updated to the kept argparse-based main(argv) / run_audit(root) API and current message wording.
  • scripts/bench_verify_distribution_consistency.py called vdc.main() with no args, which now parses the caller's own sys.argv under the kept signature — passes argv=[] explicitly instead.
  • Added tests/__init__.py / tests/unit/__init__.py: two identically-named test modules (tests/test_verify_distribution_consistency.py and tests/unit/test_verify_distribution_consistency.py, both pre-existing on master before this merge) collided under pytest's default import mode when collecting tests/ as a whole. Unrelated to the release-workflow conflict, but it blocked verifying the full suite locally, so fixed it too.

Real test evidence

$ python -m pytest tests/test_distribution_consistency.py tests/test_release_provenance.py -v
...
=================== 37 passed, 30 subtests passed in 2.38s ====================
$ python -m pytest tests/ -v
...
============== 7 failed, 81 passed, 30 subtests passed in 12.69s ==============

The 7 failures are all in tests/test_quality_helpers.py (a file untouched by this merge) and are a subprocess/stdio-handle quirk (OSError: [WinError 6] on git ls-files subprocess calls) specific to the sandboxed Windows shell used to run this locally — not a real code regression, and not something introduced by this merge.

CI status — known, unrelated to this PR

Every check on this PR is currently erroring with "The job was not started because your account is locked due to a billing issue." This is a GitHub Actions billing lock on the account, not a test or code failure — confirmed across all jobs. Resolving it requires the account owner to visit https://github.com/settings/billing; nothing in this PR's code or workflow YAML is responsible, and CI will keep failing/not-running until that's addressed.

@wesleysimplicio
wesleysimplicio merged commit f151f0e into master Jul 14, 2026
0 of 34 checks passed
@wesleysimplicio
wesleysimplicio deleted the codex/issue-10-ci-audit branch July 14, 2026 21:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant