Skip to content

Harden portable skill protocol, installer safety, and quality gates - #2

Merged
elkaix merged 4 commits into
mainfrom
hardening/portable-quality-gates
Sep 5, 2026
Merged

elkaix merged 4 commits into
mainfrom
hardening/portable-quality-gates

Conversation

@elkaix

@elkaix elkaix commented Sep 5, 2026

Copy link
Copy Markdown
Member

Scope and delivered state

Review and harden code-max without prescribing an application architecture, language, framework, or agent host. Inspected all nine tracked entries at baseline 3a7bc44658ff91e57d4a0609734832b627e9e4d0.

Four focused commits cover the portable protocol, conflict-safe optional installer, package/test/CI enforcement, and current Node 24-compatible Actions:

  • 5c21e23 refactor(skill): define portable evidence-first engineering gates
  • 73068ec fix(installer): preserve existing entries and require explicit targets
  • 47d4f8a test(quality): gate package integrity and portable utility regressions
  • 9478bf2 ci: pin verified Node 24-compatible action releases

Final head: 9478bf2f2e56732e2f818d2a027738edd6eb61e0.
The local candidate exactly matches the published Git tree: 249db1abeac2d8a6744fb14a2f5052235d057281.

Confirmed defects addressed

  • Reproduced the old ln -sfn installer replacing a regular file and nesting a link inside an existing directory.
  • Reproduced renamed checkouts installing under the wrong skill name.
  • Reproduced old CI accepting unclosed frontmatter.
  • Replaced the explicit prohibition on tests with isolated utility regression requirements.
  • Corrected Codex/Pi discovery presets against primary documentation; added explicit/custom targets and XDG support.

The isolated baseline reproduction verified installer bytes against its Git blob SHA before execution. docs/hardening.md records findings, severity, evidence, sources, trade-offs, and limits.

Implementation

  • 123-line, 9,192-byte architecture-neutral core with directly linked conditional quality gates for contracts, data, concurrency, resource lifecycle, security/privacy, UI/accessibility, performance, supply chain, and operations.
  • Evidence-backed acceptance, prompt defect reporting, preserved user work, small commits, bounded progress, honest COMPLETE/PARTIAL/BLOCKED semantics, migration/rollback, and no fabricated checks.
  • Optional Python-standard-library installer: exclusive link creation, conflict preflight, idempotency, explicit host/custom selection, write-free dry-run, owned-link uninstall, and actionable failures.
  • Offline metadata/path/link/context-budget validator with deliberately broken negative controls.
  • 30 utility regression tests and 16 behavioral scenario definitions. The latter are explicitly NOT model-executed.
  • Linux/Python 3.10 and macOS/Python 3.13 CI; retain the existing required docs aggregate. SHA-pinned checkout v7.0.1 and setup-python v7.0.0, read-only contents, disabled persisted checkout credentials, timeouts, and concurrency cancellation.
  • Updated ownership/contribution guidance, dependency-update configuration, installation/migration docs, and evidence-report template.

Observed verification

Final-head CI run 33936477669 completed successfully:

Gate Observed result
Linux / Python 3.10 package validation, utility tests, Bash syntax PASS
Linux ShellCheck PASS
macOS / Python 3.13 package validation, utility tests, Bash syntax PASS
Required aggregate docs check PASS

Local environment: Linux, Bash 5.2, Python 3.13.5.

  • python3 -B scripts/validate.py: PASS, 10 Markdown documents and 16 scenario definitions.
  • python3 -B -m unittest discover -s tests -v: PASS, 30 tests after final implementation edits.
  • bash -n skills.sh: PASS.
  • Python AST parsing with Python 3.10 grammar: PASS for all five Python files; actual Python 3.10 execution is separately established by CI.
  • git diff --cached --check: PASS.
  • git write-tree: exact match with the published final tree.

The fourth commit changed only the two Action pins after initial CI exposed deprecated Node 20 declarations; the full matrix then passed again on the final head. ShellCheck was unavailable locally but passed remotely. No independent reviewer or real model/host behavioral evaluation was performed. Utility tests do not establish model compliance or universal correctness. Windows-native installation remains unvalidated.

Migration, observability, and rollback

The skill remains instruction-only. Optional installer/validation tooling needs Python 3.10+; no third-party Python packages, hooks, network calls, or application framework were added.

Intentional CLI safety change: bare ./skills.sh exits without writes. Select --agent, an absolute parent --target, or explicitly --all. Old Codex/Pi links are not automatically deleted. Dry-run and owned-link uninstall provide an explicit adoption/reversal path.

Each operation reports its result. Preflight is not a multi-target transaction: a later I/O failure can leave earlier reported operations completed. Use trusted, user-owned directories; this is not protection against hostile concurrent filesystem replacement. Retry idempotently or remove only successful links to the same checkout. Use a clean reviewed checkout for version rollback; never reset user-owned changes.

No deployment, database migration, or branch-protection changes. License, artwork, historical research, and the CLAUDE.md symlink remain unchanged. This PR is open for review; main has not been changed by this work.

@coderabbitai

coderabbitai Bot commented Sep 5, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Team

Run ID: 6ee69dab-a082-431b-9cbd-f763fd7589bb

📥 Commits

Reviewing files that changed from the base of the PR and between 3a7bc44 and 9478bf2.

📒 Files selected for processing (19)
  • .editorconfig
  • .github/dependabot.yml
  • .github/pull_request_template.md
  • .github/workflows/ci.yml
  • .gitignore
  • AGENTS.md
  • README.md
  • SKILL.md
  • docs/hardening.md
  • evals/README.md
  • evals/scenarios.json
  • references/quality-gates.md
  • references/report-template.md
  • scripts/install.py
  • scripts/skill_meta.py
  • scripts/validate.py
  • skills.sh
  • tests/test_install.py
  • tests/test_validate.py

Included review availability: 1 review is currently available. Your included PR review attempts over the past 7 days set your current allowance at 3 reviews per hour.


📝 Walkthrough

Walkthrough

The repository adds a Python installer, package validator, evidence-based skill guidance, evaluation scenarios, documentation, regression tests, and CI controls. The shell entry point now delegates installation to Python.

Changes

Repository hardening

Layer / File(s) Summary
Skill protocol and evidence contracts
SKILL.md, AGENTS.md, docs/hardening.md, evals/*, references/*
Defines engineering workflow, evidence handling, quality gates, reporting states, evaluation scenarios, and hardening records.
Package metadata and validation
scripts/skill_meta.py, scripts/validate.py, tests/test_validate.py
Validates metadata, required files, links, symlinks, size limits, and scenario structure with regression coverage.
Safe skill installation
scripts/install.py, skills.sh, tests/test_install.py
Adds preset and custom-target installation, dry runs, uninstall support, ownership checks, conflict preservation, and filesystem safety validation.
Installation and repository documentation
README.md
Documents installation, migration, usage, validation, limitations, contribution steps, and rollback guidance.
CI and repository governance
.github/*, .editorconfig, .gitignore
Adds repository defaults, Dependabot configuration, a pull request template, and matrix-based CI checks for validation, tests, Bash syntax, and ShellCheck.

Estimated code review effort: 4 (Complex) | ~45 minutes

Merge Risk: ⚪ Minimal · up to 9478b

No unresolved material merge risk remains.

Sequence Diagram(s)

sequenceDiagram
  participant Developer
  participant skills.sh
  participant install.py
  participant Filesystem
  Developer->>skills.sh: run installation command
  skills.sh->>install.py: validate Python and forward arguments
  install.py->>Filesystem: validate metadata and destinations
  install.py->>Filesystem: create or remove owned links
  Filesystem-->>Developer: return operation status
Loading
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 2.04% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 49 functions across 6 files. (13 skipped: … Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely summarizes the main changes: hardening the skill protocol, improving installer safety, and adding quality gates.
Full details: Docstring Coverage

Explanation

Docstring coverage is 2.04% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 49 functions across 6 files. (13 skipped: 13 unsupported.)

  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch hardening/portable-quality-gates

Comment @coderabbitai help to get the list of available commands.

@elkaix
elkaix merged commit 32b3dc4 into main Sep 5, 2026
4 checks passed
@elkaix
elkaix deleted the hardening/portable-quality-gates branch September 5, 2026 01:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant