Skip to content

release: SkillSpector 2.12.0 - #550

Open
mohgupta-ship-it wants to merge 39 commits into
mainfrom
codex/release-2.11.3
Open

mohgupta-ship-it wants to merge 39 commits into
mainfrom
codex/release-2.11.3

Conversation

@mohgupta-ship-it

@mohgupta-ship-it mohgupta-ship-it commented Sep 15, 2026

Copy link
Copy Markdown
Member

Prepare SkillSpector 2.12.0 from the published v2.11.2 release by updating the package version, lockfile, changelog, and versioned release notes.

This remains a minor release because it adds backward-compatible user-facing controls (--fail-on-findings and a configurable per-artifact static-analysis allowance), OpenCode integrations, Gemini model metadata, and interactive scan progress alongside security, completeness, reporting, reliability, performance, and false-positive fixes. Findings, scores, recommendations, completeness metadata, output routing, and opt-in exit behavior can change after upgrading.

Included changes

New capabilities and CLI UX

Security, completeness, and finding integrity

Accuracy, output, and performance

Maintenance and documentation

Compatibility

  • Interactive non-verbose terminal scans now show progress and discovered files on stderr. Machine-readable stdout stays clean; --verbose retains the non-streamed diagnostic path.
  • GitHub/GitLab /blob/ inputs now scan raw bytes, so findings and recommendations can change from the prior incorrect forge-HTML result.
  • Active hooks with unmodeled payload flows now become incomplete/non-SAFE; identifier-adjacent letter spacing can lose false P3/P4 findings while retaining AE6.
  • opencode_cli and the OpenCode-native integration are opt-in. The provider requires an installed, authenticated OpenCode 1.18.30 executable; other versions fail closed. Install the native integration by copying .opencode/ from a checkout (the wheel does not install it).
  • Existing CLI providers now honor valid SKILLSPECTOR_MODEL_REGISTRY budgets; malformed values warn and retain fallback behavior.
  • The new CLI flag and configuration setting are opt-in; no new configuration is required.
  • Third-party dependency versions are unchanged from v2.11.2.
  • Explicitly use --no-llm for an intended static-only scan. Requested but unavailable or incomplete semantic analysis remains incomplete and can affect strict CLI/MCP decisions.

Candidate and validation

  • The changelog and versioned notes cover all 38 merged PRs in v2.11.2..548e5e0afd25595ef039c27cdeb283413c282c71.
  • Main baseline: 548e5e0afd25595ef039c27cdeb283413c282c71 (fix(hooks): fail closed on unmodeled payload analysis #573); the release branch is zero commits behind.
  • Release head: 3dd2defa4beee58b9342fcdad2f0fe0e1a7add64.
  • Python 3.12 make test-ci: 5,744 passed, 14 skipped, 39 deselected, 4 expected failures; 90% coverage.
  • Affected CLI, model-registry, input, security-view, hook-completeness, and release-helper suites: 1,177 passed, 10 skipped, 1 deselected.
  • OpenCode TypeScript suite: 44 passed.
  • Lock verification, skillspector --version, release-helper dry run, Ruff lint/format, package build, Twine checks, and git diff --check: pass.
  • Exact-head hosted CI: all six checks passed — change detection, lint, unit tests, OpenCode TypeScript tests, DCO, and Docker smoke (run 35267927075).

The current PR head and its GitHub checks are the source of truth for final approval. Older candidate hashes, artifact identities, test counts, and hosted runs are historical and are intentionally not treated as certification of this candidate.

Known limitations and release status

  • Public report serialization can reuse representative occurrence columns; SARIF does not preserve columns.
  • Instrumented AST fallback can raise TypeError when an optional end column is absent, although normal parsed files pass.
  • Letter-spacing reconstruction is not fail-closed. Newlines between letter-spaced characters and some alternating-width short runs can evade P3/P4 and AE6.
  • Companion-context classification is not fail-closed. PE3 and RA1 variants can still be incorrectly downgraded or missed.
  • Recursive root exclusions lack public scope-exclusion metadata.
  • Recursive Markdown stdout still emits only a summary unless --output is used; recursive JSON and SARIF now emit their merged documents.
  • Consumer summary grouping can merge distinct fingerprints or same-line occurrences even though raw findings remain present.
  • Raw and normalized views can duplicate a P9 finding at one source span, and consumer archive-path handling can replace original finding detail with an execution failure; existing gates still block these cases.
  • opencode_cli depends on the installed CLI's authentication, exact 1.18.30 behavior, model availability, and external rate limits; token-usage accounting is not currently available, and unsupported output fails closed. Multilingual batch gap-fill was validated with mocked providers; live-provider qualification remains pending.
  • Runtime-selected command and Markdown reference completeness fixes in open PRs fix: keep runtime-selected printf reconstruction incomplete #514 and fix: preserve complete Markdown reference destinations #553 are not included.
  • Deployment/provider validation, Windows validation, published-artifact validation, downstream adoption, and independent release sign-off remain separate release gates.

Publication is pending required review and the outstanding release gates. No 2.12.0 release has been published.

Signed-off-by: Mohit Gupta <mohgupta@nvidia.com>
@mohgupta-ship-it mohgupta-ship-it added the release:publish Marks an OSS release pull request for publication label Sep 15, 2026
Comment thread docs/release/skillspector-2.12.0.md
@rng1995
rng1995 marked this pull request as ready for review September 16, 2026 16:21
Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
@rng1995 rng1995 changed the title release: SkillSpector 2.11.3 release: SkillSpector 2.12.0 Sep 16, 2026
Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
@mohgupta-ship-it

Copy link
Copy Markdown
Member Author

Codex on behalf of Mohit Gupta — independent qualification update, 16 September 2026.

The freshly built 2.12.0 wheel from 65ca9bdd289e70ae6e413c205c7530c86e6b39aa has SHA256 bcb03b0aae2266aff57bac6e3fd0a5c51e83b64f866c79193c7862912aed877d. All 98 installed Python modules and 111 package files match that immutable source. The later 5134f62458fa235b487e823d1c0472be54678295 revision changes only the changelog/release notes: all package files, dependencies and build inputs match, so the runtime evidence applies by source equivalence. Artifact hashes remain tied to the actual built bytes.

Fresh results after the AS3 update:

Check Result
Focused scanner/release-helper tests 283 passed
Exact installed CLI/MCP and AS3 controls 9 passed
Wheel/sdist build and Twine Passed
Frozen 54-case edge corpus 39 passed / 15 failed
Frozen 32-case golden corpus 27 passed / 5 failed
Expanded downstream consumer 1,156 passed / 1 failed: existing PE3 read-only Docker identity-mount expectation
Real renderer 29 passed
Network-disabled downstream CI sanity 29/29 commands passed; 281 Python passes, 3 existing runtime-dependent skips

Both frozen corpora retain exactly the previous failure sets. Their behavioral expectations and fixture bytes were unchanged; only candidate identity/version provenance was refreshed. The consumer policy/limit assertions were retained. An initial provider-enabled consumer harness attempt was terminated and excluded; the reported counts come from the subsequent run with the established sanitized environment.

Draft #563 separately fixes unsupported-primary and pure/mixed newline completeness gaps. At c6aa326, it passes 5,292 required tests, 90% coverage, lint/format, packaging and local Docker smoke; the installed edge corpus improves to 43/54, with no new failures, and 18/18 CLI/MCP coexistence controls pass. Scratch integration with #514 at 63658bc and #553 at 18c897a passed 1,257 focused tests after resolving one small adjacency conflict. None of these open fixes is included in this release PR.

Current-head hosted CI/review and the remaining production/provider, Windows, published-artifact and adoption gates are still required. These results do not constitute release sign-off; no release or production pin was changed.

Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SkillSpector Review]

Reviewed current head e6bcf1044b781d9a28ae537cd8a56147edc9afa4, including the complete version/changelog/release-note diff, candidate history, existing review thread, version/lock consistency, component source identity, exact-head checks, and the stated release gates. The package and lock versions agree and all five hosted checks pass.

Changes are requested because two security guarantees in the release notes are not true of this exact candidate. Its letter-spacing implementation is byte-identical to the currently reviewed #470 implementation and still permits alternating-width spacing to evade P3/P4 and AE6. Its companion-classification analyzer blobs are identical to the unresolved #547 implementation, where four contextual fail-open paths remain. Either correct and requalify those implementations before publication or describe the precise limitations without claiming the risky cases remain covered. Deployment/provider, Windows, published-artifact, downstream-adoption, and independent release sign-off also remain merge gates.

Comment thread docs/release/skillspector-2.12.0.md Outdated
Comment thread docs/release/skillspector-2.12.0.md Outdated

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SkillSpector Review]

Re-reviewed current head ff10467cd93832bc5a9a0ee8c07a2bcd3b91e01f after the automatic synchronization with main. This merge has the previously reviewed head e6bcf1044b781d9a28ae537cd8a56147edc9afa4 and current main e19ec01cc85b9891b2976a1157fce492504cfc13 as its exact parents. The only intervening tree changes are #558's three batch-scan files; none overlaps the four release files, whose reviewed contents are unchanged.

The two existing non-outdated release-note findings therefore remain current and unresolved. The candidate still claims irregular letter spacing fails closed although alternating-width short runs can evade P3/P4 and AE6, and it still makes an unconditional companion-context guarantee despite #547's unresolved PE3 and RA1 paths. Correct and requalify those implementations or accurately document the limitations. I have not duplicated the inline findings.

Exact-head CI run 35136927518 concluded action_required without starting jobs, leaving no check results. The security-release corrections, unresolved threads, active change request, missing exact-head checks, GitHub BLOCKED state, and previously identified release qualification/sign-off gates prevent merge.

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed exact head 2b39002 after reconciling the release catalog through main 4d52048. CHANGELOG.md, the 2.12.0 versioned notes, and the PR description now cover all 31 merged PRs since v2.11.2, including the 13 recent main merges. The letter-spacing and companion-context limitations remain explicit, and all review threads are resolved. Local lock/version checks, 10 release-helper/workflow tests, and git diff validation pass. All six exact-head CI jobs pass in run 35146009149. Approved; the deployment/provider, Windows, published-artifact, downstream-adoption, and remaining release-sign-off gates recorded in the PR still apply.

SanHsien added a commit to SanHsien/SkillSpector that referenced this pull request Sep 17, 2026
Brings the fork up to upstream main c13f70e; the version is still 2.11.2.

The fork history was squashed into one commit on 2026-09-13, so it
shares no merge-base with upstream and git merge refuses. The range
diff was applied with git apply -3 instead. The fork content equals
69dcdfb plus the registered divergences, so conflicts landed only on
those seven files; the other 116 applied cleanly. FORK.md now documents
this procedure.

Divergences, resolved by each row's rule:
- static_runner.py takes upstream NVIDIA#522
  (SKILLSPECTOR_MAX_STATIC_ANALYSIS_SECONDS_PER_ARTIFACT, default 300s).
  The fork's SKILLSPECTOR_MAX_STATIC_SECONDS override and its seven
  tests are removed. Downstream gates must use the upstream name when
  their pin moves.
- test_static_yara.py, test_build_context.py and test_input_handler.py
  take upstream (NVIDIA#501-NVIDIA#505, NVIDIA#518 fix the same Windows issues); 301
  passed on Windows, rows deleted.
- test_security_end_to_end.py: upstream's version still fails
  nine_case on Windows (YARA load and SC8 budgets stay hard-coded), so
  the relaxation helper is re-applied on top; row kept and rewritten.
- .gitignore keeps the fork block; README.md stays Traditional Chinese
  and the upstream README goes to README.en.md.

Two new Windows divergences from new upstream tests:
- tests/unit/test_cli.py: a file name containing a backslash is split
  into two path parts on Windows; skipped by a capability probe added to
  tests/platform_support.py.
- test_json_container_ownership.py: oversized payloads became test ids,
  which pytest copies into PYTEST_CURRENT_TEST, over Windows' 32,767
  character environment limit; short ids added, content unchanged.

Triage: 13 of PRs NVIDIA#528-NVIDIA#580 merged into upstream main and arrive here;
29 stay open (including NVIDIA#550, release 2.12.0). Upstream closed this
fork's PR NVIDIA#486 on 2026-09-15; NVIDIA#490 builds on it and is open.

Verified on Windows in fresh-process batches against this tree:
tests/unit 1563 passed, 29 skipped; tests/nodes 3822 passed, 11
skipped, 4 xfailed (plus test_json_container_ownership 71 passed after
the id fix); remaining tests 182 passed, 16 skipped;
test_security_end_to_end.py 98 passed. ruff check and format clean,
check_divergence OK (10 diverging, 10 registered), check_pin_bounds OK.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: SanHsien <34234698+SanHsien@users.noreply.github.com>

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SkillSpector Review]

Re-reviewed current head 39d584c543028e77b9ed5bb35f92915d3dea50eb against the prior release findings, resolved threads, complete four-file release diff, current main ancestry, included PR states, version/lock consistency, and hosted checks.

The two prior security-claim findings are resolved: the letter-spacing and companion-classifier limitations are now stated explicitly without claiming fail-closed coverage.

A new exact-head release drift must be corrected. This head's second parent is current main 8028ce5d0db6ea2500a856252f82474f2103bf2b, which includes the P0 local-input security fix #562 (c3bb8132fdba) after the documented validation baseline 548e5e0afd25. However, #562 is absent from CHANGELOG.md and the release notes, and docs/release/skillspector-2.12.0.md still says the candidate was synchronized and validated only through 548e5e0. Add the included security fix to the changelog/release notes, update the exact candidate baseline, and rerun/record validation for the actual current head.

GitHub reports no checks for this exact head and mergeStateStatus=BLOCKED. The documentation/validation drift, missing exact-head checks, and remaining release qualification gates prevent merge.

Priority: P0 — release integrity and security-fix traceability require owner attention before publication.

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SkillSpector Review]

Re-reviewed current head 09c3613e06eb2bfd6cdd6e4ce27a183330ee6b6d after the final automatic synchronization with settled main d162d9b343e559be13df8ebba093df3bc9d58c90. I verified the merge parentage, complete four-file release diff, prior findings/threads, version/lock consistency, and the post-baseline main history.

The earlier letter-spacing and companion-classifier claims remain correctly qualified. The release-integrity finding remains open and has widened: this exact candidate now incorporates the post-validation merges for #562, #526, #421, #491, #517, #571, and #575 (plus documentation cleanup), while the changelog/release notes still claim validation only through 548e5e0 and omit these included changes. Update the candidate inventory and security/behavior notes, record the true exact baseline, and rerun/record release validation for this head.

GitHub currently reports no checks for this exact head and mergeStateStatus=BLOCKED. The stale release inventory/validation, missing exact-head CI, and remaining release qualification gates prevent merge.

Priority: P0 — release traceability and validation must describe the code actually being published.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release:publish Marks an OSS release pull request for publication

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants