From 24c1eecffed0b6905767d18bdc1d2f4b0f67da23 Mon Sep 17 00:00:00 2001 From: Torsten Mahr Date: Tue, 22 Sep 2026 10:29:35 +0200 Subject: [PATCH] Reassess against Repository Quality Standard 1.21.0 Up from v1.6.0. State moves from Healthy to Needs work: not a regression, but the latest published release (v1.2.0) predates a source commit that added product-identity metadata (repository URL, issue-tracker URL, licence identifier) to Info.plist. I02 and I03 now fail and R01/I04 are partial, all for the same reason: a value that exists only in source is not evidence the artifact carries it, and the artifact this assessment read does not carry it yet. The build mechanism itself is correct and verified against both a local Debug build and the published v1.2.0 zip; the fix is a new release, which is a maintainer decision this assessment does not make. Also updates for the fourteen criteria added since v1.6.0 (B14-B16, P10-P13, R07-R09, S11-S13, W09), three of which needed the fixes made in #38: B14, R08, W09. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01CdiwBVuH6DCEtFPPwxbXKc --- .github/badges/conformance.svg | 10 ++-- .github/conformance.yml | 32 ++++++++--- docs/self-assessment.md | 100 ++++++++++++++++++++++++++------- 3 files changed, 107 insertions(+), 35 deletions(-) diff --git a/.github/badges/conformance.svg b/.github/badges/conformance.svg index f874123..9133f5c 100644 --- a/.github/badges/conformance.svg +++ b/.github/badges/conformance.svg @@ -1,10 +1,10 @@ - - trsdn standard: v1.6.0 - Healthy + + trsdn standard: v1.21.0 - Needs work - - + + trsdn standard - v1.6.0 - Healthy + v1.21.0 - Needs work diff --git a/.github/conformance.yml b/.github/conformance.yml index 56f7f93..b1a01fc 100644 --- a/.github/conformance.yml +++ b/.github/conformance.yml @@ -1,6 +1,6 @@ -standard_version: "1.6.0" -assessed_on: "2026-08-31" -state: "Healthy" +standard_version: "1.21.0" +assessed_on: "2026-09-22" +state: "Needs work" evidence: "docs/self-assessment.md" criteria: @@ -17,6 +17,9 @@ criteria: B11: pass B12: pass B13: pass + B14: pass + B15: na + B16: pass P01: pass P02: pass P03: pass @@ -26,6 +29,10 @@ criteria: P07: pass P08: pass P09: pass + P10: pass + P11: pass + P12: pass + P13: pass S01: pass S02: pass S03: pass @@ -36,22 +43,28 @@ criteria: S08: pass S09: pass S10: pass + S11: pass + S12: pass + S13: na D01: na D02: na D03: na D04: na D05: na D06: na - R01: pass + R01: partial R02: pass R03: pass R04: pass R05: pass R06: pass + R07: pass + R08: pass + R09: pass I01: pass - I02: pass - I03: pass - I04: pass + I02: fail + I03: fail + I04: partial I05: pass I06: pass T01: na @@ -63,10 +76,11 @@ criteria: W02: pass W03: pass W04: pass - W05: pass - W06: pass + W05: na + W06: na W07: pass W08: pass + W09: pass G01: pass G02: pass G03: pass diff --git a/docs/self-assessment.md b/docs/self-assessment.md index cc567b5..3dcfa90 100644 --- a/docs/self-assessment.md +++ b/docs/self-assessment.md @@ -4,19 +4,28 @@ Per-criterion evidence for [`.github/conformance.yml`](../.github/conformance.ym | Field | Value | |---|---| -| Standard | [Repository Quality Standard](https://github.com/trsdn/.github/blob/v1.6.0/docs/repository-quality-standard.md) **v1.6.0** | -| Assessed on | 2026-08-31 | -| State | **Healthy** | -| Results | 74 pass · 0 partial · 0 fail · 19 n/a | +| Standard | [Repository Quality Standard](https://github.com/trsdn/.github/blob/v1.21.0/docs/repository-quality-standard.md) **v1.21.0** | +| Assessed on | 2026-09-22 | +| State | **Needs work** | +| Results | 80 pass · 2 partial · 2 fail · 23 n/a | Remediation was tracked in [#32](https://github.com/trsdn/PtionsPlus/issues/32), which followed the first assessment in [#29](https://github.com/trsdn/PtionsPlus/issues/29). -Reassessed against v1.6.0 on 2026-08-31. The only change is `P08`, which moved -from `partial` to `pass` because the standard's badge rule changed, not because -anything in this repository was relaxed. See -[Badge hosting](#badge-hosting-p08) for the reasoning. +Reassessed against v1.21.0 on 2026-09-22, jumping from v1.6.0. Fourteen +criteria are new since v1.6.0 (`B14`-`B16`, `P10`-`P13`, `R07`-`R09`, +`S11`-`S13`, `W09`); this assessment reads every criterion in the current +standard, not a diff. Three of the new ones needed a change, made in the same +pull request as this record: `B14` (a credential-exposure statement was +missing from `AGENTS.md`), `R08` (README did not tell a consumer what to check +or how), and `W09` (the site carried the vendored design language's default +palette with no project-specific override). `I02` and `I03` newly fail, not +because anything regressed, but because the latest published release, +`v1.2.0`, predates the commit that added the product-identity metadata these +criteria ask for; see [Product Identity](#product-identity). `P08` stays at +`pass` for the reason the v1.6.0 reassessment recorded; see +[Badge hosting](#badge-hosting-p08). ## Profiles @@ -49,6 +58,9 @@ Not applicable, with rationale: | B11 | pass | This record and [`.github/conformance.yml`](../.github/conformance.yml), validated in CI by [`.github/workflows/conformance.yml`](../.github/workflows/conformance.yml). | | B12 | pass | `trsdn-standard` topic. | | B13 | pass | `AGENTS.md` carries a documentation-boundaries table naming one home per fact. Commands and architecture moved out of the README into `AGENTS.md`; `CLAUDE.md` and `.github/copilot-instructions.md` are pointers, enforced by `scripts/check-documentation.sh`. | +| B14 | pass | `AGENTS.md` **Secrets and signing** names each of the five release secrets (`MACOS_CERTIFICATE`, `MACOS_CERTIFICATE_PWD`, `APPLE_ID`, `APPLE_TEAM_ID`, `APPLE_APP_PASSWORD`) and states what replaces each one and where, added in this reassessment. | +| B15 | na | README **Requirements** and `AGENTS.md` state Ptions+ has no third-party dependencies; nothing is redistributed. | +| B16 | pass | `gh api repos/trsdn/PtionsPlus/branches/main/protection` shows both force pushes and deletion blocked. | ## Public @@ -63,6 +75,10 @@ Not applicable, with rationale: | P07 | pass | Description, 14 topics, and a maintained homepage. | | P08 | pass | Required order — licence, platform, CI, release, conformance — with every badge linking to what it reports. CI uses GitHub's own workflow badge endpoint, the first-party live image the standard names. Conformance is a committed image, regenerated from the record by a repository event. Licence, platform, and release have no first-party image source and are served live by `img.shields.io`, which v1.6.0 records as a `Pass`. The hardcoded `Swift 5.9` badge this criterion exists to prevent was removed in the previous assessment. Reasoning in [Badge hosting](#badge-hosting-p08). | | P09 | pass | [`.github/stats/repo-card.svg`](../.github/stats/repo-card.svg) and `repo-card-dark.svg`, generated by [`repo-stats.yml`](../.github/workflows/repo-stats.yml) on a weekly schedule, committed, selected with a `` element, and self-contained — the only `url()` in either file is an internal filter reference. | +| P10 | pass | [`bug.yml`](../.github/ISSUE_TEMPLATE/bug.yml) collects expected behaviour, actual behaviour, numbered reproduction steps, the Ptions+ version, and the macOS version, plus mouse model and vendor-software fields specific to this product's failure modes. | +| P11 | pass | Inherited [`pull_request_template.md`](https://github.com/trsdn/.github/blob/main/.github/pull_request_template.md) covers what the change does, how it was validated, its risk and compatibility impact, and the related issue. | +| P12 | pass | `gh api repos/trsdn/PtionsPlus/vulnerability-alerts` and `automated-security-fixes` both enabled. | +| P13 | pass | CodeQL default setup, `state: active`, read from `gh api repos/trsdn/PtionsPlus/code-scanning/default-setup`. | ## Software @@ -78,28 +94,55 @@ Not applicable, with rationale: | S08 | pass | Dependabot covers GitHub Actions weekly; Dependabot security updates enabled; `@trsdn` owns triage through `CODEOWNERS`. | | S09 | pass | Two required checks protect `main`; both exist and pass. | | S10 | pass | `AGENTS.md` documents the architecture, the runtime invariant, and the product constraints; the README documents the event flow. | +| S11 | pass | Every workflow (`ci.yml`, `conformance.yml`, `release.yml`, `repo-stats.yml`) declares a `permissions` block, at the top level or on every job, no broader than the job's work. | +| S12 | pass | All 15 `uses:` references in the four workflows are pinned to a commit SHA (third-party actions) or reference a workflow within the account, satisfying the graduated table. | +| S13 | na | No workflow uses `pull_request_target` or `workflow_run`. | ## Package And Release | ID | Result | Evidence | |---|---|---| -| R01 | pass | `Info.plist` and the build settings agree with the repository metadata; verified by `scripts/verify-version.sh`. | +| R01 | partial | Name (`CFBundleName` `Ptions+`) and version (`CFBundleShortVersionString` `1.2.0`) are present in the latest published artifact and agree with the repository. Licence and repository URL are not: see `I02`/`I03` below, the same finding. `scripts/verify-version.sh` will enforce all of it starting with the next release. | | R02 | pass | README **Versioning and compatibility** states semantic versioning and what each bump means for the `config.json` schema, including that downgrade is unsupported across a major. | | R03 | pass | A `v*` tag runs [`release.yml`](../.github/workflows/release.yml), which builds, signs, notarises, staples, and uploads a ZIP, a DMG, and a SHA-256 file. | | R04 | pass | `scripts/verify-version.sh --tag` rejects a tag that does not match `MARKETING_VERSION`. | -| R05 | pass | `scripts/verify-release-artifacts.sh` rebuilds the ZIP and DMG from the stapled app and verifies them before upload; the release workflow reruns the full CI suite against the tagged commit first. | +| R05 | pass | `scripts/verify-release-artifacts.sh` rebuilds the ZIP and DMG from the stapled app and verifies them before upload; the release workflow reruns the full CI suite against the tagged commit first. This reassessment additionally downloaded the published `v1.2.0` `Ptions+.dmg` and `Ptions+.zip`, checked the DMG against its published `.sha256`, and ran `codesign --verify --deep --strict`, `xcrun stapler validate`, and `spctl --assess --type execute` against the unpacked app: all passed (`accepted`, `source=Notarized Developer ID`). Neither artifact was launched. | | R06 | pass | The release workflow gates on `scripts/changelog.sh release-notes` and publishes the `CHANGELOG.md` section of the tagged version as the release body, so a release cannot ship without described changes. | +| R07 | pass | [`release.yml`](../.github/workflows/release.yml) `Verify changelog and build release notes` reads `CHANGELOG.md` at the tagged commit and fails the release when the tag's section is missing or empty; the same extracted text becomes the published release body, so the two cannot disagree. | +| R08 | pass | README **Install** now states the mechanism (Developer ID signature plus Apple notarisation) and gives the two commands a consumer runs to check it, `codesign --verify` and `spctl --assess`, added in this reassessment. Verified directly against the `v1.2.0` artifact under `R05`. | +| R09 | pass | Secret scan: GitHub secret scanning is enabled and `gh api repos/trsdn/PtionsPlus/secret-scanning/alerts?state=open` returns none. Dependency check: `gh api repos/trsdn/PtionsPlus/dependabot/alerts?state=open` returns none, and Dependabot covers the only third-party surface (GitHub Actions); the app itself has no dependencies. | ## Product Identity +**Finding.** `PtionsPlus/Info.plist` gained `TRSRepositoryURL`, `TRSIssuesURL`, and +`TRSLicenseIdentifier` in commit `a5e9a25` (2026-08-31 13:22 CEST). The latest +published release, `v1.2.0`, was tagged at 11:25 CEST the same day — before that +commit — and `a5e9a25` is not an ancestor of the `v1.2.0` tag. This assessment +downloaded the `v1.2.0` `Ptions+.zip`, unpacked it without launching it, and read +`Contents/Info.plist` with `plutil -p`: it carries `CFBundleName`, +`CFBundleShortVersionString` (`1.2.0`), `CFBundleIdentifier`, and +`NSHumanReadableCopyright`, but none of the three `TRS*` keys, and +`Contents/Resources/` carries no bundled licence text. A local `Debug` build from +the current `main` (`scripts/validate.sh --fast`, `xcb -configuration Debug +build`) was checked the same way and does carry all three keys, confirming the +Info.plist template and build settings work; nothing is broken. This is a +released-version gap, not a broken pipeline: `scripts/verify-version.sh --app` +already fails a release whose archived app is missing any of these keys, so the +next tagged release will carry them or fail its own gate. Per +[Product Identity](https://github.com/trsdn/.github/blob/v1.21.0/docs/repository-quality-standard.md#product-identity), +"a value that appears only in source is not evidence that the artifact carries +it," so `I02` and `I03` are recorded against the artifact as published, not the +source. **Remediation is cutting a new release**, which is a maintainer +decision this assessment does not make. + | ID | Result | Evidence | |---|---|---| -| I01 | pass | `CFBundleShortVersionString` `1.2.0` and `CFBundleVersion` `7` in the built bundle, from `MARKETING_VERSION` and `CURRENT_PROJECT_VERSION`. | -| I02 | pass | `TRSRepositoryURL` and `TRSIssuesURL` in `Info.plist`, confirmed present in the built bundle with `plutil -p`. | -| I03 | pass | `NSHumanReadableCopyright` set from the build settings, `TRSLicenseIdentifier` `MIT`, and the full `LICENSE` text bundled at `Contents/Resources/LICENSE`. | -| I04 | pass | About window, reachable from the menu bar dropdown, showing version and build and linking to the repository and the issue tracker. [`AboutView.swift`](../PtionsPlus/Views/AboutView.swift) reads every value from the bundle through [`ProductIdentity.swift`](../PtionsPlus/Utilities/ProductIdentity.swift) rather than hardcoding it. | -| I05 | pass | `AppIcon.icns` embedded in the bundle; the same mark is used on the published site and as the favicon. | -| I06 | pass | `scripts/bump-version.sh` owns the version in the project file and every version marker on the site. `scripts/verify-version.sh` fails when the built bundle is missing any identity key, when a value drifts from the source plist, when the licence is not bundled, or — in CI, where `GITHUB_REPOSITORY` is set — when the embedded repository and issue URLs disagree with the repository the workflow is actually running in. All four failure paths were exercised with negative tests. | +| I01 | pass | `CFBundleShortVersionString` `1.2.0` in the `v1.2.0` artifact, read directly, matching the tag. | +| I02 | fail | No `TRSRepositoryURL` or `TRSIssuesURL`, or any equivalent key, in the `v1.2.0` artifact's `Info.plist`. Present in source since `a5e9a25`, which postdates the tag. See the finding above. | +| I03 | fail | No `TRSLicenseIdentifier` or `NSHumanReadableCopyright`-equivalent licence identifier, and no bundled licence text, in the `v1.2.0` artifact. Present in source since `a5e9a25`, which postdates the tag. See the finding above. | +| I04 | partial | [`AboutView.swift`](../PtionsPlus/Views/AboutView.swift) reads every value from the bundle through [`ProductIdentity.swift`](../PtionsPlus/Utilities/ProductIdentity.swift) rather than hardcoding it, so the About window is correctly wired in source. But `ProductIdentity.swift` reads `TRSRepositoryURL`/`TRSIssuesURL` from the bundle, which the shipped `v1.2.0` binary does not carry, so the About window in the app a user actually downloads today shows no repository or issue-tracker link until the next release. Not run; read from source, per `I04`'s own rule for an interface the assessor does not operate. | +| I05 | pass | `AppIcon.icns` embedded in the `v1.2.0` bundle (confirmed present in `Contents/Resources/`); the same mark is used on the published site and as the favicon. | +| I06 | pass | `scripts/bump-version.sh` owns the version in the project file and every version marker on the site. `scripts/verify-version.sh --app` fails a release whose archived app is missing any identity key, drifts from the source plist, ships without a bundled licence, or — in CI, where `GITHUB_REPOSITORY` is set — disagrees with the repository the workflow is running in. The one value the current artifact does carry, the version, is correctly build-derived; the mechanism for the rest is in place and gates the next release, which is what `I06` asks for. | ## Published Site @@ -111,10 +154,11 @@ Published at from `main` at `/docs`. | W02 | pass | Repository homepage points at the site; the site footer links to the repository, licence, security policy, support, issues, and changelog. | | W03 | pass | The landing view opens with the name, a one-sentence statement of what Ptions+ is, the maintained status, the version, and the download action, all above the fold. | | W04 | pass | Name and one-sentence statement; status and version; screenshot; download path; the `Y01` disclosure in its own Privacy section; links to repository, licence, security policy, and support; and a `Page last reviewed` date, which `scripts/bump-version.sh` refreshes on every release. | -| W05 | pass | Instrument Workshop, vendored into [`docs/assets/`](../docs/assets/), replacing roughly 1,050 lines of bespoke CSS. | -| W06 | pass | Version **v1.5.0** recorded in [`docs/assets/VENDORED.md`](../docs/assets/VENDORED.md) and in the header of each vendored file, with the re-vendoring procedure and a "no deviations" statement. | -| W07 | pass | The Google Fonts `preconnect` hints and stylesheet were removed and IBM Plex is self-hosted from `docs/assets/fonts/`. A network review of the rendered page recorded 11 requests, all same-origin. `scripts/check-documentation.sh` now fails the build if any resource-loading attribute or vendored stylesheet reintroduces a remote host; the guard was confirmed with a negative test. | +| W05 | na | Retired 2026-09-17 (standard 1.12.0); see [decision 0013](https://github.com/trsdn/.github/blob/v1.21.0/docs/decisions/0013-sites-are-designed-not-templated.md). | +| W06 | na | Retired 2026-09-17 (standard 1.12.0); see [decision 0013](https://github.com/trsdn/.github/blob/v1.21.0/docs/decisions/0013-sites-are-designed-not-templated.md). | +| W07 | pass | The Google Fonts `preconnect` hints and stylesheet were removed and IBM Plex is self-hosted from `docs/assets/fonts/`. A network review of the rendered page recorded 11 requests, all same-origin, plus the new `assets/site.css`, also same-origin. `scripts/check-documentation.sh` now fails the build if any resource-loading attribute or vendored stylesheet reintroduces a remote host; the guard was confirmed with a negative test. | | W08 | pass | Build-from-source instructions and the contributor-facing material were removed from the site, which now links to the repository for depth. | +| W09 | pass | Before this reassessment, the site loaded the vendored Instrument Workshop tokens unmodified — `docs/assets/core.tokens.css`'s `--identity` accent, `#29483c`, matched the design system's own default. [`docs/assets/site.css`](../docs/assets/site.css), added in this reassessment and loaded after the vendored files, overrides `--identity`/`--identity-ink` to a project-specific slate blue for light mode and its dark-mode counterpart, both checked at over 8:1 contrast against their background (WCAG requires 4.5:1). Recorded as a deviation in [`docs/assets/VENDORED.md`](../docs/assets/VENDORED.md). | ## Agent Readiness @@ -212,5 +256,19 @@ a defect. ## Open gaps -None. Every applicable criterion is at `pass`, and the 19 recorded `na` results -each carry a rationale above. +Three, all from [Product Identity](#product-identity), and all with the same +cause and the same remediation: + +- `R01` (partial), `I02` (fail), `I03` (fail): the licence identifier and the + repository/issue URLs are missing from the `v1.2.0` artifact, because the + source commit that added them postdates that tag. +- `I04` (partial): the About window is correctly wired to read those same + values, so it inherits the gap in the currently shipped build. + +**Remediation is cutting a new release.** `scripts/verify-version.sh --app` +already gates the next one on all four values being present and agreeing with +source, so the fix is publishing, not further code change. That is a +maintainer decision this assessment does not make. + +The 23 recorded `na` results each carry a rationale above, and every other +applicable criterion is at `pass`.