Skip to content

ci: fail on committed secrets and on stale generated files - #11

Merged
Sy-D merged 1 commit into
mainfrom
ci-hardening
Sep 3, 2026
Merged

ci: fail on committed secrets and on stale generated files#11
Sy-D merged 1 commit into
mainfrom
ci-hardening

Conversation

@Sy-D

@Sy-D Sy-D commented Sep 3, 2026

Copy link
Copy Markdown
Owner

Two CI jobs the competitive field has and we did not. Both are plain shell, no
third-party actions, no API key, no network call. Each was watched failing
before it was allowed to pass.

secrets

Greps the tracked tree — not the working tree, because only tracked files
can be pushed, and .gitignore is a request rather than a gate — for Solari
live keys, port-preview tokens carrying a value, JWTs of 60 characters or more,
Slack bot and app tokens, Telegram bot tokens, Discord bot tokens, and any
tracked .env.

The allowlist is empty and is meant to stay empty. Every pattern is written so
that its own text does not match it: the prefix is always followed by a
character class, never by a value. The scanner therefore needs no exemption
from its own scan.

scripts/scan-secrets.sh --self-test runs each pattern against a live sample
assembled at run time and against known-good lines that already exist in this
repo (?pt_token=x, a sha256 hex). CI runs the self-test before the scan,
because a gate nobody has watched fire is a gate nobody has tested.

Proof it fails. On a throwaway commit carrying
SOLARI_API_KEY=slr_live_9fA2kQ7ZxV1mR4tB8nL0:

credential shape found — Solari live API key:
scratch-leak.ts:1:SOLARI_API_KEY=slr_live_...
secret scan FAILED.  → exit 1

A second throwaway commit tracking .env.local failed the env-file rule the
same way. Both commits were dropped.

generated

Regenerates the embedded relay server, the embedded test app and the measured
numbers, then fails on git diff --exit-code.

The measured numbers had no generator, so this adds one.
scripts/render-measured.ts renders every table and headline count in
README.md and benchmarks/README.md from benchmarks/*.json into
<!-- generated:… --> blocks. It is a pure function of the JSON, so a second
run is byte-identical to the first. Adding the markers changed no rendered
text: the generator reproduces today's numbers exactly, and the whole diff to
both READMEs is the marker comments.

bun run lint now runs the same --check locally, alongside the two embed
checks.

Proof it fails. On a throwaway commit that hand-edited one README number
(3.5s1.2s), the job's three generator commands ran and then:

-| Agent raises its hand → ... | 1.2s | 3.6s | 3.7s |
+| Agent raises its hand → ... | 3.5s | 3.6s | 3.7s |
  → exit 1

A second throwaway commit edited src/relay/guest/server.js without
regenerating; the same job caught the stale src/relay/guest-source.ts. Both
commits were dropped.

Scope, stated plainly

The generator owns the tables and the headline counts. Prose between the blocks
is still written by hand and is not checked — which is why a claim carrying a
number belongs inside a block. Both jobs are documented in the ci.yml header.

Offline gates run: typecheck, lint, bun test src/ test-app/ e2e/ui.spec.ts
(323 pass, 0 fail), build, node scripts/dist-smoke.mjs. The live e2e was not
run — it needs a real SOLARI_API_KEY.

🤖 Generated with Claude Code

Two gates the field has and we did not. Both are plain shell; neither needs a
third-party action, an API key or a network call.

`secrets` greps the tracked tree — not the working tree, because only tracked
files can be pushed and .gitignore is a request rather than a gate — for Solari
live keys, port-preview tokens with a value, JWTs of 60 characters or more,
Slack bot and app tokens, Telegram bot tokens, Discord bot tokens, and any
tracked `.env`. The allowlist is empty and stays that way: every pattern is
written so its own text does not match it, so the scanner needs no exemption
from its own scan. `--self-test` runs each pattern against a live sample and
against known-good lines already in this repo, and CI runs it before the scan —
a gate nobody has watched fire is a gate nobody has tested.

`generated` regenerates the embedded relay server, the embedded test app and
the measured numbers, then fails on `git diff --exit-code`. The measured
numbers had no generator, so this adds one: `scripts/render-measured.ts`
renders every table and headline count in README.md and benchmarks/README.md
from `benchmarks/*.json` into `<!-- generated:… -->` blocks, byte-stably, and
`bun run lint` checks the same thing locally. Prose between the blocks is still
written by hand, which is why a claim carrying a number belongs inside a block.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@Sy-D
Sy-D merged commit fb1dd1c into main Sep 3, 2026
4 checks passed
@Sy-D
Sy-D deleted the ci-hardening branch September 3, 2026 06:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant