ci: fail on committed secrets and on stale generated files - #11
Merged
Conversation
Two gates the field has and we did not. Both are plain shell; neither needs a third-party action, an API key or a network call. `secrets` greps the tracked tree — not the working tree, because only tracked files can be pushed and .gitignore is a request rather than a gate — for Solari live keys, port-preview tokens with a value, JWTs of 60 characters or more, Slack bot and app tokens, Telegram bot tokens, Discord bot tokens, and any tracked `.env`. The allowlist is empty and stays that way: every pattern is written so its own text does not match it, so the scanner needs no exemption from its own scan. `--self-test` runs each pattern against a live sample and against known-good lines already in this repo, and CI runs it before the scan — a gate nobody has watched fire is a gate nobody has tested. `generated` regenerates the embedded relay server, the embedded test app and the measured numbers, then fails on `git diff --exit-code`. The measured numbers had no generator, so this adds one: `scripts/render-measured.ts` renders every table and headline count in README.md and benchmarks/README.md from `benchmarks/*.json` into `<!-- generated:… -->` blocks, byte-stably, and `bun run lint` checks the same thing locally. Prose between the blocks is still written by hand, which is why a claim carrying a number belongs inside a block. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two CI jobs the competitive field has and we did not. Both are plain shell, no
third-party actions, no API key, no network call. Each was watched failing
before it was allowed to pass.
secretsGreps the tracked tree — not the working tree, because only tracked files
can be pushed, and
.gitignoreis a request rather than a gate — for Solarilive keys, port-preview tokens carrying a value, JWTs of 60 characters or more,
Slack bot and app tokens, Telegram bot tokens, Discord bot tokens, and any
tracked
.env.The allowlist is empty and is meant to stay empty. Every pattern is written so
that its own text does not match it: the prefix is always followed by a
character class, never by a value. The scanner therefore needs no exemption
from its own scan.
scripts/scan-secrets.sh --self-testruns each pattern against a live sampleassembled at run time and against known-good lines that already exist in this
repo (
?pt_token=x, a sha256 hex). CI runs the self-test before the scan,because a gate nobody has watched fire is a gate nobody has tested.
Proof it fails. On a throwaway commit carrying
SOLARI_API_KEY=slr_live_9fA2kQ7ZxV1mR4tB8nL0:A second throwaway commit tracking
.env.localfailed the env-file rule thesame way. Both commits were dropped.
generatedRegenerates the embedded relay server, the embedded test app and the measured
numbers, then fails on
git diff --exit-code.The measured numbers had no generator, so this adds one.
scripts/render-measured.tsrenders every table and headline count inREADME.mdandbenchmarks/README.mdfrombenchmarks/*.jsoninto<!-- generated:… -->blocks. It is a pure function of the JSON, so a secondrun is byte-identical to the first. Adding the markers changed no rendered
text: the generator reproduces today's numbers exactly, and the whole diff to
both READMEs is the marker comments.
bun run lintnow runs the same--checklocally, alongside the two embedchecks.
Proof it fails. On a throwaway commit that hand-edited one README number
(
3.5s→1.2s), the job's three generator commands ran and then:A second throwaway commit edited
src/relay/guest/server.jswithoutregenerating; the same job caught the stale
src/relay/guest-source.ts. Bothcommits were dropped.
Scope, stated plainly
The generator owns the tables and the headline counts. Prose between the blocks
is still written by hand and is not checked — which is why a claim carrying a
number belongs inside a block. Both jobs are documented in the
ci.ymlheader.Offline gates run:
typecheck,lint,bun test src/ test-app/ e2e/ui.spec.ts(323 pass, 0 fail),
build,node scripts/dist-smoke.mjs. The live e2e was notrun — it needs a real
SOLARI_API_KEY.🤖 Generated with Claude Code