Repository navigation
setup-chromatic: build without a token and publish from the default branch - #50
CodyCBakerPhD wants to merge 6 commits into
Conversation
…efault branch setup-chromatic now writes the three-workflow form dandi/usage-page uses, for every repository rather than only those taking pull requests from forks: the Storybook and Playwright workflows build with no token and hand the build on as an artifact, and a new chromatic-publish.yml uploads it on workflow_run from the default branch. Forks' pull requests get snapshots, and a push to the default branch is accepted as the new baseline, so merges no longer leave builds waiting for review. references/fork-pull-requests.md becomes references/publish-workflow.md: why the split, the publish workflow's safety properties (now in the template), and what the three files rely on. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DQrzw3QuYxA1bV48ZgQcor
The template's own setup-chromatic skill is removed; apm.yml installs it from con/skills (pinned in apm.lock.yaml to the commit of con/skills#50, which carries the three-workflow form), and `apm install --frozen` restores it into .claude/skills/, where it is git-ignored. AGENTS.md says how to restore it and where fixes go, setup-web-app keeps the APM files and renames apm.yml, and the three Chromatic workflows now match the skill's templates (Node 22, persist-credentials: false, auto-accept keyed to the default branch), plus the template's own publish guard. .prettierignore keeps Prettier off the APM-written lock. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DQrzw3QuYxA1bV48ZgQcor
…tic from con/skills (#17) * Published Chromatic builds the way usage-page does The two Chromatic workflows now only build (Storybook, and the Playwright archive Storybook) with no token and hand the result on as an artifact; a new chromatic-publish.yml uploads it from main on workflow_run, auto-accepting builds from a push to main, so merges no longer leave builds waiting for review and pull requests from forks get snapshots. The setup-chromatic skill's step 7 now writes these three files, and setup-web-app names the publish job's template guard. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DQrzw3QuYxA1bV48ZgQcor * Named the template by its real repository in the publish guard GitHub reports this repository as brain-bbqs/bbqs-web-app-template (web-app-template only redirects), which is what the other workflow guards already match. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DQrzw3QuYxA1bV48ZgQcor * Installed setup-chromatic from con/skills instead of keeping a copy The template's own setup-chromatic skill is removed; apm.yml installs it from con/skills (pinned in apm.lock.yaml to the commit of con/skills#50, which carries the three-workflow form), and `apm install --frozen` restores it into .claude/skills/, where it is git-ignored. AGENTS.md says how to restore it and where fixes go, setup-web-app keeps the APM files and renames apm.yml, and the three Chromatic workflows now match the skill's templates (Node 22, persist-credentials: false, auto-accept keyed to the default branch), plus the template's own publish guard. .prettierignore keeps Prettier off the APM-written lock. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DQrzw3QuYxA1bV48ZgQcor --------- Co-authored-by: Claude <noreply@anthropic.com>
yarikoptic-gitmate
left a comment
There was a problem hiding this comment.
Thanks @CodyCBakerPhD. The three-workflow form reads well and the skill is consistent after the change (no stale references to the action, the fork reference or the old question numbering). CI is green.
One blocker before this is merged: the publish workflow's central claim, that nothing from the built commit executes while the tokens are in reach, does not hold. The Chromatic CLI loads Storybook configuration and spawns the package manager from its working directory, which is the fork's checkout, so a fork's pull request can read the project token. This was reproduced locally on two CLI versions. The inline comment has a two-line fix (fetch the commit, do not check its files out), which was checked locally but still needs one real run. dandi/usage-page's own chromatic-publish.yml has the same problem and wants the same change.
Also inline:
- Token selection falls back to the Storybook token when the Playwright secret is missing, so a Playwright build can land in, and become the baseline of, the Storybook project. One-line suggestion.
- Troubleshooting: the Dependabot secrets advice looks wrong per GitHub's docs, and the "no Chromatic check" row lacks its likeliest cause.
- README template: the first build is no longer accepted by hand.
- TurboSnap: dropping it is reasonable, but the snapshot cost deserves a mention.
Verified and fine as they are: a fork's commit is still fetched by actions/checkout with fetch-depth: 0; actions/checkout could not replace the hand-driven fetch in the publish job (v7 refuses a fork's commit on workflow_run); the auto-accept guard; CHROMATIC_SHA/BRANCH/SLUG match what the Chromatic action sets; uploading the archive Storybook without --playwright is equivalent.
This review went through two independent reviewer passes (GitHub Actions security; Chromatic and skill authoring) before posting.
Generated by Claude Code
Run inside a checkout of the built commit, the Chromatic CLI loads its Storybook config and starts the package manager it detects there, which honours repository-local configuration: chromatic 18.9.4 ran a script a fork's .yarnrc named, with the project token in its environment, before authenticating. The publish template now points HEAD at the commit with `git update-ref --no-deref` and writes only a stub package.json. The token is chosen by the secret's name, so a missing secret no longer falls through to the other project's token. Review follow-ups: the troubleshooting rows (Dependabot advice dropped, the publish run as the first cause of a missing check, a repository that is itself a fork), the README template's first-baseline wording, and TurboSnap's snapshot cost; publish-workflow.md loses two paragraphs that repeated step 7 and the README template. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DQrzw3QuYxA1bV48ZgQcor
yarikoptic-gitmate
left a comment
There was a problem hiding this comment.
Re-reviewed c65afe5: all six points from the first round are addressed, nothing is left stale in the skill, and CI is green. Nothing blocking remains on the code side.
Still outstanding before merge: one real run of chromatic-publish.yml for a fork's pull request, since nothing here could go past the Chromatic API. The blocker thread stays open for that.
What was checked this round, by two independent reviewer passes (GitHub Actions security; Chromatic and skill authoring):
- No-checkout fix, run locally with chromatic 18.9.4 and 18.15.1 against a mock of the Chromatic API, under
strace, with a hostile commit (.yarnrcyarn-path,.npmrcscript-shell, Storybookmain.js,chromatic.config.json,.gitattributesfilter,ext::submodule URL):- With the old
git checkout --detachthe planted script ran with the token in its environment. - With
git update-ref --no-deref HEADand the stubpackage.jsonnothing from the commit ran. The CLI spawned onlyyarn/npmversion and registry queries and read-only git commands, resolved the right commit, branch and parent build, and got as far as announcing the build. - Upload, verify and snapshot were not reached by the mock.
- With the old
- Token by name: an empty
CHROMATIC_PROJECT_TOKENstops the CLI withMissing project tokenbefore any API call. - From the CLI source only, not run: the git commands that write a worktree are behind
--patch-buildor TurboSnap, neither used here; and the stub also removes a second path, where the-cdirectory of a commit'sbuild-storybookscript is interpolated into ash -c "git log …"call. - Repo conventions:
setup-chromaticdeclares noversion:, so there is no bump to make;ci/validate_skills.pypasses for it.
One optional wording nit, in templates/README-visual-snapshots.md step 3: the colon joins "their changes were reviewed on their pull requests" to the first build, which is the one build that was not reviewed. Reads more clearly as:
The first upload comes from the merge that adds these files; that first build of each project is its baseline, so look it over in Chromatic once.
After that, the default branch's builds are accepted on their own, since their changes were reviewed on their pull requests.Generated by Claude Code
A computed secrets[...] name makes GitHub send every repository and organization secret to the runner (CodeQL: excessive secrets exposure, flagged on brain-bbqs/bbqs-web-app-template#19). Each token now sits behind its own condition instead, which keeps a missing secret empty so the CLI stops rather than uploading into the other project. Also takes the review's wording for the README template's first-baseline step. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DQrzw3QuYxA1bV48ZgQcor
AGENTS.md called `version:` in SKILL.md frontmatter "the field APM reads". apm-cli 0.31's SKILL.md parser takes only name and description from the frontmatter, and an install records the package version from apm.yml (0.1.0 for every con/skills skill). The field stays what the skills that stamp their output read it for. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DQrzw3QuYxA1bV48ZgQcor
This reverts commit 8a74a96. #51 rewrites the AGENTS.md "Skill versions" section, which covers this sentence, so this pull request stays limited to setup-chromatic. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DQrzw3QuYxA1bV48ZgQcor
…lish job's token exposure (#15) * Installed setup-chromatic from con/skills and aligned the Chromatic workflows with it The repository's own copy of setup-chromatic is removed; apm.yml installs it from con/skills (pinned in apm.lock.yaml to the commit of con/skills#50), and `apm install --frozen` restores it into .claude/skills/, where it is git-ignored. AGENTS.md says how to restore it and where fixes go, and .prettierignore keeps Prettier off the APM-written lock. The three Chromatic workflows are now the skill's templates: Node 22, persist-credentials: false on the publish job's checkout, and auto-accept keyed to the repository's default branch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DQrzw3QuYxA1bV48ZgQcor * Fetched the built commit without checking it out in the Chromatic publish job Run inside a checkout of the built commit, the Chromatic CLI loads its Storybook config and starts the package manager it detects there, which honours repository-local configuration, so a fork's pull request could read the project tokens (reproduced with chromatic 18.9.4 and a .yarnrc). The publish job now points HEAD at the commit and writes only a stub package.json, and picks the token by the secret's name. Both come from setup-chromatic at con/skills c65afe5, which apm.yml now pins. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DQrzw3QuYxA1bV48ZgQcor * Referenced the two Chromatic tokens statically in the publish job A computed secrets[...] name makes GitHub send every repository and organization secret to the runner (CodeQL: excessive secrets exposure). Each token now sits behind its own condition, which keeps a missing secret empty so the CLI stops rather than uploading into the other project. From setup-chromatic at con/skills 71e788c, which apm.yml now pins. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DQrzw3QuYxA1bV48ZgQcor --------- Co-authored-by: Claude <noreply@anthropic.com>
|
Ran Fork PR: NeurodataWithoutBorders/nwb-benchmarks-dashboard#17, from
The default-branch push path ran earlier, when dashboard#15 merged: publish #15 and publish #16. |
yarikoptic-gitmate
left a comment
There was a problem hiding this comment.
Thanks @CodyCBakerPhD. That real run was the last open item from my side, and what you report covers each thing the local test could not reach:
- the fork-only commit fetched by hash, never checked out, and the CLI still finding its branch and parent build;
- pending (not auto-accepted) changes for a fork's pull request, with
AUTO_ACCEPT: false; - each run using its own project's token;
- the statuses landing on the pull request's head commit;
- the default-branch push path.
Checked here at f6e8121: the head differs from 71e788c, which I reviewed last, by a commit and its revert, so the tree is identical and the earlier review stands. All threads are resolved, CI is green and there is no merge conflict.
Not checked: the linked runs and Chromatic builds themselves. nwb-benchmarks-dashboard is outside the repositories I can read, so I take the run results, and the templates there being identical to this PR's, as reported.
Nothing further from this review. I am leaving it as a comment, not an approval; merging is for a maintainer to decide.
Generated by Claude Code
|
@yarikoptic I tested this skill in practice and while it mostly worked well, one edge case came up regarding how to deal with fork PRs, and this is a better overall way of handling that case If you'd like to review yourself, or defer to the bots opinion (which is now satisfied) and just merge; this is slightly larger than a 'simple fix' so I didn't just go ahead and merge myself |
|
Checked ccf0b64, the merge of The earlier review stands, with nothing open from my side. Merging is for a maintainer to decide. Generated by Claude Code |
Lessons learned from the usage page on DANDI; this allows Chromatic to run from forks without too much trouble
🤖 AI text blow 🤖
setup-chromaticnow writes the three-workflow form dandi/usage-page uses, for every repository rather than only those taking pull requests from forks:chromatic-storybook.ymlandchromatic-playwright.ymlbuild with no token onpush(and on a fork'spull_request) and hand the build on as an artifact.chromatic-publish.ymluploads it onworkflow_runfrom the default branch, with--exit-zero-on-changes, and--auto-accept-changesonly for a push to the default branch.Forks' pull requests get snapshots, and merges no longer leave builds waiting for review in Chromatic (and notifications about them).
persist-credentials: false,npm ci --ignore-scriptsfrom the default branch, the built commit fetched by hash and never checked out (onlyHEADand a stubpackage.json; in a checkout the CLI runs the repository's own Storybook and package-manager configuration with the token in reach), triggering-run values only throughenv:, each token referenced statically behind its own condition (a missing one stays empty; a computedsecrets[...]name would send every secret to the runner), and auto-accept keyed togithub.event.repository.default_branch.references/fork-pull-requests.mdbecomesreferences/publish-workflow.md(why the split, those properties, what the three files rely on); step 0 drops the fork question; steps 6, 7, 10 and 11, the README template, troubleshooting and the collection README follow.dandi/usage-page's own
chromatic-publish.ymlchecks the commit out and wants the same fix.