Skip to content

docs(benchmarks): corpus-growth runbook + eval README pin corrections #435

docs(benchmarks): corpus-growth runbook + eval README pin corrections

docs(benchmarks): corpus-growth runbook + eval README pin corrections #435

Triggered via pull request August 1, 2026 21:33
Status Failure
Total duration 11m 21s
Artifacts

gate.yml

on: pull_request
Fit to window
Zoom out
Zoom in

Annotations

10 errors and 1 warning
[git-integration] cli/__tests__/ship-branch.test.mts > ship-branch.sh — --base <branch> > --base origin/studio: ships committed work as a diff vs studio, from the finalized checkout: cli/__tests__/ship-branch.test.mts#L359
AssertionError: ↳ shipping: linked .husky/_ ← /tmp/shipwt-1VBUdI/.husky/_ ↳ shipping: refreshed reviewer agents + skills from running devkit package [feat/dk1 68fe1bc] ship it 1 file changed, 1 insertion(+), 1 deletion(-) ✓ pre-commit gates ran in the ship worktree — full output: /tmp/shipwt-1VBUdI/.devkit/last-ship-gates-feat-dk1.log Review it for any SKIP / BYPASSED / ⚠️ lines — a bypassed gate verified nothing. /home/runner/work/devkit/devkit/cli/lib/ship/assert-staged-set.sh: line 105: lost: unbound variable DRY: worktree kept at /tmp/devkit-ship-feat-dk1-38343 (branch feat/dk1). Inspect, then: git worktree remove --force '/tmp/devkit-ship-feat-dk1-38343' && git branch -D 'feat/dk1' : expected 1 to be +0 // Object.is equality - Expected + Received - 0 + 1 ❯ cli/__tests__/ship-branch.test.mts:359:34
[git-integration] cli/__tests__/ship-branch.test.mts > ship-branch.sh — --base <branch> > --base studio: ships committed work as a diff vs studio, from the finalized checkout: cli/__tests__/ship-branch.test.mts#L359
AssertionError: ↳ shipping: linked .husky/_ ← /tmp/shipwt-C0JRvV/.husky/_ ↳ shipping: refreshed reviewer agents + skills from running devkit package [feat/dk1 2c5cee3] ship it 1 file changed, 1 insertion(+), 1 deletion(-) ✓ pre-commit gates ran in the ship worktree — full output: /tmp/shipwt-C0JRvV/.devkit/last-ship-gates-feat-dk1.log Review it for any SKIP / BYPASSED / ⚠️ lines — a bypassed gate verified nothing. /home/runner/work/devkit/devkit/cli/lib/ship/assert-staged-set.sh: line 105: lost: unbound variable DRY: worktree kept at /tmp/devkit-ship-feat-dk1-37945 (branch feat/dk1). Inspect, then: git worktree remove --force '/tmp/devkit-ship-feat-dk1-37945' && git branch -D 'feat/dk1' : expected 1 to be +0 // Object.is equality - Expected + Received - 0 + 1 ❯ cli/__tests__/ship-branch.test.mts:359:34
[git-integration] cli/__tests__/reship.test.mts > reship — repo path with a space (linked-worktree COMMIT_EDITMSG carries the space) > A: re-ships through a correctly-quoted commit-msg hook (path arrives intact, with its space): cli/__tests__/reship.test.mts#L709
AssertionError: ↳ shipping: linked .husky/_ ← /tmp/reship space lQoXYf/repo/.husky/_ ↳ shipping: refreshed reviewer agents + skills from running devkit package [detached HEAD 66352ff] add v2 1 file changed, 1 insertion(+), 1 deletion(-) ✓ pre-commit gates ran in the ship worktree — full output: /tmp/reship space lQoXYf/repo/.devkit/last-ship-gates-feat-pr.log Review it for any SKIP / BYPASSED / ⚠️ lines — a bypassed gate verified nothing. /home/runner/work/devkit/devkit/cli/lib/ship/assert-staged-set.sh: line 105: lost: unbound variable : expected 1 to be +0 // Object.is equality - Expected + Received - 0 + 1 ❯ cli/__tests__/reship.test.mts:709:32
[git-integration] cli/__tests__/reship.test.mts > reship — PR branch based on a non-default branch > parents the new commit on the PR-branch tip, not on origin/HEAD (main): cli/__tests__/reship.test.mts#L462
AssertionError: ↳ shipping: linked .husky/_ ← /tmp/reshipwt-JMSmED/.husky/_ ↳ shipping: refreshed reviewer agents + skills from running devkit package [detached HEAD 1ea83a4] add v2 1 file changed, 1 insertion(+), 1 deletion(-) ✓ pre-commit gates ran in the ship worktree — full output: /tmp/reshipwt-JMSmED/.devkit/last-ship-gates-feat-pr.log Review it for any SKIP / BYPASSED / ⚠️ lines — a bypassed gate verified nothing. /home/runner/work/devkit/devkit/cli/lib/ship/assert-staged-set.sh: line 105: lost: unbound variable : expected 1 to be +0 // Object.is equality - Expected + Received - 0 + 1 ❯ cli/__tests__/reship.test.mts:462:34
[git-integration] cli/__tests__/reship.test.mts > reship — exports DEVKIT_SHIP_BASE_SHA (DK-5) > is the fetched PR-branch tip, not a stale local ref: cli/__tests__/reship.test.mts#L399
AssertionError: ↳ shipping: linked .husky/_ ← /tmp/reshipwt-QMmL0E/.husky/_ ↳ shipping: refreshed reviewer agents + skills from running devkit package HOOK_BASE=7ed6b277ad38ff235489449a0650b2519d8593b2 [detached HEAD c185f0e] add v2 1 file changed, 1 insertion(+), 1 deletion(-) ✓ pre-commit gates ran in the ship worktree — full output: /tmp/reshipwt-QMmL0E/.devkit/last-ship-gates-feat-pr.log Review it for any SKIP / BYPASSED / ⚠️ lines — a bypassed gate verified nothing. /home/runner/work/devkit/devkit/cli/lib/ship/assert-staged-set.sh: line 105: lost: unbound variable : expected 1 to be +0 // Object.is equality - Expected + Received - 0 + 1 ❯ cli/__tests__/reship.test.mts:399:34
[git-integration] cli/__tests__/reship.test.mts > reship — untracked gate configs are linked into the re-ship worktree > links an untracked guard.config.json so the gate sees it (not defaults), with a notice: cli/__tests__/reship.test.mts#L345
AssertionError: ↳ shipping: linked .husky/_ ← /tmp/reshipwt-gmTsJl/.husky/_ ↳ shipping: refreshed reviewer agents + skills from running devkit package ⚠️ ship: 1 gate config(s) present in the repo but absent from the committed tree — linked into the gate worktree so gates match a normal commit (not defaults): - guard.config.json (untracked — commit it so gates are consistent for everyone) CONFIG_SEEN [detached HEAD af06765] add v2 1 file changed, 1 insertion(+), 1 deletion(-) ✓ pre-commit gates ran in the ship worktree — full output: /tmp/reshipwt-gmTsJl/.devkit/last-ship-gates-feat-pr.log Review it for any SKIP / BYPASSED / ⚠️ lines — a bypassed gate verified nothing. /home/runner/work/devkit/devkit/cli/lib/ship/assert-staged-set.sh: line 105: lost: unbound variable : expected 1 to be +0 // Object.is equality - Expected + Received - 0 + 1 ❯ cli/__tests__/reship.test.mts:345:34
[git-integration] cli/__tests__/reship.test.mts > reship — merges the re-pushed paths into the branch reconcile entry > on a real push, extends branches[$BR] with this commit content (tip blob) + new paths: cli/__tests__/reship.test.mts#L290
AssertionError: ↳ shipping: linked .husky/_ ← /tmp/reshipwt-IjXBO9/.husky/_ ↳ shipping: refreshed reviewer agents + skills from running devkit package [detached HEAD 521b167] add v2 + b 2 files changed, 2 insertions(+), 1 deletion(-) create mode 100644 b.ts ✓ pre-commit gates ran in the ship worktree — full output: /tmp/reshipwt-IjXBO9/.devkit/last-ship-gates-feat-pr.log Review it for any SKIP / BYPASSED / ⚠️ lines — a bypassed gate verified nothing. /home/runner/work/devkit/devkit/cli/lib/ship/assert-staged-set.sh: line 105: lost: unbound variable : expected 1 to be +0 // Object.is equality - Expected + Received - 0 + 1 ❯ cli/__tests__/reship.test.mts:290:32
[git-integration] cli/__tests__/reship.test.mts > reship — tracked file under a gitignored directory > re-ships a tracked-but-gitignored dist/ file without aborting on the ignore diagnostic: cli/__tests__/reship.test.mts#L217
AssertionError: ↳ shipping: linked .husky/_ ← /tmp/reshipwt-B19GTt/.husky/_ ↳ shipping: refreshed reviewer agents + skills from running devkit package [detached HEAD de14d9f] add v2 1 file changed, 1 insertion(+), 1 deletion(-) ✓ pre-commit gates ran in the ship worktree — full output: /tmp/reshipwt-B19GTt/.devkit/last-ship-gates-feat-pr.log Review it for any SKIP / BYPASSED / ⚠️ lines — a bypassed gate verified nothing. /home/runner/work/devkit/devkit/cli/lib/ship/assert-staged-set.sh: line 105: lost: unbound variable : expected 1 to be +0 // Object.is equality - Expected + Received - 0 + 1 ❯ cli/__tests__/reship.test.mts:217:34
[git-integration] cli/__tests__/reship.test.mts > reship — re-push commits onto the PR-branch tip > dry-run stacks the current content as a new commit on origin/<branch>: cli/__tests__/reship.test.mts#L158
AssertionError: ↳ shipping: linked .husky/_ ← /tmp/reshipwt-0YzZi5/.husky/_ ↳ shipping: refreshed reviewer agents + skills from running devkit package [detached HEAD f5a09e9] add v2 1 file changed, 1 insertion(+), 1 deletion(-) ✓ pre-commit gates ran in the ship worktree — full output: /tmp/reshipwt-0YzZi5/.devkit/last-ship-gates-feat-pr.log Review it for any SKIP / BYPASSED / ⚠️ lines — a bypassed gate verified nothing. /home/runner/work/devkit/devkit/cli/lib/ship/assert-staged-set.sh: line 105: lost: unbound variable : expected 1 to be +0 // Object.is equality - Expected + Received - 0 + 1 ❯ cli/__tests__/reship.test.mts:158:32
boom.test.mjs > fails: /tmp/coverage-produce-VR7r53/boom.test.mjs#L2
AssertionError: expected 1 to be 2 // Object.is equality - Expected + Received - 2 + 1 ❯ boom.test.mjs:2:37
gate
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: actions/checkout@v4. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/