Repository navigation
pull: merge into edited files, generate migrations, and the review fixes - #101
joris-sevenlab wants to merge 63 commits into
Conversation
Projects scaffolded before the NUXT_RATE_LIMIT_DISABLED gate kept a bare rateLimiter object in nuxt.config.ts, and update() never touched it, so e2e runs from one loopback IP hit nuxt-security's 429s with no way to switch the limiter off. update() now wraps an existing object literal in the gate, keeping its values; an already-gated, disabled or nested value is left alone.
An upgrade of an older project overwrote local work in several ways: a feature new to the project replaced an untracked file at the same path, the Dockerfile and plugin-written workflows were rewritten without a drift check, and every edited file was only staged as .new/.patch, where the patch replaces the whole file with upstream. Edited files are now merged three-way against the version battlestack last wrote, with diff3 markers where both sides changed the same lines. Those versions live in .battlestack/base/, keyed by sha256, with git history as the source on a project's first merge. During pull and add the recorded writers go through the same check, and after the feature pass pull restores and merges any edited file a feature still wrote over, which also covers plugins running their own copy of core. pull requires a clean working tree (--allow-dirty), refuses to start while markers from a previous pull remain, writes .battlestack/pull/REPORT.md with what is left and how to handle it, and exits with code 2 when that is not empty. A feature whose plugin did not load keeps its manifest record. Also: shared:github drops records of the deploy workflows it no longer ships, pull keeps a project's higher dependency floor, ports in committed files depend only on the project name, a project-mode command outside a project no longer starts a new project with that name, and generated projects get a battlestack-upgrade skill with matching AGENTS.md and README text.
- The overwrite check compares against HEAD, so under --allow-dirty it read uncommitted edits as feature output and restored HEAD over them. It now runs only when the tree was clean, and warns when it was skipped. - When the check stages a file instead of merging, the feature's record goes back to its previous hash and no base is stored, so the next pull still merges against the old upstream. - `add` reports what its merge-aware writes left (conflicts, staged files) in REPORT.md and exits with code 2, instead of printing plain success. - `remove` of a feature whose plugin is not loaded drops just its record and leaves its files, now that pull keeps such records. - The ledger fills in fields a plugin on another core version left out. - The upgrade skill's marker example is indented, so a team edit to it can never trip the leftover-marker check, and the outside-a-project hint points at `battlestack migrate`, which lists the conversions on offer.
A pull that rewrites a file under server/database/schema/ wrote no migration, and projects apply only committed SQL in production, so the change never reached the production database (1.4.0's new index on sessions.user_id, for one). Only scaffold ran db:generate. Pull now hashes the schema files before updating and, when they changed, runs the same db:generate helper scaffold uses. With --skip-install, or when it fails, it warns to run it by hand.
# Conflicts: # CHANGELOG.md # packages/cli/src/commands/pull.ts # packages/cli/test/pull.test.ts
…into fix/pull-fixes # Conflicts: # CHANGELOG.md # packages/cli/src/commands/pull.ts
- Print the app's server.log whenever the job fails, not only when the server does not start. - Bound each readiness probe with --connect-timeout 2 --max-time 5 so a hanging server cannot hold the job until the 30-minute timeout. - Pin rustfs/rustfs to the digest that :latest resolved to on 2026-10-19. - Rewrite the scaffold comments that predate the postgres and rustfs services and the e2e run. - Turn on strict required status checks and document the allow_update_branch setting that goes with it. Also correct the check count in the docs (eight, not six).
drizzle-kit exits 0 when it stops at a rename prompt without a TTY or cannot load the schema, so pull reported a migration that was never written. Pull now compares the migrations folder before and after, does not generate while a schema file waits for the user's merge, and puts what is left to do in REPORT.md so the run exits with code 2.
…dding ours - Match rel tokens case-insensitively, so `shortcut icon` and `Icon` count as the project's own branding. - Also scan app/app.vue and app/layouts/**/*.vue for icon rels (useHead or a <link> tag). Pages, plugins and error.vue are not scanned. - Report a note on pull when the template links are skipped. - Document the detection scope in the generated README text and wireIcons JSDoc. - Bump nuxt4:essentials to 1.2.4 (origin/main already has 1.2.3 from #96).
A file moving between features was deleted by its old owner when that owner updated first, then written back by the new owner.
… sessionHeader A runtimeConfig written as a variable or shorthand crashed pull with a proxy trap error; it now fails with an error naming nuxt.config.ts#runtimeConfig. Defaults under a non-literal nested value (e.g. `session: baseSession`) were dropped silently; patchNuxtConfig now warns with each path left out. An explicit `null` is kept as the project's value, as the comment already claimed (magicast reads it back as undefined, so presence is checked by key). On pull, nuxt4:auth notes a `sessionHeader` that is not `false`: it is kept, but a sealed session sent as a header skips the revocation check. The auth e2e template now covers a revoked session and a live session sent only as the header.
A pull note prints dimmed mid-scroll, and a report action would make every pull of a project that keeps the header on exit as needing attention. Also drops two JSDoc lines that restated the function.
…cument the migration step
nuxt4:storage patched users.ts after nuxt4:database recorded it, so every database bump staged users.ts for a manual merge, and taking the staged file dropped the avatar column. Pull also quoted a stack frame instead of drizzle-kit's error when generation stopped.
The pre-patch users.ts storage projects recorded was never committed, so the first pull after the variant split could only stage it.
… keeping a project's range
… writeRecorded writeGuarded classified against an empty owned set, so a file a project took over with `battlestack own` was merged or staged on the next bump whenever its feature wrote it via writeRecorded or copyTemplateDirRecorded instead of updateFromTemplateDirs. It now reads the feature's `owned:` state, which pull seeds from the manifest. Seeding also stops writeManifest from replacing a feature's ownedByUser with just its structural files.
dropObsolete treated any existing file on a path an older shared:github shipped as its own leftover, so a project's own production.yml, staging.yml or development.yml was reported as edited and no longer managed. Only a path this feature recorded is handled now. Recorded keys are matched separator-insensitively, both here and when checking whether another feature tracks the path.
add rewrote .battlestack/pull/REPORT.md with only its own leftovers, which erased the conflicts, staged files and migration steps a previous pull had listed. When a report already exists, add now puts its own section above it instead of replacing it.
writeGuarded is exported, so the owned set a caller passes is now unioned with the feature's owned state rather than dropped from the signature.
…ded once the project has its own
…repos - Scope every git call to the project directory: `diff --relative`, `status -- .`, and `HEAD:./<rel>` reads, so a project in a subdirectory of a repository is repaired and preflighted against its own files. - Read HEAD and history blobs with `cat-file --filters`, so eol and smudge conversion match the working-tree bytes the manifest hashes. - findBaseInGit is async: one `git log --raw` for distinct blob ids, then one filtered read per blob until a hash matches. `cat-file --batch --filters` was not usable: its headers report the unfiltered size. - repairOverwrites skips `--force` and `--overwrite`, and the conflict marker preflight no longer blocks them. - A staged repair restores the writing feature's own start hash, resolved from fqid to bare id, under the key its map uses; it now writes a `.patch` next to the `.new`, as the report says. - The report lists a restored file once; restores through writers whose report a feature discards are found by comparing against files missing at start. Per-feature lines include ledger merges, conflicts and staged files those writers recorded. - Core: `hashContent`, `toPosixRel`, `recordedFiles(ctx)` and `trackersOf(ctx, rel)` replace local walks of the `files:*` state maps and duplicate sha256 helpers; `stageUpstream` and `isBinaryPath` are exported for staging outside templates.ts. - pull.ts keeps per-run report lists in a local, not ctx.state, and drops a dead `--skills-only` check.
…directory project
# Conflicts: # packages/cli/src/commands/pull.ts
…ding git With core.autocrlf, a file battlestack wrote as LF and committed came back as CRLF from cat-file --filters, so on Windows the git-history merge base was never found and pristine files looked edited.
Pull restores a file the way git restore does, which is CRLF on a Windows runner with core.autocrlf.
…e new pull behaviour
It is a local merge cache. New projects ignore it, and a project that committed it gets a step in REPORT.md to stop tracking it.
…st added A project scaffolded before the policy can lock packages younger than it allows, and pnpm then refuses every script. Pull skips lint and db:generate and lists the lockfile rebuild in REPORT.md.
…age project Takes the drop out of the migration and drizzle's snapshots, and adds a migration that puts the column back where the drop already ran, so avatars survive wherever it has not been deployed yet.
landing-shell and build-info emit with `this.id`, so the template-dir regex missed them and their changes needed no bump. landing-shell's layout changed in this PR without one, so existing projects would never get it.
# Conflicts: # CHANGELOG.md
JoeyHoutenbos
left a comment
There was a problem hiding this comment.
Reviewed at 6a6bf7d. pnpm tsc, pnpm test (1179 passed) and bump:check are green, and replaying a real 1.3 → 1.4 project pull with this build keeps the moved auth e2e test and the project name. Docs, changelog and the upgrade skill match the code. Four things need fixing before this merges, though.
Blockers
1. A file moving to an earlier-stage feature is deleted and never comes back (packages/core/src/utils/templates.ts, deferredToLater in sharedRelOwnership + removeQueuedObsoleteFiles)
The mirror of the case #100 was about. Old owner A (later stage) ships moved.ts at v1; at v1.1 it moves to B (earlier stage). B still sees the rel in A's seeded files: map, so it defers to the "later owner" and neither writes nor records it. A then drops it as obsolete, nobody tracks it, and the queued deletion removes it. Pull exits 0, and a second pull does not restore it.
Repro: in packages/cli/test/pull.test.ts "a file moving to a later feature", swap the stages in setup (from = STAGE.AUTH, to = STAGE.NAMING); "keeps a pristine file and hands its record to the new owner" fails with ENOENT. Suggest: ignore a later feature's rel in deferredToLater when that feature's new version no longer ships it, and add the swapped-stage variants as tests.
2. The avatar_key repair rewrites a migration that may already be applied (packages/preset-nuxt4/src/utils/migrations.ts, repairDroppedAvatarColumn)
The project's own templates/database/tools/migrate.mjs skips a migration only when its sha256 is in __drizzle_migrations. After the repair the file has a new hash, so the next deploy runs it again. With the PR's own fixture (DROP COLUMN "avatar_key" + CREATE INDEX "users_email_idx") that is "relation already exists", rollback, failed deploy. Only a migration whose sole statement was the drop survives (SELECT 1;). Safer: leave applied SQL alone and only add the ADD COLUMN IF NOT EXISTS migration, or report the drop and let the user decide.
3. With npm, the repair overwrites a real schema migration (migrations.ts, the generateMigration(ctx, ['--custom', '--name=restore_avatar_key']) call and the write of RESTORE_SQL)
npm run db:generate --custom --name=… hands the flags to npm, not the script (no --), so drizzle-kit runs a plain generate. If the pull also changed the schema, that writes the real migration, exactly one new file appears, and it is overwritten with the avatar SQL. The snapshot already holds the real change, so the next generate says "No schema changes" and the change never reaches the database. The fallback hint gives npm users the same command. Suggest: put -- before extra args, and only write RESTORE_SQL into an empty file named *_restore_avatar_key.sql.
4. On a core.autocrlf=true checkout an untouched file becomes one whole-file conflict (templates.ts, classifyForUpdate / mergeInPlace)
The working copy is CRLF, the recorded hash is of the LF bytes, so the file counts as drifted; findBase finds the LF blob and git merge-file turns every line into one diff3 conflict, written in place. Before this PR it was staged untouched. repairOverwrites already compares the --filters form; the drift check needs the same. The test "finds a file battlestack wrote as LF under core.autocrlf" never re-checks the file out, so the working copy stays LF and this path is not exercised.
Should fix
- Owned files get merged into via a second tracker (
writeGuarded→ownedRelsFor(ctx, featureId)):battlestack ownrecords ownership on the first tracker only, so a second feature writing the same rel (e.g.app/app.config.ts, nuxt-ui and landing-shell) merges into the owned file. Check ownership across every tracker. - Merging only works on the machine that ran the previous merge: after a merge the manifest records the upstream hash, but those bytes live only in
.battlestack/base/, which is now gitignored. On another clone or in CIfindBaseInGitfinds no blob with that hash, so every later pull of an edited file stages instead of merging. - A later pull wipes unfinished work from
REPORT.mdand exits 0: pull stagesa.ts(exit 2), a same-version re-run skips every feature and rewrites the report to "Nothing needs attention" while.battlestack/pull/a.ts.newis still there. Same for the--skip-install"run db:generate" action: the next pull sees no schema change and the reminder is gone. Carry unresolved items forward, or leave the report alone when nothing ran. - The drop match ignores the table (
AVATAR_COLUMNand the statement filter): a deliberateALTER TABLE "teams" DROP COLUMN "avatar_key"gets rewritten too, and theusers.tscheck is a plainincludes('avatar_key'). Withprefix: 'timestamp',parseIntof the file name makes the snapshot repair skip every snapshot while the SQL drop is still removed. - The new e2e test "a session revoked from another device is refused…" cannot fail on revocation: with
sessionHeader: falseevery header-only request is 401, as the next test asserts. It passes with revocation broken. Send the cookie and assert the revocation check refuses it, or drop it. runtimeConfig: { public: pub }aborts the feature's pull (nuxt-config.ts/ts-file.ts) with "runtimeConfig.public is not an object literal", while any other non-literal nested parent is a warning. Treatpublicthe same, or document it.
Nits
- An obsolete, locally edited file gives "Nothing needs attention" and exit 0 alongside a "No longer managed" section asking for a decision.
add's warning can read "merging into 0 existing file(s)" when only actions are left.- The
sessionHeaderwarning runs onupdateonly, not onadd. rateLimiter: { … } as any/satisfies Xstays ungated without a note.- Icon detection misses
:rel="'icon'"and unquotedrel=icon. - The lockfile check:
execFile('pnpm')cannot startpnpm.cmdon Windows, and offline orERR_PNPM_OUTDATED_LOCKFILEboth read as "not blocked". - Migration detection reads only
server/database/migrations, so a project with anotheroutgets "wrote no migration" when one was written. pull-safety.test.ts"pull with --allow-dirty skips the overwrite check" never callsrepairOverwritesor pull, so it passes with the guard removed; the merge outcome ofrepairOverwriteshas no test.
What this changes
One PR for the
battlestack pullwork that was spread over #94, #97, #100 and #89, plus the fixes their reviews asked for. After this merges,pullmerges upstream changes into files you edited, never overwrites local work, generates a migration when it changes the schema (and says so honestly when it could not), and reports what is left to do in.battlestack/pull/REPORT.mdwith exit code 2.Supersedes and closes #94, #97 and #100. Closes #89.
Why
The three PRs fixed overlapping parts of
pullfrom different angles: #100's ordering bug is already fixed at the root by #94's hash handling, and #97 sat on top of the same lines inpull.ts. Reviewing and merging them one by one meant three rounds of conflicts in the same file. Testing them together against a real scaffold also turned up bugs none of them caught on their own (see below).What is in here
From #94 (seamless upgrade), unchanged apart from the merge with
main: three-way merge into edited files, the overwrite check after the update, the clean-tree preflight,REPORT.mdand exit code 2. The merge with #93 keeps the rename re-render; its warning now points atREPORT.mdinstead of patch files.From #97 (migrations), reworked for the review in Slack:
.sqlfile appears. drizzle-kit exits 0 when it stops at a rename prompt without a TTY, so the old "ran db:generate" line showed up with no migration written. Scaffold's initial migration uses the same check.REPORT.md.REPORT.mdand exit code 2, instead of a warning that changed nothing.drizzle.config.tsreads are hashed, and the hint names the project's package manager.From #100: its test only. The reorder is not needed on top of #94. Separately, pull now removes files no feature ships any more only after every feature has updated, so a file moving between features is never deleted and written back in the same run, whatever the order.
From #89 (rateLimiter gate), reworked: the gate is applied through the config AST instead of a regex, so a limiter with nested options is gated too, a commented-out one is ignored, and
false, an already-gated value or a variable is left alone.Found while testing against a real
nuxt4-fullstackscaffold: everynuxt4:databasebump stagedserver/database/schema/users.tsfor a manual merge, becausenuxt4:storagepatches inavatarKeyafter the database feature recorded the file. Taking the staged file and runningdb:generatewould drop theavatar_keycolumn.nuxt4:databasenow ships ausers.tsvariant withavatarKeywhen storage is on (same pattern ashealth/with-db), and a test pins it byte-for-byte to storage's patch.Review follow-ups from #95, #96, #98 and #99:
runtimeConfigthat is not an inline object gives a clear error instead of a proxy crash, skipped nested defaults are warned about, an explicitnullis kept.setRuntimePublicDefaultgoes through the same merge.sessionHeaderother thanfalseis kept with a warning; two more e2e cases for the session header (revoked on another device, live session in the header only).shortcut icon, any casing, anduseHeadinapp/app.vueand layouts; pull notes when it skips the template links.server.logwhen the tests fail, the readiness curl has timeouts, stale comments fixed, and the ruleset file turns on strict status checks.Also: a rename stays pending when a feature fails or its plugin does not load during the re-render (edge case from the #93 review), and
.claude/worktrees/is ignored.Fixes from the in-depth review of this PR
--relative,HEAD:./<rel>).git cat-file --filters, socore.autocrlforeol=crlfno longer make pristine files look edited. The history lookup is async and reads each blob once.battlestack ownare skipped by every recorded writer again (the Dockerfile was merged), and a feature with structural files no longer dropsownentries from the manifest.shared:githubignores a retired workflow path it never wrote (a project's ownproduction.yml).battlestack addputs its section above an earlier pull'sREPORT.mdinstead of erasing it.--force/--overwritestart over conflict markers; the overwrite repair keeps a.bakunder--forceand stays out of the way under--overwrite. Its staged fallback restores the writing feature's own hash and writes the promised.patch.users.tsgets a stored merge base, and pull repairs a migration that dropsusers.avatar_key, which earlier pulls could have led to: it takes the drop out of the migration and drizzle's snapshots, and adds a migration (ADD COLUMN IF NOT EXISTS, made withdrizzle-kit generate --custom) that puts the column back where the drop already ran. Verified with real drizzle-kit: a freshdb:generateafterwards finds no schema changes.^0.xranges: a higher 0.x minor is not kept over a feature's range.trackersOf/recordedFiles) replaces four copies of the recorded-files walk; duplicated sha256 helpers are gone.Finishing touches
.battlestack/base/is a local merge cache and stays out of git: new projects ignore it, the docs and the upgrade skill no longer say to commit it, and pull listsgit rm -r --cached .battlestack/basefor a project that committed it.pnpm-workspace.yamlcan have a lockfile pnpm then rejects. The pull that adds the policy checks it withpnpm install --frozen-lockfile --lockfile-only, and when it fails, skips lint anddb:generateand lists the lockfile rebuild inREPORT.md. Rare in practice, since older projects had a policy in.npmrcthat pnpm did not enforce, so only recently updated packages trip it.battlestack-upgradeskill covers migrations, set-aside schema files,--forcebackups, monorepos and checking that a second pull exits with code 0.users.tskeptavatarKey, and the overwrite repair caught five files a feature wrote over.Type of change
AI assistance
Agent, service and model, if you know it: Claude Code, Opus 5.5
How much of this is agent-written?
Feature versions
version:nuxt4:auth1.10.8,nuxt4:essentials1.2.4,nuxt4:database1.7.5,nuxt4:storage1.3.4,shared:security1.0.5, plus pull: merge into edited files, never overwrite local work #94's own bumpscollectDocsorcollectEnvcontribution, and also bumped the feature that writes the target fileChangelog
## Unreleasedsaying what changes for someone using battlestackTesting
Against a real
nuxt4-fullstackscaffold (installed, committed), running the CLI from source:plain pull with no template change: every feature up to date, exit 0, nothing in
REPORT.mdindex added to the auth
sessions.tstemplate:generated 0001_*.sql, exit 0, SQL is theCREATE INDEXcolumn renamed: no SQL,
REPORT.mdquotes drizzle-kit, exit 2 (the stack-frame quoting found here is fixed)sessions.tsedited by the user: a clean merge generates the migration with both edits; a conflict skips generation and lists the merge, exit 2nuxt4:databasebump with storage on: reproduced theusers.tsstaging above, now fixedpnpm tscpassespnpm testpassesI removed the behaviour each new assertion covers, watched the test fail, and put it back
Checklist
docs/,README.md)packages/*/templates/**are LFNot one concern on purpose, see Why. Each fix is its own commit, and the merges of #94, #96, #97 and #89 are kept as merge commits, so the history reads per topic.
Anything else
pnpm tsc; the scaffold e2e job in CI is their first run..github/rulesets/require-ci-to-pass.json. Someone with admin rights still needs to apply the ruleset and enableallow_update_branch(docs/repository-settings.mdhas the command).setRuntimePublicDefaultchanges reach projects at the next bump of nuxt-ui, auth-verification, build-info or health; they were not bumped for it.battlestack addandremovedo not generate migrations yet;--skip-installalways skipsdb:generate;pull --dry-runlists no per-file preview; k8s files are staged, not merged.·, and therunningLowstrings indashboard.jsonwere reworded.:latest: Dependabot does not update images in a workflow'sservices:, so a digest pin would go stale.