Skip to content

npm_publishing.md

Latest commit

 

History

History
228 lines (179 loc) · 9.35 KB

npm_publishing.md

File metadata and controls

228 lines (179 loc) · 9.35 KB

npm Publishing

Status: Reference / runbook Last updated: 2026-06-03

0. TL;DR

  • npm organization: sqlanvil (free tier, public packages)
  • Placeholders published 2026-05-27: @sqlanvil/cli@0.0.1, @sqlanvil/core@0.0.1 (name reservations only)
  • Versioning is sqlanvil's own SemVer line: SQLANVIL_VERSION in version.bzl (currently 1.0.0). The Bazel build stamps the npm packages from it. DF_VERSION (currently 3.0.59) is the upstream dataform release this fork is synced to — kept as metadata (surfaced in sqlanvil --version as sqlanvil 1.0.0 (Dataform core 3.0.59) and exported as dataformVersion from @sqlanvil/core), not the package version. Bump SQLANVIL_VERSION for sqlanvil releases; bump DF_VERSION on upstream syncs. See §4.
  • Real publish target: Bazel-built tarball at bazel-bin/packages/@sqlanvil/cli/package.tar.gz (and matching for core)
  • For testing, don't publish — install the built tarball locally (npm i ./sqlanvil-cli.tgz). Only publish to npm when you specifically need the install-from-registry path (the stateless sqlanvilCoreVersion flow), and then publish a -beta.N prerelease under --tag beta so you never burn a real version number. See §3.0.

1. npm Organization

  • Name: sqlanvil
  • Owner: Ivan Histand (ihistand on npm)
  • Plan: Free (unlimited public packages, no private packages)
  • Created: 2026-05-27
  • URL: https://www.npmjs.com/org/sqlanvil

Free plan is sufficient — sqlanvil is OSS, no private packages needed.

2. Published Packages

Package Version Date Status
@sqlanvil/cli 0.0.1 2026-05-27 placeholder (name reservation)
@sqlanvil/core 0.0.1 2026-05-27 placeholder (name reservation)

Both placeholders print "not yet published — see GitHub" and exit non-zero. Source not committed — one-shot scaffolding under ~/sqlanvil-npm-placeholders/ on Ivan's Mac.

Note: The next real publish will be 1.0.0 (SQLANVIL_VERSION), not 0.0.2 — sqlanvil's own SemVer line, decoupled from the Dataform base. See §4. The 0.0.1 placeholders are superseded the first time a real 1.x build is published.

Names to consider claiming later

Don't grab speculatively (npm anti-squatting policy may reclaim dormant placeholders after 6+ months). Claim when implementation is within 1–2 phases of needing them:

Package Probable timing Purpose
@sqlanvil/postgres Phase 3b–4 If Postgres adapter ships standalone
@sqlanvil/supabase Phase 5 Supabase variant adapter
@sqlanvil/protos Phase 4 Generated TS proto types as standalone module
sqlanvil (unscoped) optional Vanity name — easier npm i sqlanvil

3. Workflow

3.0. Testing without publishing (preferred)

For almost all testing you do not need npm. Build the tarball and install it locally — it's the identical artifact you'd publish, with zero registry impact:

cd ~/projects-ivan/sqlanvil
./scripts/docker-bazel build //packages/@sqlanvil/cli:package_tar \
                             //packages/@sqlanvil/core:package_tar \
                             --jobs=2 --local_ram_resources=2048

# Extract from the Docker bazel volume to the host (see §3.2), then in a
# throwaway test project:
npm i /path/to/sqlanvil-core.tgz
npm i -g /path/to/sqlanvil-cli.tgz

The one thing a local install can't exercise: the stateless install path, where cli/api/commands/compile.ts runs npm i @sqlanvil/core@<ver> from the registry (triggered by sqlanvilCoreVersion: in workflow_settings.yaml). To validate that flow end-to-end the package must be on npm — publish a beta (§3.4), never the real version, for testing.

3.1. Build the tarball inside Docker

Native macOS Bazel is currently broken on this host (the wrapped_clang toolchain hits a dyld LC_UUID error when compiling protobuf's C++), so all builds go through scripts/docker-bazel (Linux toolchain):

cd ~/projects-ivan/sqlanvil
./scripts/docker-bazel build //packages/@sqlanvil/cli:package_tar \
                             --jobs=2 --local_ram_resources=2048
# Output: bazel-bin/packages/@sqlanvil/cli/package.tar.gz

--jobs=2 --local_ram_resources=2048 avoids OOM-killing the in-container Bazel JVM during the webpack bundling step.

The tarball is platform-agnostic — pure JS bundle, no native deps in the tarball itself. Native modules like pg are listed in package.json and rebuilt by npm on the consumer's machine.

3.2. Extract from container to host

bazel-bin/ symlinks into the Docker named volume, so you can't cp from the host. Run the copy inside a container with the workspace mounted:

docker run --rm \
  -v "$PWD:/workspace" \
  -v sqlanvil-bazel-cache:/root/.cache/bazel \
  -v sqlanvil-bazel-disk:/root/.cache/bazel-disk \
  sqlanvil-dev \
  cp /workspace/bazel-bin/packages/@sqlanvil/cli/package.tar.gz \
     /workspace/sqlanvil-cli.tgz

Tarball now at ./sqlanvil-cli.tgz (host filesystem). Repeat for core.

3.3. Inspect before publishing

tar tzf sqlanvil-cli.tgz | head -20
# Should show: package/package.json, package/bundle.js, etc.

tar xzf sqlanvil-cli.tgz -C /tmp/inspect && cat /tmp/inspect/package/package.json
# Verify name + version are what you expect (version == SQLANVIL_VERSION, e.g. 1.0.0)

3.4. Publish

npm login                                  # 2FA device-auth flow (see §6)

# Real release (version == SQLANVIL_VERSION, e.g. 1.0.0):
npm publish ./sqlanvil-cli.tgz --access public

# Beta / test release (set SQLANVIL_VERSION = "1.0.0-beta.0" first, rebuild):
npm publish ./sqlanvil-cli.tgz --access public --tag beta

--tag beta keeps the prerelease off the latest dist-tag, so npm i @sqlanvil/cli users never receive it; testers opt in with npm i @sqlanvil/cli@beta. Publish core first, then cli.

4. Version Policy

sqlanvil has its own SemVer line, SQLANVIL_VERSION (in version.bzl, currently 1.0.0) — this is the published @sqlanvil/* version. DF_VERSION (currently 3.0.59) is the upstream dataform-co/dataform release the fork is synced to, kept as metadata only: surfaced in sqlanvil --version (sqlanvil 1.0.0 (Dataform core 3.0.59)) and exported as dataformVersion from @sqlanvil/core. The two are independent (you can't express "our build N on top of 3.0.59" in 3-segment SemVer like 3.0.59.1, so the package version is decoupled).

Why decouple (supersedes the earlier "track DF_VERSION" policy): SemVer is three segments, so pinning the package to the exact upstream patch leaves no room for sqlanvil's own iterative releases between upstream syncs. @sqlanvil/core is a different npm package than @dataform/core, so there is no version collision to avoid — mirroring DF_VERSION only ever helped human signaling, which the --version / dataformVersion metadata now provides.

The CLI's compile-time cli/core compatibility check derives its floor from the CLI's own version (cli/vm/compile.ts requires a matching major + core ≥ cli.major.minor.0), so cli@X + core@X (both stamped from SQLANVIL_VERSION) stay in lockstep automatically. Dataform-capability gates inside the SQL generators (e.g. incremental pre/post-ops at > 1.4.8, the pre-3.0.57 caller-file shim) compare against the Dataform version (dataformVersion), not the package version — so decoupling does not disable them.

  • sqlanvil release: bump SQLANVIL_VERSION in version.bzl (e.g. 1.0.0 → 1.0.1 / 1.1.0).
  • Upstream sync: bump DF_VERSION (the scripts/update_version helper) — this changes the displayed Dataform base, not the package version.
  • Prerelease / test: SQLANVIL_VERSION = "X.Y.Z-beta.N", published --tag beta. Never reuse a real version number — npm versions are permanent and immutable.
  • Don't publish at all for local testing — use §3.0.

5. Future: Automate via CI

Once GitHub Actions is wired, publish becomes a tag-triggered CI step:

# .github/workflows/release.yml
on:
  push:
    tags: ['v*']
jobs:
  publish:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: ./scripts/docker-bazel build //packages/@sqlanvil/cli:package_tar
      - run: docker cp $(docker create sqlanvil-dev):/workspace/bazel-bin/packages/@sqlanvil/cli/package.tar.gz ./pkg.tgz
      - uses: actions/setup-node@v4
        with:
          registry-url: https://registry.npmjs.org
      - run: npm publish ./pkg.tgz --access public
        env:
          NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

Token setup: npm → Profile → Access Tokens → Generate (Granular, scoped to @sqlanvil/*) → store as NPM_TOKEN GitHub secret.

6. Auth Notes

  • 2FA is enabled by default for new npm accounts. Each npm publish from a CLI triggers a browser device-auth flow.
  • For automation, use granular access tokens (npm Profile → Access Tokens → "Granular") scoped only to @sqlanvil/* packages. Less blast radius than classic legacy tokens.

7. References