Source-adjacent generated code + structure-preserving output layout

[omridevk]

Maintainer order (2026-07-27)

1. Generated source must be exposed OUTSIDE the .napl/ state directory, directly
   next to the prompt that generated it. Where the prompt-to-output relationship is
   one-to-many or many-to-many, symlinks may express the relationship.
2. Generated output must preserve the directory structure of the .napl sources.

Standing principle for this feature, verbatim from the maintainer: do this right and
do not hide anything ugly. The generated code is a first-class product of the
language, not a build byproduct, and the repo layout must say so.

Why (the usability failure today)

Today src/store/store_hash.napl generates src/.napl/src/rust/store_hash/src/lib.rs.
Three problems:

• The output is buried in a dot-directory two levels deep; a reader browsing the
  sources cannot see the code a prompt produced without knowing toolchain internals.
• The generated tree is FLAT (rust/<module>/), discarding the authored hierarchy
  (store/, cli/, core/, ...). The authored structure carries meaning; the
  output throws it away.
• The mapping prompt->output lives only in map.json, i.e. in metadata instead of
  in the filesystem where humans and tools look first.

Requirements

• R1: For every module, its generated output is reachable at a stable path adjacent
  to its authoring .napl file, visible in a normal file tree (no dot-prefix).
• R2: The generated tree mirrors the authored tree: src/store/store_hash.napl ->
  a generated unit under src/store/ (exact naming below), recursively for any
  authored nesting depth.
• R3: Multi-target modules (one prompt -> rust and ts outputs) and any future
  many-to-many prompt/output relationships must be expressible; symlinks are the
  sanctioned mechanism where a single physical location must appear in more than
  one place.
• R4: Nothing about the store changes: history, blame, and drift stay
  content-addressed in .napl/store/. This feature is about where LIVE outputs
  sit, not where history sits.
• R5: No hidden mirrors. If a path exists in two places, one is a symlink and
  ls -l says so. No copies, no generated-then-synced trees.

Candidate designs (to be settled in the spec)

A. Physical adjacency (preferred starting point)

Generated code physically lives next to its prompt:

src/
  store/
    store_hash.napl
    store_hash.rust/        <- the generated crate (Cargo.toml, src/lib.rs), 0444
  cli/
    blame_render.napl
    blame_render.rust/

• The cargo workspace enumerates members across the authored tree (workspace
  member globs or a generated members list in the workspace manifest).
• <module>.<target>/ naming makes target multiplicity trivial (.rust/, .ts/).
• Symlinks only appear for genuine m:n cases (R3), not as the primary mechanism.
• Ugliness surfaced honestly: generated dirs sit in your face in the tree. That is
  the point; editors can fold them, and their read-only bit marks them.

B. Symlink views over the current physical layout

Keep physical outputs under .napl/src/, place src/store/store_hash.rust -> ../.napl/src/rust/store_hash symlinks next to prompts.

• Cheaper migration, but violates the spirit of "not hidden": the real files stay
  buried, and the visible tree is a facade of links. Also symlink-in-git has
  platform caveats (Windows checkout).
• Retained as a fallback if design A hits a hard blocker (e.g. cargo workspace
  constraints that cannot express the mirrored layout).

Blast radius (honest list; every consumer of the flat layout)

• paths_core (all generated-path derivation), discovery, map schema/map.json
• cmd_gen_io (write paths, scope gate = "module writes only its own dir"),
  statusclass_io (observed generated tree), driftdetect_*, healing_io (moved-file
  heal must understand the mirrored layout - it may get SIMPLER, since authored
  moves and generated moves become the same move)
• store tree paths for live-tree derivation (store objects record relative paths;
  decide whether stored paths are authored-relative, which is the natural choice
  post-migration)
• migrate/one-shot relayout of the existing tree + git mv-equivalent handling
• cargo workspace manifest, CI --locked builds, conformance runner paths,
  equivalence suite paths, LSP file resolution, VS Code extension watcher globs
• prek/lint ignores, tsdown/vitest globs on the ts side

Sequencing

Strictly AFTER: C4 store activation, morning retirement pass, and the
metadata-fold slice (ir/ + mapl/ + attribution/ + map.json into the store).
Rationale: every artifact retired before this lands is one fewer consumer of paths
to relayout; the store's authored-relative path decision (R4/blast list) is the
natural foundation. This is the "make the language usable" slice that follows the
"make the language sound" slices.

Open questions for the spec

1. Naming: <module>.rust/ vs <module>.gen/rust/ vs bare <module>/ when
   single-target. Leaning <module>.<target>/ for greppability and multiplicity.
2. Workspace mechanics: cargo member globs across arbitrary depth vs a
   toolchain-maintained members list in the workspace Cargo.toml (generated,
   reviewed, committed).
3. Read-only enforcement stays 0444 per file; does the dir itself get marked?
4. Does the ts target mirror identically under the same rule at the same time, or
   in a follow-up?
5. m:n semantics: what authored construct produces many-to-many today or
   near-term? (If none concrete, R3 stays a design constraint, not built
   machinery - no speculative symlink framework.)

The full design spec (Fable-authored, per campaign discipline) lands in
docs/specs/ once C4 + retirement close and the store path decisions above are
observable facts rather than predictions. This issue is the tracking anchor.

[omridevk]

Design spec landed: docs/specs/2026-07-27-source-adjacent-layout-design.md

Key decisions: physical adjacency (<module>.<target>/ next to its .napl, mirrored tree), toolchain-generated src/Cargo.toml workspace manifest, store paths authored-relative, and R6 prompt purity: the 'Where this code lives' boilerplate is deleted from all authored prompts and injected by the gen harness at gen time (P1, lands before the relayout so the layout flip touches zero authored prompts). The relayout itself executes as a healing pass recording Move events, so blame history stays continuous and no module is re-generated. Sequenced after store activation, retirement, and the metadata fold.

[omridevk]

Maintainer requirement added (2026-07-27): the 1:1 napl-file-to-generated-crate coupling is an implementation artifact, not a language idea. One .napl document must be able to produce multiple source files. Design direction agreed with the maintainer:

• Decouple AUTHORING granularity (the file) from GENERATION granularity (the unit that is genned, gated, and regenerated). A .napl file becomes a document of module sections; each section is an independent gen unit with its own output(s), so regen cost stays bounded at section size.
• File-level frontmatter with per-section overrides (deps, model targeting).
• Blame/attribution/drift are line-based over the block index and carry over unchanged.
• Composes with the adjacency layout: one document -> one adjacent generated dir containing its sections' outputs; the current ~33 cli/ files collapse to a handful of subsystem documents.

Spec amendment lands in docs/specs/2026-07-27-source-adjacent-layout-design.md alongside the layout phases; sequencing unchanged (after store activation + retirement + metadata fold).

[omridevk]

Two more maintainer decisions recorded for the layout slice (2026-07-27):

1. .napl/ sits at the PROJECT ROOT, next to src/, not inside it. Project root = where the tool is inited/run; sources live in subdirectories under it; the toolchain discovers .napl files by walking the project tree. Current src/.napl placement is a defect to fix in this slice.
2. Derived state is never committed: committed surface = prompts + generated code + .napl/store/ + lock.json. All derived indexes (map.json successor and any future hot caches) live under .napl/cache/, ignored via a .napl/.gitignore, disposable and rebuilt on demand (the .git/index pattern). The tracked map.json gets git rm --cached when the CLI learns the new path (metadata-fold slice).

[omridevk]

Scope update from the maintainer (2026-07-27): the ADJACENCY MOVE ITSELF IS PINNED - moving generated code next to the .napl files (the ./ adjacent-folder layout, spec design A) is deferred pending a later maintainer decision; do not schedule it.

Approved and standing, in this issue's scope:

1. Plain .napl file with no frontmatter and no metadata blocks; napl generate emits as many output files as the code needs (multi-file output per gen).
2. .napl/ state dir moves to the project root, next to src/.
3. Derived state is never committed: map.json successor and all caches live under .napl/cache/, ignored via .napl/.gitignore, disposable and rebuilt from the store.
4. Prompt purity: 'Where this code lives' boilerplate deleted from all authored prompts; the gen harness injects placement/scope/deps paths at gen time.
5. Retirement/fold prerequisites as already sequenced (prompts-at-gen + journal into the store; ir/mapl/attribution + map.json fold).

The spec doc's design A (physical adjacency) and its downsides analysis (PR noise via linguist-generated, 0444 + drift defense, merge-by-regen convention) remain the recorded recommendation for whenever the pin is revisited.

[omridevk]

Folding #42 into this issue per the maintainer - quality scoring is part of the same language contract, not a separate track.

6. SCORE-FIRST STEP (fallow-integral): every gen/reconcile/heal step produces a {score, grade, verdict} BEFORE acceptance, recorded in the step's store event alongside usage telemetry; napl health is the sibling query to napl cost over the same event graph. Real fallow (repo dep 2.98.0) scores TS/JS output; the rust target gets a clippy-complexity-lint scorer normalized to the same contract until a rust-capable fallow exists. Gate semantics mirror fallow audit: findings INTRODUCED by the change fail the step, pre-existing debt reports without blocking. Fallow joins the gen agent's sanctioned tool allowlist so agents self-score during the attempt loop. Root causes of current generated-code quality (correctness-only gate, additive-patch accretion, monolith output, std-only hand-rolling) and the periodic from-scratch-regen remedy are recorded in Generated Rust quality gate: complexity lints in the gen loop + periodic from-scratch regens #42's description; that issue is closed as folded into this one.

[omridevk]

Entry-point options for the fallow integration (item 6), from fallow's actual surface (loaded via the repo's tanstack intent skill, fallow#fallow; fallow already in package.json intent.skills):

Option A - typed CLI envelope as the step gate (recommended authority). napl invokes fallow audit --base <ref> --format json --quiet per step through the same process seam that spawns engines today. The envelope is a typed contract: root kind field, pinned SchemaVersion, exit codes 0/1/2 (1 = findings, not error). Score source: fallow health --score (0-100 + grade, CRAP metric, complexity breakdown). Baselines (--save-baseline, --dead-code-baseline, --health-baseline, --dupes-baseline) give exactly the new-findings-fail / old-debt-reports semantics item 6 pinned. Zero coupling, version-pinned, sub-second.

Option B - in-attempt self-scoring via the agent tool whitelist (complement to A). The gen harness adds fallow to the injected --allowedTools set plus a preamble instruction: run fallow audit before finishing an attempt. The agent fixes findings before the authoritative gate (A) ever sees them - fewer burned attempts. fallow's own agent rules apply verbatim (--format json --quiet 2>/dev/null, || true, never watch).

Option C - repo-level hooks, available today with zero napl work. fallow hooks install --target agent / --target git gates commits and pushes on fallow audit at the repo level (Claude Code PreToolUse). Covers the site/ts lanes immediately, independent of the language integration.

Option D - CI surface. --format sarif into GitHub Code Scanning; fallow ci reconcile-review auto-resolves stale PR review threads keyed by finding fingerprint. Post-gate reporting, not the gate itself.

Option E - the deep one: link fallow's Rust engine into napl-cli. fallow is Rust-native (cargo install fallow-cli). Investigate consuming it as a crate dependency instead of a subprocess: the scorer becomes part of the language binary itself - no npm/node requirement for scoring ts output, one process, shared config loading. Needs investigation of whether fallow-rs publishes a library crate with a stable API or only the CLI binary. If library linkage exists, E supersedes A's subprocess while keeping A's envelope semantics as the internal contract.

Scope limit that shapes all options: fallow analyzes TS/JS only. For the rust target the {score, grade, verdict} contract is fed by the clippy-based scorer regardless of entry point; fallow scores the ts target and site lanes.

Recommendation: B now (whitelist + preamble, trivial), C now (repo hooks), A as the language step-gate when item 6 lands, E investigated during that build and adopted if the crate surface exists. Scores persist to store events per item 6 in every variant.

[omridevk]

Maintainer order (2026-07-27) for item 6's rust scorer: MAXIMUM clippy, NO inline suppressions.

Lint policy:

• Deny by default: warnings, clippy::all, clippy::pedantic, clippy::nursery, clippy::cargo.
• Curated additions from the restriction group (it cannot be enabled wholesale - it contains mutually contradictory lints): unwrap_used, expect_used, panic, todo, unimplemented, dbg_macro, and print_stdout/print_stderr denied everywhere except the designated CLI output seams.
• Complexity thresholds via clippy.toml: cognitive-complexity, too-many-lines, too-many-arguments, type-complexity - tuned once against freshly written (not accreted) modules, then frozen.
• NO INLINE ESCAPES: #[allow(...)] is banned in generated code, enforced twice - clippy's allow_attributes lint set to deny, plus a harness grep gate that fails the gen if any allow/expect attribute appears. Any genuinely needed relaxation lives ONLY in the central workspace lints table / clippy.toml: one reviewed file, visible in diffs, maintainer-approved. This extends the standing no-escape-hatches rule from agent conduct into the code itself.
• Config home: [workspace.lints] in the toolchain-generated workspace manifest + clippy.toml beside it; every generated crate inherits lints.workspace = true.

Rollout (per-crate, not flag-day):

1. Gate scope is the gen's own crate (cargo clippy -p --all-targets), matching the planned per-module gate narrowing. A gen passes only if ITS crate is clean under the full policy.
2. Existing crates are not retro-gated; they become clean as the from-scratch regen wave reaches them, scheduled against real cost data from napl cost. Expected side effect per tonight's evidence (gen 330): from-scratch regens under a quality gate come out structurally better, not just lint-clean.
3. Pilot the thresholds on 2-3 modules before enforcing campaign-wide, since pedantic+nursery on large modules will cost attempts; measure the attempt-rate delta on sonnet vs opus via the telemetry A/B before locking tiering for gated regens.
4. The clippy finding counts/densities normalize into the {score, grade, verdict} contract recorded on the step's store event - same schema as fallow's scores on ts output.

[omridevk]

Rust-scorer sourcing survey (2026-07-27): no single fallow-equivalent exists for Rust; the scorer composes per-axis tools behind the {score, grade, verdict} contract.

Chosen stack:

• clippy = enforcement layer (the max-lints no-suppression policy above). 700+ lints, ships with the toolchain.
• mozilla/rust-code-analysis = numeric score source: per-function cyclomatic, cognitive, Halstead, and maintainability-index metrics, JSON output, available as CLI AND as a Rust library crate - linkable directly into napl-cli like the fallow-engine option. MUST-VERIFY before adoption: maintenance status has slowed; pin a rev and check tree-sitter grammar currency for modern Rust syntax.
• cargo-machete (fast) or cargo-udeps (thorough) for unused dependencies; rustc dead_code covers unused items.
• Duplication is the honest gap: PMD CPD / jscpd have token-based Rust support as a bolt-on; alternative is a small token-fingerprinting pass inside the scorer (fallow-style hashing over the parse tree we already have via rust-code-analysis).

Rejected for this use: SonarQube (server-shaped platform, not a CLI gate), Qlty (aggregator wrapping the same underlying tools - adds a layer without adding an engine), CodeScene (commercial; its churn-x-complexity hotspots concept is worth imitating from store event history instead, where napl already has better provenance data than git alone).

Sources: analysis-tools.dev Rust tool rankings, code-quality.io 2026 Rust static-analysis roundup, mozilla.github.io/rust-code-analysis docs (metrics + JSON commands).

[omridevk]

Item 7, maintainer-ordered (2026-07-27): DETERMINISTIC TOOLCHAIN - ONE LLM CALL PER GEN.

Finding that triggered it: every gen today runs the coding agent PLUS a stack of hidden llm_complete derivation calls (mapl line-map alignment, IR, attribution assist), all on the gen model, all invisible pre-telemetry. This is cost, latency, and nondeterminism in a content-addressed toolchain.

Principle: the coding agent is the ONLY model call in a gen. Every derivation becomes deterministic or a verified byproduct of that one call:

• mapl: the coding agent emits the prompt-section -> code-line mapping as a byproduct of the session it is already in (it knows what it is implementing as it writes). A DETERMINISTIC validator checks the emitted map (ranges exist, coverage complete, monotonic) - agent self-reports are verified, never trusted. No post-hoc alignment call.
• IR: extracted from the generated code by deterministic parse (syn / tree-sitter): functions, types, signatures are facts, not judgments.
• attribution: pure bookkeeping derived from the mapl + gen event; no model involvement.
• drift: hash comparison always-on (exists today); semantic comparison only on explicit user demand, never in the gen loop.

Consequences: gen = one agent session + pure Rust (parse, validate, hash, commit). Reproducible: same prompt + same agent output -> byte-identical derived artifacts. The checkModel split landing in tonight's telemetry batch is TRANSITIONAL - it cheapens the derivation calls until this item deletes them.

Sequencing: after store activation + retirement (do not rip out derivation mid-campaign); pairs naturally with the multi-file-output rework since the agent-emitted mapping must speak tree paths anyway.

[omridevk]

Item 8 (maintainer question 2026-07-27: why one store writer?): PARALLEL GENS. The journal's serialization was structural (append-only file); the store's is policy. Object writes are conflict-free by construction (content-addressed, idempotent); the only critical section is the HEAD advance. Design: commit path gains compare-and-swap semantics on HEAD (re-read, re-parent event, retry on contention) using the existing locks/attempts transaction machinery; per-gen cargo target dirs remove the build-lock contention (disk-for-throughput trade). Then N gens on disjoint modules run genuinely concurrently - multiplies regen-wave throughput. Sequenced after the commit path has telemetry (so contention retries are measured) and after one uncontended campaign wave proves the single-writer path; the CAS change itself is spec-changing (store_txn_core/store_io/cmd_gen_io prompts, opus class).