Forkable docs (f05): spec and implementation — fork/unfork/status/update/diff/list

[jlevy]

Summary

This PR delivers the complete forkable-docs workflow (format f05): the full design spec (docs/project/specs/done/plan-2026-06-11-forkable-docs.md) together with its implementation through all phases (0–5). The spec's golden maps are captured from the built CLI; no phase-contract placeholders remain for shipped behavior.

What ships

• Format f05: stamp-only f04→f05 migration with seamless shared-layout co-migration; older CLIs refuse f05 repos; documented abort/rollback path
• The tbd docs surface: bare overview (three postures: hidden / curated / everything, shared menu with setup), list (cross-kind, state markers), show <name> (kind-agnostic, --section/--sections, provenance notes), manual, sync (canonical; tbd sync --docs deprecated alias), add <docref> (canonical docrefs recorded; per-kind --add kept as aliases), fork (by name, --all, or --category from declared frontmatter), unfork, update (three-way merge, --merge/--keep-ours), diff (--base/--upstream), status
• Fork kernel: committed manifest + base snapshots, hash-derived states (upstream/forked/customized/conflicted/missing/local), POSIX paths on every platform, version-skew guards, out-of-band deletion handling, generated fork-dir README
• docs_cache.local_dirs: serve additional in-repo doc directories between the fork dir and the cache (state local, not forkable)
• One data model: every inventory and read surface builds docmap entries through a single constructor; per-kind --list --json emits the same docmap filtered by kind
• reference kind: docref-format and docmap-format reference docs served from the cache; the self-docs tbd-docs/tbd-design registered as forkable reference docs with bundled fallback
• tbd doctor: a Forked docs health-check group (corrupt manifest, missing/orphaned, base integrity, conflict markers, reserved names, gitignored fork dir) with --fix
• Reporting: tbd status Docs line (forks only; zero-fork output byte-identical), setup --auto Docs summary, sync drift notice; --interactive removed
• Agent & docs surface: skill routing rows, two-axis welcome-user onboarding, suggest-upstream-improvements playbook, README/development.md/docs-overview updates, manual chapters (two modes, sync taxonomy, configuration reference), CHANGELOG f05 entry
• Hardening: tbd itself now ignores ambient GIT_DIR and always operates on the cwd repository (with a one-time notice) — safe inside any git hook; the pre-push test suite carries two independent scrub layers; cross-platform fork e2e runs on Windows CI
• Release readiness: tests/qa/release-v0.3.0-forkable-docs.qa.md — the manual end-to-end playbook (real-repo f04→f05 upgrade, fork lifecycle, hook safety, release cut, global swap)

Review and tracking

• Full senior review: Forkable docs (f05): spec and implementation — fork/unfork/status/update/diff/list #169 (comment) (41/41 findings resolved)
• Merge-readiness ledger: Forkable docs (f05): spec and implementation — fork/unfork/status/update/diff/list #169 (comment)
• Spec epic tbd-67ek closed with all 12 phase beads; tbd-tgwi (ambient GIT_DIR) fixed in this PR. docs_cache.fork_dir remains a reserved, documented in-era key; the older docs-config-redesign spec is retitled to its f06+ era.

CI green on ubuntu/macos/windows + coverage/lint + benchmark; suite at 1,313 unit tests + 881 tryscript blocks. PR #170 was combined into this one.

https://claude.ai/code/session_01X8S12JzmmxEfLpYzgH8Y7E
https://claude.ai/code/session_01QPsCSYGtwR8JtX2R1aaxyh

[deepsource-io]

DeepSource Code Review

We reviewed changes in 64b2831...acce5aa on this pull request. Below is the summary for the review, and you can see the individual issues we found as inline review comments.

See full review on DeepSource ↗

PR Report Card

Overall Grade

Security   Reliability   Complexity   Hygiene

Code Review Summary

Analyzer	Status	Updated (UTC)	Details
Secrets		Jun 13, 2026 1:57a.m.	Review ↗

---

Important

AI Review is run only on demand for your team. We're only showing results of static analysis review right now. To trigger AI Review, comment @deepsourcebot review on this thread.

[github-actions]

Coverage Report for packages/tbd

Status	Category	Percentage	Covered / Total
🔵	Lines	33.72%	2849 / 8447
🔵	Statements	33.66%	2950 / 8763
🔵	Functions	38.8%	449 / 1157
🔵	Branches	29.76%	1342 / 4508

File Coverage

File	Stmts	Branches	Functions	Lines	Uncovered Lines
Changed Files
packages/tbd/src/cli/commands/docs-fork.ts	2.36%	0%	2.12%	2.56%	67-71, 90-745, 761-765, 778-784, 792-795, 802, 812-818, 831
packages/tbd/src/cli/commands/docs.ts	3.75%	0%	0%	3.96%	61-426, 434, 445, 455-458, 470, 477
packages/tbd/src/cli/commands/doctor.ts	3.68%	4.91%	3.65%	3.85%	220-2106, 2119-2120
packages/tbd/src/cli/commands/guidelines.ts	1.81%	0%	0%	1.96%	28-122, 138-139
packages/tbd/src/cli/commands/setup.ts	2.18%	0%	0%	2.23%	80-103, 123-152, 170-229, 361-388, 425-450, 474-1732, 1768-2231, 2249-2287
packages/tbd/src/cli/commands/shortcut.ts	0.69%	0%	0%	0.7%	49-371, 390-391
packages/tbd/src/cli/commands/status.ts	0.74%	0%	0%	0.76%	94-461, 468-469
packages/tbd/src/cli/commands/sync.ts	1.08%	0%	6.45%	1.09%	101-1198, 1215-1216
packages/tbd/src/cli/commands/uninstall.ts	0.79%	0%	0%	0.8%	27-301, 311-312
packages/tbd/src/cli/lib/data-context.ts	18.18%	7.5%	28.57%	19.23%	97-102, 106-109, 123-220, 236, 239-247, 263-264, 298-310
packages/tbd/src/cli/lib/doc-command-handler.ts	0%	0%	0%	0%	65-402
packages/tbd/src/cli/lib/doc-serve.ts	6.45%	0%	0%	6.66%	51-157
packages/tbd/src/cli/lib/docs-menu.ts	0%	100%	0%	0%	22-29
packages/tbd/src/cli/lib/docs-sync-output.ts	0%	0%	0%	0%	17-102
packages/tbd/src/cli/lib/output.ts	53.48%	52.84%	53.57%	51.51%	69-71, 77-79, 111-113, 198-282, 305, 309-333, 371-374, 430, 460-465, 541-607
packages/tbd/src/docmap/docmap.ts	97.56%	88.88%	100%	97.29%	135
packages/tbd/src/docmap/index.ts	100%	100%	100%	100%
packages/tbd/src/docref/docref.ts	96.34%	93.47%	100%	96.29%	112, 136, 279
packages/tbd/src/docref/index.ts	100%	100%	100%	100%
packages/tbd/src/file/common-dir-layout.ts	11.53%	11.42%	11.11%	11.53%	34-155, 203-241
packages/tbd/src/file/doc-add.ts	87.75%	79.41%	100%	87.75%	86, 106, 140, 143-146, 153-154
packages/tbd/src/file/doc-fork.ts	90.97%	82.89%	78.26%	93.6%	75, 257-266, 304, 348, 365, 367, 406, 429, 488
packages/tbd/src/file/doc-sync.ts	54.45%	40%	73.68%	54.45%	72, 94-107, 198-206, 218-222, 235, 248-249, 263, 323-325, 334-339, 344-369, 384, 399-402, 446-494, 635-726
packages/tbd/src/file/fork-manifest.ts	94.79%	93.1%	96.15%	94.38%	166-168, 331, 372
packages/tbd/src/file/fork-update.ts	94.28%	91.37%	87.5%	94.28%	142, 329-332
packages/tbd/src/file/git.ts	70.82%	60.06%	66.29%	71.15%	164-187, 215, 237, 239-240, 264, 274-362, 470, 615-616, 781-838, 907-910, 944, 1060-1067, 1088-1094, 1101-1108, 1120, 1124-1134, 1139-1146, 1159, 1177, 1189, 1200-1252, 1268-1274, 1278, 1283-1287, 1313, 1348, 1400, 1439-1452, 1494-1547, 1584, 1690, 1693, 1697, 1773, 1834, 1844, 1855, 1918-1921, 2027-2077, 2143-2146, 2170, 2173-2174, 2180-2181, 2269, 2286, 2306, 2316-2320
packages/tbd/src/file/github-fetch.ts	95.83%	81.81%	87.5%	95.83%	121, 242
packages/tbd/src/lib/doc-categories.ts	16.66%	0%	0%	16.66%	13-24
packages/tbd/src/lib/git-env.ts	78.57%	66.66%	100%	83.33%	56, 59-62
packages/tbd/src/lib/paths.ts	85.27%	69.69%	60%	85.93%	164, 168, 172, 429-443, 515, 551-553, 569-592, 633-643
packages/tbd/src/lib/schemas.ts	100%	100%	100%	100%
packages/tbd/src/lib/tbd-format.ts	100%	97.22%	100%	100%

Generated in workflow #1002 for commit acce5aa by the Vitest Coverage Report Action

[jlevy]

Senior Review: PR #169 — Forkable Docs (f05) Spec and Kernel

Reviewed commit: 25bc008 · Date: 2026-06-12 · Also saved in-repo at docs/project/reviews/review-2026-06-12-pr169-forkable-docs.md (branch claude/jolly-hawking-ttkxgw).

Scope: (1) adversarial senior review of the spec and the shipped code; (2) holistic review of documentation state across the new surface area; (3) analysis of two open design questions — copy-all-plus-gitignore vs export-only forking, and doc-map registry vs search-path resolution; (4) deep review of the DocRef and DocMap abstractions.

All critical findings were verified empirically against the CLI built from this branch (pnpm build, fresh sandbox repo), not just by code reading.

1. PR Review

CI: all 5 checks green (ubuntu/macos/windows tests, benchmark, coverage+lint). DeepSource grade A. No unresolved human review comments (only bot comments). Unit coverage of the new core modules is 89–97%; the new CLI layer is ~3% under vitest but covered by tryscripts — which CI runs only on ubuntu (ci.yml matrix runs vitest run per-OS; tryscripts only run inside the ubuntu Coverage & Lint job).

Summary: The spec is genuinely strong — the doc model in tbd-design.md §2.9 (copies table, seven invariants, derived-not-stored state) is the best artifact in the PR, and the merge/update design (stored bases, decision table, version-skew guard) is sound and exhaustively unit-tested (129 tests, verified locally).

Scope note (settled during review): this PR is committed to being the complete, full f05 experience and will ship as the next release — there is no separate shipping step. The spec's phases therefore serve as the strict validation checklist for driving this branch to done (finding 4 audits the branch against them). Within that frame, there are two user-facing bugs (verified end-to-end against the built CLI) plus one data-clobbering edge case that are release blockers.

Critical findings (verified empirically)

1. Forked shortcuts are never served — the feature's core promise (G2) is broken for one of three kinds. doc-sync.ts:561–566 persists docs_cache.lookup_path into every repo's config on setup (verified in a fresh sandbox repo). shortcut.ts:78 then does config.docs_cache?.lookup_path ?? DEFAULT_SHORTCUT_PATHS — the persisted key wins, and it contains only cache paths. Result, reproduced with the built PR CLI: fork review-code, edit it, and tbd docs list shows [forked, customized] while tbd shortcut review-code serves the upstream copy. Guidelines and templates work because they don't honor lookup_path. This is precisely the "lookup_path zombie" the spec itself cites as a lesson, and it falsifies §2.9 invariant 1 as written. The tryscripts miss it because they only fork a guideline. Fix: prepend the fork dir structurally in shortcut.ts regardless of config ([FORK_SHORTCUTS_DIR, ...(lookup_path ?? CACHE_SHORTCUT_PATHS)]), and add a fork-a-shortcut serve assertion to the tryscript.

2. The CLI recommends a flag that doesn't exist — and the golden test pins it as correct. docs-fork.ts:357 (zero-fork tbd docs status) prints Make some visible: tbd docs fork --category=general (and your languages). --category is a Phase 4 feature; running the suggested command errors with unknown option '--category=general' (verified). cli-docs-fork.tryscript.md:36 golden-tests this hint, so CI enshrines a broken recommendation that agents will follow verbatim. Fix: either implement --category now (the spec says it reuses existing frontmatter metadata, so it's small) or change the hint to name-based forking until Phase 4.

3. git merge-file error exits are misread as conflict counts → can overwrite a customized doc with empty content. fork-update.ts:66–79 treats any positive exit code as a conflict count. Verified: git merge-file exits 255 on errors (e.g. binary-content refusal), and git documents the conflict count as truncated to 127 — so exit 255 means error, not "255 conflicts". On that path stdout is empty, updateOne returns merged-conflict, and the handler writes empty content over the user's customized fork (git-recoverable, but still a silent clobber). Fix: treat code > 127 as an error; one-line guard plus a unit test.

Major findings

4. Phase-completion audit: the branch is partway through every phase — use the phases as strict completion gates, and reconcile the golden maps. This PR delivers the complete f05 experience as the next release, so the spec's phases are the in-PR validation checklist rather than separate shipping steps. Audited against the spec, the branch currently stands:

• Phase 0 (contracts + docs): spec contracts authored ✔; tbd-design.md §2.9/§4.13 ✔; tbd-docs.md drift table + abort recipe ✔. Missing: the tbd docs command-group manual section, the three-sync taxonomy table and .tbd/ layout contract in tbd-docs.md, the docs-overview.md and README forkable-docs rows, references/docref-format.md, references/docmap-format.md, suggest-upstream-improvements.md, and item 0.5 (lock golden maps against real output).
• Phase 1 (format + kernel): f05 stamp + migration ✔ (but stamp-only: no .tbd/.gitignore refresh, no generated .tbd/README.md, and FORMAT_HISTORY.f05 omits the docs_cache.fork_dir/local_dirs keys the spec defines — finding 5); docref/docmap modules ✔ (but wired into nothing — see §4); fork-manifest ✔; fork/unfork ✔; precedence wiring ✔ for guidelines/templates, broken for shortcuts (finding 1). Missing: tbd docs sync subcommand, the serve provenance note (Decision 18), per-kind --list markers.
• Phase 2 (status/browse/doctor): status, list, diff ✔. Missing: bare tbd docs overview (old viewer still the default action), show/manual, the shared docmap renderer + per-kind reader migration, local_dirs, docs add <docref>, grouped sync, all doctor checks, the tbd status Docs line.
• Phase 3 (update/merge): merge module + update command ✔ (with the exit-code bug, finding 3); pending-update reporting in setup missing; the tbd sync drift notice is a good addition not in the spec — add it there.
• Phase 4 (categories/setup): nothing landed yet, but --category is already recommended by shipped output (finding 2); setup Docs summary and --interactive removal pending.
• Phase 5 (agent surface): nothing landed (skill upgrade hints only); reference kind + self-docs, skill routing rows, welcome-user onboarding, and CHANGELOG pending.

Two consequences. First, the golden maps and the shipped output must be reconciled into one source of truth now, because they already disagree in a dozen places (e.g. spec: Updated 2 forked docs: / impl: Updated 1 forked doc(s):; spec: 1 doc is missing (forked file deleted): / impl: 1 doc(s) missing (forked file deleted or renamed):; spec fork output has a Recorded base in .tbd/doc-forks/… line the impl never prints; spec unfork refusal points at tbd docs diff, impl points at tbd docs status; spec list --json includes stale and word_count, impl emits neither). Recommendation: update the spec maps to the (mostly better) shipped wording, then treat them as binding for the remaining phases — validating each phase against its golden block as it lands. Second, the PR title/description should state this scope (full experience, next release) and carry the phase checklist; as of 25bc008 they still described a spec-only PR.

5. The f05 format definition drifts from its own spec, and a release cut now would ship a confusing hybrid.

• FORMAT_HISTORY.f05 (tbd-format.ts) omits docs_cache.fork_dir, local_dirs, and the generated .tbd/README.md that the spec's contract table defines as part of f05; the migration is stamp-only (no .tbd/.gitignore refresh, no .tbd/README.md), and fork_dir is not configurable at all — FORK_DIR is a hard constant (paths.ts:361), contradicting Resolved Decision 6. Additive later landing is probably fine, but then amend the spec so f05's definition matches what f05 actually stamps.
• The old docs surface coexists with the new: tbd docs --list (sections) and tbd docs list (docs) both work with different meanings; the command description still says "use tbd sync --docs". The spec's safety argument is "everything ships in the same release behind the f05 gate" — and since this PR is that release vehicle, the argument holds only once the surface re-homing lands here too. Concretely: docs: Address distributed systems review feedback in ceads-design #1, feat: tbd-cli v1 complete implementation #2, and the disposition of all four old tbd docs behaviors are release bars for this PR.
• The config migration also reorders keys (lookup_path moved to the bottom of docs_cache in this repo's own diff) — harmless but contradicts "metadata-only stamp" minimal-churn expectations.

6. Windows is untested for the whole feature and has at least one real defect. FORK_DIR = join(DOCS_DIR, 'tbd') is docs\tbd on Windows; forkRelPath() then records docs\tbd/guidelines/x.md (mixed separators) into the committed manifest and CLI output. Meanwhile the unit tests use a different constant — DEFAULT_FORK_DIR = 'docs/tbd' (POSIX literal in doc-fork.ts:44) — so Windows CI green proves nothing about production paths: tests exercise a value production never uses. And tryscripts don't run on Windows at all. Fix: one POSIX-string constant for repo-relative semantics (join only at fs boundaries), delete the duplicate, and run the fork tryscripts in the OS matrix. (Related: docref rejects C:/... as "unknown scheme" — see §4.)

Minor findings

7. conflicted never clears in the stored manifest. After resolving markers, state computes correctly (flag AND markers), but the committed forks.yml keeps conflicted: true until some later update writes the entry (docs-fork.ts:461–465 only clears when an update applies). Spec says "auto-clears". Cosmetic but confusing in a committed file.
8. tbd docs update <typo> silently reports "All forked docs are up to date" — unknown names are filtered out without error (docs-fork.ts:418–419).
9. Path-traversal hardening: unforkDoc/updateOne compute fs paths from committed manifest name/kind without validating for separators/.. — a hostile forks.yml in a cloned repo can direct rm/writes outside the fork dir. Validate names (no /, \, ..) on manifest read.
10. pathExists reads the whole file to test existence, then callers re-read it (doc-fork.ts:69–76); it also swallows non-ENOENT errors. Use stat, or read once and branch on ENOENT.
11. Fork-conflict error is not actionable (raw "already exists and is not an unmodified fork", no options) — violates the spec golden and error-handling-rules ("tell users what to do next"); contrast unfork, which does it well.
12. tbd docs list --kind=bogus / fork --kind=bogus silently produce empty results or a misleading "No doc found" (KIND_CACHE_PATHS[kind] ?? []). Validate the kind.
13. FORK_KINDS includes 'reference' but KIND_CACHE_PATHS doesn't — a forked reference doc (Phase 5) would permanently read orphaned. Latent trap; add a comment or a guard now.
14. The sync drift notice writes via process.stderr.write (sync.ts ~218), bypassing the output layer the spec's style contract mandates; update --json returns prose strings in needsDecision rather than names.
15. UpdateAction includes a never-returned 'noop' member; the update tryscript uses GNU-only sed -i (fails if run locally on macOS).
16. Spec is behind the code in one place: the version-skew guard (skip-newer-base, a good design addition, documented in §2.9 invariant 7) has no row in the spec's update decision table. Add it.

Strengths worth keeping

The derived-state design (no stored tracking ⇒ git can't desync it) is the right call and §2.9 articulates it precisely; the stored-base three-way merge is the correct answer to the shadcn-has-no-update-story problem and Alternatives #5 justifies the cost honestly; out-of-band deletion as a supported state with exactly two resolutions is excellent UX thinking; the decision-table unit tests cover every row × strategy including the skew guard; the abort-upgrade recipe with its state-inventory table is the kind of operational doc most projects never write; and the drift tryscript (rename → missing+local, prune-on-empty) tests realistic mess, not just happy paths.

2. Holistic Documentation Review

Surface inventory after this PR:

Surface	Audience	State
tbd-design.md §2.9 + §4.13	contributors/design	Strong, new canonical doc model
tbd-docs.md "Forked Docs in Your Repo" + "Aborting a Format Upgrade"	users	Good, but see below
README "Upgrading" + skill-baseline upgrade hints	users/agents	Good; upgrade ergonomics now first-class
development.md format-upgrade section	contributors	Good
The plan spec	design	Strong but now diverges from code (§1, finding 4)
docs-overview.md	contributors	Stale — still describes only the old --add flags; its Phase 0 contract row was not executed
tbd-docs.md command reference	users	Missing the new commands entirely — fork/unfork/status/update/diff/list ship in this PR but are documented nowhere in the manual; discovery is --help only
welcome-user.md, skill routing rows, suggest-upstream-improvements.md, docref-format.md, docmap-format.md	agents/users	Absent (Phases 0/5 promised, not landed)

Code currently leads docs on this branch: the commands exist but the manual, the onboarding, and the agent routing don't yet — today an agent discovers tbd docs fork via --help and the (broken) --category hint with zero guidance. Since this PR is the complete experience, the docs must catch up before release; concretely:

1. Re-couple agent surface to command surface. The minimal skill routing rows ("make guidelines visible" → fork; "update the guidelines" → update; "I deleted a forked file" → status/restore/finalize) and a short tbd-docs.md "Managing docs" section are part of this PR's release bar — a command without its routing row is invisible to the primary operator (agents).
2. One first-principles "Managing docs" chapter in tbd-docs.md, opening with the two-mode model users actually face — hidden cache (default: docs live in gitignored .tbd/docs/, always active, zero repo footprint) vs forked (tracked in docs/tbd/, visible on GitHub, editable, mergeable) — then the scope axis (all vs by category), then commands, then the drift table. The content exists today but is scattered across the spec (two-axis framing), §2.9 (model), and tbd-docs (drift table). The spec's Documentation Contract table already prescribes exactly this; execute it.
3. Avoid the dual-drift-table trap. The user-action table in tbd-docs.md and the drift matrix in §2.9 describe the same truths in different words for different audiences — fine per the ownership/audience rule, but make one canonical (suggest §2.9) and have the other cite it, so the next state addition doesn't fork them.
4. Upgrade workflow is now well covered (README → manual troubleshooting → development.md → §2.9, correctly layered by audience). One gap: the README "Upgrading" section should add one line: "if you've forked docs, tbd sync will tell you when upstream moved — run tbd docs update."
5. docs-overview.md needs its promised rewrite (the tbd docs group, docref-based add, fork mention) — it's the repo's own orientation doc and currently teaches the superseded surface.
6. The three-sync taxonomy (sync vs setup vs docs sync vs docs update) is the single most confusion-prone area for users; the spec's 4-row table is the right artifact and should land in tbd-docs.md verbatim as the contract says (today a 3-row variant lives only in design §4.13).

3. The Two Design Questions

Q1: Copy-all-and-gitignore-some vs export-only-the-forked

Recommendation: keep the current export-only model. Don't build the gitignore-workflow variant, even as an option.

• Gitignored mirrors don't actually deliver the visibility goal. The original complaint is "can't browse them on GitHub, can't check them in." Gitignored files appear in neither GitHub nor PRs — so under copy-all, the unforked majority is exactly as invisible to the team as .tbd/docs/ is today; the cache has only moved to a prettier path. The only incremental benefit is local-editor browsing, which .tbd/docs/ (plain local files) already provides.
• It creates the worst silent failure mode in the design space. A user or agent will edit a gitignored mirror file (they sit right next to tracked ones, and agents grep first and check ignore rules never). The edit works locally, is served (fork dir has top precedence), and silently never reaches the team — no commit, no PR review, no record. The current design makes visibility an explicit act (fork = start tracking), so divergence is impossible without a git-visible artifact. That property is load-bearing; the copy-all option destroys it.
• Gitignore-as-state-machine is fragile in well-known ways: ignoring an already-tracked file does nothing (the docs: Address distributed systems review feedback in ceads-design #1 gitignore confusion); git add -f accidents; "is doc X forked?" becomes a predicate over two systems (index + ignore rules) instead of one manifest — and the derived-state achievement (§2.9 invariant 5: no sequence of git operations can desync tbd) stops holding, because ignore rules are not content.
• Upgrade and deletion semantics get contradictory. Today "nothing is ever silently re-created against the user's deletion" is a clean principle. With a mirror, tbd docs sync must re-write unforked mirror files on every upgrade — so deleting one either resurrects (violating the principle) or requires tombstones (new state machinery).
• "See everything, then choose" is already served — and better: tbd docs list (with sizes/descriptions) is the catalog; tbd docs show <name> (Phase 2 — worth pulling earlier, since it's the browse-without-forking command); and crucially tbd docs fork --all is the copy-all option in tracked form: every doc visible in docs/tbd/, on GitHub, with the manifest tracking all of them — and unfork (or just not committing) is the undo. A user who wants the all-visible experience can have it today with one command and real visibility, instead of a fake one.

The fallback intuition — tbd should always use docs/tbd as the first source and fall back to its internal cached copies — is exactly what's implemented (the precedence list, §2.9 invariant 1) and is correct independent of this choice; it's also what makes out-of-band deletion degrade gracefully. The action item from §1 is just to make it true for shortcuts.

One simplification to consider: since fork --all is the sanctioned "show me everything" path, make the zero-fork tbd docs overview present three named postures — hidden (default), curated (--category), everything (--all) — so the catalog→choice flow is explicit without any gitignore machinery.

Q2: Granular doc-map registry vs search path

What the PR actually builds is a hybrid, and the split is right — but it should be stated as a principle, in the docs, because right now it has to be reverse-engineered. The principle is:

Resolve by convention; track only what cannot be derived; publish the inventory as a generated view.

• Resolution = search path (fork dir → [future local_dirs] → cache, first-match-wins, names-are-identity, flat kind dirs). Right because: zero registration ceremony (drop a file → served, the local state for free), and a registry can't be wrong if there is no registry — every stale-registry failure mode (file says X, disk says Y) is structurally impossible. The lookup_path bug found in §1 is the cautionary tale for the alternative: the one place resolution is config-state-driven is exactly where the feature broke.
• The manifest tracks only the non-derivable fact: which upstream a fork came from and the base snapshot at the fork point. A merge base cannot be recomputed from disk; everything else (customized/stale/missing/local) is derived by hashing. So the manifest is minimal by construction — it's not a doc registry, it's a provenance ledger, and only for docs that have an upstream relationship.
• docmap = generated view, so it's always true. Making it authoritative (file-by-file management in docmap format) would mean: every add/rename/delete must update the map (agents and humans will skip this), map merge conflicts, and the stale-registry failure class — in exchange for capabilities not yet needed.

What is given up, honestly stated: (1) arbitrary layouts — a team with an existing guidelines tree in non-tbd layout can't map names→paths; they must move files into <kind>/<name>.md (or wait for local_dirs); an authoritative map could redirect per-doc. (2) Per-doc metadata overrides and ordering (title/description live in frontmatter only). (3) A committed machine-readable inventory for external consumers to read from the repo at rest (today they'd have to run tbd docs list --json).

Why the loss is acceptable and reversible: the docmap format already carries everything an authoritative registry would need (type, name, path, source). So the future move — "tbd can also read a committed docmap as a doc source" (#117's framework, as 'operations over docmaps') — adds a consumer without changing the format or breaking anything shipped. The option stays open at zero cost. What should not happen is making the docmap authoritative preemptively: wait for a concrete user with the arbitrary-layout problem.

Two refinements to make now so the story is crisp (both feed §4):

• Fix the location inconsistency: tbd's own docs list --json currently emits upstream entries with neither path nor source (docs-fork.ts:573–581), while the format's definition says every entry has a location. Emit source: internal:… for upstream docs (already computed for forking). Then the docmap is genuinely usable as an inventory by third parties.
• Write the one-paragraph "resolution is by search path; the docmap is a view, not an input (today)" statement into docmap-format.md and §2.9, so nobody later "fixes" the system into registry-driven resolution by accident.

4. Deep Review: DocRef and DocMap Abstractions

Both modules are well-built as code: dependency-free as claimed, small public APIs, spec-mirror tests, extraction-ready. This section is about the abstractions — where they're exactly right, and where v0.1 currently overcommits or under-specifies in ways that are cheap to fix now and expensive later. Verdict: DocRef needs four cuts/clarifications to be the universal grammar intended; DocMap needs two tightenings and one deletion. Neither needs more features.

DocRef (src/docref/docref.ts)

What's right and should be preserved: single-string, totally-parsed, typed result; the // repo/path separator is genuinely better than GitHub's own blob-URL ambiguity (branch names containing / parse correctly — github:o/r@feature/x//path works because the separator is unambiguous); normalization of blob/raw web URLs to one canonical form is exactly the right kind of opinion; idempotent parse∘format round-trips are tested.

Issue 1 — cut the git: scheme (overcommitment). git:owner/repo//path has no hostname, so it's unresolvable — there's nothing a consumer can fetch. Worse, the natural reading git:host.com/owner/repo//path mis-parses today (owner=host.com, repo=owner, path swallows the rest). Shipping an unresolvable, mis-parsing scheme in a v0.1 grammar is exactly the "complexity that might not be correct later" risk: once any manifest contains a git: ref it must be supported forever. Cut it from v0.1; add a host-bearing form (git:host/owner/repo@ref//path) when a non-GitHub/GitLab need actually appears.

Issue 2 — decide the bare-path question deliberately (currently the grammar validates almost nothing). parseDocRef('hello world') succeeds as a local path, so isDocRef() is true for nearly any string and validation is toothless. For a universal address format the grammar should be strict — local paths must start with ./, ../, or / — letting each consumer decide to coerce bare strings at its own boundary (tbd can keep accepting guidelines/python-rules.md in config by prepending ./ before parse). Strict grammar + lenient consumers composes; lenient grammar can never be tightened. If the lenient rule stays, document it as a deliberate decision in docref-format.md, because it surprises.

Issue 3 — two local-path holes: (a) ~/ parses as local but no expansion semantics are defined anywhere — define ("consumers expand to the user home") or reject in v0.1 (recommend reject; it's a config-file convenience that can come later); (b) Windows drive-letter paths (C:/Users/...) hit the unknown scheme rejection (docref.ts:172) since C: matches the scheme regex. CI runs Windows; a Windows user's absolute path is a legitimate address. Either special-case ^[A-Za-z]:[\\/] as local, or document "absolute paths are POSIX-style" — but choose explicitly.

Issue 4 — URL fragments are silently dropped during normalization. https://github.com/o/r/blob/main/f.md#testing normalizes to github:o/r@main//f.md — the #testing vanishes (gitRefFromUrl reads pathname only). For documents, fragments are meaningful (tbd itself has --section). v0.1 can rule fragments out of scope, but silent data loss in a normalizer is the one behavior a format can't afford. Either preserve (add an optional fragment to the git/url kinds — small) or reject refs with fragments with a clear error. Recommend preserve: it's one optional field and it future-proofs section addressing.

Smaller notes: docRefsEqual is syntactic — fine, but say so in the format doc (GitHub owners are case-insensitive; case is deliberately not normalized). internal: is fine to keep in the universal grammar if the format doc defines it app-relatively ("the consuming tool's bundled collection") rather than as tbd-specific. And the reference doc should cite purl (package-url) as prior art and say why it doesn't fit (package-centric identity, no good in-repo file story) — reviewers will ask.

The biggest DocRef gap is not in the code: references/docref-format.md doesn't exist, so the grammar's only spec is a module docstring — while Resolved Decision 10 declares it a "hard rule with no exceptions" and nothing in the shipped code parses a docref anywhere (zero imports outside the module; manifest source strings are built by string concatenation, docs-fork.ts:81–89). Before the next release: write the reference doc, and wire tryParseDocRef validation into at least manifest read and docs status so the hard rule is enforced somewhere real.

DocMap (src/docmap/docmap.ts)

What's right: one object, one entry shape; passthrough() for extension fields with "consumers must ignore unknown fields"; identity uniqueness enforced; self-identifying version tag. This is the right size for v0.1.

Tighten 1 — require a location. The format's own definition says each entry has "a location (path, and/or a provenance source)" but the schema makes both optional and tbd's first producer emits entries with neither (§3 Q2). Add a zod refinement: at least one of path/source per entry. An inventory whose entries can't be located isn't an inventory; this is the single change that makes hand-authored docmaps in other repos actually consumable.

Tighten 2 — pin path-relativity. Nothing says what path is relative to. For a committed docmap file the only sane answer is relative to the docmap file's own directory (the sitemap convention); for generated/streamed docmaps (tbd's --json), relative to a stated collection root. One paragraph in docmap-format.md; without it, two consumers will disagree on day one.

Delete 1 — drop word_count from the core format. It's the only presentation field with a unit opinion baked in, and tbd — the format's first and only producer — doesn't emit it (it renders bytes + approx tokens instead, docs-fork.ts:568). A core field the reference implementation skips is a credibility leak. Let size/length metrics be extension fields (size_bytes, approx_tokens, word_count — whatever a producer has); keep core = identity + location + title/description.

Smaller notes: parseDocMap accepts any docmap/* version — fine for 0.x, but state the policy ("readers accept docmap/0.*, warn on others"). entryKey/groupByType/filterByType are good minimal helpers; resist adding more (no merge/diff helpers until an operation needs them — that's the #117 layer, deliberately deferred).

Using this use case to refine the format: the fork feature is teaching the right lesson — tbd needed state/stale and put them in extension fields, not the core. That's the pattern holding up well. The two places the use case exposes real format gaps are exactly location-requiredness and path-relativity above; fix those and docmap/0.1 is something another tool could adopt as-is.

5. Suggested Next Steps

This PR ships the complete f05 experience as the next release; in priority order:

1. Fix the three correctness blockers — shortcut serving precedence, the --category hint, the merge-file exit-code guard.
2. Reconcile the golden maps with shipped output (one source of truth); from then on validate each phase against its golden block as it lands — the Phase 0.5 discipline, applied in-PR.
3. Complete the remaining phase items per the audit in finding 4; the old-surface re-homing (Phase 2) and the agent surface (Phase 5) are the release-critical tail.
4. Apply the DocRef/DocMap tightenings (§4) while nothing depends on them.
5. Keep the PR title/description current with the phase checklist as items complete.

---

Review session: https://claude.ai/code/session_01QPsCSYGtwR8JtX2R1aaxyh

---

Generated by Claude Code

[jlevy]

Adversarial-review fix pass — Tier 1 & 2 findings resolved

This round closes out the adversarial-review findings (the Review/* beads under the Spec: Forkable Docs Workflow (f05) epic), pushed as two commits:

• 6b4d266 — code fixes + doc corrections
• cf5beae — the two missing unit tests (S4, S6)

Code / state-machine fixes (each with unit or golden coverage)

Finding	Fix
F1 doctor false-positive + half-migrate	An older-but-upgradeable common-dir layout is now a pending migration (warning, exit 0), not a mismatch error (was exit 1, which broke CI on un-migrated repos). doctor --fix routes through prepareDataSyncContext, migrating both config and layout (never layout-only). Corrupt layout.yml is rewritten from config.
S1 no lock on forks.yml	Read-modify-write serialized under withForkManifestLock (shared lock) across fork/unfork/update.
S2 / S8 unvalidated name/kind/path	isSafeDocName rejects path-traversal/unsafe names; readForkManifest parses per-entry, dropping unsafe/malformed entries with a warning instead of aborting the whole read.
S3 / D1 update swallows skips	tbd docs update now surfaces skipped docs (conflicted/orphaned/missing/no-base/newer-base) instead of reporting "up to date".
S4 version-skew bypass on refresh	forkDoc refresh path guards against silent downgrade (skips unless --force).
S5 CRLF spurious conflict	mergeContents LF-normalizes all three inputs before git merge-file.
S6 README index injection	Frontmatter blurbs sanitized, link targets percent-encoded, local filenames escaped.
S7 legit markers stuck conflicted	hasUnresolvedConflict keys off tbd's own marker labels, so docs that contain example conflict markers aren't stuck.
D3 bogus --category hint	Empty-status hint now points at tbd docs fork <name> / --all.
F5 corrupt layout not fixable	Corrupt layout.yml is doctor-fixable with remediation text.

Doc corrections (6b4d266)

• F2 / F4 — abort recipe (tbd-docs.md): added a "quiesce other tbd processes" note (a concurrent write re-stamps the layout and can undo an abort; revert config before deleting the stamp); clarified that reverting config alone drops the format gate even when forks were already committed.
• development.md — layout.yml "mirrors the config's tbd_format" (was a stale "f04").
• Re-ran a docs-vs-code audit (sub-agent); the remaining doc-correction findings were verified accurate against current code.

Beads

12 Review/* beads closed (F1, S1, S2/S8, S3/D1, S4, S5, S6, S7, D3, F5, F2/F4, and the doc-correction bead). 2 left open as Tier-3 follow-ups with notes:

• tbd-nt2c — doctor forks.yml corruption check (folds into the doctor fork-checks bead; readForkManifest is already tolerant, so not a crash risk).
• tbd-m72a — S9 (README-prune cosmetic) + F7 (accepts f01, theoretical). F3 verified not a gap: docs subcommands are format-gated via readConfig → checkFormatCompatibility, which throws IncompatibleFormatError on a newer-than-supported repo.

Verification

Full suite green locally — 1280 vitest + 874 tryscript, typecheck, eslint. CI on cf5beae: ubuntu/macOS tests + benchmark green; Windows test and Coverage/Lint still running at time of posting.

---

Generated by Claude Code

[jlevy]

Review follow-up: bead map and takeover status

All review findings are now tracked as beads under the feature epic tbd-67ek (synced via tbd sync). Head is e8b5112 with five takeover commits; CI is green on all 5 checks for it (run 27434046101).

Note on the red run for 1132a75: that commit was never intended content — a pre-push hook test run corrupted a local worktree branch and the bad head was briefly pushed, then immediately force-replaced with e8b5112. The incident is tracked as tbd-a1lc (P1, infra): git-using tests inherit the hook's GIT_* env and operate on the real repo (rewrote local main, flipped core.bare, overwrote ids.yml with a corruption fixture — all repaired and verified). Until fixed: run tests manually and push with --no-verify/SKIP=test.

Epic tbd-5wv9 — correctness and polish fixes (18)

Bead	Item	Status
tbd-62qe	Forked shortcuts not served (lookup_path precedence)	✅ fixed 4301220, golden-tested
tbd-k6m2	--category hint for nonexistent flag	✅ fixed upstream 6b4d266, verified
tbd-xbpe	merge-file exit >127 misread as conflicts	✅ fixed 4301220 + unit test
tbd-8gt5	Reconcile spec golden maps with shipped output	🔲 open
tbd-wngu	f05 definition drift (migration extras, fork_dir config)	🔲 open
tbd-g0hu	Re-home old tbd docs viewer surface	🔲 open
tbd-iqm1	Windows fork-dir paths + tryscripts in OS matrix	🔲 open
tbd-y85r	conflicted flag never cleared in manifest	✅ fixed a3a5b37, verified e2e
tbd-fywy	docs update unknown names silently no-op	✅ fixed a3a5b37, verified
tbd-62nc	Manifest path-traversal hardening	✅ fixed upstream 6b4d266, verified
tbd-znnn	pathExists full-read + error swallowing	✅ fixed 83fe4bb
tbd-y1kp	Actionable fork-overwrite refusal	✅ fixed a3a5b37, verified
tbd-00wl	Validate --kind values	✅ fixed a3a5b37, verified
tbd-roz6	reference kind guard	✅ resolved a3a5b37
tbd-cab2	Drift notice via output layer; --json names not prose	🔲 open
tbd-d9l0	Dead noop action; GNU-only sed -i	✅ fixed 83fe4bb/e8b5112
tbd-69g0	Spec: skew-guard row + drift notice	✅ done e8b5112
tbd-3h1s	Zero-fork overview: three postures	🔲 open

Epic tbd-vbd3 — docs completeness for the f05 release (7)

Bead	Item	Status
tbd-msh3	Skill routing rows (fork/update/missing-file)	🔲 open
tbd-j7im	tbd-docs.md Managing Docs chapter	🔲 open
tbd-koe4	One canonical drift table (§2.9), other cites it	✅ done e8b5112
tbd-jznb	README Upgrading: forked-docs line	✅ done e8b5112
tbd-slcn	docs-overview.md rewrite	🔲 open
tbd-low8	Three-sync taxonomy table in tbd-docs.md	🔲 open (blocked on tbd docs sync existing)
tbd-a3pj	Capture design rationale (export-only; resolve-by-convention)	✅ done e8b5112

Epic tbd-3neh — DocRef v0.1 hardening (5)

Bead	Item	Status
tbd-s6tb	Drop git: scheme (future-protocols note)	✅ fixed 6b6949e
tbd-z9hs	Strict anchored local paths; reject bare/~	✅ fixed 6b6949e
tbd-devl	Windows drive-letter paths parse as local	✅ fixed 6b6949e
tbd-0n4l	Preserve URL fragments through normalization	✅ fixed 6b6949e
tbd-vu3d	docref-format.md reference doc + wire validation into tbd	🔲 open

Epic tbd-ss3p — DocMap v0.1 tightening (5)

Bead	Item	Status
tbd-qylm	Require location (path and/or source) per entry	✅ fixed a3a5b37
tbd-7mxx	Drop word_count from core schema	✅ fixed a3a5b37
tbd-xnbl	docs list --json emits source for upstream entries	✅ fixed a3a5b37, verified 27/27 located
tbd-hayb	Version policy: accept docmap/0.* only	✅ fixed a3a5b37
tbd-arsr	docmap-format.md reference doc (path-relativity, view-not-input)	🔲 open

Totals: 40 beads (4 epics + 35 children + 1 infra bug). 24 closed, 16 open. Each close carries the fixing commit and verification evidence; the upstream hardening commit 6b4d266 accounted for two of them (tbd-k6m2, tbd-62nc), independently verified against the built CLI before closing.

Remaining release-bar tail (per the phase audit in the review): old-surface re-homing (tbd-g0hu), golden-map reconciliation (tbd-8gt5), f05 definition (tbd-wngu), Windows (tbd-iqm1), and the docs-completeness epic.

https://claude.ai/code/session_01QPsCSYGtwR8JtX2R1aaxyh

---

Generated by Claude Code

[jlevy]

Merge-readiness accounting — full ledger

Head: e5ce028. All review beads are now resolved except one deliberate follow-up; below is the complete honest accounting of what is done, what remains, and why.

Review beads: 40 of 41 closed

Every bead from the four review epics is closed with the fixing commit and verification evidence, and all four epics are closed:

• tbd-5wv9 correctness/polish (18/18) — the three P0s (forked-shortcut serving, --category hint, merge-file exit guard) fixed and verified end-to-end; golden maps reconciled (tbd-8gt5); f05 spec'd as the stamp-only migration it is (tbd-wngu); old tbd docs surface fully re-homed per the disposition table in 3c718c0 (bare overview with three postures, show/manual with --section/--sections, canonical tbd docs sync, old flags retired, all goldens rewritten); Windows fork paths unified to POSIX constants with a new cross-platform vitest e2e (tbd-iqm1).
• tbd-vbd3 docs completeness (7/7) — "Managing Docs: Two Modes" first-principles chapter + the four-row sync-taxonomy table in tbd-docs.md; skill routing rows regenerated into all skill copies; docs-overview.md rewritten; rationale captured in spec Alternatives + design §2.9.
• tbd-3neh DocRef (5/5) — git: dropped, strict anchored locals, drive letters, fragment preservation, references/docref-format.md authored, and the docref-everywhere rule now enforced (manifest source fields validate as docrefs on read).
• tbd-ss3p DocMap (5/5) — location required, word_count out of core, version policy, producer conformance, references/docmap-format.md authored (path relativity, extension fields, view-not-input).

Open (1): tbd-tgwi — tbd itself misresolves its data dir under an ambient GIT_DIR (any user running tbd inside a git hook). Pre-existing product behavior, not introduced here; fix options sketched on the bead (recommend: honor GIT_DIR only when it agrees with cwd discovery). Not a merge bar for this PR. The test-suite half is fixed and proven (7f35ba8, red/green vs a victim repo + a hooked push with zero ref drift).

Remaining for the full f05 experience — spec phase items (not review findings)

These are the spec's own Phase 2–5 build items; each is annotated as a (Phase N contract) in the reconciled golden maps, so nothing user-visible references them before they exist:

Item	Spec home
docs_cache.local_dirs + tbd docs add <docref> + grouped source sync	Phase 2, item 10
tbd doctor fork checks (missing/orphaned/base/conflicts/reserved names/gitignored fork dir)	Phase 2, item 12
Shared docmap renderer; per-kind --list --json → docmap; per-kind reader provenance notes	Phase 2, item 9 (Decisions 18/21/22)
tbd status Docs line + setup pending-update report	Phase 3, item 14b
--category fork selection (frontmatter curation)	Phase 4, item 15
Setup Docs summary (three-posture menu) + --interactive removal	Phase 4, item 16
reference kind + self-docs/format-docs served through the cache	Phase 5, item 18
welcome-user two-axis onboarding; suggest-upstream-improvements playbook	Phase 5, items 17/19
CHANGELOG / release notes	Phase 5, item 20
fork_dir configurability, .tbd/.gitignore refresh, generated .tbd/README.md	in-era additive (spec'd as such)

Blocked externally: tryscript goldens on Windows (tryscript executes blocks via the platform shell — cmd — an upstream limitation; the new fork-cross-platform-e2e.test.ts provides the cross-OS coverage instead).

State of the branch

With e5ce028 the PR is internally consistent: no command references a flag that doesn't exist, the old and new docs surfaces no longer coexist, the spec's golden maps match shipped output (with explicit phase-contract annotations for the rest), docs/skill/agent surfaces describe exactly what's built, and the f05 gate's "everything ships together" safety argument holds for the shipped surface. Full suite green: 1,294 unit tests + 881 tryscript blocks + lint/typecheck; CI green on ubuntu/macos at time of writing (Windows in progress — the new cross-platform e2e is the first real Windows exercise of the fork surface).

The Phase 2–5 table above is the remaining build list for "complete experience"; every item is additive and independently landable on this branch.

https://claude.ai/code/session_01QPsCSYGtwR8JtX2R1aaxyh

---

Generated by Claude Code

[jlevy]

Phases 2–5 complete — the full f05 experience now ships in this PR

Head: 3406821, CI green on all five checks (including Windows). Every numbered item in the spec's implementation plan is built; epic tbd-67ek and all 12 phase beads are closed with commit-level evidence.

Landed since the last ledger (commits a6f54f5, 3406821):

Item	Spec home
reference kind; docref/docmap format docs served; self-docs (tbd-docs, tbd-design) registered as forkable reference docs with bundled fallback	Phase 5, item 18
One-docmap rendering: shared entry constructor across list / overview / show / per-kind readers; per-kind --list --json → docmap; entry+content read shape; per-kind provenance notes	Phase 2, item 9 (Decisions 18/21/22)
tbd docs add <docref> (canonical docrefs recorded; explicit @ref required for git sources; local docrefs offline; per-kind --add as aliases)	Phase 2, item 10
docs_cache.local_dirs served between fork dir and cache (state local, provenance note, not forkable)	Phase 2, item 10
Sync grouped per git repo+ref: per-group failure isolation, one revision capture per group, never prunes on fetch failure	Phase 2, item 10
tbd doctor Forked docs group (7 checks, --fix finalize/clean), silent for non-users	Phase 2, item 12
tbd status Docs line (forks only; zero-fork byte-identical)	Phase 3, item 14b
Declared frontmatter categories on all 27 guidelines; inference retired; tbd docs fork --category	Phase 4, item 15
Setup Docs summary (shared three-posture menu incl. the category line — flag and hint landed together); --interactive removed everywhere	Phase 4, item 16
suggest-upstream-improvements playbook; two-axis welcome-user; README/development.md; CHANGELOG f05 entry	Phase 5, items 17/19/20

Two real bugs were found and fixed during this work: the docs-cache refresh dropped sibling docs_cache keys on config rewrite, and a serve-path double-sanitize silently emptied configured local dirs.

Spec hygiene: all golden-map blocks for shipped behavior are now captured from the built CLI; the (Phase N contract) annotation convention has no remaining instances for shipped surfaces.

Still open, deliberately: tbd-tgwi (tbd's own resolution under ambient GIT_DIR — pre-existing product behavior with a recommended fix sketched) and the reserved docs_cache.fork_dir key (documented as in-era, not yet read). Neither blocks this PR.

https://claude.ai/code/session_01QPsCSYGtwR8JtX2R1aaxyh

---

Generated by Claude Code