feat: browse the files behind a source - #104
Merged
Merged
Conversation
The Files view existed but nothing pointed at it: Sources offered Sync, Rename and Remove, the file list always showed every layer at once, and no file had a URL. A source's file count and root path were already in the /api/files payload the console fetched, and were being dropped on the floor. - Sources' detail panel now reads that payload: a Files row (with the honest "none on this machine" for a remote graph or a REST-read repo), a Location row, and Browse files as the primary action. - The Files view takes a scope, so "browse this source" means one source rather than a search box over all of them. Clearing the scope keeps the open file — widening the navigator is not closing a document. - #/files/<layer>/<rel> is a real address: bookmarkable, survives reload, and Back restores what the hash says. A file is carried in the URL only while the navigator is scoped to the layer that holds it, so parse and serialize are exact inverses — the same bare/deep split Concepts makes. - One palette entry per source, and a Go to Files item bound to the same chord in the desktop View menu and the browser build. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: John Siracusa <siracusa5@users.noreply.github.com>
A 3,000-note vault rendered as 3,000 buttons of "Daily Notes/2024-01-05.md" — every one of them in the DOM, rebuilt on every mount, with no structure to read. It is now a directory tree, windowed, with no new dependency. Shape follows apps/site/src/lib/pack-explorer.ts, which already proved the flattening for the pack explorer, plus what a windowed, layer-aware navigator additionally needs: sibling position for ARIA and a subtree count. Measured against the field vault: 3,030 files across 9 folders render as 10 rows collapsed, 47 in the DOM with a folder of 250 open. Two things were easy to get wrong and are handled deliberately. Windowing rules out nested role="group" wrappers, since the rows that exist are a moving slice. Rows carry aria-level/setsize/posinset instead — the flattened form the tree pattern defines for exactly this — with aria-expanded on directories and the standard arrow/Home/End/Enter semantics, one roving tab stop, and no invented interaction. Focus has to survive the window. Keyboard movement scrolls its target into view before focusing it, and the active row is rendered in its own fixed slot, deliberately excluded from the slice: a row that crossed between the two would be unmounted and remounted, and the browser drops focus to <body> when the old node leaves the document. Scrolling the list past a focused row now leaves it mounted and focused. Empty states teach rather than shrug: a source that keeps no files here says why (remote graph, or a repo read over the API) and points at Concepts; a search with no hits names the search and the scope. Landing on Files opens a file but no longer expands its folders — that buried the overview under 250 siblings before the user had done anything. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: John Siracusa <siracusa5@users.noreply.github.com>
PATCH /api/sources now accepts `path` for the local/files kinds: expandHome, then the same cheap probeFolder the add path uses (folder-missing and not-a-folder fail; size never does), then the manifest write. Name and level survive a path-only patch, and a refused patch leaves the manifest exactly as it was. Refused where there is no folder to repoint: an MCP source is reached by command, a github source is read from its repo, and a clone-backed layer's folder belongs to Sync — gitCloneOrPull writes CACHE_DIR/<slug>, never layer.path, so repointing it would leave a source that reads one folder and pulls into another. Each refusal says which case it is, because "remove and add it again" is still the right advice for exactly these. A new folder is a new content identity, so adoptIndexes finds nothing to carry and the entry re-indexes from zero. Measured on a 3,000-note vault: the PATCH itself answers in 22ms, then the row reads indexing / 0 concepts for ~16s while it re-reads, and the old folder's concepts stop resolving immediately. That is the honest outcome — the snapshot adoption would have carried indexes a folder this source no longer reads. The response says `reindexing: true` so a client can name the cause before the row goes blue. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: John Siracusa <siracusa5@users.noreply.github.com>
…n process
shell.showItemInFolder opens a Finder window on whatever it is handed, so a
channel that accepted a path would let a compromised renderer point the user's
Finder at any file on the machine. This one takes a source NAME and a path
relative to it — the preload cannot express an absolute path at all.
src/main/reveal.mjs reads the manifest from disk, builds the same
profile-unified layer view every engine read site builds, and runs the engine's
own guards: layerRootMap (mcp and REST-read github layers are structurally
absent — they have no folder) and assertInsideRoot (".." , an absolute rel, and
symlinks pointing out of the root). Imported by path rather than copied, so the
containment rule keeps exactly one implementation. An escaping path is refused,
never clamped back to the root: clamping would answer a request nobody made,
and quietly. The IPC is main-window-only in the trusted-window policy.
navigation-test.mjs covers the policy, the preload shape, and the resolver
against a real layer root: traversal, deep traversal, an absolute rel, a symlink
out of the root, a layer with no folder, an unknown layer, a missing file, and
non-string arguments.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: John Siracusa <siracusa5@users.noreply.github.com>
Files and concepts are two ends of one thing, and until now you could only walk it in one direction. An open document names the concept it becomes: the file's rel minus its document extension, matched against a loaded concept id. Checked against a real files layer — 3,000 of 3,000 markdown notes matched, and the 30 attachments beside them matched nothing, which is the point: a file the cascade does not read gets no strip rather than a link to a concept that isn't there. A contested concept says so in words and an icon, the same marker the canvas node uses, never the amber alone. The other direction needs the contributor's real source name, not its lane, so ConceptSection and Dissent now carry sourceLayer — two sources can share a lane and only the name identifies the one holding the file. The "Open file" link is drawn from the /api/files listing rather than guessed as <id>.md, so it appears only where a file actually exists: a files-kind layer may hold the concept as .mdx or .txt, and an MCP or REST-read contributor keeps nothing here at all. Both directions run the navigation guard exactly once, through store actions rather than setView plus a setter — the second ask was the bug that moved the selection whether or not the user said yes. Reveal in Finder rides along on the file header, hidden rather than disabled outside the desktop app: a control that can never do anything says less than no control at all. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: John Siracusa <siracusa5@users.noreply.github.com>
…ng it The detail panel said, verbatim, "The path, repository, or command is fixed for this source. To change it, remove the source and add it again." For a folder-backed source that is no longer true, so it is gone; for a repo or an MCP command it still is, so each of those keeps its own version of the sentence saying which case it is. The folder is a labelled field in the existing rename/level panel — inline, where the other two live, not a new dialog — with the native picker beside it in the desktop app. Only a real move is sent: an untouched field must not re-key the index entry and put a settled source through a full re-read for nothing. A move answers with reindexing:true, and the panel names the cause before the row turns blue on its own. The control that opens the panel now names what the panel can change; a button called "Rename / level" over a form that also repoints the source would hide the very thing this adds, and the visible words have to be inside the accessible name. Reveal in Finder sits beside Browse files, for the sources that keep something on this machine, and only inside the desktop app. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: John Siracusa <siracusa5@users.noreply.github.com>
The navigator was live-only, so the one place people evaluate ContextCake without installing it — the public Web Demo — showed an explanatory gate where the feature is. The demo now renders the same navigator, read-only: the tree, folder counts, scoping, deep links, the rendered document, and the file ⇄ concept links in both directions. build-demo-data.mjs produces the fixture by calling the engine's own listFilesApi/readFileApi over the demo bundle, the same way it already shells out to resolver.mjs for the cascade — the snapshot is generated engine output, never hand-authored, and it captures only the two GET answers, so there is no write path for the demo to fake. Saving is gated on `live && file.editable`, which also unbinds ⌘S and makes the editor readOnly. A binary in the snapshot says it carries text rather than spinning on a raw fetch that cannot be served. Sources keeps its read-only demo panel but still offers the way in, and the palette entry per source goes with it: browsing is a read. Layer roots are rewritten repo-relative in the fixture — the bundle ships publicly and the build machine's home directory has no business in it. The demo corpus gains one file, personal/notes/scratch.txt: a plain .txt in an OKF bundle is listed by the navigator and read by no adapter, which is exactly the case the "resolves to" strip has to stay silent about. Signed-off-by: John Siracusa <siracusa5@users.noreply.github.com>
The Files view had no user-facing documentation — the only file explorer described anywhere was the playground's, which is a different surface for a different route. Adds a guide under Guides (first, ahead of the playground tour: the app is the recommended route) covering the way in from a source, the tree and its keyboard contract, scoping, deep links, reading and editing with the limits that are deliberate, the file ⇄ concept rule, Reveal in Finder, repointing a source's folder, and which source kinds keep no files to browse at all. The demo page promised only resolved concepts and conflicts; it shows the navigator now, so the third card says so. The console README predated both the Files and Sources views and listed neither. Signed-off-by: John Siracusa <siracusa5@users.noreply.github.com>
…one is added
The three views that read /api/files all keyed their effect on
`sources.length`, and a reload() replaces the sources array without ever
changing its length. So the two writes this PR added were invisible to the
panel it added:
- rename `notes` → `notes-2`: filesByLayer is still keyed by `notes`, so a
source with 3,000 files renders "None on this machine" and its Browse
files button disappears.
- repoint `notes` from /a to /b: the PATCH lands and the engine re-indexes,
but Location and the file count keep quoting /a — and the edit panel
prefills the folder field with it.
Both self-heal on remount, which is why nothing caught them.
`filesRevalidation()` is now the one answer to that question, used at all
three call sites. The source names cover add/remove/rename, including a change
made outside this app that arrives through the poll rather than a write; the
store's `reloadKey` covers a repoint, where the name is the one thing that did
not change.
Three tests, each pinning one half: rename and repoint go red on
`sources.length` with exactly the reported symptoms, and the out-of-band add
keeps the coverage the count used to give.
Signed-off-by: John Siracusa <siracusa5@users.noreply.github.com>
…othing `paddingLeft: 8 + depth * 13` had no cap and `.cc-tree-name` had no floor, inside a `minmax(220px, 300px)` column with `overflow-x: hidden`. Measured against the navigator's own CSS at the column's *maximum* width: depth=10 indent=138px name=112.9px visible depth=14 indent=190px name= 72.0px visible depth=20 indent=268px name= 0.0px blank depth=30 indent=398px name= 0.0px blank A file 20 folders down was a row you could click and focus with nothing written on it. `walkAll` caps files, not depth, so a docs monorepo or a foldered vault reaches this; the 3,000-note test vault maxes out at depth 3, which is why it would have shipped. The indent stops growing at 10 levels — the deepest inset that still leaves a name room at the column's narrowest — and `.cc-tree-name` gets a 4ch floor so flex cannot take it to zero either way. Past the cap rows share an inset; `aria-level` and the row's `title` still carry the real depth, so the picture flattens and the tree's account of itself does not. Signed-off-by: John Siracusa <siracusa5@users.noreply.github.com>
…Demo `graph.manifest.path` was the absolute manifest path, and it flows into demo-cascade.json and gets inlined into the shipped JS chunk. From a local publish that puts a developer's home directory and repo layout on a public artifact: $ npm run build && grep -rhoE '/Users/[^"]+' dist/assets/*.js /Users/john/repos/context-cake/.claude/worktrees/…/apps/playground/manif… The generator already does the right thing one field over — `layer.root` is rewritten repo-relative with a comment explaining why. Same treatment here. The built dist/ now contains no /Users/, /home/ or /private/ at all. Signed-off-by: John Siracusa <siracusa5@users.noreply.github.com>
`{"path": ["/etc"]}` returned 200, because String(["/etc"]) is "/etc" — a
single-element array walked straight through the trim and the probe. Harmless
in practice (two elements become "/a,/b" and 400), but the rest of this
handler validates its input and this is one line.
Signed-off-by: John Siracusa <siracusa5@users.noreply.github.com>
…URL back
Two history bugs that composed into a three-step way to lose typed text.
The unsaved-changes guard only fired when a popstate changed the *view*, so
Back or Forward between two `#/files` entries — same view, different document
— walked straight through it: zero confirm dialogs, the draft gone. The guard
now also fires when the entry names a different open file, and still asks
nothing when nothing is dirty (it is a cancelable event with no listener until
the editor has edits to lose).
Cancelling a guarded Back then rewrote the URL as `#/${view}`, which dropped
the scope and the open file, and did it with replaceState — onto the entry the
popstate had already moved to, i.e. the *neighbouring* page. So the URL stopped
describing the screen (and a reload landed on a bare `#/files`), and the entry
behind it became a Files URL instead of the Sources view the user had visited
— which is precisely the adjacent-Files-entries shape the first bug exploited.
The refusal now pushes the full current hash, through one `currentHash()` the
URL effect shares, so neither neighbour is touched.
Covered by real jsdom session-history traversal, not a synthesized PopStateEvent
— a hand-fired event leaves the stack untouched and cannot see which entry a
refusal wrote to.
Signed-off-by: John Siracusa <siracusa5@users.noreply.github.com>
…r the rest resolveRevealTarget read the manifest strictly, so a single malformed layer — the kind the console already lists as a broken row and browses around — threw during validation and every reveal, including a plain healthy file in a healthy folder, came back "ContextCake could not read its list of sources." It now reads through the engine's own readContextManifestQuarantined, the same read-path tolerance service.mjs adopted after validation-inside-openSources answered 500 on the two screens a user needs to fix the problem. Quarantine only removes: the bad layer is simply absent, so it is refused by name — and says so, rather than blaming the whole source list. Signed-off-by: John Siracusa <siracusa5@users.noreply.github.com>
buildTree keyed both directories and files by `<layer>/<rel>`, and the pre-order
walk descends by `children.get(node.id)`, so a collision makes a row walk into
somebody else's subtree. Two shapes were reachable:
- a file beside a folder of the same name (`notes`, `notes/a.md`,
`notes/b.md`) emitted five file rows for three files, with duplicate React
keys and duplicate `rows.current`/`indexById` entries;
- a layer literally named `a/b` beside a layer `a` holding `b/c.md` and
`b/d.md` — validateContextManifest accepts that name and /api/files lists
it — made `c.md` and `d.md` disappear from the tree entirely.
Ids now carry the layer's index and whether the row is a directory or a file,
which neither a filename nor a layer name can forge. `ancestorsOf(path)` becomes
`ancestorsOfId(id)` for the same reason: splitting a path could not tell where a
slash-bearing layer name ended.
Signed-off-by: John Siracusa <siracusa5@users.noreply.github.com>
The active row was excluded from the windowed slice and appended after it, in a fixed trailing slot. Rows are absolutely positioned by index, so the picture was right; the accessibility tree was not. Sequential and browse-mode reading of a flattened role="tree" follows DOM order, so the focused row was announced last — out of sequence with the aria-posinset it reports. It is now spliced in at its own index, and still rendered when it lies outside the window, which is the part that keeps focus alive. That costs nothing: every row is keyed by its id and emitted in ascending index order, and React only re-inserts a keyed child that has to move backwards past a sibling. Scrolling, expanding and filtering all add and remove at the edges, leaving the survivors in the same relative order — so the focused node is never detached and re-attached. The focus-across-virtualization tests are unchanged and still pass. Signed-off-by: John Siracusa <siracusa5@users.noreply.github.com>
Two defects in the same corner, both worst during the flow the navigator is used in most. `isExpanded` checked `expandAll` before `collapsed`, so while a filter was active every folder was permanently open: ArrowLeft always took the collapse branch and could never walk to the parent, and clicking a folder was a visible no-op that silently dropped the folder's stored state on the way out. The filter's open-everything is now a default rather than a veto, and every close is recorded rather than inferred from a missing open marker — required, because two things default to open and for those "no marker" already means open. And when the active row left the visible set — the listing refetched while the user was mid-navigation, its row deleted on disk — the tab stop moved to the first row but focus did not, so it fell to <body> and the keyboard went dead with no way back but the mouse. Focus now follows the tab stop, but only when it was already inside the tree: the same reset fires while the user is typing in the search box, and taking focus out of the box would be worse than the bug. Whether focus is in the tree is tracked as it moves, since a removed node hands focus to <body> without firing anything an effect could read after the fact. Signed-off-by: John Siracusa <siracusa5@users.noreply.github.com>
The module header claimed a filter shrinking the list left the focused row in the DOM with focus — it does not; the row is gone, and focus follows the tab stop instead. It also said nothing about DOM order now that visual order is a contract. The guide said a search opens every folder, which is still true as a starting point but no longer a lock. Signed-off-by: John Siracusa <siracusa5@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Field feedback on the shipped Mac app: "I thought we would have some sort of source navigator built in. That doesn't seem to be a thing."
A file browser did exist, but it was effectively unreachable — Sources never linked to it, it couldn't be scoped to one source, it rendered every layer as one flat list of path strings, it had no deep links, and the Sources panel told you to "remove the source and add it again" to change a path. This turns it into a real navigator.
Builds on #100.
main@2905f6c.Changes
Sources → files. The detail panel now shows the file count and root path it was already fetching and discarding, plus a Browse files action that opens the navigator scoped to that source. Command-palette entries per source, and a Go to Files menu item in the Mac app.
A tree instead of a wall of paths. Collapsible directories with counts, derived client-side from each file's
rel— the shape already proven by the marketing site's Pack explorer. On a real 3,000-note Obsidian vault this is a 10-row collapsed overview instead of 3,030 rows.Virtualized, and still keyboard-navigable. ~47 DOM rows for a 262-row expanded model, hand-rolled windowing with no new dependency. It's a real
role="tree"widget —treeitem,aria-expanded,aria-level, roving tabindex, standard ↑↓→←/Home/End — because the register here is earned familiarity, not an invented control. The active row renders in a fixed slot outside the windowed slice, so a focused row that scrolls away is never unmounted and focus is never stranded on<body>.Deep links.
#/files/<layer>/<rel>round-trips: scoped chip, folder revealed, file open.Reveal in Finder (desktop only, hidden on web). The bridge takes
{layer, rel}and cannot express an absolute path; the main process resolves it against the manifest using the engine's ownlayerRootMap+assertInsideRoot(imported, not copied), so traversal, absoluterel, and out-of-root symlinks are refused rather than clamped.Files ↔ concepts, both directions. An open document names the concept it resolves to with its conflict count; each contributor in a concept gets an "Open file" link. Both are derived from the real
/api/fileslisting rather than guessing<id>.md, so an MCP or REST-read contributor correctly gets no affordance instead of one that opens on an error. Conflict state carries an icon and words, never color alone.Editable source folder.
PATCH /api/sourcesacceptspathforlocal/fileskinds — cheap probe, same as add, no walk on the request path. Name and level survive; kinds where it makes no sense refuse with their own reason.Web-demo parity. The demo bundle now carries a files snapshot generated from real engine output (never hand-authored), so the navigator renders read-only in the public demo — where evaluation actually happens. Saving and Reveal are honestly absent, not faked.
Test Plan
npm testend to end exits 0; retrieval eval unchanged (recall@1 0.895, recall@5 1.000, mrr 0.947)service-test.shfor the path patch: re-indexes against the new folder, old folder's concepts stop resolving, name/level survive, bad paths fail without mutating the manifest,github/mcprefuseAttachments/,.obsidian/) driving the actual console against the actual engine — collapsed overview, expansion, keyboard tree, deep link, cross-links../../../../etc/passwd, absoluterel, out-of-root symlink, unknown layer all refusedReviewed by two independent passes (adversarial + combined code/security), both driving the real console against the real engine over a 3,000-note vault. Eleven findings fixed on the branch. Two mattered most, and both were the kind that only show up when you actually use the thing:
#/files*entries skipped it entirely — and the guard's own Cancel path rewrote the URL to bare#/filesand clobbered the neighbouring history entry, which is what manufactured the adjacent-URL state in the first place. The two composed into a 3-step data-loss path. Fixed together, tested as a composition, driven through real jsdom session history rather than a synthesized event.sources.length, which a rename never changes.Also fixed: one invalid layer disabled Reveal in Finder for every source (re-introducing the failure mode #100 removed — now uses the same quarantined read); the focused tree row was emitted last in the DOM, so screen readers read it out of sequence;
buildTreesilently lost files when a filename collided with a sibling folder; focus fell to<body>when the active row left the filtered set; ArrowLeft and the folder chevron were dead during a search; and the Web Demo bundle shipped the build machine's home directory.Security verdict Clear: Reveal's containment was probed with 27 hostile inputs — traversal, absolute
rel, out-of-root symlinks (file and directory), NUL bytes, 9,000-character paths,__proto__layer names,mcp/github-restlayers — every escape refused and never clamped, no absolute path or filesystem error returned to the renderer, IPC restricted to the main window's main frame. The path patch reuses the add path's validation with no atomicity gap.Notes
indexingwith live progress instead. Documented inCLAUDE.md.role="group". Windowing means the rendered rows are a moving slice, so the tree uses the flattened form the ARIA pattern defines for that case (aria-level/setsize/posinset).setWindowOpenHandlermakes cmd-click on an anchor a question this PR didn't need to answer. URLs are identical.github-rest/mcpsources (needs an engine tree/proxy endpoint; MCP browsing is concept-shaped, not file-shaped), and file create/delete/rename APIs (layer-files.mjsrefuses creation deliberately; needs concept-id and conflict-log semantics specced first).console-preview.ymlfilters onapps/console/**, so editing the demo corpus orpackages/core/src/layer-files.mjsalone won't trigger a Web Demo preview redeploy.