Dispatch card+json GET and HEAD to the read operation

[habdelra]

The card+json GET and HEAD handlers become dispatchers into the operations core's read. GET keeps every byte it serves today; HEAD stops being a stub and answers the headers its GET would carry.

What GET delegates, and what it keeps

The document is the read operation's: link expansion, links.self, prefix-form ids, the freshly joined meta.generation and meta.screenshots, the file-metadata answer for a path that holds bytes, and which absence a missing index row is. The handler keeps the response around it — the index drain, the .json redirect, the If-None-Match fast path, the validator, the cache directive, the creation time, and the mapping from a refusal to a status.

Two things the handler used to treat as one now arrive apart, because the read distinguishes them:

• A missing row splits into "nothing at this path" and "the source is on disk ahead of the index". Both already produced distinct responses; the split just moves to where the decision is made.
• The file-metadata answer is its own assembly outcome. It is derived from the bytes on disk and has no index row, so it is served with a content type and nothing else — no validator, no cache directive, never retained — which is what it has always been.

For a read to produce both a body and a validator that describes it, read now reports the index row its answer came from. Previously only the headers-only mode did, so a caller wanting both had to peek the index a second time and take on the skew of a write landing in between.

Both read modes ask in the handler's order — the instance row first, the bytes on disk behind it. Classifying by the URL's extension before asking the index would be cheaper and is wrong for a card whose id ends in a registered one: a card's id normally comes from a UUID, but nothing enforces that, since a source write names the path it writes and the indexer makes an instance row out of any .json holding a card resource. Getting that backwards made a card at notes.md answer 404 from the assembly while the conditional fast path, which peeks the row directly, answered 304 for the same URL.

HEAD becomes a read, for callers who may read

Today a HEAD on a card URL answers 200 with the realm-identity headers, for any path, without authentication. Realm discovery depends on that — a client works out which realm serves a URL, and whether it is public, before it has credentials for it — so it stays, for every Accept bucket and for every caller who may not read the realm.

In the card+json bucket a permitted caller now gets the real thing: validator, modification time, content type, creation time, and a 304 for a matching If-None-Match. It runs read in headers-only mode, so no card document is assembled and no link is expanded, and Content-Length is left off because knowing one means serializing the document. A path with nothing at it answers 404; a file URL answers exactly as its GET does.

A HEAD is a read exactly where the GET is the card read. _search is the one card+json path whose GET answers a query instead, so a HEAD of it keeps the discovery answer rather than reporting that the realm has no card there.

Which of the two a caller gets turns on permission, so the check is asked rather than enforced. The realm-wide HEAD exemption exists to let an unauthenticated discovery probe through, and a caller riding it has shown nothing about what it may read — so the probe does not take it. A refusal there is an ordinary answer with a fallback, not a failed request, so it is not logged as one.

Consequences worth reviewing

One caller sees real 404s now. boxel-cli's anonymous-browse probe sends a card+json HEAD to decide whether a URL serves a published card. Against a public realm it is a permitted caller, so a path that names no card now 404s where the stub answered 200. That is the behavior its own doc comment describes — it already has a text/html shell re-probe for exactly this case, which the stub's unconditional 200 had made unreachable. A real published card URL is unaffected.

A card GET now costs a definition lookup it did not before. Dispatch resolves the target's type through the definition cache so an author can specialize read, which adds an index-row peek and a CachingDefinitionLookup call per request. Two details worth a reviewer's judgement: the dispatch peek is not shared with cardDocument, so a conditional GET reads the row three times; and on a cache miss the lookup populates through the prerenderer, which a card GET has not previously been able to block on. The cache is rebuilt on demand, so a miss is a one-time rebuild rather than a failure.

A specialized read is refused, not served. A card type declaring read with an input/output/program stage answers 501 rather than serving the plain document under the author's name; one that failed lowering answers 422. No card declares one, and the projection runners are separate work — but "an author declares a read" breaking that card's REST read surface deserves a deliberate answer rather than a serve-time refusal.

Test plan

The leading commit pins the read-side behaviors nothing asserted — Content-Type and Vary on a 200, no body on a 304, an unsupported Accept falling through to the module/file fallback and 404ing rather than 406ing, a GET straight after a write serving the written state, and the HEAD discovery contract. It is green against the unchanged handlers, all six shards: https://github.com/cardstack/boxel/actions/runs/35015050497

After the swap, card-endpoints-test.ts passes with no edits to any assertion — 83/83 locally, alongside the operations core, dispatch and response-cache suites at 82/82.

New card-head-test.ts covers the HEAD contract: the same validator, modification time, content type, cache directive and creation time as the GET; no body and no Content-Length; a spy proving cardDocument is never reached; a matching If-None-Match answering 304; a missing card answering 404 in step with its GET; _search keeping the discovery answer; a file URL answering as its GET does; a card whose id carries a registered file extension served as a card by the plain GET, the conditional GET and the HEAD alike; and, on a private realm, realm identity alone without credentials against real headers with them. card-endpoints-test.ts also gains a case for an errored row that recorded no title, which the error body must not invent one for.

🤖 Generated with Claude Code

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-09-15T19:44:01.355746Z	32e6031	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 32e6031d18

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Leave indexing in flight before testing the read gate

This PATCH does not set X-Boxel-Skip-Index-Wait, so patchCardInstance takes its default synchronous path and waits for the incremental index update before returning. The following GET will therefore observe Van Gogh even if the card-read handler stops draining in-flight indexing, allowing the exact regression this test claims to prevent. Exercise the deferred-write path with the same authenticated principal and ensure indexing remains pending when the GET starts.

AGENTS.md reference: AGENTS.md:L286-L290

Useful? React with 👍 / 👎.

[habdelra]

[Claude Code 🤖]

Correct, and thank you — fixed in 628e700045.

I verified the mechanism rather than taking it on faith: patchCardInstance computes answerFromEcho = duringPrerender || isSkipIndexWaitRequest(request) and only then passes waitForIndex: false to writeMany. Without the header the write awaits its incremental index before responding, so by the time the GET ran there was nothing in flight and drainRequestersOwnIndexing returned at its if (!anyPending) guard. The assertion held for a reason unrelated to the gate it named.

Your second condition turned out to be load-bearing in a way I had not appreciated until I read the gate. It does not merely help — without it the test still cannot work, for two separate reasons:

• requestContext.anonymous short-circuits with skipped-not-writer, on the grounds that a credential-less caller has no write in flight to wait on. The module these tests live in is a public-writable realm and sends no Authorization header, so every request in it was anonymous.
• Even authenticated, the gate calls incrementalIndexingInitiatedBy(requester) and waits only on deferreds tagged with that user. A different principal finds nothing pending and is excused.

So the fix sends X-Boxel-Skip-Index-Wait on the PATCH and the same john JWT on both requests. writeMany receives initiatingUser from the request context, which is what tags the deferred the GET then waits on.

One limitation I will state rather than let the green run imply otherwise: this is still a timing test. If indexing happens to settle between the PATCH returning and the GET starting, it passes without the gate doing work. What changed is that it can no longer pass by construction — the deferred path guarantees there is something to drain, where the previous version guaranteed there was not.

Renamed to "a read issued straight after a deferred write serves the written state", since the old name described a weaker thing than the test now does. Passes 83/83 in card-endpoints-test.ts.

[habdelra]

[Claude Code 🤖]

Review findings

A review pass over the full branch (the swap commits are still local; they go up once this run is green). Findings that need a change or a decision, most severe first. I re-verified the mechanism behind 1, 2 and 6 by reading the code directly rather than taking the pass's word for it.

1. read classifies by URL extension before consulting the index, and the two GET paths disagree

readDocument tests urlNamesFile(url) before reaching cardDocument; the handler asked the index first and fell back to file metadata only when there was no row. urlNamesFile keys purely on the extension against FILEDEF_CODE_REF_BY_EXTENSION, which registers .md, .txt, .png, .ts, .csv and ~45 more.

So a card instance stored at notes.md.json — whose instance URL is notes.md — reads as a file: fileMetaOrMissing → nonJsonFileExists sees the sibling .json → missingTarget → 404, where the handler served the card document.

The sharper half is that getCard's conditional fast path still peeks instance() directly and does find that row. Same URL, same state: a GET with a matching If-None-Match answers 304, a plain GET answers 404.

dispatch.ts states the assumption this rests on — "a card's id never carries a registered extension — ids are minted from a UUID" — and it held while nothing routed through it. Routing the card+json GET here promotes it to the root read path, where nothing enforces it: card+source POST writes an arbitrary local path and the indexer makes an instance row out of whatever .json holds a card resource. The peek/assembly disagreement is worth closing regardless of how reachable such an id is.

2. The headers mode answers a file from the index row; the document mode answers it from disk with the realm's refusals applied

readHeaders's file branch reads indexQueryEngine.file(url). readDocument's goes through fileMetaDocument, which applies two refusals the index row knows nothing about: openFileForMetadata refuses an _-prefixed path and a .json, and nonJsonFileExists refuses a path with a sibling .json.

For an indexed _-prefixed file with a registered extension, HEAD answers 200 + content-type: application/vnd.card+json while its GET answers 404. That breaks this change's own stated contract — the headers a GET would send — by construction, because the two modes consult different sources for the same question.

3. A permitted caller loses the discovery answer on every card+json path that is not a live card

The route is /.*, so it claims the whole bucket. HEAD <realm>/_search (or _info, a directory, any path with no card) from a caller that passes the read check now runs a card read and answers 404 where the identity stub answered 200.

That follows from "a missing card → 404", but it is broader than "not permitted, or another Accept bucket, keeps the stub", and nothing pins it. It also makes boxel-cli's servesPublishedCard probe pay a text/html re-probe where the stub used to short-circuit — which is the fallback its own doc comment describes, so arguably a fix, but it deserves a test rather than an argument.

4. A declared read turns a card's whole REST read surface into a refusal

A type declaring read with an input/output/program stage answers 501 on a plain GET; one that failed lowering answers 422; one with params answers 400. No type declares operations today, so this is latent — but "author declares a read" silently breaking that card's GET for every client wants an explicit decision (fall back to the built-in at the facade? refuse at declaration time?) rather than a serve-time 501.

5. Cost, sharper than I described it

Per card GET this adds a dispatch peek plus a lookupDefinition. Two details I had not accounted for:

• The dispatch peek is not shared with cardDocument — the scope memo reaches readHeaders, not readDocument — and getInstance selects i.*, hydrating pristine_doc/search_doc/deps for a row it reads one field off. A conditional GET now reads the row three times.
• readFromDatabaseCache is a DB query per call with no in-process memo, and on a miss it populates through the prerenderer, with a 5s HEAD visibility probe for a foreign-realm type. A card GET has not been able to block on headless Chrome before, and readDocument never reads the definition for anything.

Worth considering lookupCachedDefinition on this path so a miss degrades to the built-in, and threading the dispatch peek into cardDocument.

6. An errored card's GET body changes for a row with no recorded title

title: row?.title ?? error.title cannot distinguish "row present, title undefined" from "no row", so it falls through to the refusal's title, which is already defaulted to Error. For an errored index row whose failure was a plain Error, the body goes from title absent and … from index: undefined - <msg> to "title": "Error" and … from index: Error - <msg>. Against a byte-identical bar that is a break. row ? row.title : error.title.

Nothing in the repo asserts the errored-card GET body — grep "cannot return card" hits only the handler — so this needs a test alongside the fix.

7. permittedToRead does not mirror the gate it stands in for

internalHandle skips checkPermission entirely when isLocal; headCard calls it unconditionally, so an in-process card+json HEAD against a non-public realm would take the stub instead of the card's headers. Nothing does that today. Separately, the probe rescues only AuthenticationError/AuthorizationError, so a DB failure inside isSessionRevoked turns a discovery HEAD into a 500 where it previously never touched the DB.

Checked and clean

probe cannot weaken enforcement on any non-HEAD path, and warnRefusal wraps only the refusal-path warnings. An unpermitted HEAD leaks nothing new — the refusal returns before any index read. HEAD never assembles a document, and its ETag inputs match the GET's exactly. isDocumentResult/isHeadResult still discriminate. The response cache's retain/sizeOf treat all four non-document outcomes as the single missing outcome was treated. The not-found / not-indexed split matches the handler's own checks including the empty-source case, and the errored-row status clamp is unchanged.

Test gaps

Beyond the errored-body gap above: nothing pins what a permitted card+json HEAD does on a non-card path, and the file-URL HEAD test's etag/last-modified assertions are undefined on both sides, so they would pass against a HEAD that returned no headers at all.

[habdelra]

[Claude Code 🤖]

Review findings, now anchored to the code they are about. The earlier top-level comment predates the implementation commits being pushed, so nothing it described was in the diff to anchor to; treat this as the version to read.

Fixed, with the commit that did it:

• The index, not the URL's extension, decides whether a path is a card — 61db737d5a. Both read modes asked urlNamesFile before the index, so a card whose id ends in a registered file extension read as a file: notes.md.json is a card at notes.md, and the assembly answered 404 while the If-None-Match fast path peeked the row directly and answered 304 for the same URL. Both modes now ask in the handler's order — instance row first, bytes on disk behind it. Covered by a test that asserts the plain GET, the conditional GET and the HEAD all serve it as a card.
• The two read modes can no longer disagree about what a path is — 61db737d5a. Same change: the headers mode consulted the indexed file row where the document mode consulted disk with the realm's refusals applied, so a HEAD could answer 200 for a path whose GET answered 404.
• A HEAD is a read only where the GET is the card read — 7ef14cdbe1. The route claimed /.*, including _search, whose GET answers a query; a HEAD of it reported "no card here" instead of the realm identity a discovery caller is reading. It keeps the discovery answer, registered ahead of the catch-all.
• An errored row that recorded no title no longer gets one invented — 61db737d5a. row?.title ?? error.title could not tell "row present, title undefined" from "no row", so it fell through to the refusal's own 'Error' default and changed both the body's title and the response message. The row is now read whole, with a test in card-endpoints-test.ts.

Open, inline below — one cost question and two latent behavior questions, none of which I think this PR should decide unilaterally.

Deliberate and worth knowing: boxel-cli's anonymous-browse probe sends a card+json HEAD and, against a public realm, is a permitted caller — so a path naming no card now 404s where the stub answered 200. That is what its own doc comment describes, and it already has a text/html shell re-probe for the case, which the stub's unconditional 200 had made unreachable. Worth a reviewer's eye rather than a fix.

[habdelra]

[Claude Code 🤖]

Cost this adds to the hottest read path — wants a decision, not a fix from me.

runOperation resolves the target's type before it picks a behavior, so every card+json GET now pays for two things it did not before:

• An index-row peek that is not shared with the assembly. resolveOperation → definitionFor → adoptsFromOf → scope.peekInstance, and the scope memo reaches readHeaders but not readDocument, so cardDocument does its own instance() lookup afterwards. A conditional GET reads the same row three times: the If-None-Match peek above, the dispatch peek, and cardDocument's. getInstance selects i.*, so each of those hydrates pristine_doc / search_doc / deps to read one field — instance.meta.adoptsFrom.
• A CachingDefinitionLookup.lookupDefinition per request. readFromDatabaseCache is a DB query with no in-process memo, and on a miss it populates through the prerenderer — with a HEAD visibility probe bounded by REALM_PROBE_TIMEOUT_MS for a foreign-realm type. A card GET has not previously been able to block on headless Chrome.

The read never consults the definition for anything: readDocument and readHeaders both work from the index alone. The lookup exists only so a declaration can override the built-in.

Two things that would cut it without giving that up — lookupCachedDefinition here, so a miss degrades to the built-in instead of populating; and threading the dispatch peek into cardDocument so the row is read once. Both change dispatch for every caller, not just this one, which is why I have left them alone.

[habdelra]

[Claude Code 🤖]

Disposition: not changed in this PR, and here is the reasoning to argue with.

The project decided "rely on rebuild-on-miss" for the definition cache, so paying a lookup here is the intended design rather than an oversight. What that decision did not anticipate is this being on the card+json GET, which is the hottest read the realm has — the decision was made when the core had no HTTP surface routed to it.

I left it because every fix changes dispatch for every caller:

• lookupCachedDefinition here would make a miss degrade to the built-in instead of populating. That is almost certainly right for a read, and almost certainly wrong for a write, where silently falling back to the built-in would ignore an author's declaration. Dispatch cannot tell those apart today.
• Threading the dispatch peek into cardDocument is a bigger change to RealmIndexQueryEngine's contract than a facade PR should make.

What would change my mind: a measurement. If the definition cache runs cold in production more than rarely, the prerender-population path turns a card GET into a multi-second request and the argument stops being theoretical. boxel:client-perf's server-request latency by endpoint would show it.

Worth noting the cost is bounded rather than unbounded — REALM_PROBE_TIMEOUT_MS caps the foreign-realm probe — but a bounded several seconds on a card read is still a different thing from what this endpoint did before.

[habdelra]

[Claude Code 🤖]

CI data that bears on this, posted before I can attribute it — please do not read it as a conclusion.

Two runs of this branch, differing only by the implementation commits:

	32e6031 (tests only)	628e700 (full)
Host shards	12–18 min	15–21 min — every shard +2 to +6 min
Realm Server shards	15/15/12/16/16/19	16/16/12/17/17/19 — flat
patch-fields "patches do not await persistence"	pass, 3.3s	fail, 9.6s, waitUntil timed out

Both shard groups ran in overlapping windows in each run, so "the second run was later" does not account for it on its own. The base-realm fetch noise in the failing log is not the cause either — the passing run had 193 Failed to fetch / FetchHeaderTimeout lines against realm-server.ci.localhost/base/*, the failing one 242.

What would make this finding's mechanism the cause: the host runs a Realm in-browser for its test realms, so every store.get is a card+json GET through the changed path, now paying a peek plus a lookupDefinition. A ~25% host slowdown with one timing-sensitive test tipping over is the shape that would produce.

What argues against it, and why I am not claiming it yet: the realm-server suite exercises this path far harder than the host does and is flat (3,091 tests, 0 failures). If the per-GET cost were large enough to move a suite 25%, it should show there first and more. And 16 concurrent browser shards contend for a runner far more than 6 node shards do, so variance is not excluded — I could not find an external control, because host shards are path-gated and neither main nor the other branches active in that window ran them.

Next step: re-run shard 11 alone on the same commit once the run completes. Green at ~12 min is evidence for variance; red again at ~17 min is evidence for this finding, and the mitigation is the lookupCachedDefinition option above — a cold miss degrading to the built-in rather than populating through the prerenderer.

[habdelra]

[Claude Code 🤖]

Resolved, in one direction only. The test failure was flaky; the slowdown is still unattributed — and I am no longer claiming it points here.

Shard 11 re-run on the identical commit, no code change:

	shard 11	result
32e6031 (tests only)	12 min	pass
628e700 attempt 1	17 min	fail — waitUntil timed out
628e700 attempt 2	20 min	pass

Same commit, opposite outcomes, so patch-fields "patches do not await persistence" is flaky rather than a deterministic regression. That is verified rather than assumed — it is the whole reason I re-ran instead of dismissing it.

The third data point also undercuts my own hypothesis in a way worth recording: attempt 2 was slower than the attempt that failed (20 min vs 17) and passed anyway. So duration is not what tips this test over, and same-commit variance on this shard is at least 3 minutes without any code differing. That widens the noise band enough that I would not now read 12 → 17 as a signal on its own.

Weighing it with the rest: the realm-server suite exercises the changed GET path far harder than the host does, ran 3,091 tests green, and its shard durations moved 15/15/12/16/16/19 → 16/16/12/17/17/19. If this finding's cost were large enough to move a suite by a quarter, that is where it would show, and it does not.

So: I am not claiming a measured regression, and I am equally not claiming the cost is free. The mechanism above is real and unmeasured — an extra row read plus a lookupDefinition per card GET, with a cold miss populating through the prerenderer. What would settle it is a measurement rather than more CI archaeology: boxel:client-perf's server-request latency for the card+json endpoint, before and after. I have not done that, and I would rather say so than let a green re-run read as an answer to a question it did not ask.

Only percy/-cardstack-host remains red, which is not a signal anyone here chases.

[habdelra]

[Claude Code 🤖]

A card type that declares its own read loses its whole REST read surface. Latent, but it wants a deliberate answer.

Every refusal this maps is one a plain GET <card> can now return, because dispatch resolves a declaration before it reaches the built-in:

• a type declaring read with a program / input / output / fill / of / query stage → 501, since refuseUnservedStages refuses rather than serving the plain document under the author's name
• a declaration that could not be lowered → 422
• a declared read carrying params → 400 from validateParams

#respondToCardJsonOutcome passes each straight through as the response status, and headCard reaches the same mapping, so the card's HEAD goes with it.

Nothing in the base realm declares operations today, so none of this is reachable. What makes it worth answering now rather than when it bites: the failure mode is "an author adds a declaration and that card stops being readable by every client", which is not obviously what an author declaring a projection expects. The alternatives are to fall back to the built-in at the facade, or to refuse the declaration at authoring/lowering time where the author is looking. Serving a 501 is the one option that fails at the point furthest from the person who caused it.

[habdelra]

[Claude Code 🤖]

Disposition: not changed in this PR. It belongs to the transform-runner work, and here is why I did not paper over it.

The tempting fix is for the facade to catch the 501 and serve the built-in document anyway. That is wrong, and the executor's own comment says why better than I can: serving the plain document when the author declared a projection "hands the caller a well-formed answer to a different question". A card whose author wrote a redacting output would quietly serve the unredacted document to everyone. Given this project defers enforcement entirely — identity-aware but unenforced — a facade that silently ignores projections is the worst of both worlds.

So the real options are:

1. Refuse at declaration/lowering time, where the author is looking. invalid: true already exists on a lowered declaration for exactly this shape of problem, so the machinery is there.
2. Implement the projection runners, at which point the refusal disappears on its own.

Both are outside a facade-parity PR. The reason I raised it rather than leaving it silent: the 501 is reachable the moment anyone declares a read, and the person who hits it will be an author who thinks they wrote a projection, not someone reading this diff.

One correction to my own finding above, having re-read validateParams: the 400 for a declared read carrying params fires before the executor, so it is a different refusal from the 501 with a different remedy. Same conclusion, but they are not one bug.

[habdelra]

[Claude Code 🤖]

This probe does not mirror the gate it stands in for, in two ways.

1. isLocal is ignored. internalHandle skips checkPermission entirely for in-process dispatch, which stamps X-Boxel-Assume-User and sends no Authorization header. This calls it unconditionally, so an in-process card+json HEAD against a non-public realm would fail the probe and take the discovery answer instead of the card's headers. Nothing issues one today, so this is a latent divergence rather than a live bug — but the probe is supposed to answer "would the gate have let this through", and for that caller it answers the opposite.

2. A non-auth failure becomes a 500. Only AuthenticationError and AuthorizationError are rescued. A DB failure inside isSessionRevoked, or inside the permission check, now turns an unauthenticated discovery HEAD into a 500 — a request that previously touched neither the DB nor an index. Whether that is right depends on which you would rather a discovery probe see when the realm is unhealthy: an honest 500, or the realm-identity answer it asked for. I have left it honest, but it is a behavior change for the one caller class this route exists to keep working, so it should be a decision rather than a default.

[habdelra]

[Claude Code 🤖]

Disposition: the isLocal half is genuinely unfixable from here; the error half is a deliberate choice. Both want your call.

On isLocal: a router handler receives (request, requestContext) and nothing else — isLocal is a parameter of internalHandle, not something on the request or the context. So this cannot consult it without threading a new signal down to every handler, which is more surface than the bug deserves while nothing issues an in-process card+json HEAD. If one ever does, the symptom is mild and self-announcing: a local caller gets the discovery answer instead of the card's headers, on a realm it could have read.

On the error half, I want to argue against my own finding slightly. I framed "a DB failure becomes a 500" as a regression, and it is — but the alternative is worse in a specific way. Catching everything and returning the discovery answer would mean a realm whose session store is down answers 200 to a probe asking "can I read this?", and the caller proceeds as if the realm were merely private. A 500 says "ask again later", which is true. So I would keep it unless you disagree.

The narrow version I would accept: rescue the auth errors as now, and let everything else through — which is exactly what it does. My finding was right that it is a behavior change and wrong to imply it should be reverted.

[habdelra]

[Claude Code 🤖]

The four findings that were fixed, now anchored to the code that fixes them rather than summarized in a review body. Each names the commit and the failure it prevents, so a reviewer reading the diff meets the reasoning where the code is.

Together with the three open findings already inline, every finding from the review pass is now on a line. Nothing further lives only in a top-level comment — the two earlier summaries are superseded by these threads.

[habdelra]

[Claude Code 🤖]

Finding — fixed in 61db737d5a. This ordering is load-bearing; please don't let it drift back.

This read tested urlNamesFile(url) before reaching cardDocument, and answered file metadata whenever the extension matched. urlNamesFile keys purely on the extension against FILEDEF_CODE_REF_BY_EXTENSION, which registers .md, .txt, .png, .ts, .csv and ~45 more.

The failure that produced: a card stored at notes.md.json has instance URL notes.md. visit-file.ts indexes any .json whose data is a card resource, so the row exists. With the extension test first: fileMetaDocument('notes.md') → nonJsonFileExists sees the sibling .json → refuses → missingTarget → 404, permanently, for a card that is sitting in the index.

Worse, it made the handler disagree with itself. getCard's If-None-Match fast path peeks instance() directly and does find that row, so the same URL in the same state answered 304 to a conditional GET and 404 to a plain one.

dispatch.ts states the assumption this rested on — "a card's id never carries a registered extension — ids are minted from a UUID" — and it held while nothing routed through here. Routing the card+json GET promoted it to the root read path, where nothing enforces it: card+source POST writes an arbitrary local path.

Regression test: card-head-test.ts "the plain GET, the conditional GET and the HEAD all serve it as a card".

[habdelra]

[Claude Code 🤖]

Verified, not just asserted. card-head-test.ts builds a realm whose fileSystem contains notes.md.json — a card at notes.md — alongside reference.md, a path that really does hold bytes. The test drives all three reads at the first URL and requires them to agree: the plain GET serves a card with the stored attributes, the conditional GET answers 304 against that card's validator, and the HEAD reports the same validator. The second test holds the file case still, so the fix cannot have been "treat everything as a card".

Against the pre-fix code the first of those fails on the plain GET alone (404 where the conditional GET says 304), which is the disagreement that made this worth fixing rather than just wrong.

One thing this does cost, stated plainly: a read of a genuine file now consults the index before falling back to disk, where it used to short-circuit on the extension. That is one row lookup per file read, and it is the price of the index being the thing that decides. The handler already paid it.

[habdelra]

[Claude Code 🤖]

Finding — fixed in 61db737d5a, same change as the document mode above.

The headers mode had the same extension-first test, plus a second divergence: where it did decide a path was a file, it answered from indexQueryEngine.file(url) while the document mode answered from core.fileMetaDocument(localPath). Those are different sources with different refusals — openFileForMetadata refuses an _-prefixed path and a .json, and nonJsonFileExists refuses a path with a sibling .json; an index row knows none of that.

So a HEAD could answer 200 + content-type: application/vnd.card+json — "a GET would serve you a file-meta document" — for a path whose GET answered 404. That breaks this PR's own stated contract, the headers a GET would send, by construction rather than by accident: two modes consulting different sources cannot be relied on to agree.

Both now ask the same question in the same order, instance row → indexed file row → bytes on disk, so the answer to "what is at this path" comes from one place. The indexed-file row is kept as the cheap middle step, so a HEAD on an indexed file still costs no disk read.

One thing I did not change and want your call on: openFileForMetadata's _-prefix refusal. #operationStoredFile argues in its own comment that the refusal "does not describe the byte routes" — the card+source GET/HEAD refuses no name and upsertCardSource writes whatever path it is given — so an indexed _-prefixed file is arguably a path the document mode should serve rather than one the headers mode should stop reporting. That is a pre-existing question about the shipped executor, not something this PR should decide.

[habdelra]

[Claude Code 🤖]

On the open question above — the _-prefix refusal — I am leaving it and want to be clear that is a decision, not an oversight.

Having traced it: openFileForMetadata refuses _-prefixed paths and .json, and #operationStoredFile deliberately does not, arguing in its own comment that the _ refusal "does not describe the byte routes". Those two are already inconsistent with each other on main, independently of anything here.

What this PR changed is only that both read modes now consult the same source, so they cannot disagree with each other. Whether that shared source has the right refusals is the older question, and answering it means changing what a card+json GET of an _-prefixed file returns — a parity change, in a PR whose whole contract is strict parity.

Reachability, since it decides how urgent this is: it needs an _-prefixed file with a registered extension that is also indexed as a file row. I did not establish that such a row is produced — the indexer path for file entries is visit-file.ts → indexFileWithResults, and I stopped short of proving whether _-prefixed paths reach it. So this may well be unreachable today. Worth someone confirming before it is treated as a live defect.

[habdelra]

[Claude Code 🤖]

Finding — fixed in 7ef14cdbe1. This registration order matters.

The card+json HEAD route was /.* alone, so it claimed every path in the bucket — including the one whose GET is not the card read. _search has its own card+json GET route (searchEntriesResponse, registered above getCard's /.*), so GET /_search answers a query, or 400 without one. HEAD /_search fell through to the card read and answered 404: "this realm has no card here", to a caller whose entire purpose is reading the realm-identity headers.

The rule the fix encodes: a HEAD is a read exactly where the GET is the card read. _search is the only card+json path that isn't, today.

Two things for a future reader:

• Order is the mechanism. lookupRouteTable returns the first pattern that matches, iterating the routes Map in insertion order, so this must stay registered ahead of the /.* below it.
• This does not self-maintain. A new card+json GET route added above getCard would need a matching HEAD here, or a HEAD of it starts answering 404. I considered introspecting the router instead of listing the path, and it was worse — the explicit registration is at least visible and testable.

Caught by the run rather than by reading: card-head-test.ts originally asserted HEAD /_search matched its GET's status and got 404 vs 400. Pinned now by "a card+json path whose GET is not the card read keeps the discovery answer".

[habdelra]

[Claude Code 🤖]

How this was caught is the part worth keeping. Not by reading — by the test run. I wrote card-head-test.ts's "a path the realm serves no card at answers in step with its GET" to pin the finding that a permitted HEAD now 404s on non-card paths, listed /_search among those paths, and it failed: HEAD 404 against GET 400.

My first reading was that the test expectation was wrong. It was — but only because the premise underneath it was: I had assumed /.* was the whole card+json GET surface, and _search has its own route above getCard. The assertion I wrote to pin one finding surfaced a different, real one.

Which is the argument for writing the "in step with its GET" assertion as a comparison against a live GET rather than against a literal status. A literal 404 would have passed and hidden this.

The scope of the fix is exactly one path, and I checked that by enumerating every SupportedMimeType.CardJson registration rather than assuming: _search (GET + QUERY), createCard (POST), getCard (GET /.*), patchCardInstance (PATCH), removeCard (DELETE). Only _search is a GET that is not the card read.

[habdelra]

[Claude Code 🤖]

Finding — fixed in 61db737d5a. A byte-parity break, and a small one worth understanding.

This read the row member by member with ??:

title: row?.title ?? error.title,

SerializedError.title is optional, and serializableError passes a non-CardError through as best-effort, so a row whose failure was a plain Error records no title. ?? cannot distinguish "row present, title undefined" from "no row", so it fell through to error.title — which errorRowFailure had already defaulted to 'Error'.

For an errored index row with no recorded title, against a byte-identical bar:

	before	after the break
body title	absent (dropped by JSON.stringify)	"Error"
message	… from index: undefined - boom	… from index: Error - boom

Reading the row whole is the fix, not a different coalescing operator: a row that recorded no title is reporting that it has none, and that is information the body is supposed to carry through.

This one is the argument for the test that now sits in card-endpoints-test.ts ("an errored row that recorded no title reports none"). Nothing in the repo asserted the errored-card GET body at all — grep "cannot return card" hit only the handler — so this rebuilt exactly the payload nothing was watching.

[habdelra]

[Claude Code 🤖]

The gap this sat in is the real finding, and it is now closed.

Nothing in the repo asserted the errored-card GET body — grep "cannot return card" hit only the handler. The existing errored-row test pins the status mapping thoroughly (eight cases, including the 404→500 and out-of-range fallbacks) and never looks at the body beyond errors[0].status. So this rebuilt a payload that no test was watching, which is exactly how a ?? slipped through.

card-endpoints-test.ts now carries "an errored row that recorded no title reports none": it writes an error_doc with no title to both boxel_index and boxel_index_working, then asserts the body's title is absent and its message is the row's own. Against the pre-fix code that fails with "Error".

Worth being honest about the severity: this is a cosmetic difference in an error body, not a correctness problem for any caller I can identify. I fixed it because the bar this PR set for itself is byte-identical responses, and "cosmetic" is how parity erodes — not because I think a client is reading title on a 500.

[github-actions]

Host Test Results

290 tests   285 ✅  11m 4s ⏱️
  1 suites    5 💤
  1 files      0 ❌

Results for commit 628e700.

Realm Server Test Results

    1 files    226 suites   1h 23m 58s ⏱️
3 091 tests 3 091 ✅ 0 💤 0 ❌
3 130 runs  3 130 ✅ 0 💤 0 ❌

Results for commit 628e700.