Skip to content

Commit d574164

Browse files
easeevclaude
andcommitted
publish: give every project a stable URL that renders
A release directory is named after its content digest, so republishing the same inputs lands on the same URL and different inputs get a new one. That is what makes a release immutable, and it is also why a link to one rots: a new viewer bundle is enough to move it, since the bundle is part of the digest. `latest.json` never moves but serves JSON, so it was not a link anyone could be given. `publish` now writes a forwarding page beside the pointer, and reports it as `entry_url`. The page reads `latest.json` in the browser rather than naming a release, so its bytes depend only on the target's prefix: every publish writes the same page, and the entry point cannot fall behind the pointer whatever order concurrent publishers finish in, or if a publish promotes and then fails. Nothing can drift because nothing is remembered. It needs script, as does the viewer it forwards to. It is written at two keys, `<prefix>/index.html` and `<prefix>/`, which is what makes the bare URL portable rather than a property of one CDN. A host that resolves directories looks for index.html, the name every static host agrees on; an object store serves keys and needs one of exactly that name. Both travel with the bucket. A prefix-less target has no directory key and no filesystem allows a file named that way, so those report the explicit index.html URL, and a store that rejects the key loses only the tidier form. An index.html already at the key without chainplot's marker is left alone and `entry_point_written` comes back false, so publishing into a bucket that serves a site of its own does not replace its front page. On s3 that refusal is a conditional write and holds against a concurrent writer; on directory it is a check and a rename, which docs/capabilities.md says rather than claiming otherwise. A precondition failure is re-read before reporting, so another chainplot publisher is not mistaken for a foreign owner. The page and the pointer are written no-cache, must-revalidate. A cache that serves either without asking would show an older release, which is the one thing the stable URL exists to prevent. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent d5136ca commit d574164

8 files changed

Lines changed: 724 additions & 1 deletion

File tree

‎README.md‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -244,6 +244,39 @@ as a second agent. Both ingest examples are published live:
244244
Publishing more than one project into a single bucket needs a `prefix` on the
245245
target; without one, each project's `latest.json` overwrites the others'.
246246

247+
## The link you can share
248+
249+
A release directory is named after its content digest, so the same inputs
250+
republish to the same URL and different inputs get a new one. That makes a
251+
release immutable, and it also means a link to one moves whenever the release
252+
does — a new viewer bundle is enough, since the bundle is part of the digest.
253+
254+
So `publish` writes an `index.html` beside `latest.json` that reads the
255+
pointer in the browser and forwards to whichever release it names. The page
256+
holds no release of its own, so it is the same bytes every publish and cannot
257+
fall behind. The publish root is the stable link:
258+
259+
```text
260+
https://<host>/<prefix>/ → always the current release
261+
https://<host>/<prefix>/latest.json → the pointer it reads
262+
```
263+
264+
The page is written twice, at `<prefix>/index.html` and at `<prefix>/`, which
265+
is what makes the bare URL portable. A host that resolves a directory looks
266+
for `index.html`, the one name every static host agrees on; an object store
267+
serves keys and nothing else, so it needs a key of that exact name. Writing
268+
both means the same link works either way, and it travels with the bucket
269+
rather than living in a rewrite rule at whichever CDN is in front of it.
270+
271+
A target publishing to the bucket root has no directory key to write, and no
272+
filesystem allows a file named `<prefix>/`, so `directory` targets and
273+
prefix-less ones report the explicit `index.html` URL instead.
274+
275+
`publish` returns it as `entry_url`. If an `index.html` is already at that key
276+
and chainplot did not write it, it is left alone and `entry_point_written`
277+
comes back false — a bucket that serves a site of its own keeps its front
278+
page.
279+
247280
## Look and feel
248281

249282
The viewer transcribes the Chainstack design tokens from `cp-ui-kit`

‎docs/capabilities.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,7 @@ Machine-readable source of truth: `chainplot capabilities --json`.
3131
- Event sources: explicit address lists, one chain per project, rindexer adapter (pinned image, linux/amd64)
3232
- End policies: `pinned` (finality-checked at plan time), `follow_finalized` (requires `finalized` chain policy)
3333
- Publish targets: `directory` (atomic `latest.json`), `s3` (conditional write; verified on Cloudflare R2). Set `prefix` on a target when one bucket or directory holds more than one project — `latest.json` is otherwise a single key at the root and the projects overwrite each other's pointer.
34+
- Publish root: beside `latest.json`, `publish` writes an `index.html` that forwards to the release the pointer names. A release directory is content-addressed, so its URL moves whenever the release does — a change to the viewer bundle is enough, since the bundle is part of the digest. The root is the URL that stays put, and `publish` returns it as `entry_url`. The page is written both at `<prefix>/index.html` and at `<prefix>/`: a host that resolves directories finds the former, an object store that serves keys needs the latter, and writing both keeps the bare URL working on either without a rewrite rule at the CDN. A prefix-less target has no directory key and a `directory` target cannot name a file that way, so those report the explicit `index.html` URL. An `index.html` already at that key without chainplot's marker is left alone and `entry_point_written` comes back false, so publishing into a bucket that serves a site of its own does not replace that site's front page. On `s3` that refusal is enforced by a conditional write and holds against a concurrent writer; on `directory` it is a check followed by a rename, so a foreign file written into the same key mid-publish is overwritten. The page reads `latest.json` in the browser rather than naming a release, so its bytes depend only on the prefix: every publish writes the same page, and the entry point cannot fall behind the pointer whatever order concurrent publishers finish in. It needs script, as does the viewer it forwards to. Both the page and `latest.json` are written `no-cache, must-revalidate`, since a cache that serves either without asking would show an older release.
3435
- Charts: `line`, `bar`, `area`, `kpi`, `table` (allowlisted encodings only)
3536
- Dataset modes: `results_only` (default), `dataset_referenced`, `dataset_included`. The default publishes the page and its results only. `dataset_referenced` uploads the parquet beside the release and records a release-relative path and checksum, so `fork` fetches and verifies it on demand. `dataset_included` copies it into the release. Set per build (`--mode`) or per project (`policy.release_mode`).
3637
- Panel presentation: `title`, `description`, `span` (`half`/`full`), `hide_columns`, `unit`

‎src/publish/directory.ts‎

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,11 @@ import fs from "node:fs";
22
import path from "node:path";
33
import type { LatestPointer } from "./target.js";
44
import type { PublishTarget } from "./target.js";
5+
import { ENTRY_POINT_MARKER } from "./entryPoint.js";
56

67
const LATEST = "latest.json";
78
const TEMP_PREFIX = ".latest-tmp-";
9+
const ENTRY_POINT = "index.html";
810

911
export class DirectoryTarget implements PublishTarget {
1012
constructor(
@@ -17,6 +19,11 @@ export class DirectoryTarget implements PublishTarget {
1719
return path.join(this.rootDir, this.keyPrefix, LATEST);
1820
}
1921

22+
/** The forwarding page, beside the pointer. */
23+
private entryPointPath(): string {
24+
return path.join(this.rootDir, this.keyPrefix, ENTRY_POINT);
25+
}
26+
2027
async uploadFiles(
2128
releaseDir: string,
2229
prefix: string,
@@ -68,6 +75,57 @@ export class DirectoryTarget implements PublishTarget {
6875
return JSON.parse(fs.readFileSync(file, "utf8")) as LatestPointer;
6976
}
7077

78+
async promoteEntryAlias(): Promise<boolean> {
79+
// A file cannot be named `<prefix>/`. Anything serving a directory of
80+
// files resolves the bare path to index.html by itself anyway.
81+
return false;
82+
}
83+
84+
async promoteEntryPoint(html: string): Promise<boolean> {
85+
const entry = this.entryPointPath();
86+
const existed = fs.existsSync(entry);
87+
if (existed && !fs.readFileSync(entry, "utf8").includes(ENTRY_POINT_MARKER)) {
88+
return false;
89+
}
90+
// Temp-then-publish, so a reader never sees a half-written page.
91+
//
92+
// Claiming a free key is atomic below; replacing our own page is not,
93+
// because POSIX has no compare-and-replace. A foreign writer that
94+
// replaces our page between the marker read above and the rename below
95+
// loses its file. That window is a local directory being written by two
96+
// processes at once, which the S3 target rules out with a conditional
97+
// write and this one cannot; `docs/capabilities.md` says so rather than
98+
// claiming a guarantee that is not here.
99+
const temp = path.join(
100+
path.dirname(entry),
101+
`${TEMP_PREFIX}entry-${process.pid}-${Date.now()}`,
102+
);
103+
fs.mkdirSync(path.dirname(entry), { recursive: true });
104+
fs.writeFileSync(temp, html);
105+
try {
106+
if (existed) {
107+
// Replacing our own page: rename overwrites, atomically.
108+
fs.renameSync(temp, entry);
109+
} else {
110+
// Claiming a free key: link fails if anything appeared since the
111+
// check above, so a site's own index.html is never erased.
112+
fs.linkSync(temp, entry);
113+
fs.unlinkSync(temp);
114+
}
115+
} catch (err) {
116+
fs.rmSync(temp, { force: true });
117+
if ((err as NodeJS.ErrnoException).code !== "EEXIST") throw err;
118+
// Something appeared since the check. Another chainplot publisher wrote
119+
// the same bytes, which is the outcome we wanted; anything else means
120+
// there is no entry point to report.
121+
return (
122+
fs.existsSync(entry) &&
123+
fs.readFileSync(entry, "utf8").includes(ENTRY_POINT_MARKER)
124+
);
125+
}
126+
return true;
127+
}
128+
71129
async promoteLatest(pointer: LatestPointer): Promise<void> {
72130
// Atomic on the same filesystem: write temp, rename over latest.json.
73131
const latest = this.latestPath();

‎src/publish/entryPoint.ts‎

Lines changed: 140 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,140 @@
1+
// The stable entry point a reader can bookmark.
2+
//
3+
// A release directory is content-addressed, so its URL changes whenever the
4+
// release does — including when only the viewer bundle changed, since the
5+
// bundle is part of the digest. That is what makes a release immutable, and
6+
// it is also why a link to one rots. `latest.json` never moves but serves
7+
// JSON, so it cannot be the link you hand someone.
8+
//
9+
// This page sits beside `latest.json` at the publish root and forwards to
10+
// whichever release that pointer names, so `https://host/<prefix>/index.html`
11+
// renders the current dashboard and keeps doing so across republishes.
12+
//
13+
// It reads the pointer in the browser rather than carrying a release baked
14+
// into it, and that is the whole design: the bytes depend only on the
15+
// target's prefix, so every publish writes the same page and there is no
16+
// state to keep in step with the pointer. A page that named a release would
17+
// have to be rewritten on every publish and could fall behind — two
18+
// publishers racing, or a publish that promoted and then failed before
19+
// rewriting the page — and a dashboard silently showing an older release is
20+
// the worst way for this to break. Nothing here can drift, because nothing
21+
// here remembers anything.
22+
//
23+
// Reading the pointer needs script. So does the release it forwards to: the
24+
// viewer renders nothing without it. That costs no reader anything, and the
25+
// no-script path says so rather than linking somewhere equally unusable.
26+
27+
/**
28+
* Marks the page as ours. `promoteEntryPoint` refuses to overwrite an
29+
* index.html without it, so publishing into a bucket that already serves a
30+
* site of its own does not clobber that site's front page.
31+
*/
32+
export const ENTRY_POINT_MARKER = "<!-- chainplot:entry-point -->";
33+
34+
/** How long the page stays blank before admitting something went wrong. */
35+
const FALLBACK_DELAY_SECONDS = 4;
36+
37+
/**
38+
* Embed a string in a script as a JSON literal.
39+
*
40+
* `</script>` inside a string ends the element early whatever JSON thinks, so
41+
* the slash is escaped too.
42+
*/
43+
function scriptLiteral(value: string): string {
44+
return JSON.stringify(value).replaceAll("/", "\\/");
45+
}
46+
47+
/**
48+
* The path from the publish root to a release, which is what the page
49+
* forwards to. Relative on purpose: the same bytes work under a bucket root,
50+
* a custom domain, or a host that serves the bucket under some path of its
51+
* own.
52+
*
53+
* The result goes into a URL without percent-encoding, which is safe because
54+
* neither half can carry a character that would change the URL's shape: a
55+
* target prefix is `^[A-Za-z0-9][A-Za-z0-9._-]*(/[A-Za-z0-9._-]+)*$` in the
56+
* project schema, and a release id is hex. No `#`, `?`, `%` or space can
57+
* reach here. Widen that pattern and this needs encoding.
58+
*/
59+
export function relativeReleasePath(
60+
releasePrefix: string,
61+
keyPrefix: string | null,
62+
): string {
63+
if (!keyPrefix) return releasePrefix;
64+
const base = `${keyPrefix}/`;
65+
if (!releasePrefix.startsWith(base)) {
66+
throw new Error(
67+
`release prefix ${releasePrefix} is not under the target prefix ${keyPrefix}`,
68+
);
69+
}
70+
return releasePrefix.slice(base.length);
71+
}
72+
73+
/**
74+
* The forwarding page for a target.
75+
*
76+
* Depends only on the prefix, so republishing a project writes identical
77+
* bytes however many times it runs.
78+
*/
79+
export function entryPointHtml(keyPrefix: string | null): string {
80+
const prefix = scriptLiteral(keyPrefix ?? "");
81+
const delay = FALLBACK_DELAY_SECONDS;
82+
// `replace` rather than `assign`: the forwarding page must not become a
83+
// history entry, or Back from the dashboard lands here and bounces forward
84+
// again. `no-store` because a cached pointer is the staleness this page
85+
// exists to avoid.
86+
return `<!doctype html>
87+
${ENTRY_POINT_MARKER}
88+
<html lang="en">
89+
<head>
90+
<meta charset="utf-8">
91+
<meta name="viewport" content="width=device-width, initial-scale=1">
92+
<title>Chainplot</title>
93+
<meta name="robots" content="noindex">
94+
<style>
95+
body { margin: 0; font: 14px/1.5 system-ui, sans-serif; color: #33383d; }
96+
/* Hidden until something has gone wrong, so a redirect that works shows
97+
nothing at all rather than a message the reader cannot act on. */
98+
#fallback { visibility: hidden; animation: reveal 0s ${delay}s forwards; padding: 2rem; }
99+
@keyframes reveal { to { visibility: visible; } }
100+
</style>
101+
<script>
102+
(function () {
103+
var PREFIX = ${prefix};
104+
function relative(releasePrefix) {
105+
var base = PREFIX ? PREFIX + "/" : "";
106+
if (base && releasePrefix.lastIndexOf(base, 0) !== 0) return null;
107+
return releasePrefix.slice(base.length);
108+
}
109+
function fail(message) {
110+
var note = document.getElementById("reason");
111+
if (note) note.textContent = message;
112+
}
113+
if (typeof fetch !== "function") return;
114+
fetch("latest.json", { cache: "no-store" })
115+
.then(function (response) {
116+
if (!response.ok) throw new Error("latest.json returned " + response.status);
117+
return response.json();
118+
})
119+
.then(function (pointer) {
120+
var path = pointer && pointer.release_prefix && relative(pointer.release_prefix);
121+
if (!path) throw new Error("latest.json names no release under this prefix");
122+
location.replace(path + "/index.html");
123+
})
124+
.catch(function (err) {
125+
fail(String((err && err.message) || err));
126+
});
127+
})();
128+
</script>
129+
</head>
130+
<body>
131+
<div id="fallback">
132+
<p><strong>This page forwards to the current release.</strong></p>
133+
<p>It did not, which means <a href="latest.json">latest.json</a> could not be read.
134+
<span id="reason"></span></p>
135+
<noscript><p>It needs JavaScript, as does the dashboard it forwards to.</p></noscript>
136+
</div>
137+
</body>
138+
</html>
139+
`;
140+
}

‎src/publish/publishRelease.ts‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ import type {
1212
export type { PublishResult, LatestPointer };
1313

1414
import { latestPointer } from "./latestPointer.js";
15+
import { entryPointHtml } from "./entryPoint.js";
1516
import { DirectoryTarget } from "./directory.js";
1617
import { S3Target, s3EnvFromProcess } from "./s3.js";
1718

@@ -170,8 +171,28 @@ export async function publishRelease(
170171
const pointer: LatestPointer = latestPointer(prefix, body);
171172
await target.promoteLatest(pointer);
172173

174+
// The release directory is content-addressed, so its URL moves whenever the
175+
// release does — a viewer change is enough. The forwarding page at the
176+
// publish root is what stays put, so there is a link worth sharing.
177+
//
178+
// The page reads the pointer in the browser, so its bytes depend only on the
179+
// prefix. Every publish writes the same page, and nothing can drift out of
180+
// step with the pointer because the page holds no release of its own.
181+
const entryHtml = entryPointHtml(targetDoc.prefix ?? null);
182+
const entryWritten = await target.promoteEntryPoint(entryHtml);
183+
// The same page at `<prefix>/` as well, so the bare URL resolves on a store
184+
// that serves keys. Where it is not written the `index.html` above still is,
185+
// which is the name every static host agrees on.
186+
const aliasWritten = entryWritten ? await target.promoteEntryAlias(entryHtml) : false;
187+
173188
const publicBase = targetDoc.public_base_url?.replace(/\/$/, "") ?? null;
174189
const dashboardUrl = publicBase ? `${publicBase}/${prefix}/index.html` : null;
190+
// The bare form once the directory key exists: a store that serves keys
191+
// finds it there, and a host that resolves directories finds index.html by
192+
// itself. Without that key only the explicit name is safe to hand out.
193+
const entryUrl = publicBase
194+
? `${publicBase}/${base}${aliasWritten ? "" : "index.html"}`
195+
: null;
175196
// verifyFiles proved the bytes are in the bucket. It says nothing about the
176197
// URL handed back: a base URL naming a different bucket, a bucket with public
177198
// access off, or an endpoint that folded the bucket into every key all pass
@@ -185,6 +206,8 @@ export async function publishRelease(
185206
release_prefix: prefix,
186207
latest_url: publicBase ? `${publicBase}/${base}latest.json` : null,
187208
dashboard_url: dashboardUrl,
209+
entry_url: entryWritten ? entryUrl : null,
210+
entry_point_written: entryWritten,
188211
files_uploaded: files.length + referenced.length,
189212
datasets_referenced: referenced.map((r) => r.path),
190213
promoted: true,

0 commit comments

Comments
 (0)