Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 48 additions & 0 deletions .github/workflows/deploy-freshness.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
name: Deploy freshness

# OPS-A guard: prove the LIVE site matches main HEAD. The daily sync commits
# fresh data and relies on a workflow-triggering push token to fire pages.yml
# (see sync-stars.yml's OPS-A invariant). If that ever breaks, the commit lands
# but no deploy runs and the site silently freezes on stale data with no failed
# run to notice. This job turns that invisible failure into a red scheduled run.
on:
schedule:
# A few hours after the 05:23 sync + its deploy, so a healthy deploy has
# settled. A one-off drift can be a deploy in flight; a persistent red here
# is the silent-freeze signal.
- cron: '41 9 * * *'
workflow_dispatch:

# Strictly read-only: this monitor never writes to the repo, needs no Pages or
# id-token permission, and uses NO secret — it reads committed data and fetches
# only the PUBLIC published artifact.
permissions:
contents: read

concurrency:
group: deploy-freshness
cancel-in-progress: true

jobs:
freshness:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
with:
ref: main

- uses: pnpm/action-setup@v6
- uses: actions/setup-node@v5
with:
node-version-file: .nvmrc
cache: pnpm

- run: pnpm install --frozen-lockfile

# Compares the public live dataset-meta.json against main HEAD's committed
# copy. Exits non-zero on drift (GitHub notifies on a failed scheduled run)
# or if the live site is unreachable/malformed — we never conclude "fresh"
# from a site we could not read. The live URL is derived from
# GITHUB_REPOSITORY; pass --url to override for a custom domain.
- name: check deploy freshness
run: pnpm exec tsx packages/deploy/src/cli.ts freshness --data .
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -23,3 +23,6 @@ classifier-state.json
.claude/
.idea/
.vscode/

# Playwright MCP browser session artifacts
.playwright-mcp/
7 changes: 5 additions & 2 deletions apps/dashboard/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,13 @@
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<!-- CSP backstop (SEC-B): Pages cannot set headers, so this meta is the only
place a policy can live. Asserted by the deploy verify step; see
packages/deploy/src/verify.ts before loosening any directive. -->
packages/deploy/src/verify.ts before loosening any directive. NB:
frame-ancestors is intentionally omitted — it (like X-Frame-Options) is
ignored when delivered via <meta>, so clickjacking protection is simply
not available on Pages; keeping it here only logged a console error. -->
<meta
http-equiv="Content-Security-Policy"
content="default-src 'none'; script-src 'self'; style-src 'self'; img-src 'self' data:; font-src 'self'; connect-src 'self'; base-uri 'none'; form-action 'none'; frame-ancestors 'none'"
content="default-src 'none'; script-src 'self'; style-src 'self'; img-src 'self' data:; font-src 'self'; connect-src 'self'; base-uri 'none'; form-action 'none'"
/>
<title>StarLedger</title>
</head>
Expand Down
55 changes: 37 additions & 18 deletions docs/P3-ai-spec.md
Original file line number Diff line number Diff line change
Expand Up @@ -458,17 +458,32 @@ with no secret/backend and acceptable size + latency — see
runs produced and merged PRs #17, #18, and #20. Each changed only the public
artifact pair, passed CI plus the structural and provenance gates, and deployed
through Pages. The public artifact bytes and metadata hashes were verified against
`main`; the current published coverage is five annotations out of 492 canonical
repositories. The third run exercised the configured `max_total_per_run: 3`
budget without exceeding it.

**Final operational closeout remains pending.** `verify-ai-provenance` remains
required alongside `verify-agent-artifacts` and CI on `main`. Continue bounded,
manual backfill with exactly one executor enabled. Before P3 is fully closed,
visually confirm the public dashboard's category/tag facets and secondary summary,
then run the executor unchanged after the current collection is fully accounted
for and prove it produces neither an artifact PR nor byte churn. Existing fixture
and no-op tests support this behavior but do not replace the required live replay.
`main`; the published coverage was five annotations out of the 492 canonical
repositories present at the time — still five today, now against 550, as no
executor has run since. The third run exercised the configured
`max_total_per_run: 3` budget without exceeding it.

**Operational closeout — status (updated 2026-07-08).** `verify-ai-provenance`
remains required alongside `verify-agent-artifacts` and CI on `main`. Continue
bounded, manual backfill with exactly one executor enabled.

- ✅ **Public-dashboard visual confirmation — DONE.** Verified live at
`https://f-e-u-e-r.github.io/starledger/` under the new strict CSP: React
mounts and the stylesheet applies (the CSP does not break the app), and the
category/tag facets, AI badge, secondary summary, AI tags, the "N of M
enriched" counter (`5 of 550`), and AI-aware search all render.
- ✅ **No-churn replay, deterministic half — DONE (offline).**
`classifier verify-artifacts` passes on the committed pair, re-serializing the
committed annotations is byte-identical, and the meta `annotations_sha256` +
`annotation_count` match. So the assembler produces zero byte churn on an
unchanged set.
- ⏳ **No-churn replay, live half — PENDING (credentialed).** Running
`classifier plan --current ai-annotations.json` against live README OIDs at the
current base (to prove the planner emits zero jobs) needs a `STAR_SYNC_TOKEN`
and is the owner's step; the offline checks above support but do not replace it.

Existing fixture and no-op tests support this behavior but do not replace the
required live planner replay.

`pnpm p3-gate` is the aggregate gate: typecheck, lint, format, the full test suite
(AI schema drift, fingerprint/planner, injection fixtures, structural + provenance
Expand All @@ -478,8 +493,9 @@ build, and generated-schema drift.
## P3 exit conditions

The contract, gate, and implementation conditions below are met and tested. Live
artifact publication is also verified; the visible-dashboard check and no-churn
replay remain pending the operational closeout above.
artifact publication is verified, and (2026-07-08) the visible-dashboard check is
DONE and the deterministic no-churn replay is verified offline; only the live
credentialed planner replay remains pending the operational closeout above.

- canonical stars remain untouched by AI;
- jobs are generated only by trusted deterministic code;
Expand All @@ -490,9 +506,11 @@ replay remain pending the operational closeout above.
- the dashboard works fully without AI (fail-soft) and fails closed only on
canonical data;
- public AI artifact deployment is hash-verified against `main`;
- live AI facets, secondary summaries, and AI-aware search still need visual
confirmation on the public dashboard;
- an unchanged live run produces no artifact churn after bounded backfill;
- live AI facets, secondary summaries, and AI-aware search are visually confirmed
on the public dashboard (2026-07-08);
- an unchanged run produces no artifact churn: proven offline via the
deterministic assembler replay; the live credentialed planner replay is still
pending;
- semantic search is explicitly deferred by ADR-001 (lexical search shipped).

## Subsequent milestones
Expand All @@ -512,5 +530,6 @@ replay remain pending the operational closeout above.
- **P3.5 (ADR and live artifact publication delivered; final closeout pending):**
vector search is deferred and lexical search shipped. Three validated Claude
Routine artifact PRs have merged and deployed five annotations under bounded
budgets. Public dashboard visual confirmation and the post-backfill no-churn
replay remain pending.
budgets. Public dashboard visual confirmation is DONE (2026-07-08) and the
deterministic no-churn replay is verified offline; only the live credentialed
planner replay remains pending.
49 changes: 47 additions & 2 deletions packages/deploy/src/cli.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
import { existsSync, readFileSync } from 'node:fs';
import { appendFileSync, existsSync, readFileSync } from 'node:fs';
import { resolve } from 'node:path';
import { argv, env, exit } from 'node:process';
import { verifyDatasetIntegrity } from './dataset';
import { evaluateDeployFreshness } from './freshness';
import { writeFixtureDataset } from './fixture';
import {
DATASET_META_FILE,
Expand All @@ -26,6 +27,17 @@ function derivedBase(): string {
return env.GITHUB_ACTIONS && repo ? `/${repo}/` : '/';
}

/** Best-effort append to the GitHub Actions run summary (no-op off CI). */
function appendStepSummary(markdown: string): void {
const path = env.GITHUB_STEP_SUMMARY;
if (!path) return;
try {
appendFileSync(path, markdown + '\n');
} catch {
/* the summary is diagnostic only; never let it fail the check */
}
}

async function main(): Promise<void> {
const cmd = argv[2];
const dist = flag('dist') ?? DEFAULT_DIST;
Expand Down Expand Up @@ -75,9 +87,42 @@ async function main(): Promise<void> {
console.log(`[deploy] wrote fixture dataset (${r.repoCount} repos) → ${out}`);
break;
}
case 'freshness': {
// OPS-A: compare the PUBLIC live dataset-meta.json against main HEAD's
// VERIFIED fingerprint (evaluateDeployFreshness re-hashes stars.json so a
// stale committed meta can't self-certify). Read-only; needs no secret.
// Exit 3 on drift so the scheduled run fails visibly (a one-off can be a
// deploy in flight; a persistent red is a silent freeze). Exit 1 (via
// main().catch) if the dataset/live site is unverifiable — never "fresh".
const outcome = await evaluateDeployFreshness({
dataDir: data,
url: flag('url'),
repoSlug: env.GITHUB_REPOSITORY,
});
if (outcome.status === 'fresh') {
console.log(
`[deploy] freshness OK: live site matches main HEAD (${outcome.expectedSha.slice(0, 12)}…)`,
);
break;
}
console.error(
`[deploy] DRIFT: live site is serving ${outcome.liveSha.slice(0, 12)}… but main HEAD is ` +
`${outcome.expectedSha.slice(0, 12)}… — deploy drift (OPS-A)`,
);
appendStepSummary(
`### ⚠️ Deploy freshness drift (OPS-A)\n\n` +
`- live \`stars_sha256\`: \`${outcome.liveSha}\`\n` +
`- main HEAD \`stars_sha256\`: \`${outcome.expectedSha}\`\n\n` +
`The published site does not match main HEAD. A single occurrence can be a ` +
`deploy in flight; a persistent drift is a silent freeze — see the OPS-A ` +
`invariant in \`.github/workflows/sync-stars.yml\`.`,
);
exit(3);
break;
}
default:
console.error(
'usage: deploy <stage|verify|smoke|fixture> [--data dir] [--dist dir] [--base /x/] [--out dir]',
'usage: deploy <stage|verify|smoke|fixture|freshness> [--data dir] [--dist dir] [--base /x/] [--out dir] [--url URL]',
);
exit(2);
}
Expand Down
141 changes: 141 additions & 0 deletions packages/deploy/src/freshness.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
import { existsSync, readFileSync } from 'node:fs';
import { join } from 'node:path';
import { DatasetMetaSchema } from '@starred/schema';
import { verifyDatasetIntegrity } from './dataset';
import { DATASET_META_FILE, STARS_FILE } from './stage';

/**
* Deploy-freshness guard (OPS-A). The daily sync commits fresh data to main and
* relies on a workflow-triggering push token to fire pages.yml. If that
* invariant ever breaks (e.g. someone swaps the App token for GITHUB_TOKEN), the
* commit still lands but NO deploy runs — the live site silently freezes on
* stale data with no failed run to notice. This guard makes that failure
* OBSERVABLE by comparing the live site's published fingerprint against main
* HEAD. It is strictly read-only: no writes, no secret, only the public artifact.
*/

export interface FreshnessResult {
status: 'fresh' | 'drift';
/** stars_sha256 the live site is currently serving. */
liveSha: string;
/** stars_sha256 committed at main HEAD — what the site SHOULD be serving. */
expectedSha: string;
}

/**
* Parse a fetched dataset-meta.json body and return its stars_sha256, validated
* against the canonical schema. Throws (fail-closed) on malformed JSON or a body
* that does not satisfy the committed contract: a monitor that cannot read the
* live fingerprint must alarm, never silently conclude "fresh".
*/
export function parseLiveStarsSha(liveMetaText: string): string {
let raw: unknown;
try {
raw = JSON.parse(liveMetaText);
} catch {
throw new Error('live dataset-meta.json is not valid JSON');
}
const parsed = DatasetMetaSchema.safeParse(raw);
if (!parsed.success) {
throw new Error('live dataset-meta.json failed dataset-meta schema validation');
}
return parsed.data.stars_sha256;
}

/** Pure comparison of the live fingerprint against the expected (main HEAD) one. */
export function compareFreshness(liveSha: string, expectedSha: string): FreshnessResult {
return { status: liveSha === expectedSha ? 'fresh' : 'drift', liveSha, expectedSha };
}

export interface CheckFreshnessOptions {
/** Absolute URL of the live dataset-meta.json, e.g. https://<owner>.github.io/<repo>/dataset-meta.json. */
url: string;
/** The stars_sha256 committed at main HEAD — what the live site should be serving. */
expectedSha: string;
/** Injectable for tests; defaults to the global fetch. */
fetchImpl?: typeof fetch;
}

/**
* Fetch the live dataset-meta.json and compare its fingerprint to main HEAD. A
* network failure or non-200 THROWS — the monitor cannot conclude "fresh" from
* an unreachable site, and a false "fresh" would defeat the guard's whole point.
*/
export async function checkFreshness(opts: CheckFreshnessOptions): Promise<FreshnessResult> {
const doFetch = opts.fetchImpl ?? fetch;
let res: Response;
try {
res = await doFetch(opts.url);
} catch (err) {
throw new Error(
`could not reach the live site (${opts.url}): ${
err instanceof Error ? err.message : String(err)
}`,
);
}
if (!res.ok) {
throw new Error(`live dataset-meta.json → HTTP ${res.status} (${opts.url})`);
}
const liveSha = parseLiveStarsSha(await res.text());
return compareFreshness(liveSha, opts.expectedSha);
}

/**
* Public URL of the live dataset-meta.json for a project Pages site, or undefined
* when the slug is missing/malformed. Pages subdomains are the owner login
* lowercased; the repo path keeps its case. Pure so it is unit-testable without
* the environment.
*/
export function deriveLiveMetaUrl(repoSlug: string | undefined): string | undefined {
if (!repoSlug || !repoSlug.includes('/')) return undefined;
const [owner, name] = repoSlug.split('/');
if (!owner || !name) return undefined;
return `https://${owner.toLowerCase()}.github.io/${name}/${DATASET_META_FILE}`;
}

export interface EvaluateFreshnessOptions {
/** Directory holding main HEAD's stars.json + dataset-meta.json. */
dataDir: string;
/** Live URL override; when absent it is derived from repoSlug. */
url?: string;
/** Repo slug ("owner/repo") for URL derivation when url is absent. */
repoSlug?: string;
/** Injectable for tests; defaults to the global fetch. */
fetchImpl?: typeof fetch;
}

export interface FreshnessOutcome extends FreshnessResult {
/** The live URL that was actually checked. */
url: string;
}

/**
* The full freshness evaluation used by the CLI/workflow. It derives main HEAD's
* VERIFIED fingerprint by re-hashing stars.json and confirming dataset-meta
* agrees (verifyDatasetIntegrity) — so a stale or corrupt committed meta cannot
* "self-certify" the site as fresh against itself — then resolves the live URL,
* fetches, and compares. Every failure mode (missing data, an inconsistent local
* dataset, an underivable URL, an unreachable/malformed live artifact) THROWS,
* fail-closed: the monitor never reports "fresh" from a state it could not verify.
*/
export async function evaluateDeployFreshness(
opts: EvaluateFreshnessOptions,
): Promise<FreshnessOutcome> {
const starsPath = join(opts.dataDir, STARS_FILE);
const metaPath = join(opts.dataDir, DATASET_META_FILE);
if (!existsSync(starsPath) || !existsSync(metaPath)) {
throw new Error(
`canonical dataset (stars.json + dataset-meta.json) not found in ${opts.dataDir}`,
);
}
const expectedSha = verifyDatasetIntegrity(
readFileSync(starsPath, 'utf8'),
readFileSync(metaPath, 'utf8'),
).sha256;
const url = opts.url ?? deriveLiveMetaUrl(opts.repoSlug);
if (!url) {
throw new Error('could not derive the live URL; pass --url <https://…/dataset-meta.json>');
}
const result = await checkFreshness({ url, expectedSha, fetchImpl: opts.fetchImpl });
return { ...result, url };
}
Loading