Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,10 +65,10 @@ Each service documents its own contract and operational detail in its own `READM
- Bindings, cron, and vars are defined in `wrangler.jsonc` (D1, KV, R2, Durable Objects, rate limiters, service bindings); keep names aligned with `CloudflareBindings`.
- Use Wrangler secrets for production tokens instead of committing them.
- `ASSERTION_SIGNING_SECRET`: HMAC key the extensions v2 API uses to verify short-lived bearer assertions minted by the extensions site (`src/lib/auth/bearer-assertion.ts`). Not sent over the wire — only signs/verifies server-side in each Worker. Add `ASSERTION_SIGNING_SECRET="..."` to `.dev.vars` for local dev; set via `wrangler secret put ASSERTION_SIGNING_SECRET` in production, matching the value configured in the extensions site's Worker.
- `EXTENSIONS_REVALIDATE_SECRET`: bearer token the extensions v2 API sends to the extensions site's `POST /api/revalidate` (over the `EXTENSIONS_FRONTEND` service binding) to purge the site's CDN-cached catalogue pages after content mutations. Must match the value configured in the extensions site's Worker. Same local/prod setup as `ASSERTION_SIGNING_SECRET`.
- `EXTENSIONS_REVALIDATE_SECRET`: bearer token the extensions v2 API sends to the extensions site's `POST /api/revalidate` (over the `EXTENSIONS_FRONTEND` service binding) to purge the site's CDN-cached extension pages after content mutations. Must match the value configured in the extensions site's Worker. Same local/prod setup as `ASSERTION_SIGNING_SECRET`.

## Cache Revalidation

Any endpoint that mutates catalogue-visible content (revision approve/reject, moderator-correct, delist/relist, developer approve, developer profile upsert via `PUT /developers/me`, profile deletion via `DELETE /developers/me`, claim approve/reject, extension withdraw) must call `revalidateCatalogue(c)` from `src/services/extensions/v2/revalidate.ts` after a successful write.
Any endpoint that mutates publicly visible content (revision approve/reject, moderator-correct, delist/relist, developer approve, developer profile upsert via `PUT /developers/me`, profile deletion via `DELETE /developers/me`, claim approve/reject, extension withdraw) must call `revalidateExtensions(c)` from `src/services/extensions/v2/revalidate.ts` after a successful write.

Skipping it does not break correctness — the site's `maxAge`/`stale-while-revalidate` windows bound staleness — but changes then take minutes instead of seconds to appear. Profile deletion keeps its purge even though the `developers` tag is currently a no-op.
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# FOSSBilling API Worker

This is the API service that acts as the central hub for FOSSBilling instances. It serves version checks, system-wide alerts, the extensions catalogue, preview builds, and release stats.
This is the API service that acts as the central hub for FOSSBilling instances. It serves version checks, system-wide alerts, the extensions directory, preview builds, and release stats.

Everything is built on [Hono](https://hono.dev), making it lightweight and fast. While we currently deploy this to Cloudflare Workers, the code is designed to be platform-agnostic.

Expand Down Expand Up @@ -71,15 +71,15 @@ We use [Cloudflare D1](https://developers.cloudflare.com/d1/) and [KV](https://d
- **R2 Bucket** (`DOWNLOAD_BUCKET`): Shared by versions, previews, and stats for release/preview zips — see [`src/services/previews/v1/README.md`](src/services/previews/v1/README.md) and the comment in `wrangler.jsonc` for which bucket this points at and why.
- **Durable Object** (`PREVIEW_GITHUB_BUDGET`): Token bucket limiting previews GitHub calls. Policy lives in `src/services/previews/v1/budget.ts`.
- **Rate limiters** (`EXTENSION_WRITE_RATE_LIMITER`, `PROFILE_CREATION_RATE_LIMITER`): Edge pacing for extension writes and profile creation. Durable quotas are enforced in D1; see [extensions v2 README](src/services/extensions/v2/README.md#limits-and-budgets).
- **Service binding** (`EXTENSIONS_FRONTEND`): Private binding to the Extensions site worker, used to purge cached catalogue pages after mutations.
- **Service binding** (`EXTENSIONS_FRONTEND`): Private binding to the Extensions site worker, used to purge cached extension pages after mutations.
- **Cron** (`0 * * * *`): Hourly extension resource maintenance. See [extensions v2 README](src/services/extensions/v2/README.md#revisions-and-retention).

### Environment Variables

- `GITHUB_TOKEN`: A GitHub Personal Access Token (classic) with public repo read access.
- `ASSERTION_SIGNING_SECRET`: Shared HMAC secret used to verify the short-lived bearer assertions minted by the Extensions site. Configure the same value in both Workers; it is never sent to clients.
- `ASSERTION_SIGNING_SECRET_PREVIOUS`: Optional previous HMAC secret accepted during a signing-key rotation.
- `EXTENSIONS_REVALIDATE_SECRET`: Bearer token the API sends to the Extensions site's `POST /api/revalidate` (over the `EXTENSIONS_FRONTEND` service binding) to purge the site's CDN-cached catalogue pages after content mutations. Configure the same value in both Workers; it is never sent to clients.
- `EXTENSIONS_REVALIDATE_SECRET`: Bearer token the API sends to the Extensions site's `POST /api/revalidate` (over the `EXTENSIONS_FRONTEND` service binding) to purge the site's CDN-cached extension pages after content mutations. Configure the same value in both Workers; it is never sent to clients.
- `EXTENSIONS_V2_EMAIL_PROVIDER`: `mxroute` or `resend` to send moderator notification emails, or unset/`disabled` to skip sending.
- `EXTENSIONS_V2_EMAIL_FROM`: Sender address for notification emails (default `noreply@fossbilling.org`).
- `EXTENSIONS_V2_EMAIL_REPLY_TO`: Optional Reply-To address for a monitored inbox. Omit when unset (no Reply-To header is sent) — notification emails do not invite replies.
Expand Down
7 changes: 0 additions & 7 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 0 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,6 @@
"drizzle-orm": "0.45.4",
"hono": "4.13.14",
"semver": "7.8.5",
"spdx-license-ids": "3.0.24",
"zod": "4.6.7"
},
"devDependencies": {
Expand Down
95 changes: 71 additions & 24 deletions src/services/extensions/v1/database.ts
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
import { minVersion, validRange } from "semver";
import { and, eq, isNotNull, isNull, sql } from "drizzle-orm";
import { ExtensionsDb } from "../../../lib/db";
import { extensions, developers } from "../v2/db/schema";
import { DatabaseResult } from "../../../lib/interfaces";
import { sortReleasesDescending } from "../../../lib/releases";
import { parseJSON } from "../../../lib/json";
import type { Release as ManifestRelease } from "../v2/schemas/extensions";
import { Extension, Release, Author, Repository } from "./interfaces";

const EXTENSION_COLUMNS = {
Expand Down Expand Up @@ -34,10 +36,10 @@ const EXTENSION_COLUMNS = {
// non-null here because both queries below filter on published_at IS NOT NULL
// and extensions_published_content_check makes that filter sufficient - a
// published row cannot be missing any of them. That filter is also what keeps
// unreviewed extensions out of the v1 catalogue. delisted_at IS NULL is the
// other half: v1 shares this table with v2's public catalogue and must stay
// unreviewed extensions out of the v1 extension listing. delisted_at IS NULL is the
// other half: v1 shares this table with v2's public extension listing and must stay
// in sync with what v2 hides, or a moderator delisting an extension would
// pull it from the v2 catalogue while it stayed visible here.
// pull it from the v2 extension listing while it stayed visible here.
interface ExtensionRow {
id: string;
type: string;
Expand All @@ -63,6 +65,10 @@ export class ExtensionsDatabase {
// Shared catch shape for this module's D1 reads: driver message surfaced
// under the generic DATABASE_ERROR code.
private databaseError(error: unknown): DatabaseResult<never> {
console.error(
"[extensions-v1] Cannot read canonical extension data:",
error
);
return {
data: null,
error: {
Expand Down Expand Up @@ -101,10 +107,17 @@ export class ExtensionsDatabase {
.select(EXTENSION_COLUMNS)
.from(extensions)
.innerJoin(developers, eq(extensions.developerId, developers.id))
.where(type ? and(published, eq(extensions.type, type)) : published);
.where(
type
? and(
published,
eq(extensions.type, type === "mod" ? "module" : type)
)
: published
);
// Offset pagination needs a deterministic order (the unpaginated
// default deliberately stays unordered) - (LOWER(id), id) matches
// the published-catalogue index and is unique.
// the published-extension index and is unique.
rows = page
? ((await base
.orderBy(sql`LOWER(${extensions.id})`, extensions.id)
Expand All @@ -118,10 +131,14 @@ export class ExtensionsDatabase {
const hasMore = page ? rows.length > page.limit : false;
const trimmed = page && hasMore ? rows.slice(0, page.limit) : rows;

return {
data: { extensions: trimmed.map(parseExtensionRow), hasMore },
error: null
};
try {
return {
data: { extensions: trimmed.map(parseExtensionRow), hasMore },
error: null
};
} catch (error) {
return this.databaseError(error);
}
}

async getExtensionById(id: string): Promise<DatabaseResult<Extension>> {
Expand All @@ -147,7 +164,11 @@ export class ExtensionsDatabase {
return this.notFound(id);
}

return { data: parseExtensionRow(row), error: null };
try {
return { data: parseExtensionRow(row), error: null };
} catch (error) {
return this.databaseError(error);
}
}

// Badge/version endpoints read only these two columns, and badges are the
Expand Down Expand Up @@ -180,24 +201,26 @@ export class ExtensionsDatabase {
return this.notFound(id);
}

const releases = sortReleasesDescending(
parseJSON<Release[]>(row.releases, [])
);
return {
data: {
latestRelease: releases[0] ?? null,
license: parseJSON(row.license, { name: "" })
},
error: null
};
try {
const releases = sortReleasesDescending(toV1Releases(row.releases));
return {
data: {
latestRelease: releases[0] ?? null,
license: toV1License(row.license)
},
error: null
};
} catch (error) {
return this.databaseError(error);
}
}
}

function parseExtensionRow(row: ExtensionRow): Extension {
const releases = parseJSON<Release[]>(row.releases, []);
const releases = toV1Releases(row.releases);
return {
id: row.id,
type: row.type as Extension["type"],
type: (row.type === "module" ? "mod" : row.type) as Extension["type"],
name: row.name,
description: row.description,
author: {
Expand All @@ -212,11 +235,35 @@ function parseExtensionRow(row: ExtensionRow): Extension {
} as Author,
releases: sortReleasesDescending(releases),
website: row.website,
license: parseJSON(row.license, { name: "" }),
icon_url: row.iconUrl ?? "",
license: toV1License(row.license),
// Package-relative manifest icons have no public URL in the v1 contract.
icon_url:
row.iconUrl && /^https?:\/\//i.test(row.iconUrl) ? row.iconUrl : "",
readme: row.readme,
source: parseJSON<Repository>(row.source, { type: "custom", repo: "" }),
version: row.version,
download_url: row.downloadUrl
};
}

// Storage is canonical after migration 0029; these only adapt the v1 contract.
function toV1License(stored: string): { name: string } {
return { name: JSON.parse(stored) as string };
}

function toV1Releases(stored: string): Release[] {
return (JSON.parse(stored) as ManifestRelease[]).map(
({ requires, ...release }) => {
const range = requires.fossbilling.trim()
? validRange(requires.fossbilling)
: null;
const minimum = range === null ? null : minVersion(range);
if (!minimum) {
throw new Error(
`Release "${release.tag}" requires manual correction: FOSSBilling constraint "${requires.fossbilling}" has no representable v1 minimum`
);
}
return { ...release, min_fossbilling_version: minimum.version };
}
);
}
2 changes: 1 addition & 1 deletion src/services/extensions/v1/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ extensionsV1.use("/*", trimTrailingSlash());

// The assembled list is ~126KB of readme + release history per entry, rebuilt
// from a full-table D1 read (per the documented legacy contract) on every
// request - yet catalogue content changes at human speed. Cache the serialized
// request - yet extension content changes at human speed. Cache the serialized
// response body in CACHE_KV and let concurrent cold misses share one build per
// isolate; a cached response also freezes the unpaginated query's otherwise
// nondeterministic row order.
Expand Down
15 changes: 8 additions & 7 deletions src/services/extensions/v1/interfaces.ts
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
import { EXTENSION_TYPES } from "../v2/schemas/extensions";

export type Extension = {
id: string;
// Derived from v2 rather than restated: both describe the same
// extensions.type column (v1/database.ts already reads v2's db/schema), so a
// new extension type would otherwise pass v2's runtime validator while v1's
// type silently disagreed.
type: (typeof EXTENSION_TYPES)[number];
type:
| "mod"
| "theme"
| "payment-gateway"
| "server-manager"
| "domain-registrar"
| "hook"
| "translation";
name: string;
description: string;
author: Author;
Expand Down
4 changes: 2 additions & 2 deletions src/services/extensions/v1/list-cache.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
// Cache-key surface for the v1 list body cache (see index.ts). Kept separate
// so extensions/v2's revalidateCatalogue can invalidate it after catalogue
// so extensions/v2's revalidateExtensions can invalidate it after extension directory
// mutations without importing the v1 app module (which would drag the whole
// Hono app into v2's import graph for one function).
import { EXTENSION_TYPES } from "../v2/schemas/extensions";
Expand Down Expand Up @@ -33,7 +33,7 @@ export function invalidateListCache(
// Mirrors listCacheKey's encoding: the "all" sentinel for the unfiltered
// list, "%"-prefixed encoded keys per type.
const keys = [listCacheKey(undefined, undefined)].concat(
EXTENSION_TYPES.map((type) => listCacheKey(type, undefined))
[...EXTENSION_TYPES, "mod"].map((type) => listCacheKey(type, undefined))
);
for (const key of keys) {
waitUntil(kv.delete(key).catch(() => {}));
Expand Down
Loading
Loading