Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions src/services/extensions/v2/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,15 @@ New v2 writes use the [FOSSBilling manifest v1 fields](https://github.com/FOSSBi

API publishing fields `readme`, `source`, `download_url`, and `releases` remain alongside the manifest metadata. Each release carries its own `requires` object instead of `min_fossbilling_version`. Constraints are retained verbatim, including Composer ranges and module dependencies; package CI remains responsible for semantic constraint/license validation and confirming that icon files exist. The API adds size bounds to metadata (author names: 200 characters; license and requirement strings: 1,000; URLs and icon paths: 2,048).

`requires.fossbilling` is mandatory. `requires.php` is an optional additional restriction: when omitted, FOSSBilling’s PHP requirements apply; when supplied, both the core and extension constraints must be satisfied.

`author` is package attribution. `developer` identifies the authenticated owner and remains controlled by the API; an author field never grants ownership.

Run the read-only `db/manifest-preflight.sql` query before deployment. It reports extension IDs and zero-based release indexes needing manual correction. Every published release must have an explicitly reviewed `requires` object, including its PHP constraint; migration does not invent `*`, an author, or a license placeholder. Use the package manifest/release metadata to fill missing requirements. A deliberate `*` is allowed only when it accurately represents the reviewed requirement.
Run the read-only `db/manifest-preflight.sql` query before deployment. It reports extension IDs and zero-based release indexes needing manual correction.

Migration `0029` fills a missing `requires.fossbilling` from the legacy minimum as a `>=` constraint, preserving existing PHP and module constraints. It also adds verified PHP constraints from the BTCPay 0.1.3–0.1.5 and OpenReceive 0.4.29 packages. Explicitly invalid values require correction using verified package metadata.

Migration `0029_align_manifest.sql` stops with `manifest_data_requires_manual_correction` if the preflight reports issues, rolling back the entire migration. Correct those rows and retry. Once prepared, it converts `mod` to `module`, converts legacy license objects to strings (using `spdx_id`, otherwise `name`), copies the current release's exact requirements and the developer's attribution, and removes obsolete release minimum fields. Physical `website` storage remains for v1 compatibility. Reviewed and pending revision bodies retain their original audit content; old pending revisions must be resubmitted using the new contract before approval. Published reads expect canonical storage; historical audit reads remain separate.
Migration `0029_align_manifest.sql` stops with `manifest_data_requires_manual_correction` if the preflight reports issues, rolling back the entire migration. Correct those rows and retry. Once prepared, it converts `mod` to `module`, converts legacy license objects to strings (using `spdx_id`, otherwise `name`), copies the current release's requirements and the developer's attribution, and removes obsolete release minimum fields. Physical `website` storage remains for v1 compatibility. Reviewed and pending revision bodies retain their original audit content; old pending revisions must be resubmitted using the new contract before approval. Published reads expect canonical storage; historical audit reads remain separate.

For example, after checking an affected package's actual requirements, an operator can add `requires` to one release without rewriting its remaining fields:

Expand Down
14 changes: 10 additions & 4 deletions src/services/extensions/v2/db/manifest-preflight.sql
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,13 @@ WITH published AS (
WHERE e.published_at IS NOT NULL
), release_rows AS (
SELECT p.id, r.key AS release_index, r.type AS release_type,
CASE WHEN r.type = 'object' THEN r.value ELSE '{}' END AS release
CASE WHEN r.type = 'object' THEN CASE
WHEN (json_type(r.value,'$.requires') IS NULL
OR (json_type(r.value,'$.requires') = 'object' AND json_type(r.value,'$.requires.fossbilling') IS NULL))
AND json_type(r.value,'$.min_fossbilling_version') = 'text'
AND length(trim(json_extract(r.value,'$.min_fossbilling_version'))) BETWEEN 1 AND 100
THEN json_set(r.value,'$.requires.fossbilling','>=' || trim(json_extract(r.value,'$.min_fossbilling_version')))
ELSE r.value END ELSE '{}' END AS release
FROM published p, json_each(CASE WHEN json_valid(p.releases) THEN p.releases ELSE '[]' END) r
)
SELECT p.id, NULL AS release_index, problem.value AS issue
Expand All @@ -34,13 +40,13 @@ FROM published p, json_each(json_array(
THEN 'version: identify exactly one current release matching version or v+version' END
)) problem WHERE problem.value IS NOT NULL
UNION ALL
SELECT id, release_index, 'requires: set explicit FOSSBilling and PHP constraints for this release; missing values are not guessed'
SELECT id, release_index, 'requires: supply a FOSSBilling constraint or legacy minimum and a valid optional PHP constraint'
FROM release_rows WHERE release_type != 'object'
OR COALESCE(json_type(release,'$.requires'),'') != 'object'
OR COALESCE(json_type(release,'$.requires.fossbilling'),'') != 'text'
OR COALESCE(json_type(release,'$.requires.php'),'') != 'text'
OR COALESCE(length(trim(json_extract(release,'$.requires.fossbilling'))),0) NOT BETWEEN 1 AND 1000
OR COALESCE(length(trim(json_extract(release,'$.requires.php'))),0) NOT BETWEEN 1 AND 1000
OR (json_type(release,'$.requires.php') IS NOT NULL AND (json_type(release,'$.requires.php') != 'text'
Comment thread
admdly marked this conversation as resolved.
Resolved
OR length(trim(json_extract(release,'$.requires.php'))) NOT BETWEEN 1 AND 1000))
OR EXISTS (SELECT 1 FROM json_each(CASE WHEN json_type(release,'$.requires') = 'object' THEN json_extract(release,'$.requires') ELSE '{}' END)
WHERE key NOT IN ('fossbilling','php','modules'))
OR (json_type(release,'$.requires.modules') IS NOT NULL AND json_type(release,'$.requires.modules') != 'object')
Expand Down
49 changes: 44 additions & 5 deletions src/services/extensions/v2/db/migrations/0029_align_manifest.sql
Original file line number Diff line number Diff line change
Expand Up @@ -167,7 +167,13 @@ WITH published AS (
WHERE e.published_at IS NOT NULL
), release_rows AS (
SELECT p.id, r.key AS release_index, r.type AS release_type,
CASE WHEN r.type = 'object' THEN r.value ELSE '{}' END AS release
CASE WHEN r.type = 'object' THEN CASE
WHEN (json_type(r.value,'$.requires') IS NULL
OR (json_type(r.value,'$.requires') = 'object' AND json_type(r.value,'$.requires.fossbilling') IS NULL))
AND json_type(r.value,'$.min_fossbilling_version') = 'text'
AND length(trim(json_extract(r.value,'$.min_fossbilling_version'))) BETWEEN 1 AND 100
THEN json_set(r.value,'$.requires.fossbilling','>=' || trim(json_extract(r.value,'$.min_fossbilling_version')))
ELSE r.value END ELSE '{}' END AS release
FROM published p, json_each(CASE WHEN json_valid(p.releases) THEN p.releases ELSE '[]' END) r
)
SELECT p.id, NULL AS release_index, problem.value AS issue
Expand All @@ -188,13 +194,13 @@ FROM published p, json_each(json_array(
THEN 'version: identify exactly one current release matching version or v+version' END
)) problem WHERE problem.value IS NOT NULL
UNION ALL
SELECT id, release_index, 'requires: set explicit FOSSBilling and PHP constraints for this release; missing values are not guessed'
SELECT id, release_index, 'requires: supply a FOSSBilling constraint or legacy minimum and a valid optional PHP constraint'
FROM release_rows WHERE release_type != 'object'
OR COALESCE(json_type(release,'$.requires'),'') != 'object'
OR COALESCE(json_type(release,'$.requires.fossbilling'),'') != 'text'
OR COALESCE(json_type(release,'$.requires.php'),'') != 'text'
OR COALESCE(length(trim(json_extract(release,'$.requires.fossbilling'))),0) NOT BETWEEN 1 AND 1000
OR COALESCE(length(trim(json_extract(release,'$.requires.php'))),0) NOT BETWEEN 1 AND 1000
OR (json_type(release,'$.requires.php') IS NOT NULL AND (json_type(release,'$.requires.php') != 'text'
OR length(trim(json_extract(release,'$.requires.php'))) NOT BETWEEN 1 AND 1000))
OR EXISTS (SELECT 1 FROM json_each(CASE WHEN json_type(release,'$.requires') = 'object' THEN json_extract(release,'$.requires') ELSE '{}' END)
WHERE key NOT IN ('fossbilling','php','modules'))
OR (json_type(release,'$.requires.modules') IS NOT NULL AND json_type(release,'$.requires.modules') != 'object')
Expand All @@ -212,6 +218,38 @@ DROP TABLE _manifest_migration_check;
--> statement-breakpoint
DROP VIEW _manifest_migration_issues;
--> statement-breakpoint
-- Preserve legacy minimums as lower-bound constraints. No extra PHP constraint
-- means the extension inherits the PHP requirements of its FOSSBilling runtime.
UPDATE extensions SET releases = (
SELECT json_group_array(json(CASE
WHEN r.type = 'object' AND (json_type(r.value,'$.requires') IS NULL
OR (json_type(r.value,'$.requires') = 'object' AND json_type(r.value,'$.requires.fossbilling') IS NULL))
AND json_type(r.value,'$.min_fossbilling_version') = 'text'
AND length(trim(json_extract(r.value,'$.min_fossbilling_version'))) BETWEEN 1 AND 100
THEN json_set(r.value,'$.requires.fossbilling','>=' || trim(json_extract(r.value,'$.min_fossbilling_version')))
ELSE r.value END))
FROM json_each(CASE WHEN json_valid(extensions.releases) THEN extensions.releases ELSE '[]' END) r
) WHERE published_at IS NOT NULL AND json_valid(releases) AND json_type(releases) = 'array';
--> statement-breakpoint
-- Composer requirements from the distributed release packages:
-- https://github.com/ChristianGabs/btcpay-fossbilling/releases (0.1.3-0.1.5)
-- https://github.com/OpenReceive/openreceive/releases/tag/v0.4.29
UPDATE extensions SET releases = (
SELECT json_group_array(json(CASE
WHEN json_extract(value,'$.tag') IN ('0.1.3','v0.1.3','0.1.4','v0.1.4','0.1.5','v0.1.5')
AND json_type(value,'$.requires.php') IS NULL
THEN json_set(value,'$.requires.php','>=8.1') ELSE value END))
FROM json_each(extensions.releases)
) WHERE id = 'BTCPay' AND published_at IS NOT NULL;
--> statement-breakpoint
UPDATE extensions SET releases = (
SELECT json_group_array(json(CASE
WHEN json_extract(value,'$.tag') IN ('0.4.29','v0.4.29')
AND json_type(value,'$.requires.php') IS NULL
THEN json_set(value,'$.requires.php','^8.2') ELSE value END))
FROM json_each(extensions.releases)
) WHERE id = 'openreceive' AND published_at IS NOT NULL;
--> statement-breakpoint
UPDATE extensions SET
type = CASE WHEN type = 'mod' THEN 'module' ELSE type END,
license = json_quote(CASE WHEN json_type(license) = 'object'
Expand All @@ -225,7 +263,8 @@ UPDATE extensions SET
)
WHERE published_at IS NOT NULL;
--> statement-breakpoint
-- Requirements have been explicitly reviewed. Remove only the obsolete v1 field.
-- Requirements retain the legacy minimum or an explicit constraint.
-- Remove only the obsolete v1 field.
UPDATE extensions SET releases = (
SELECT json_group_array(json(json_remove(value,'$.min_fossbilling_version')))
FROM json_each(extensions.releases)
Expand Down
6 changes: 5 additions & 1 deletion src/services/extensions/v2/schemas/manifest.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,11 @@ const versionConstraint = z
export const ManifestRequirementsSchema = z
.object({
fossbilling: versionConstraint,
php: versionConstraint,
php: versionConstraint
.optional()
.describe(
"Additional PHP constraint; when omitted, FOSSBilling runtime requirements apply"
),
modules: z
.record(z.string().regex(/^[a-z0-9_-]+$/), versionConstraint)
.optional()
Expand Down
38 changes: 37 additions & 1 deletion test/services/extensions/v2/manifest.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,42 @@ describe("manifest v1 publishing", () => {
});
});

it("publishes without an additional PHP restriction", async () => {
await seedDeveloper("owner", "user-1");
await insertUser(db, { id: "moderator", is_moderator: 1 });
const fixture = sampleCreate();
const requires = { fossbilling: "^0.8.9" };
const content = {
...fixture,
requires,
releases: fixture.releases.map((release) => ({ ...release, requires }))
};
const created = await post(
"/extensions/v2/extensions",
await authHeaders("user-1"),
content
);
expect(created.status).toBe(201);
const { result } = (await created.json()) as {
result: { revision_id: string };
};
expect(
(
await post(
`/extensions/v2/extensions/${content.id}/revisions/${result.revision_id}/approve`,
await authHeaders("moderator"),
{}
)
).status
).toBe(200);
const published = (await (
await get(`/extensions/v2/extensions/${content.id}`, {})
).json()) as { result: typeof content };
expect(published.result.requires).toEqual(requires);
expect(published.result.releases[0].requires).toEqual(requires);
expect((await get(`/extensions/v1/${content.id}`, {})).status).toBe(200);
});

it("keeps the historical minimum-version field bounded in revision schemas", () => {
const content = (minimum: string) => ({
releases: [
Expand Down Expand Up @@ -178,7 +214,7 @@ describe("manifest v1 publishing", () => {
{ version: "1.0.0+" + "x".repeat(100) },
{ description: "x".repeat(501) },
{ name: "x".repeat(101) },
{ requires: { fossbilling: "^1" } },
{ requires: { fossbilling: "^1", php: null } },
{ requires: { fossbilling: " ", php: ">=8.3" } },
{ requires: { fossbilling: "^1", php: " " } },
{
Expand Down
111 changes: 106 additions & 5 deletions test/services/extensions/v2/migrations.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1004,8 +1004,7 @@ it("migrates published manifest metadata without changing reviewed audit bodies"
{
tag: "1.0.0",
date: "2026-01-01",
download_url: "https://example.com/a.zip",
min_fossbilling_version: "0.8.9"
download_url: "https://example.com/a.zip"
},
{ ...olderRelease, min_fossbilling_version: "0.7.0" }
]);
Expand Down Expand Up @@ -1036,7 +1035,7 @@ it("migrates published manifest metadata without changing reviewed audit bodies"
{
id: "legacy",
release_index: 0,
issue: expect.stringContaining("set explicit FOSSBilling and PHP")
issue: expect.stringContaining("supply a FOSSBilling constraint")
}
]);
// Existing over-quota collections must survive a structural backfill.
Expand All @@ -1052,7 +1051,7 @@ it("migrates published manifest metadata without changing reviewed audit bodies"
expect(
db.prepare("SELECT type FROM extensions WHERE id='legacy'").get()?.type
).toBe("mod");
const incompleteCorrection = `UPDATE extensions SET releases=json_set(releases,'$[0].requires',json('{"fossbilling":">=0.8.9"}')) WHERE id='legacy';`;
const incompleteCorrection = `UPDATE extensions SET releases=json_set(releases,'$[0].requires',json('{"fossbilling":">=0.8.9","php":null}')) WHERE id='legacy';`;
db.exec("BEGIN");
expect(() =>
db.exec(
Expand Down Expand Up @@ -1205,7 +1204,9 @@ it.each([
],
[
"releases",
JSON.stringify([{ tag: "1.0.0", requires: { fossbilling: ">=0.8.9" } }]),
JSON.stringify([
{ tag: "1.0.0", requires: { fossbilling: ">=0.8.9", php: null } }
]),
"requires:",
0
],
Expand Down Expand Up @@ -1273,3 +1274,103 @@ it.each([
}
}
);

it.each(["", "v"])(
"migrates legacy requirements with %s-prefixed release tags",
(prefix) => {
const db = new DatabaseSync(":memory:");
try {
db.exec("PRAGMA foreign_keys=ON");
for (const name of migrationNames.filter((name) => name < "0029"))
db.exec(migration(name));
db.exec(
"INSERT INTO developers(id,type,name) VALUES('dev','organization','Developer')"
);
const fixtures = [
{
id: "legacy",
tags: ["1.0.0"],
minimum: "0.6",
initial: undefined,
expected: { fossbilling: ">=0.6" }
},
{
id: "BTCPay",
tags: ["0.1.5", "0.1.4", "0.1.3"],
minimum: "0.6",
initial: undefined,
expected: { fossbilling: ">=0.6", php: ">=8.1" }
},
{
id: "openreceive",
tags: ["0.4.29"],
minimum: "0.8.0",
initial: undefined,
expected: { fossbilling: ">=0.8.0", php: "^8.2" }
},
{
id: "partial",
tags: ["1.0.0"],
minimum: "0.6",
initial: { php: ">=8.4", modules: { branding: "^1" } },
expected: {
fossbilling: ">=0.6",
php: ">=8.4",
modules: { branding: "^1" }
}
}
];
for (const fixture of fixtures) {
const releases = fixture.tags.map((tag) => ({
tag: prefix + tag,
date: "2026-01-01",
download_url: "https://example.com/release.zip",
min_fossbilling_version: fixture.minimum,
requires: fixture.initial
}));
db.prepare(
`INSERT INTO extensions(id,developer_id,type,name,description,releases,website,license,readme,source,version,download_url,published_at)
VALUES(?,'dev','mod','Legacy','Description',?,'','{"name":"MIT"}','README','{}',?,'https://example.com/release.zip','now')`
).run(fixture.id, JSON.stringify(releases), fixture.tags[0]);
}
const preflight = readFileSync(
join(migrationsDirectory, "../manifest-preflight.sql"),
"utf8"
);
expect(db.prepare(preflight).all()).toEqual([]);
db.exec("BEGIN");
db.exec(migration("0029_align_manifest.sql"));
db.exec("COMMIT");
expect(db.prepare(preflight).all()).toEqual([]);
for (const fixture of fixtures) {
const row = db
.prepare("SELECT releases,manifest FROM extensions WHERE id=?")
.get(fixture.id)!;
const releases = JSON.parse(row.releases as string);
expect(releases.map((release: { tag: string }) => release.tag)).toEqual(
fixture.tags.map((tag) => prefix + tag)
);
for (const release of releases) {
expect(release.requires).toEqual(fixture.expected);
expect(release).not.toHaveProperty("min_fossbilling_version");
}
expect(JSON.parse(row.manifest as string).requires).toEqual(
fixture.expected
);
}
expect(
db
.prepare(
readFileSync(
join(migrationsDirectory, "../resource-inventory.sql"),
"utf8"
)
)
.all()
).toEqual([]);
expect(db.prepare("PRAGMA foreign_key_check").all()).toEqual([]);
} finally {
db.close();
}
}
);
Loading