From 331bf0ec07eb1f349fb98f11c2d3146e825c43bd Mon Sep 17 00:00:00 2001 From: Per Nyfelt Date: Mon, 16 Nov 2026 21:58:50 +0100 Subject: [PATCH 1/3] Document current Portal capacity verification procedure --- AGENTS.md | 5 +- req/phase8-4-portal-capacity.md | 79 +++++++++++++++++++++++++++++++ req/phase8-4-release-checklist.md | 6 ++- tools/release-all/README.md | 3 +- 4 files changed, 89 insertions(+), 4 deletions(-) create mode 100644 req/phase8-4-portal-capacity.md diff --git a/AGENTS.md b/AGENTS.md index be597cc5..836ee81e 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -291,8 +291,9 @@ driven by `./releaseAll.sh`: the root `combinedRelease.modules` list names all n modules, the plan derives the candidate set (SNAPSHOT modules are excluded and reported), the script gates the plan's selected set against the expected release tuple, and `releaseAll` runs the selected modules' checks, signs, stages and bundles — including the `.module` each POM -advertises — and submits one Portal deployment (Central allows 10 deployments per month, and -the combined flow spends one slot for the whole tuple). The auxiliary modes are +advertises — and submits one Portal deployment for the tuple. Current monthly size, file-count +and release-count limits are organization-specific; verify the Usage Center and the +[capacity procedure](req/phase8-4-portal-capacity.md) before publication. The auxiliary modes are `./releaseAll.sh --dry-run` (plan + report, warning-only signing), `--bundle-only` (through bundling, no upload) and `--resume ` (polls an already-submitted deployment, journaled in `build/releaseAll/deployment.json`, to `PUBLISHED` without re-uploading). A diff --git a/req/phase8-4-portal-capacity.md b/req/phase8-4-portal-capacity.md new file mode 100644 index 00000000..d915239d --- /dev/null +++ b/req/phase8-4-portal-capacity.md @@ -0,0 +1,79 @@ +# Phase 8.4 Portal capacity check + +Reviewed 2026-11-16 at PR #101 merge `102e03ea43e508042ec4d43a61802b448a525267`. +Status: research complete; account capacity and signed-candidate measurement pending. +This procedure does not authorize publication or close the native-license gate. + +## Current official sources + +The [upload documentation](https://central.sonatype.org/publish/publish-portal-upload/) +allows a ZIP up to 1 GB containing multiple components. Our existing bundle verifier caps +the actual ZIP at 1,000,000,000 bytes. No separate per-file cap was established from this page. + +The [publishing limits](https://central.sonatype.org/publish/maven-central-publishing-limits/) +track monthly organization-wide file count, release size and release count. Current thresholds +come from the account, rather than a universal ten-deployment allowance. The public percentile +table is context, not proof of this account's entitlement. The policy also distinguishes +community publishing from commercial-nature artifacts; the latter require Publisher Pro +from 2026-10-01 independently of volume. Confirm the applicable account status before release. + +The [Usage Center documentation](https://central.sonatype.org/publish/publish-portal-usage-center/) +defines size as published-file bytes, files individually, and release events as distinct +publish operations. Select the organization containing `se.alipsa` at +[Publishing → Usage Center](https://central.sonatype.com/publishing/usage). +Retain a dated non-secret snapshot of current usage and limits for all three metrics, +organization/namespace scope, plan, adjusted-limit or exemption expiry, and any grace/enforcement +banner. Account for other releases in the same month and refresh immediately before publishing. + +## Measure the signed candidate + +After the runbook's preliminary-record and bundle validation succeeds, run this from the +candidate checkout. It reads the ZIP without extracting it and writes only local evidence. +The bundle digest must match the preliminary record and eventual final receipt. + +```sh +python3 - <<'PY' +import hashlib +import json +import zipfile +from pathlib import Path + +bundle = Path("build/releaseAll/jmlx-mega-bundle.zip") +digest = hashlib.sha256() +with bundle.open("rb") as stream: + for chunk in iter(lambda: stream.read(1024 * 1024), b""): + digest.update(chunk) +with zipfile.ZipFile(bundle) as archive: + files = [entry for entry in archive.infolist() if not entry.is_dir()] + if not files or len({entry.filename for entry in files}) != len(files): + raise SystemExit("expected non-empty bundle without duplicate files") + report = { + "bundleSha256": digest.hexdigest(), + "bundleBytes": bundle.stat().st_size, + "withinConservativeUploadLimit": bundle.stat().st_size <= 1_000_000_000, + "zipFileCount": len(files), + "zipEntryBytes": sum(entry.file_size for entry in files), + "files": [{"name": entry.filename, "bytes": entry.file_size} + for entry in sorted(files, key=lambda entry: entry.filename)], + } +Path("build/releaseAll/portal-bundle-measurement.json").write_text( + json.dumps(report, indent=2) + "\n", encoding="utf-8") +if not report["withinConservativeUploadLimit"]: + raise SystemExit("bundle exceeds conservative Portal upload limit") +PY +``` + +`zipEntryBytes` measures files inside the upload ZIP, including already-compressed JARs; +it does not expand the JARs' native payloads. `zipFileCount` includes signatures and checksum +sidecars if present. These are local inventory measurements, not authenticated Portal usage. +Confirm how Portal accounts for sidecars or generated files when reconciling the account. +The manifest requires five artifacts and five signatures per new coordinate: at least 90 +files for the proposed nine-module tuple, before optional checksum sidecars. + +Compare the inventory with remaining monthly capacity for the selected scope. The combined +flow submits one deployment; confirm its release-event accounting rather than equating nine +coordinates with nine events. The cited pages do not establish rejected-upload accounting; +obtain clarification if retries affect available capacity. Retain the snapshot, measurement, +candidate SHA/digest and accounting decision outside `build/` before running `clean`. +Close the checklist gate only when the exact signed bundle and current account evidence agree. +No signed candidate or account-specific capacity verdict exists yet. diff --git a/req/phase8-4-release-checklist.md b/req/phase8-4-release-checklist.md index eec67135..69274f2b 100644 --- a/req/phase8-4-release-checklist.md +++ b/req/phase8-4-release-checklist.md @@ -46,6 +46,8 @@ BellSoft 21.0.12.1; the matching-toolchain audit reproduces every JAR exactly. T [MLX issue #4902](https://github.com/ml-explore/mlx/issues/4902); awaiting an upstream response and review of any applicable grant or compliance decision. - [ ] Verify current official Portal size limits/accounting against actual signed bundle. + [Capacity procedure and current sources](phase8-4-portal-capacity.md), reviewed 2026-11-16; + organization Usage Center values and the exact signed bundle remain outstanding. - [x] Integrate released upstream plugin 2.6.0 and verify the clean historical 8.1 fixture. [Compatibility evidence](phase8-4-plugin-2.6.0-compatibility.md) records the unpushed scratch commit, expected published-tuple refusal, eligible-three dry-run and HTTPS upload fixture. @@ -99,7 +101,7 @@ exclusions, evidence and unresolved provenance; review its findings before closi Logs/receipts belong under ignored `build/releaseAll/` or outside the checkout. Never put credentials in the evidence or publish the temporary candidate draft Release. -## Portal limits research (2026-11-14) +## Portal limits research (2026-11-16) The official [upload documentation](https://central.sonatype.org/publish/publish-portal-upload/) states a 1 GB archive limit. The bundle checker conservatively caps compressed ZIP bytes at @@ -110,3 +112,5 @@ Current [publishing limits](https://central.sonatype.org/publish/maven-central-p and [Usage Center](https://central.sonatype.org/publish/publish-portal-usage-center/) require account-specific capacity verification. The older ten-deployments statement is not used as proof of present entitlement or rejected-upload accounting; that gate remains open. +The [capacity procedure](phase8-4-portal-capacity.md) separates upload bytes from monthly +published-file usage and records the account information needed to close this gate. diff --git a/tools/release-all/README.md b/tools/release-all/README.md index 9b9ad489..f9ec8891 100644 --- a/tools/release-all/README.md +++ b/tools/release-all/README.md @@ -137,7 +137,8 @@ The native publication's unconditional unstaged guard also applies to combined s The Portal upload documentation states a 1 GB archive limit; account-level publishing limits must be checked against current Usage Center data rather than assuming a universal deployment -quota. See the dated readiness record for sources and pending account-specific verification. +quota. Follow the [capacity procedure](../../req/phase8-4-portal-capacity.md) to retain the +account snapshot and measure the exact signed bundle. Account verification remains pending. Workflow run IDs and container image IDs are operator attestations, not authenticated remote provenance. Review actual workflow runs and the declared host before publication. Both macOS From 51ab165109931a370bcbe4d0ca9f81747596a7c6 Mon Sep 17 00:00:00 2001 From: Per Nyfelt Date: Tue, 17 Nov 2026 19:22:02 +0100 Subject: [PATCH 2/3] Address PR #102 review: counted sidecars, digest assertion, dated addendum - File count: signatures and checksums count toward the Portal file count per the publishing-limits FAQ; the pinned plugin's .asc/.md5/.sha1 sidecars are not optional. Expect at least 180 files (about 270 if the .asc files carry checksums) for the nine-module tuple; the exact figure comes from the measurement, not the estimate - The embedded script takes the preliminary record as an argument and asserts the bundle SHA-256 matches it before measuring, so a stale or rebuilt bundle fails before any evidence is written - testzip() verifies the archive's recorded CRCs before the directory sizes are trusted - The checklist keeps the historical 2026-11-14 research date; the 2026-11-16 capacity-procedure reference is a dated addendum, not a rewritten heading - State the grace-period rule (30 days, resets after 2 consecutive months below limits) so a banner can be read with its snapshot - Name the repository root of the candidate checkout as the script's working directory Verified: re-ran the embedded script against a synthetic bundle - happy path, stale record, CRC corruption and wrong-cwd cases all behave as documented. --- req/phase8-4-portal-capacity.md | 43 ++++++++++++++++++++++++------- req/phase8-4-release-checklist.md | 8 +++--- 2 files changed, 38 insertions(+), 13 deletions(-) diff --git a/req/phase8-4-portal-capacity.md b/req/phase8-4-portal-capacity.md index d915239d..91f4e8fa 100644 --- a/req/phase8-4-portal-capacity.md +++ b/req/phase8-4-portal-capacity.md @@ -23,27 +23,39 @@ publish operations. Select the organization containing `se.alipsa` at [Publishing → Usage Center](https://central.sonatype.com/publishing/usage). Retain a dated non-secret snapshot of current usage and limits for all three metrics, organization/namespace scope, plan, adjusted-limit or exemption expiry, and any grace/enforcement -banner. Account for other releases in the same month and refresh immediately before publishing. +banner. The limits page states that exceeding a threshold starts a 30-day grace period that +resets "after 2 consecutive months of being below limits", so read any grace/enforcement +banner together with its snapshot, not in isolation. Account for other releases in the same +month and refresh immediately before publishing. ## Measure the signed candidate After the runbook's preliminary-record and bundle validation succeeds, run this from the -candidate checkout. It reads the ZIP without extracting it and writes only local evidence. -The bundle digest must match the preliminary record and eventual final receipt. +repository root of the candidate checkout, passing the runbook's preliminary record +(`build/releaseAll/preliminary-.json`). It reads the ZIP without extracting it +and writes only local evidence. The script asserts the bundle digest matches the preliminary +record; a stale or rebuilt bundle fails before anything is measured, and the eventual final +receipt repeats the comparison. ```sh -python3 - <<'PY' +python3 - build/releaseAll/preliminary-.json <<'PY' import hashlib import json +import sys import zipfile from pathlib import Path bundle = Path("build/releaseAll/jmlx-mega-bundle.zip") +record = json.loads(Path(sys.argv[1]).read_text(encoding="utf-8")) digest = hashlib.sha256() with bundle.open("rb") as stream: for chunk in iter(lambda: stream.read(1024 * 1024), b""): digest.update(chunk) +if record.get("bundleSha256") != digest.hexdigest(): + raise SystemExit("bundle digest differs from the preliminary record") with zipfile.ZipFile(bundle) as archive: + if archive.testzip() is not None: + raise SystemExit("bundle ZIP failed CRC verification") files = [entry for entry in archive.infolist() if not entry.is_dir()] if not files or len({entry.filename for entry in files}) != len(files): raise SystemExit("expected non-empty bundle without duplicate files") @@ -63,12 +75,23 @@ if not report["withinConservativeUploadLimit"]: PY ``` -`zipEntryBytes` measures files inside the upload ZIP, including already-compressed JARs; -it does not expand the JARs' native payloads. `zipFileCount` includes signatures and checksum -sidecars if present. These are local inventory measurements, not authenticated Portal usage. -Confirm how Portal accounts for sidecars or generated files when reconciling the account. -The manifest requires five artifacts and five signatures per new coordinate: at least 90 -files for the proposed nine-module tuple, before optional checksum sidecars. +`zipEntryBytes` reports the sizes recorded in the ZIP directory for the files inside the +upload ZIP, including already-compressed JARs; it does not expand the JARs' native payloads, +and the script's `testzip` verifies the archive's recorded CRCs before either number is +trusted. `zipFileCount` includes the `.asc`, `.md5` and `.sha1` sidecars the pinned +nexus-release-plugin generates, and Sonatype's publishing-limits FAQ (its "Why does file +count include signatures, checksums, source jars, javadocs, and related files?" entry) +states these files "are already reflected in the usage data the thresholds are based on" — +they are not optional. The plugin's `BundleTask` computes MD5 and SHA-1 for every artifact +and its `BundleValidator` checks `.asc`, `.md5` and `.sha1` entries; +`req/phase8-1-release-notes.md` records the 8.1 bundle carrying the `.module` "with signature +and checksums"; the upload page's example tree shows `.md5`/`.sha1` next to each `.jar`, +`.pom` and `.asc`. The manifest's five artifacts and five signatures per new coordinate are +therefore at least 180 files for the proposed nine-module tuple (the 90 artifact/signature +files plus a `.md5` and a `.sha1` for each of the 45 artifacts), about 270 if the `.asc` +files also carry checksums as that example shows. Take the exact figure from the measurement +against the Usage Center snapshot, not from this estimate. These are local inventory +measurements, not authenticated Portal usage. Compare the inventory with remaining monthly capacity for the selected scope. The combined flow submits one deployment; confirm its release-event accounting rather than equating nine diff --git a/req/phase8-4-release-checklist.md b/req/phase8-4-release-checklist.md index 69274f2b..e8b63f9b 100644 --- a/req/phase8-4-release-checklist.md +++ b/req/phase8-4-release-checklist.md @@ -101,7 +101,7 @@ exclusions, evidence and unresolved provenance; review its findings before closi Logs/receipts belong under ignored `build/releaseAll/` or outside the checkout. Never put credentials in the evidence or publish the temporary candidate draft Release. -## Portal limits research (2026-11-16) +## Portal limits research (2026-11-14) The official [upload documentation](https://central.sonatype.org/publish/publish-portal-upload/) states a 1 GB archive limit. The bundle checker conservatively caps compressed ZIP bytes at @@ -112,5 +112,7 @@ Current [publishing limits](https://central.sonatype.org/publish/maven-central-p and [Usage Center](https://central.sonatype.org/publish/publish-portal-usage-center/) require account-specific capacity verification. The older ten-deployments statement is not used as proof of present entitlement or rejected-upload accounting; that gate remains open. -The [capacity procedure](phase8-4-portal-capacity.md) separates upload bytes from monthly -published-file usage and records the account information needed to close this gate. + +Addendum (2026-11-16): the [capacity procedure](phase8-4-portal-capacity.md) separates upload +bytes from monthly published-file usage and records the account information needed to close +this gate. From 96005773bc042bdda8978241ce7f9d41ac6ac980 Mon Sep 17 00:00:00 2001 From: Per Nyfelt Date: Tue, 17 Nov 2026 20:52:42 +0100 Subject: [PATCH 3/3] Address PR #102 review round 2: 270-file estimate, quoted placeholder, revision line - File count: the 8.1 combined bundle (nexus-release-plugin 2.5.1) carries exactly 30 files per coordinate - five artifacts and five signatures, each with .md5 and .sha1, including checksums of the .asc files themselves - so the nine-module tuple is expected at 270 files; the exact figure is confirmed from the 2.6.0 measurement (verified against the local 8.1 bundle inventory before stating it) - Quote the RECORD placeholder assignment: the unquoted form is parsed as a redirection in bash and sh (value truncated to 'preliminary-') and in zsh (value emptied) before the script runs; the quoted form preserves the literal placeholder in all three shells, and pasting it as-is now fails with a clear FileNotFoundError naming the path to set - Add the 'Revised 2026-11-17 (review rounds 1 and 2).' header line --- req/phase8-4-portal-capacity.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/req/phase8-4-portal-capacity.md b/req/phase8-4-portal-capacity.md index 91f4e8fa..ce50c42e 100644 --- a/req/phase8-4-portal-capacity.md +++ b/req/phase8-4-portal-capacity.md @@ -1,6 +1,7 @@ # Phase 8.4 Portal capacity check Reviewed 2026-11-16 at PR #101 merge `102e03ea43e508042ec4d43a61802b448a525267`. +Revised 2026-11-17 (review rounds 1 and 2). Status: research complete; account capacity and signed-candidate measurement pending. This procedure does not authorize publication or close the native-license gate. @@ -38,7 +39,8 @@ record; a stale or rebuilt bundle fails before anything is measured, and the eve receipt repeats the comparison. ```sh -python3 - build/releaseAll/preliminary-.json <<'PY' +RECORD='build/releaseAll/preliminary-.json' # set to the runbook's record +python3 - "$RECORD" <<'PY' import hashlib import json import sys @@ -83,15 +85,13 @@ nexus-release-plugin generates, and Sonatype's publishing-limits FAQ (its "Why d count include signatures, checksums, source jars, javadocs, and related files?" entry) states these files "are already reflected in the usage data the thresholds are based on" — they are not optional. The plugin's `BundleTask` computes MD5 and SHA-1 for every artifact -and its `BundleValidator` checks `.asc`, `.md5` and `.sha1` entries; -`req/phase8-1-release-notes.md` records the 8.1 bundle carrying the `.module` "with signature -and checksums"; the upload page's example tree shows `.md5`/`.sha1` next to each `.jar`, -`.pom` and `.asc`. The manifest's five artifacts and five signatures per new coordinate are -therefore at least 180 files for the proposed nine-module tuple (the 90 artifact/signature -files plus a `.md5` and a `.sha1` for each of the 45 artifacts), about 270 if the `.asc` -files also carry checksums as that example shows. Take the exact figure from the measurement -against the Usage Center snapshot, not from this estimate. These are local inventory -measurements, not authenticated Portal usage. +and its `BundleValidator` checks `.asc`, `.md5` and `.sha1` entries; the 8.1 combined bundle +(nexus-release-plugin 2.5.1; its coordinates are published on Central and its composition is +recorded in `req/phase8-1-release-notes.md`) carries exactly 30 files per coordinate — five +artifacts and five signatures, each with a `.md5` and a `.sha1`, including checksums of the +`.asc` files themselves. Expect 270 files for the proposed nine-module tuple and confirm the +exact figure from the 2.6.0 measurement. These are local inventory measurements, not +authenticated Portal usage. Compare the inventory with remaining monthly capacity for the selected scope. The combined flow submits one deployment; confirm its release-event accounting rather than equating nine