From a595a50f5305070aeda2db0e2752e4d884a8e36b Mon Sep 17 00:00:00 2001 From: Nate Chadwick <263952448+natechadwick-intsof@users.noreply.github.com> Date: Sun, 2 Aug 2026 09:37:32 -0400 Subject: [PATCH 1/6] feat(build): generate THIRD-PARTY inventory via license-maven-plugin Replace hand-curated third-party component lists and version pins in NOTICE.txt and thirdPartyCopyright with a stable product pointer. The versioned dependency/license inventory is now produced at build time by org.codehaus.mojo:license-maven-plugin (aggregate-add-third-party) and copied into the perc-distribution-tree assembly as THIRD-PARTY.txt. Resolves #1689. > Co-Authored by Grok Build using grok-4.5 with agent main. --- NOTICE.txt | 27 ++- .../1689-license-maven-plugin-erlang.md | 57 ++++++ modules/perc-distribution-tree/pom.xml | 50 +++++ .../ThirdPartyInventoryPackagingTest.java | 119 ++++++++++++ pom.xml | 46 +++++ src/license/README.md | 48 +++++ src/license/THIRD-PARTY.properties | 17 ++ .../server/PSStringResources.properties | 20 +- .../server/PSThirdPartyCopyrightTest.java | 177 ++++++++++++++++++ 9 files changed, 534 insertions(+), 27 deletions(-) create mode 100644 docs/ai-generated/code-reviews/1689-license-maven-plugin-erlang.md create mode 100644 modules/perc-distribution-tree/src/test/java/com/percussion/distribution/install/ThirdPartyInventoryPackagingTest.java create mode 100644 src/license/README.md create mode 100644 src/license/THIRD-PARTY.properties create mode 100644 system/src/test/java/com/percussion/server/PSThirdPartyCopyrightTest.java diff --git a/NOTICE.txt b/NOTICE.txt index 453b604804..eb2741c106 100644 --- a/NOTICE.txt +++ b/NOTICE.txt @@ -1,17 +1,14 @@ Percussion CMS Copyright 1999-2026 Percussion Software, Inc. - -This product includes software developed by the Apache Software Foundation (http://www.apache.org/). -Copyright (c) 2004 The Apache Software Foundation. All rights reserved. - -GNU Runtime Libraries are included in this product and are covered under the GNU LGPL (http://www.gnu.org/licenses/lgpl.html). - -This product includes the jTDS driver, which is released under the terms of the GNU LGPL. - -XStream Copyright (c) 2003-2005, Joe Walnes. All rights reserved. -ASM Copyright (c) 2000-2005 INRIA, France Telecom All rights reserved. -Lato font Copyright (c) 2012, Lukasz Dziedzic -with Reserved Font Name Lato. -This Font Software is licensed under the SIL Open Font License, Version 1.1. -This license is copied below, and is also available with a FAQ at: -http://scripts.sil.org/OFL +Additional contributions and ongoing maintenance by Intersoft Data Labs Pvt. Ltd. +(https://www.intsof.com), 2023-present. + +This product is licensed under the Apache License, Version 2.0 +(https://www.apache.org/licenses/LICENSE-2.0). A copy of the license is provided +in the LICENSE.txt file in the product distribution. + +This product includes third-party open source software. A complete, versioned +inventory of third-party dependencies and their licenses is generated from the +Maven reactor dependency set at build time and is shipped as THIRD-PARTY.txt +in the product distribution. Do not hand-maintain version pins or component +lists here — the build-generated inventory is authoritative. diff --git a/docs/ai-generated/code-reviews/1689-license-maven-plugin-erlang.md b/docs/ai-generated/code-reviews/1689-license-maven-plugin-erlang.md new file mode 100644 index 0000000000..85b01bce9e --- /dev/null +++ b/docs/ai-generated/code-reviews/1689-license-maven-plugin-erlang.md @@ -0,0 +1,57 @@ +# Erlang review: #1689 license-maven-plugin THIRD-PARTY inventory + +| Field | Value | +|--------------------|----------------------------------| +| **Date** | 2026-08-02 | +| **Branch** | `feat/1689-license-maven-plugin` | +| **Scope** | Uncommitted work for issue #1689 | +| **Recommendation** | approve | +| **Gate** | May commit/push: **yes** | +| **Blocking bugs** | 0 | + +## Summary + +Adopts `org.codehaus.mojo:license-maven-plugin` on the reactor root to generate a versioned +`THIRD-PARTY.txt` inventory from the dependency set. Hand-curated component lists and version pins +are removed from `NOTICE.txt` and `thirdPartyCopyright`; both become stable pointers only. The +installer module copies `LICENSE.txt`, `NOTICE.txt`, and the generated inventory into the assembly +root. Behavioral tests cover the blurb policy and packaging when the inventory is present. + +## Scope + +- `pom.xml` — plugin version property + root-only aggregate execution +- `NOTICE.txt` — stable product notice + pointer +- `system/.../PSStringResources.properties` — thin `thirdPartyCopyright` / copyright year +- `system/.../PSThirdPartyCopyrightTest.java` — new +- `modules/perc-distribution-tree/pom.xml` — copy license artifacts into assembly +- `modules/perc-distribution-tree/.../ThirdPartyInventoryPackagingTest.java` — new +- `src/license/*` — missing-license map + README +- Out of scope discarded: `modules/perc-i18n/scripts/cache/i18n_translate.json` (unrelated drift) + +**Memory patterns hit:** non-portable path joins (checked clean — uses `Path`/`Files`); missing +behavioral tests (present for blurb + packaging); incomplete change-class (packaging companion +included). + +**Cross-platform path review:** clean. Tests resolve repo root via `Path` walk/`resolve`; no +hardcoded `/` or `\` filesystem joins; no Unix-only absolute roots; no line-ending fragile multi-line +file equality assertions. + +## Issues + +### suggestion — packaging soft-skips when inventory missing + +- **File:** `modules/perc-distribution-tree/src/test/java/com/percussion/distribution/install/ThirdPartyInventoryPackagingTest.java` +- **Note:** `assumeTrue` means standalone Surefire without a prior root aggregate pass does not fail. + Acceptable for this monorepo (AC targets full reactor). Documented in pom comment + `src/license/README.md`. + Full-reactor / process-resources path was verified locally (LICENSE + NOTICE + THIRD-PARTY in assembly). + +### nit — copyright year pin in test + +- **File:** `system/.../PSThirdPartyCopyrightTest.java` (`1999-2026`) +- **Note:** Will need a yearly bump; matches product prose. Acceptable. + +## Gate + +No bugs. No missing behavioral tests for the changed policy. Path I/O portable. + +**May commit/push: yes** diff --git a/modules/perc-distribution-tree/pom.xml b/modules/perc-distribution-tree/pom.xml index 1b3f52a0c6..ac986ac261 100644 --- a/modules/perc-distribution-tree/pom.xml +++ b/modules/perc-distribution-tree/pom.xml @@ -393,6 +393,56 @@ + + + org.apache.maven.plugins + maven-resources-plugin + ${maven.resources.plugin.version} + + + + copy-license-inventory + + copy-resources + + generate-resources + + ${assembly-directory} + + + ${maven.multiModuleProjectDirectory} + + LICENSE.txt + NOTICE.txt + + false + + + ${maven.multiModuleProjectDirectory}/target/generated-sources/license + + THIRD-PARTY.txt + + false + + + + + + maven-dependency-plugin diff --git a/modules/perc-distribution-tree/src/test/java/com/percussion/distribution/install/ThirdPartyInventoryPackagingTest.java b/modules/perc-distribution-tree/src/test/java/com/percussion/distribution/install/ThirdPartyInventoryPackagingTest.java new file mode 100644 index 0000000000..bc657c6fb9 --- /dev/null +++ b/modules/perc-distribution-tree/src/test/java/com/percussion/distribution/install/ThirdPartyInventoryPackagingTest.java @@ -0,0 +1,119 @@ +/* + * Copyright 1999-2026 Percussion Software, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.percussion.distribution.install; + +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertTrue; +import static org.junit.jupiter.api.Assumptions.assumeTrue; + +import java.io.IOException; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.nio.file.Paths; +import org.junit.jupiter.api.Test; + +/** + * Packaging guard for the build-generated third-party license inventory (issue #1689). + * + *

When the reactor root has already produced {@code + * target/generated-sources/license/THIRD-PARTY.txt} (full reactor build or {@code mvn + * license:aggregate-add-third-party} without {@code -N}), the distribution assembly root must also + * contain that file after {@code generate-resources} (copy-license-inventory). Standalone module + * test runs without a prior generation step skip the assembly assertion rather than inventing a + * hand-curated inventory. + */ +public class ThirdPartyInventoryPackagingTest { + + @Test + void assemblyShipsGeneratedThirdPartyInventoryWhenAvailable() throws IOException { + Path repoRoot = resolveRepoRoot(); + Path generated = + repoRoot + .resolve("target") + .resolve("generated-sources") + .resolve("license") + .resolve("THIRD-PARTY.txt"); + assumeTrue( + Files.isRegularFile(generated) && Files.size(generated) > 0, + "THIRD-PARTY.txt not generated yet — run from repo root (no -N):" + + " mvnw license:aggregate-add-third-party"); + + Path assemblyCopy = + Paths.get("target") + .resolve("classes") + .resolve("distribution") + .resolve("THIRD-PARTY.txt") + .toAbsolutePath() + .normalize(); + assumeTrue( + Files.isRegularFile(assemblyCopy), + "Assembly copy not present yet (generate-resources / copy-license-inventory not run)."); + + String text = Files.readString(assemblyCopy, StandardCharsets.UTF_8); + assertFalse(text.isBlank(), "Packaged THIRD-PARTY.txt must not be blank"); + assertTrue( + text.toLowerCase().contains("license") || text.contains("("), + "Packaged THIRD-PARTY.txt should look like a license inventory"); + } + + @Test + void assemblyShipsStableLicenseAndNoticeWhenPresent() throws IOException { + Path repoRoot = resolveRepoRoot(); + Path licenseSrc = repoRoot.resolve("LICENSE.txt"); + Path noticeSrc = repoRoot.resolve("NOTICE.txt"); + assumeTrue(Files.isRegularFile(licenseSrc), "LICENSE.txt missing at repo root"); + assumeTrue(Files.isRegularFile(noticeSrc), "NOTICE.txt missing at repo root"); + + Path assemblyDir = + Paths.get("target").resolve("classes").resolve("distribution").toAbsolutePath().normalize(); + assumeTrue( + Files.isDirectory(assemblyDir), + "Assembly directory not built yet — generate-resources not run"); + + Path licenseOut = assemblyDir.resolve("LICENSE.txt"); + Path noticeOut = assemblyDir.resolve("NOTICE.txt"); + assumeTrue( + Files.isRegularFile(licenseOut) && Files.isRegularFile(noticeOut), + "LICENSE.txt/NOTICE.txt not yet copied into assembly (copy-license-inventory not run)"); + + assertFalse(Files.readString(licenseOut, StandardCharsets.UTF_8).isBlank()); + String notice = Files.readString(noticeOut, StandardCharsets.UTF_8); + assertTrue( + notice.contains("THIRD-PARTY.txt"), + "Packaged NOTICE.txt must point at the generated inventory"); + } + + /** + * Resolves the monorepo root whether Surefire runs from {@code modules/perc-distribution-tree} or + * another cwd. Portable Path API only. + */ + private static Path resolveRepoRoot() { + Path cwd = Paths.get("").toAbsolutePath().normalize(); + Path probe = cwd; + for (int i = 0; i < 8 && probe != null; i++) { + if (Files.isRegularFile(probe.resolve("LICENSE.txt")) + && Files.isRegularFile(probe.resolve("pom.xml"))) { + return probe; + } + probe = probe.getParent(); + } + // Fallback: module is two levels under root. + return cwd.resolve("..").resolve("..").normalize(); + } +} diff --git a/pom.xml b/pom.xml index c7f211ae14..f509ea9862 100644 --- a/pom.xml +++ b/pom.xml @@ -183,6 +183,8 @@ 1.3.1 4.13.2 6.0.2 + + 2.7.1 1.0.0 5.0.1 4.17.0 @@ -3044,6 +3046,50 @@ false + + + org.codehaus.mojo + license-maven-plugin + ${license.maven.plugin.version} + false + + ${maven.multiModuleProjectDirectory}/target/generated-sources/license + THIRD-PARTY.txt + ${project.build.sourceEncoding} + true + true + false + + com\.percussion|com\.intsof + test,system + + false + true + ${maven.multiModuleProjectDirectory}/src/license/THIRD-PARTY.properties + + false + + Apache License, Version 2.0|The Apache Software License, Version 2.0|Apache 2|Apache-2.0|ASL, version 2|The Apache License, Version 2.0 + MIT License|The MIT License|MIT + BSD-3-Clause|BSD Licence 3|The BSD 3-Clause License|BSD 3-clause|3-Clause BSD License|New BSD License + Eclipse Public License - v 2.0|EPL 2.0|Eclipse Public License 2.0|Eclipse Public License v2.0 + + + + + aggregate-third-party-inventory + + aggregate-add-third-party + + generate-resources + + + org.apache.maven.plugins diff --git a/src/license/README.md b/src/license/README.md new file mode 100644 index 0000000000..1f09b0c45f --- /dev/null +++ b/src/license/README.md @@ -0,0 +1,48 @@ +# Third-party license inventory (build-generated) + +## What is authoritative + +The **versioned** third-party dependency / license inventory is **not** hand-edited. + +It is produced at build time by +[`org.codehaus.mojo:license-maven-plugin`](https://www.mojohaus.org/license-maven-plugin/) +(`aggregate-add-third-party`) configured on the **reactor root** `pom.xml` +(issue [#1689](https://github.com/intersoftdatalabs-in/percussioncms/issues/1689)): + +| Artifact | Location | +|-------------------------------|-----------------------------------------------------------------------------------------------------| +| Generated inventory | `${repo-root}/target/generated-sources/license/THIRD-PARTY.txt` | +| Shipped with installer | `THIRD-PARTY.txt` at the root of the `perc-distribution-tree` assembly (copied from the path above) | +| Product notice (stable prose) | root `NOTICE.txt` — product copyright + pointer only | +| Startup / About blurb | `system` resource key `thirdPartyCopyright` — same pointer, **no version pins** | + +Do **not** reintroduce hand-curated component lists or dependency version pins into +`NOTICE.txt` or `thirdPartyCopyright`. That was the drift failure mode this automation +replaces. + +## Manual generation + +From the repository root (JDK 21 + Maven wrapper). **Do not pass `-N` / `--non-recursive`** +— that only loads the empty root POM and produces an empty inventory: + +```bash +# Loads the full reactor, then runs the aggregate goal on the root only +./mvnw license:aggregate-add-third-party +# Windows: +mvnw.cmd license:aggregate-add-third-party +``` + +A full reactor build also runs the goal during the root `generate-resources` phase +(before child modules package). + +## `THIRD-PARTY.properties` (this directory) + +This file is **not** the inventory. It is an optional **missing-license map** used when +an upstream POM does not declare a license. Entries look like: + +```properties +groupId--artifactId--version=Some License Name +``` + +Only add rows for dependencies the plugin reports as missing a license. Never use this +file to re-list the full dependency set by hand. diff --git a/src/license/THIRD-PARTY.properties b/src/license/THIRD-PARTY.properties new file mode 100644 index 0000000000..48ea421c17 --- /dev/null +++ b/src/license/THIRD-PARTY.properties @@ -0,0 +1,17 @@ +# Missing-license overrides for org.codehaus.mojo:license-maven-plugin. +# Format: groupId--artifactId--version=License Name +# +# Only list dependencies whose POM does not declare a license. Do not hand-maintain +# a full inventory here — the plugin generates THIRD-PARTY.txt from the reactor. +# +# Entries below are the small set currently reported as "Unknown license" by the plugin. + +avalon-framework--avalon-framework--4.1.5=Apache Software License, Version 1.1 +classworlds--classworlds--1.1-alpha-2=Apache Software License, Version 1.1 +org.codehaus.plexus--plexus-container-default--1.0-alpha-9-stable-1=Apache License, Version 2.0 +jakarta-regexp--jakarta-regexp--1.4=Apache License, Version 2.0 +net.htmlparser--jericho-html--2.1=Eclipse Public License 1.0 +oro--oro--2.0.8=Apache License, Version 2.0 +saxon--saxon--6.5.3=Mozilla Public License 1.0 +stax--stax--1.2.0=Apache License, Version 2.0 +xpp3--xpp3--1.1.3.3=Indiana University Extreme! Lab Software License diff --git a/system/src/main/resources/com/percussion/server/PSStringResources.properties b/system/src/main/resources/com/percussion/server/PSStringResources.properties index 884962b69f..ac7fc9a801 100644 --- a/system/src/main/resources/com/percussion/server/PSStringResources.properties +++ b/system/src/main/resources/com/percussion/server/PSStringResources.properties @@ -7,18 +7,14 @@ ########################################################################### copyright=Percussion CMS \ - Copyright (C) Percussion Software, Inc. 1999-2023 -thirdPartyCopyright=This product includes software developed by the Apache Software Foundation (http://www.apache.org/). \ - Copyright (c) 2000 The Apache Software Foundation. All rights reserved. \ - GNU Runtime Libraries are included in this product and are covered under the GNU LGPL (http://www.gnu.org/licenses/lgpl.html). \ - This product includes the jTDS driver v1.2.2, which is released under the terms of the GNU LGPL. \ - XStream Copyright (c) 2003-2005, Joe Walnes. All rights reserved. \ - ASM Copyright (c) 2000-2005 INRIA, France Telecom All rights reserved. \ - Lato font Copyright (c) 2012, Lukasz Dziedzic \ - with Reserved Font Name Lato. \ - This Font Software is licensed under the SIL Open Font License, Version 1.1. \ - This license is copied below, and is also available with a FAQ at: \ - http://scripts.sil.org/OFL + Copyright (C) Percussion Software, Inc. 1999-2026 \ + Additional contributions and ongoing maintenance by Intersoft Data Labs Pvt. Ltd. (https://www.intsof.com), 2023-present. +# Stable, version-agnostic pointer only. The authoritative versioned inventory is +# build-generated by org.codehaus.mojo:license-maven-plugin as THIRD-PARTY.txt +# (see root pom.xml, issue #1689). Do not list components or pin versions here. +thirdPartyCopyright=This product is licensed under the Apache License, Version 2.0 (https://www.apache.org/licenses/LICENSE-2.0).\n\ + Additional contributions and ongoing maintenance by Intersoft Data Labs Pvt. Ltd. (https://www.intsof.com), 2023-present.\n\ + This product includes third-party open source software. A complete, versioned inventory of third-party dependencies and their licenses is generated from the Maven reactor dependency set at build time and is shipped as THIRD-PARTY.txt in the product distribution (alongside LICENSE.txt and NOTICE.txt). # Trace option names and descriptions traceBasicRequestInfo_dispname=Basic Request Info traceBasicRequestInfo_desc=The Basic Request Information trace logs the type of request (POST or GET) and the complete URL of the request. diff --git a/system/src/test/java/com/percussion/server/PSThirdPartyCopyrightTest.java b/system/src/test/java/com/percussion/server/PSThirdPartyCopyrightTest.java new file mode 100644 index 0000000000..14dd516baf --- /dev/null +++ b/system/src/test/java/com/percussion/server/PSThirdPartyCopyrightTest.java @@ -0,0 +1,177 @@ +/* + * Copyright 1999-2026 Percussion Software, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.percussion.server; + +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertNotNull; +import static org.junit.jupiter.api.Assertions.assertTrue; + +import java.io.IOException; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.nio.file.Paths; +import java.util.Locale; +import java.util.ResourceBundle; +import java.util.regex.Pattern; +import org.junit.jupiter.api.Test; + +/** + * Verifies the third-party copyright blurb emitted at server startup. + * + *

Policy (issue #1689): the resource-bundle text and root {@code NOTICE.txt} are a stable, + * version-agnostic pointer only. The authoritative versioned inventory is build-generated by {@code + * org.codehaus.mojo:license-maven-plugin} as {@code THIRD-PARTY.txt}. Do not reintroduce + * hand-curated component lists or dependency version pins here. + */ +public class PSThirdPartyCopyrightTest { + + /** Matches {@code vMAJOR.MINOR.PATCH}-style pins (e.g. {@code v1.3.1}, {@code v2.3.232}). */ + private static final Pattern VERSION_PIN = Pattern.compile("\\bv\\d+\\.\\d+\\.\\d+\\b"); + + /** Matches bare {@code MAJOR.MINOR.PATCH} pins (no leading {@code v}). */ + private static final Pattern BARE_VERSION_PIN = + Pattern.compile("(? Date: Sun, 2 Aug 2026 09:56:11 -0400 Subject: [PATCH 2/6] feat(build): merge npm production licenses into THIRD-PARTY.txt Extend the #1689 inventory so the shipped file covers both Maven and npm. - license-maven-plugin writes THIRD-PARTY-MAVEN.txt - scripts/generate-third-party-inventory.py reads product package-lock.json production packages (WebUI SPA lockfile) and merges both halves into a single THIRD-PARTY.txt for the installer - Unit tests for the merge script; packaging test asserts npm section > Co-Authored by Grok Build using grok-4.5 with agent main. --- .../ThirdPartyInventoryPackagingTest.java | 7 + pom.xml | 39 ++- scripts/README.md | 30 ++ scripts/generate-third-party-inventory.bat | 9 + scripts/generate-third-party-inventory.py | 284 ++++++++++++++++++ .../test_generate_third_party_inventory.py | 117 ++++++++ src/license/README.md | 68 +++-- src/license/npm-package-locks.txt | 11 + 8 files changed, 542 insertions(+), 23 deletions(-) create mode 100644 scripts/generate-third-party-inventory.bat create mode 100644 scripts/generate-third-party-inventory.py create mode 100644 scripts/test_generate_third_party_inventory.py create mode 100644 src/license/npm-package-locks.txt diff --git a/modules/perc-distribution-tree/src/test/java/com/percussion/distribution/install/ThirdPartyInventoryPackagingTest.java b/modules/perc-distribution-tree/src/test/java/com/percussion/distribution/install/ThirdPartyInventoryPackagingTest.java index bc657c6fb9..c6cef7f2d2 100644 --- a/modules/perc-distribution-tree/src/test/java/com/percussion/distribution/install/ThirdPartyInventoryPackagingTest.java +++ b/modules/perc-distribution-tree/src/test/java/com/percussion/distribution/install/ThirdPartyInventoryPackagingTest.java @@ -70,6 +70,13 @@ void assemblyShipsGeneratedThirdPartyInventoryWhenAvailable() throws IOException assertTrue( text.toLowerCase().contains("license") || text.contains("("), "Packaged THIRD-PARTY.txt should look like a license inventory"); + // Merged inventory (Maven + npm) — issue #1689 + assertTrue( + text.contains("Maven third-party dependencies"), + "Packaged THIRD-PARTY.txt should include the Maven inventory section"); + assertTrue( + text.contains("npm third-party dependencies") || text.contains("npm:"), + "Packaged THIRD-PARTY.txt should include the npm production inventory"); } @Test diff --git a/pom.xml b/pom.xml index f509ea9862..906f35a16d 100644 --- a/pom.xml +++ b/pom.xml @@ -185,6 +185,8 @@ 6.0.2 2.7.1 + + python 1.0.0 5.0.1 4.17.0 @@ -3048,9 +3050,10 @@ org.codehaus.mojo @@ -3059,7 +3062,8 @@ false ${maven.multiModuleProjectDirectory}/target/generated-sources/license - THIRD-PARTY.txt + + THIRD-PARTY-MAVEN.txt ${project.build.sourceEncoding} true true @@ -3090,6 +3094,33 @@ + + + org.codehaus.mojo + exec-maven-plugin + ${maven.exec.plugin.version} + false + + + merge-third-party-inventory + generate-resources + + exec + + + + ${python.executable} + ${maven.multiModuleProjectDirectory} + + ${maven.multiModuleProjectDirectory}/scripts/generate-third-party-inventory.py + --root + ${maven.multiModuleProjectDirectory} + --require-maven + + + + + org.apache.maven.plugins diff --git a/scripts/README.md b/scripts/README.md index 002af10f02..8455c60c60 100644 --- a/scripts/README.md +++ b/scripts/README.md @@ -12,6 +12,36 @@ Out of scope for spec 994 (must NOT be touched): ## Scripts +### `generate-third-party-inventory.py` / `generate-third-party-inventory.bat` + +Merge **Maven** and **npm** third-party license inventories into a single +`THIRD-PARTY.txt` (issue [#1689](https://github.com/intersoftdatalabs-in/percussioncms/issues/1689)). + +- **Purpose**: `license-maven-plugin` only sees Maven GAVs. Product UI also ships npm + production deps from `package-lock.json`. This script writes the npm half and merges + both into one file for the installer. +- **Usage**: + + ```bash + # After Maven inventory exists: + ./mvnw license:aggregate-add-third-party + python3 scripts/generate-third-party-inventory.py --require-maven + + # Windows: + mvnw.cmd license:aggregate-add-third-party + scripts\generate-third-party-inventory.bat --require-maven + ``` + +- **Inputs**: + - `target/generated-sources/license/THIRD-PARTY-MAVEN.txt` (from license-maven-plugin) + - `src/license/npm-package-locks.txt` (list of product package-lock.json paths) +- **Outputs** (under `target/generated-sources/license/`): + - `THIRD-PARTY-NPM.txt` — npm production intermediate + - `THIRD-PARTY.txt` — **merged** inventory (shipped) +- **Tests**: `python3 -m pytest scripts/test_generate_third_party_inventory.py -v` + (or `python3 scripts/test_generate_third_party_inventory.py`) +- **Docs**: `src/license/README.md` + ### `prune-stale-worktrees.py` / `prune-stale-worktrees.bat` List or remove **stale git worktrees** left by agent sessions (Kilo / Grok / etc.). diff --git a/scripts/generate-third-party-inventory.bat b/scripts/generate-third-party-inventory.bat new file mode 100644 index 0000000000..cb110b9d02 --- /dev/null +++ b/scripts/generate-third-party-inventory.bat @@ -0,0 +1,9 @@ +@echo off +REM Windows launcher for generate-third-party-inventory.py (issue #1689) +SETLOCAL +cd /d "%~dp0\.." +python scripts\generate-third-party-inventory.py %* +IF ERRORLEVEL 1 ( + python3 scripts\generate-third-party-inventory.py %* +) +ENDLOCAL diff --git a/scripts/generate-third-party-inventory.py b/scripts/generate-third-party-inventory.py new file mode 100644 index 0000000000..18566eea3e --- /dev/null +++ b/scripts/generate-third-party-inventory.py @@ -0,0 +1,284 @@ +#!/usr/bin/env python3 +"""Merge Maven + npm third-party license inventories into a single THIRD-PARTY.txt. + +Issue #1689: org.codehaus.mojo:license-maven-plugin produces the Maven half; +this script reads product package-lock.json files for production npm deps and +writes one merged inventory consumed by the installer assembly. + +Cross-platform (Windows / Linux / macOS). Python 3.9+. Stdlib only. +""" + +from __future__ import annotations + +import argparse +import json +import sys +from dataclasses import dataclass +from pathlib import Path +from typing import Iterable + + +DEFAULT_MAVEN_NAME = "THIRD-PARTY-MAVEN.txt" +DEFAULT_NPM_INTERMEDIATE = "THIRD-PARTY-NPM.txt" +DEFAULT_MERGED_NAME = "THIRD-PARTY.txt" +DEFAULT_LOCK_LIST = "src/license/npm-package-locks.txt" + + +@dataclass(frozen=True, order=True) +class NpmPackage: + name: str + version: str + license: str + source: str # lockfile path (repo-relative) for provenance + + def format_line(self) -> str: + lic = self.license if self.license else "Unknown license" + ver = self.version if self.version else "unknown" + return f" ({lic}) {self.name} (npm:{self.name}:{ver} - {self.source})" + + +def repo_root_from_script() -> Path: + return Path(__file__).resolve().parent.parent + + +def read_lock_list(list_file: Path, root: Path) -> list[Path]: + if not list_file.is_file(): + return [] + locks: list[Path] = [] + for raw in list_file.read_text(encoding="utf-8").splitlines(): + line = raw.strip() + if not line or line.startswith("#"): + continue + # Portable: list entries use forward slashes; resolve via Path parts. + rel = Path(*line.replace("\\", "/").split("/")) + path = (root / rel).resolve() + locks.append(path) + return locks + + +def _package_name_from_lock_key(key: str) -> str | None: + """Map package-lock 'packages' key to npm package name. + + Keys look like: + node_modules/react + node_modules/@scope/pkg + node_modules/foo/node_modules/bar + """ + if not key or key == ".": + return None + # Use the last node_modules segment (nested installs). + marker = "node_modules/" + if marker not in key.replace("\\", "/"): + # Workspace / file: link entry (e.g. ../../../vendor/mkd-language) + # Prefer packages that are real registry installs under node_modules. + return None + norm = key.replace("\\", "/") + idx = norm.rfind(marker) + name = norm[idx + len(marker) :] + return name or None + + +def collect_npm_from_lock(lock_path: Path, root: Path) -> list[NpmPackage]: + if not lock_path.is_file(): + raise FileNotFoundError(f"package-lock.json not found: {lock_path}") + data = json.loads(lock_path.read_text(encoding="utf-8")) + packages = data.get("packages") + if not isinstance(packages, dict): + raise ValueError(f"Unsupported package-lock (missing packages map): {lock_path}") + + try: + rel_source = lock_path.resolve().relative_to(root.resolve()).as_posix() + except ValueError: + rel_source = str(lock_path) + + found: dict[tuple[str, str], NpmPackage] = {} + for key, meta in packages.items(): + if not isinstance(meta, dict): + continue + if meta.get("dev") is True or meta.get("devOptional") is True: + continue + name = _package_name_from_lock_key(str(key)) + if not name: + continue + version = str(meta.get("version") or "").strip() + if not version: + # file: linked packages sometimes omit version in the packages map + continue + license_raw = meta.get("license") + if isinstance(license_raw, dict): + # rare { "type": "MIT", "url": "..." } + license_str = str(license_raw.get("type") or license_raw.get("name") or "Unknown license") + elif isinstance(license_raw, list): + license_str = " OR ".join(str(x) for x in license_raw) + elif license_raw: + license_str = str(license_raw) + else: + # Optional fallback: node_modules package.json when present + license_str = _license_from_node_modules(lock_path.parent, name) or "Unknown license" + + pkg = NpmPackage(name=name, version=version, license=license_str, source=rel_source) + found[(name, version)] = pkg + + return sorted(found.values()) + + +def _license_from_node_modules(package_root: Path, name: str) -> str | None: + # Portable join of scoped package path segments. + parts = name.split("/") + pkg_json = package_root.joinpath("node_modules", *parts, "package.json") + if not pkg_json.is_file(): + return None + try: + meta = json.loads(pkg_json.read_text(encoding="utf-8")) + except (OSError, json.JSONDecodeError): + return None + lic = meta.get("license") + if isinstance(lic, dict): + return str(lic.get("type") or lic.get("name") or "") or None + if isinstance(lic, list): + return " OR ".join(str(x) for x in lic) or None + if lic: + return str(lic) + return None + + +def format_npm_section(packages: Iterable[NpmPackage]) -> str: + pkgs = list(packages) + lines = [ + f"Lists of {len(pkgs)} third-party npm dependencies (production).", + "", + ] + for p in pkgs: + lines.append(p.format_line()) + lines.append("") + return "\n".join(lines) + + +def merge_inventories(maven_text: str, npm_text: str) -> str: + maven_body = maven_text.strip() + npm_body = npm_text.strip() + parts = [ + "Percussion CMS third-party dependency license inventory", + "Generated at build time — do not hand-edit. See src/license/README.md (issue #1689).", + "", + "================================================================================", + "Maven third-party dependencies", + "================================================================================", + "", + maven_body if maven_body else "(no Maven inventory — run license:aggregate-add-third-party first)", + "", + "================================================================================", + "npm third-party dependencies (production)", + "================================================================================", + "", + npm_body if npm_body else "(no npm production dependencies found)", + "", + ] + return "\n".join(parts) + + +def generate( + root: Path, + out_dir: Path, + maven_name: str = DEFAULT_MAVEN_NAME, + npm_name: str = DEFAULT_NPM_INTERMEDIATE, + merged_name: str = DEFAULT_MERGED_NAME, + lock_list: Path | None = None, + require_maven: bool = False, +) -> Path: + out_dir.mkdir(parents=True, exist_ok=True) + maven_path = out_dir / maven_name + npm_path = out_dir / npm_name + merged_path = out_dir / merged_name + + if maven_path.is_file(): + maven_text = maven_path.read_text(encoding="utf-8") + else: + if require_maven: + raise FileNotFoundError( + f"Maven inventory missing: {maven_path}. " + "Run mvnw license:aggregate-add-third-party first." + ) + maven_text = "" + + list_file = lock_list if lock_list is not None else root / DEFAULT_LOCK_LIST + locks = read_lock_list(list_file, root) + npm_packages: dict[tuple[str, str], NpmPackage] = {} + missing_locks: list[Path] = [] + for lock in locks: + if not lock.is_file(): + missing_locks.append(lock) + continue + for pkg in collect_npm_from_lock(lock, root): + npm_packages[(pkg.name, pkg.version)] = pkg + + if missing_locks: + missing = ", ".join(str(p) for p in missing_locks) + print(f"WARNING: package-lock.json not found (skipped): {missing}", file=sys.stderr) + + npm_sorted = sorted(npm_packages.values()) + npm_text = format_npm_section(npm_sorted) + npm_path.write_text(npm_text, encoding="utf-8", newline="\n") + + merged = merge_inventories(maven_text, npm_text) + merged_path.write_text(merged, encoding="utf-8", newline="\n") + print( + f"Wrote {merged_path} " + f"(Maven present={maven_path.is_file()}, npm packages={len(npm_sorted)})" + ) + return merged_path + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument( + "--root", + type=Path, + default=None, + help="Repository root (default: parent of scripts/)", + ) + parser.add_argument( + "--out-dir", + type=Path, + default=None, + help="Output directory (default: /target/generated-sources/license)", + ) + parser.add_argument( + "--lock-list", + type=Path, + default=None, + help=f"File listing package-lock.json paths (default: /{DEFAULT_LOCK_LIST})", + ) + parser.add_argument( + "--require-maven", + action="store_true", + help="Fail if THIRD-PARTY-MAVEN.txt is missing", + ) + args = parser.parse_args(argv) + + root = (args.root or repo_root_from_script()).resolve() + out_dir = ( + args.out_dir.resolve() + if args.out_dir + else root / "target" / "generated-sources" / "license" + ) + lock_list = args.lock_list.resolve() if args.lock_list else root / DEFAULT_LOCK_LIST + + try: + generate( + root=root, + out_dir=out_dir, + lock_list=lock_list, + require_maven=args.require_maven, + ) + except FileNotFoundError as exc: + print(f"ERROR: {exc}", file=sys.stderr) + return 1 + except (OSError, ValueError, json.JSONDecodeError) as exc: + print(f"ERROR: {exc}", file=sys.stderr) + return 1 + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/test_generate_third_party_inventory.py b/scripts/test_generate_third_party_inventory.py new file mode 100644 index 0000000000..df613e2824 --- /dev/null +++ b/scripts/test_generate_third_party_inventory.py @@ -0,0 +1,117 @@ +#!/usr/bin/env python3 +"""Unit tests for generate-third-party-inventory.py (issue #1689).""" + +from __future__ import annotations + +import importlib.util +import json +import tempfile +import unittest +from pathlib import Path + +# Load hyphenated sibling script as a module (stdlib only). +import sys + +SCRIPTS = Path(__file__).resolve().parent +_SPEC = importlib.util.spec_from_file_location( + "generate_third_party_inventory", + SCRIPTS / "generate-third-party-inventory.py", +) +assert _SPEC and _SPEC.loader +g = importlib.util.module_from_spec(_SPEC) +# dataclasses frozen/order needs the module registered before exec_module +sys.modules[_SPEC.name] = g +_SPEC.loader.exec_module(g) + + +class CollectNpmFromLockTest(unittest.TestCase): + def test_production_packages_include_react_exclude_dev(self) -> None: + with tempfile.TemporaryDirectory() as tmp: + root = Path(tmp) + lock_dir = root / "frontend" + lock_dir.mkdir() + lock = { + "lockfileVersion": 3, + "packages": { + "": {"name": "app", "version": "1.0.0"}, + "node_modules/react": { + "version": "19.2.8", + "license": "MIT", + }, + "node_modules/vitest": { + "version": "4.1.0", + "license": "MIT", + "dev": True, + }, + "node_modules/@scope/pkg": { + "version": "1.2.3", + "license": "Apache-2.0", + }, + }, + } + lock_path = lock_dir / "package-lock.json" + lock_path.write_text(json.dumps(lock), encoding="utf-8") + + pkgs = g.collect_npm_from_lock(lock_path, root) + names = {p.name for p in pkgs} + self.assertIn("react", names) + self.assertIn("@scope/pkg", names) + self.assertNotIn("vitest", names) + react = next(p for p in pkgs if p.name == "react") + self.assertEqual(react.version, "19.2.8") + self.assertEqual(react.license, "MIT") + self.assertIn("npm:react:19.2.8", react.format_line()) + + +class MergeInventoriesTest(unittest.TestCase): + def test_merge_contains_both_sections(self) -> None: + maven = "Lists of 1 third-party dependencies.\n (MIT) foo (g:a:1 - )" + npm = "Lists of 1 third-party npm dependencies (production).\n (MIT) react (npm:react:1 - x)" + merged = g.merge_inventories(maven, npm) + self.assertIn("Maven third-party dependencies", merged) + self.assertIn("npm third-party dependencies (production)", merged) + self.assertIn("foo", merged) + self.assertIn("react", merged) + self.assertIn("do not hand-edit", merged) + + +class GenerateEndToEndTest(unittest.TestCase): + def test_generate_writes_merged_file(self) -> None: + with tempfile.TemporaryDirectory() as tmp: + root = Path(tmp) + out = root / "out" + # Minimal maven half + out.mkdir() + (out / g.DEFAULT_MAVEN_NAME).write_text( + "Lists of 1 third-party dependencies.\n (Apache License, Version 2.0) guava\n", + encoding="utf-8", + ) + # Lock list + lock + lock_dir = root / "ui" + lock_dir.mkdir() + lock = { + "lockfileVersion": 3, + "packages": { + "": {}, + "node_modules/jquery": {"version": "3.7.1", "license": "MIT"}, + }, + } + (lock_dir / "package-lock.json").write_text(json.dumps(lock), encoding="utf-8") + list_file = root / "locks.txt" + list_file.write_text("ui/package-lock.json\n", encoding="utf-8") + + merged = g.generate( + root=root, + out_dir=out, + lock_list=list_file, + require_maven=True, + ) + text = merged.read_text(encoding="utf-8") + self.assertIn("guava", text) + self.assertIn("jquery", text) + self.assertIn("npm:jquery:3.7.1", text) + self.assertTrue((out / g.DEFAULT_NPM_INTERMEDIATE).is_file()) + + +if __name__ == "__main__": + unittest.main() diff --git a/src/license/README.md b/src/license/README.md index 1f09b0c45f..2a604a6108 100644 --- a/src/license/README.md +++ b/src/license/README.md @@ -4,17 +4,24 @@ The **versioned** third-party dependency / license inventory is **not** hand-edited. -It is produced at build time by -[`org.codehaus.mojo:license-maven-plugin`](https://www.mojohaus.org/license-maven-plugin/) -(`aggregate-add-third-party`) configured on the **reactor root** `pom.xml` -(issue [#1689](https://github.com/intersoftdatalabs-in/percussioncms/issues/1689)): - -| Artifact | Location | -|-------------------------------|-----------------------------------------------------------------------------------------------------| -| Generated inventory | `${repo-root}/target/generated-sources/license/THIRD-PARTY.txt` | -| Shipped with installer | `THIRD-PARTY.txt` at the root of the `perc-distribution-tree` assembly (copied from the path above) | -| Product notice (stable prose) | root `NOTICE.txt` — product copyright + pointer only | -| Startup / About blurb | `system` resource key `thirdPartyCopyright` — same pointer, **no version pins** | +It is produced at build time and written to a **single** merged file: + +| Piece | How | +|-------|-----| +| Maven / Java dependencies | [`org.codehaus.mojo:license-maven-plugin`](https://www.mojohaus.org/license-maven-plugin/) `aggregate-add-third-party` → `THIRD-PARTY-MAVEN.txt` | +| npm production dependencies | `scripts/generate-third-party-inventory.py` reads product `package-lock.json` files listed in `npm-package-locks.txt` → `THIRD-PARTY-NPM.txt` | +| **Shipped inventory** | Same script **merges** both halves → **`THIRD-PARTY.txt`** | + +Issue [#1689](https://github.com/intersoftdatalabs-in/percussioncms/issues/1689). + +| Artifact | Location | +|----------|----------| +| Merged inventory (authoritative) | `${repo-root}/target/generated-sources/license/THIRD-PARTY.txt` | +| Maven intermediate | `…/THIRD-PARTY-MAVEN.txt` | +| npm intermediate | `…/THIRD-PARTY-NPM.txt` | +| Shipped with installer | `THIRD-PARTY.txt` at the root of the `perc-distribution-tree` assembly | +| Product notice (stable prose) | root `NOTICE.txt` — product copyright + pointer only | +| Startup / About blurb | `system` resource key `thirdPartyCopyright` — same pointer, **no version pins** | Do **not** reintroduce hand-curated component lists or dependency version pins into `NOTICE.txt` or `thirdPartyCopyright`. That was the drift failure mode this automation @@ -22,26 +29,49 @@ replaces. ## Manual generation -From the repository root (JDK 21 + Maven wrapper). **Do not pass `-N` / `--non-recursive`** -— that only loads the empty root POM and produces an empty inventory: +From the repository root (JDK 21 + Maven wrapper + Python 3.9+). + +**Do not pass `-N` / `--non-recursive` to `license:aggregate-add-third-party`** — that only +loads the empty root POM and produces an empty Maven inventory. ```bash -# Loads the full reactor, then runs the aggregate goal on the root only +# 1) Maven half (full reactor dependency graph) ./mvnw license:aggregate-add-third-party +# 2) npm half + merge into THIRD-PARTY.txt +python3 scripts/generate-third-party-inventory.py --require-maven + # Windows: mvnw.cmd license:aggregate-add-third-party +scripts\generate-third-party-inventory.bat --require-maven ``` -A full reactor build also runs the goal during the root `generate-resources` phase -(before child modules package). +A full reactor build also runs both steps on the root `generate-resources` phase +(license plugin first, then the merge script). CLI +`license:aggregate-add-third-party` alone does **not** run the merge — always follow +with the Python script when generating by hand. + +If `python` is not on `PATH`, set `-Dpython.executable=python3` on the Maven command +that runs the merge execution, or invoke `python3` on the script directly. + +## npm package locks + +Edit `npm-package-locks.txt` to add product-shipped frontend lockfiles. **Do not** list +QA-only trees (`modules/perc-qa-automation`) or pure build tooling. + +The WebUI SPA is built from `WebUI/src/main/frontend` — that lockfile is listed once. +Do not also list `WebUI/package-lock.json` (duplicate production set). + +Production packages are those in the lockfile `packages` map that are **not** marked +`dev` / `devOptional`. Licenses come from the lockfile `license` field (with optional +`node_modules/.../package.json` fallback when present). ## `THIRD-PARTY.properties` (this directory) -This file is **not** the inventory. It is an optional **missing-license map** used when -an upstream POM does not declare a license. Entries look like: +This file is **not** the inventory. It is an optional **missing-license map** for Maven +dependencies whose POM does not declare a license. Entries look like: ```properties -groupId--artifactId--version=Some License Name +groupId--artifactId--version=License Name ``` Only add rows for dependencies the plugin reports as missing a license. Never use this diff --git a/src/license/npm-package-locks.txt b/src/license/npm-package-locks.txt new file mode 100644 index 0000000000..32bb03b2be --- /dev/null +++ b/src/license/npm-package-locks.txt @@ -0,0 +1,11 @@ +# Product-shipped npm package-lock.json paths (relative to repo root). +# One path per line. Comments (#) and blank lines are ignored. +# +# Only product UI / delivery frontends belong here — not QA automation +# (modules/perc-qa-automation) or pure build tooling lockfiles. +# +# The WebUI SPA is built from src/main/frontend (see WebUI/pom.xml +# frontend-maven-plugin workingDirectory). Do not also list WebUI/package-lock.json +# or production packages will be double-counted. + +WebUI/src/main/frontend/package-lock.json From 9dcba97c2b055675261291b4231f00e96b2ba0b0 Mon Sep 17 00:00:00 2001 From: Nate Chadwick <263952448+natechadwick-intsof@users.noreply.github.com> Date: Sun, 2 Aug 2026 10:02:28 -0400 Subject: [PATCH 3/6] style: Spotless sort root pom for license inventory plugin config Pre-push spotless:check requires property alphabetical order and execution element order for the merge-third-party-inventory plugin. > Co-Authored by Grok Build using grok-4.5 with agent main. --- pom.xml | 6 +++--- scripts/README.md | 1 - src/license/README.md | 26 +++++++++++++------------- 3 files changed, 16 insertions(+), 17 deletions(-) diff --git a/pom.xml b/pom.xml index 906f35a16d..b013f52d7d 100644 --- a/pom.xml +++ b/pom.xml @@ -185,8 +185,6 @@ 6.0.2 2.7.1 - - python 1.0.0 5.0.1 4.17.0 @@ -245,6 +243,8 @@ 42.7.12 UTF-8 UTF-8 + + python 2.5.2 2.1.0 6.5.3 @@ -3103,10 +3103,10 @@ merge-third-party-inventory - generate-resources exec + generate-resources ${python.executable} diff --git a/scripts/README.md b/scripts/README.md index 8455c60c60..6a2bfdecd4 100644 --- a/scripts/README.md +++ b/scripts/README.md @@ -31,7 +31,6 @@ Merge **Maven** and **npm** third-party license inventories into a single mvnw.cmd license:aggregate-add-third-party scripts\generate-third-party-inventory.bat --require-maven ``` - - **Inputs**: - `target/generated-sources/license/THIRD-PARTY-MAVEN.txt` (from license-maven-plugin) - `src/license/npm-package-locks.txt` (list of product package-lock.json paths) diff --git a/src/license/README.md b/src/license/README.md index 2a604a6108..091f9f147b 100644 --- a/src/license/README.md +++ b/src/license/README.md @@ -6,22 +6,22 @@ The **versioned** third-party dependency / license inventory is **not** hand-edi It is produced at build time and written to a **single** merged file: -| Piece | How | -|-------|-----| -| Maven / Java dependencies | [`org.codehaus.mojo:license-maven-plugin`](https://www.mojohaus.org/license-maven-plugin/) `aggregate-add-third-party` → `THIRD-PARTY-MAVEN.txt` | -| npm production dependencies | `scripts/generate-third-party-inventory.py` reads product `package-lock.json` files listed in `npm-package-locks.txt` → `THIRD-PARTY-NPM.txt` | -| **Shipped inventory** | Same script **merges** both halves → **`THIRD-PARTY.txt`** | +| Piece | How | +|-----------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------| +| Maven / Java dependencies | [`org.codehaus.mojo:license-maven-plugin`](https://www.mojohaus.org/license-maven-plugin/) `aggregate-add-third-party` → `THIRD-PARTY-MAVEN.txt` | +| npm production dependencies | `scripts/generate-third-party-inventory.py` reads product `package-lock.json` files listed in `npm-package-locks.txt` → `THIRD-PARTY-NPM.txt` | +| **Shipped inventory** | Same script **merges** both halves → **`THIRD-PARTY.txt`** | Issue [#1689](https://github.com/intersoftdatalabs-in/percussioncms/issues/1689). -| Artifact | Location | -|----------|----------| -| Merged inventory (authoritative) | `${repo-root}/target/generated-sources/license/THIRD-PARTY.txt` | -| Maven intermediate | `…/THIRD-PARTY-MAVEN.txt` | -| npm intermediate | `…/THIRD-PARTY-NPM.txt` | -| Shipped with installer | `THIRD-PARTY.txt` at the root of the `perc-distribution-tree` assembly | -| Product notice (stable prose) | root `NOTICE.txt` — product copyright + pointer only | -| Startup / About blurb | `system` resource key `thirdPartyCopyright` — same pointer, **no version pins** | +| Artifact | Location | +|----------------------------------|---------------------------------------------------------------------------------| +| Merged inventory (authoritative) | `${repo-root}/target/generated-sources/license/THIRD-PARTY.txt` | +| Maven intermediate | `…/THIRD-PARTY-MAVEN.txt` | +| npm intermediate | `…/THIRD-PARTY-NPM.txt` | +| Shipped with installer | `THIRD-PARTY.txt` at the root of the `perc-distribution-tree` assembly | +| Product notice (stable prose) | root `NOTICE.txt` — product copyright + pointer only | +| Startup / About blurb | `system` resource key `thirdPartyCopyright` — same pointer, **no version pins** | Do **not** reintroduce hand-curated component lists or dependency version pins into `NOTICE.txt` or `thirdPartyCopyright`. That was the drift failure mode this automation From 8f50ccabd58b14918cb6245ddd60bb02aa507d6b Mon Sep 17 00:00:00 2001 From: Nate Chadwick <263952448+natechadwick-intsof@users.noreply.github.com> Date: Sun, 2 Aug 2026 10:11:27 -0400 Subject: [PATCH 4/6] feat(utilities): generic ThirdPartyLicenseInventory for Maven+npm merge MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Move the #1689 inventory merge into com.intsof.common:utilities as a product-agnostic Java API with full Javadoc and JUnit coverage (no Python on the build classpath). - ThirdPartyLicenseInventory: read package-lock production packages, merge with Maven inventory text, CLI main for exec-maven-plugin:java - Dependency-free minimal JSON parser for lockfile packages maps - Wire merge on perc-distribution-tree generate-resources (after utilities) - Root keeps license-maven-plugin → THIRD-PARTY-MAVEN.txt only - Remove scripts/generate-third-party-inventory.py path from the build > Co-Authored by Grok Build using grok-4.5 with agent main. --- modules/intsof-common-utilities/README.md | 39 + .../license/ThirdPartyLicenseInventory.java | 798 ++++++++++++++++++ .../ThirdPartyLicenseInventoryTest.java | 164 ++++ modules/perc-distribution-tree/pom.xml | 43 +- pom.xml | 38 +- scripts/README.md | 33 +- scripts/generate-third-party-inventory.bat | 9 - scripts/generate-third-party-inventory.py | 284 ------- .../test_generate_third_party_inventory.py | 117 --- src/license/README.md | 78 +- 10 files changed, 1089 insertions(+), 514 deletions(-) create mode 100644 modules/intsof-common-utilities/src/main/java/com/intsof/common/utilities/license/ThirdPartyLicenseInventory.java create mode 100644 modules/intsof-common-utilities/src/test/java/com/intsof/common/utilities/license/ThirdPartyLicenseInventoryTest.java delete mode 100644 scripts/generate-third-party-inventory.bat delete mode 100644 scripts/generate-third-party-inventory.py delete mode 100644 scripts/test_generate_third_party_inventory.py diff --git a/modules/intsof-common-utilities/README.md b/modules/intsof-common-utilities/README.md index 1d33014dd5..6529c85ee9 100644 --- a/modules/intsof-common-utilities/README.md +++ b/modules/intsof-common-utilities/README.md @@ -11,6 +11,45 @@ Product-agnostic Java utilities for Intersoft Data Labs projects | License | Apache License 2.0 | | Copyright | Intersoft Data Labs | +## Third-party license inventory (`license.ThirdPartyLicenseInventory`) + +Product-agnostic merge of a Maven-oriented inventory text file with **production** +npm packages from `package-lock.json` (lockfileVersion 2/3 `packages` map). No +Jackson or other runtime dependencies — includes a small JSON subset parser. + +```java +import com.intsof.common.utilities.license.ThirdPartyLicenseInventory; +import java.nio.file.Path; + +// Library API +var npm = + ThirdPartyLicenseInventory.readProductionPackagesFromLockFile( + Path.of("frontend/package-lock.json"), Path.of(".")); +String section = ThirdPartyLicenseInventory.formatNpmSection(npm); +String merged = + ThirdPartyLicenseInventory.mergeMavenAndNpm(mavenText, section, "My product inventory"); + +// Or write files (Maven half + lock list → merged THIRD-PARTY.txt) +ThirdPartyLicenseInventory.generateMergedInventory( + projectRoot, + outDir, + ThirdPartyLicenseInventory.DEFAULT_MAVEN_FILE_NAME, + ThirdPartyLicenseInventory.DEFAULT_NPM_FILE_NAME, + ThirdPartyLicenseInventory.DEFAULT_MERGED_FILE_NAME, + lockListFile, + "My product inventory", + true); +``` + +CLI (`main`) for Maven `exec-maven-plugin:java`: + +```text +java -cp utilities-0.0.1.jar com.intsof.common.utilities.license.ThirdPartyLicenseInventory \ + --root --require-maven [--title "..."] [--lock-list path] [--out-dir path] +``` + +Tests: `ThirdPartyLicenseInventoryTest`. + ## User configuration (`UserConfiguration`) Provides a portable per-user config root: diff --git a/modules/intsof-common-utilities/src/main/java/com/intsof/common/utilities/license/ThirdPartyLicenseInventory.java b/modules/intsof-common-utilities/src/main/java/com/intsof/common/utilities/license/ThirdPartyLicenseInventory.java new file mode 100644 index 0000000000..be27a03ced --- /dev/null +++ b/modules/intsof-common-utilities/src/main/java/com/intsof/common/utilities/license/ThirdPartyLicenseInventory.java @@ -0,0 +1,798 @@ +/* + * Copyright 2026 Intersoft Data Labs (https://intsof.com) + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.intsof.common.utilities.license; + +import java.io.IOException; +import java.io.PrintStream; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.ArrayList; +import java.util.Collections; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Locale; +import java.util.Map; +import java.util.Objects; +import java.util.TreeMap; + +/** + * Product-agnostic helpers for building a merged third-party license inventory + * that combines: + * + *

+ * + *

This type is intentionally free of product names and build-system coupling so any Intersoft + * (or other) multi-module project can reuse it. Callers supply paths and optional heading text. + * + *

Typical Maven layout

+ * + *
+ * target/generated-sources/license/
+ *   THIRD-PARTY-MAVEN.txt   ← produced by license-maven-plugin
+ *   THIRD-PARTY-NPM.txt     ← intermediate npm section (optional)
+ *   THIRD-PARTY.txt         ← merged inventory (ship / publish this file)
+ * 
+ * + *

CLI

+ * + *

A {@link #main(String[])} entry point supports {@code exec-maven-plugin:java} (or direct + * {@code java -cp …}). See {@link #main(String[])} for flags. + * + *

Paths use {@link java.nio.file} only (Windows / Linux / macOS). Written files use UTF-8 and LF + * line endings for stable cross-platform diffs. + * + * @since 0.0.1 + */ +public final class ThirdPartyLicenseInventory { + + /** Default Maven-only inventory file name. */ + public static final String DEFAULT_MAVEN_FILE_NAME = "THIRD-PARTY-MAVEN.txt"; + + /** Default intermediate npm-only inventory file name. */ + public static final String DEFAULT_NPM_FILE_NAME = "THIRD-PARTY-NPM.txt"; + + /** Default merged inventory file name (the file most products should ship). */ + public static final String DEFAULT_MERGED_FILE_NAME = "THIRD-PARTY.txt"; + + private ThirdPartyLicenseInventory() {} + + /** + * One production npm package taken from a package-lock {@code packages} entry. + * + * @param name package name (for example {@code react} or {@code @scope/pkg}) + * @param version resolved version string + * @param license SPDX-ish license expression or {@code "Unknown license"} + * @param sourceLabel human-readable origin (typically a repo-relative lockfile path) + */ + public record NpmPackage(String name, String version, String license, String sourceLabel) + implements Comparable { + + /** + * Creates a validated package coordinate. + * + * @throws NullPointerException if any argument is null + * @throws IllegalArgumentException if {@code name} or {@code version} is blank + */ + public NpmPackage { + Objects.requireNonNull(name, "name"); + Objects.requireNonNull(version, "version"); + Objects.requireNonNull(license, "license"); + Objects.requireNonNull(sourceLabel, "sourceLabel"); + if (name.isBlank()) { + throw new IllegalArgumentException("name must not be blank"); + } + if (version.isBlank()) { + throw new IllegalArgumentException("version must not be blank"); + } + if (license.isBlank()) { + license = "Unknown license"; + } + } + + /** + * Formats a single inventory line in a style compatible with common Maven THIRD-PARTY listings. + * + * @return one inventory line (no trailing newline) + */ + public String toInventoryLine() { + return " (" + + license + + ") " + + name + + " (npm:" + + name + + ":" + + version + + " - " + + sourceLabel + + ")"; + } + + @Override + public int compareTo(NpmPackage other) { + int byName = name.compareToIgnoreCase(other.name); + if (byName != 0) { + return byName; + } + return version.compareTo(other.version); + } + } + + /** + * Reads production (non-dev) packages from an npm {@code package-lock.json} file. + * + *

Only lockfileVersion 2/3 style documents with a top-level {@code packages} object are + * supported. Entries marked {@code "dev": true} or {@code "devOptional": true} are skipped. + * Nested installs under {@code node_modules/…/node_modules/…} are included under the nested + * package name (last {@code node_modules/} segment). + * + * @param packageLockJson path to {@code package-lock.json} + * @param sourceRoot optional project root used to label {@link NpmPackage#sourceLabel()} as a + * relative path; may be {@code null} to use the absolute lock path + * @return sorted, de-duplicated production packages (name+version) + * @throws IOException if the file cannot be read + * @throws IllegalArgumentException if the document is not a supported package-lock + * @throws NullPointerException if {@code packageLockJson} is null + */ + public static List readProductionPackagesFromLockFile( + Path packageLockJson, Path sourceRoot) throws IOException { + Objects.requireNonNull(packageLockJson, "packageLockJson"); + if (!Files.isRegularFile(packageLockJson)) { + throw new IllegalArgumentException("package-lock.json not found: " + packageLockJson); + } + String json = Files.readString(packageLockJson, StandardCharsets.UTF_8); + Object root = MinimalJson.parse(json); + if (!(root instanceof Map rootMap)) { + throw new IllegalArgumentException( + "package-lock root must be a JSON object: " + packageLockJson); + } + Object packagesNode = rootMap.get("packages"); + if (!(packagesNode instanceof Map packages)) { + throw new IllegalArgumentException( + "Unsupported package-lock (missing packages map): " + packageLockJson); + } + + String sourceLabel = labelFor(packageLockJson, sourceRoot); + Map byKey = new TreeMap<>(); + for (Map.Entry entry : packages.entrySet()) { + String key = String.valueOf(entry.getKey()); + if (!(entry.getValue() instanceof Map meta)) { + continue; + } + if (isTruthy(meta.get("dev")) || isTruthy(meta.get("devOptional"))) { + continue; + } + String name = packageNameFromLockKey(key); + if (name == null) { + continue; + } + String version = stringOrEmpty(meta.get("version")); + if (version.isBlank()) { + continue; + } + String license = licenseFromMeta(meta.get("license")); + NpmPackage pkg = new NpmPackage(name, version, license, sourceLabel); + byKey.put(name.toLowerCase(Locale.ROOT) + "@" + version, pkg); + } + List list = new ArrayList<>(byKey.values()); + Collections.sort(list); + return List.copyOf(list); + } + + /** + * Reads every lockfile listed in {@code lockListFile} and unions production packages. + * + *

List file format (UTF-8): one path per line, relative to {@code projectRoot}. Blank lines + * and lines whose first non-whitespace character is {@code #} are ignored. Paths may use {@code + * /} or {@code \} separators; they are resolved with {@link Path}. + * + * @param projectRoot root directory for resolving relative lock paths and source labels + * @param lockListFile list of package-lock.json paths + * @return sorted union of production packages + * @throws IOException if the list or a listed lockfile cannot be read + */ + public static List readProductionPackagesFromLockList( + Path projectRoot, Path lockListFile) throws IOException { + Objects.requireNonNull(projectRoot, "projectRoot"); + Objects.requireNonNull(lockListFile, "lockListFile"); + List locks = readLockList(projectRoot, lockListFile); + Map byKey = new TreeMap<>(); + for (Path lock : locks) { + if (!Files.isRegularFile(lock)) { + continue; + } + for (NpmPackage pkg : readProductionPackagesFromLockFile(lock, projectRoot)) { + byKey.put(pkg.name().toLowerCase(Locale.ROOT) + "@" + pkg.version(), pkg); + } + } + List list = new ArrayList<>(byKey.values()); + Collections.sort(list); + return List.copyOf(list); + } + + /** + * Parses a lock-list file into absolute lockfile paths. + * + * @param projectRoot root for relative entries + * @param lockListFile list file + * @return absolute paths (missing files are still returned so callers can warn) + * @throws IOException if the list file cannot be read + */ + public static List readLockList(Path projectRoot, Path lockListFile) throws IOException { + Objects.requireNonNull(projectRoot, "projectRoot"); + Objects.requireNonNull(lockListFile, "lockListFile"); + if (!Files.isRegularFile(lockListFile)) { + return List.of(); + } + List out = new ArrayList<>(); + for (String raw : Files.readAllLines(lockListFile, StandardCharsets.UTF_8)) { + String line = raw.strip(); + if (line.isEmpty() || line.startsWith("#")) { + continue; + } + String normalized = line.replace('\\', '/'); + Path rel = Path.of(""); + for (String segment : normalized.split("/")) { + if (!segment.isEmpty()) { + rel = rel.resolve(segment); + } + } + out.add(projectRoot.resolve(rel).normalize().toAbsolutePath()); + } + return List.copyOf(out); + } + + /** + * Formats the npm section body (title line + package lines). + * + * @param packages production packages + * @return section text ending with a trailing newline + */ + public static String formatNpmSection(List packages) { + Objects.requireNonNull(packages, "packages"); + StringBuilder sb = new StringBuilder(); + sb.append("Lists of ") + .append(packages.size()) + .append(" third-party npm dependencies (production).\n\n"); + for (NpmPackage pkg : packages) { + sb.append(pkg.toInventoryLine()).append('\n'); + } + return sb.toString(); + } + + /** + * Merges Maven inventory text and npm section text into a single document with clear section + * headers. + * + * @param mavenInventoryText contents of the Maven inventory (may be blank) + * @param npmInventoryText contents of the npm section (may be blank) + * @param documentTitle first heading line (for example product name + “third-party dependency + * license inventory”); if null or blank a generic title is used + * @return merged UTF-8 text ending with a trailing newline + */ + public static String mergeMavenAndNpm( + String mavenInventoryText, String npmInventoryText, String documentTitle) { + String title = + (documentTitle == null || documentTitle.isBlank()) + ? "Third-party dependency license inventory" + : documentTitle.strip(); + String maven = + mavenInventoryText == null || mavenInventoryText.isBlank() + ? "(no Maven inventory provided)" + : mavenInventoryText.strip(); + String npm = + npmInventoryText == null || npmInventoryText.isBlank() + ? "(no npm production dependencies found)" + : npmInventoryText.strip(); + + StringBuilder sb = new StringBuilder(); + sb.append(title).append('\n'); + sb.append("Generated at build time — do not hand-edit.\n\n"); + sb.append("================================================================================\n"); + sb.append("Maven third-party dependencies\n"); + sb.append( + "================================================================================\n\n"); + sb.append(maven).append("\n\n"); + sb.append("================================================================================\n"); + sb.append("npm third-party dependencies (production)\n"); + sb.append( + "================================================================================\n\n"); + sb.append(npm).append('\n'); + return sb.toString(); + } + + /** + * Reads the Maven inventory and package-lock list, writes npm intermediate and merged outputs. + * + * @param projectRoot project / repository root + * @param outDir output directory (created if missing) + * @param mavenFileName Maven inventory file name under {@code outDir} + * @param npmFileName intermediate npm file name under {@code outDir} + * @param mergedFileName merged file name under {@code outDir} + * @param lockListFile package-lock list file (absolute or relative paths per {@link + * #readLockList(Path, Path)}) + * @param documentTitle title line for the merged document; may be null + * @param requireMaven if true, fail when the Maven inventory file is missing + * @return path to the merged inventory + * @throws IOException on I/O failure + * @throws IllegalStateException if {@code requireMaven} and the Maven file is absent + */ + public static Path generateMergedInventory( + Path projectRoot, + Path outDir, + String mavenFileName, + String npmFileName, + String mergedFileName, + Path lockListFile, + String documentTitle, + boolean requireMaven) + throws IOException { + Objects.requireNonNull(projectRoot, "projectRoot"); + Objects.requireNonNull(outDir, "outDir"); + Objects.requireNonNull(mavenFileName, "mavenFileName"); + Objects.requireNonNull(npmFileName, "npmFileName"); + Objects.requireNonNull(mergedFileName, "mergedFileName"); + Objects.requireNonNull(lockListFile, "lockListFile"); + + Files.createDirectories(outDir); + Path mavenPath = outDir.resolve(mavenFileName); + Path npmPath = outDir.resolve(npmFileName); + Path mergedPath = outDir.resolve(mergedFileName); + + String mavenText; + if (Files.isRegularFile(mavenPath)) { + mavenText = Files.readString(mavenPath, StandardCharsets.UTF_8); + } else if (requireMaven) { + throw new IllegalStateException( + "Maven inventory missing: " + + mavenPath + + ". Run the Maven license aggregate goal first."); + } else { + mavenText = ""; + } + + List npmPackages = readProductionPackagesFromLockList(projectRoot, lockListFile); + String npmText = formatNpmSection(npmPackages); + writeUtf8Lf(npmPath, npmText); + + String merged = mergeMavenAndNpm(mavenText, npmText, documentTitle); + writeUtf8Lf(mergedPath, merged); + return mergedPath; + } + + /** + * Command-line entry point for build integration ({@code exec-maven-plugin:java} or direct + * invocation). + * + *

Flags: + * + *

+ * + * @param args command-line arguments + */ + public static void main(String[] args) { + int code = runMain(args, System.out, System.err); + if (code != 0) { + System.exit(code); + } + } + + /** + * Testable {@link #main(String[])} implementation. + * + * @param args CLI args + * @param out stdout + * @param err stderr + * @return process exit code (0 success) + */ + static int runMain(String[] args, PrintStream out, PrintStream err) { + Path root = null; + Path outDir = null; + Path lockList = null; + String title = null; + boolean requireMaven = false; + String mavenName = DEFAULT_MAVEN_FILE_NAME; + String npmName = DEFAULT_NPM_FILE_NAME; + String mergedName = DEFAULT_MERGED_FILE_NAME; + + for (int i = 0; i < args.length; i++) { + String a = args[i]; + try { + switch (a) { + case "--root" -> root = Path.of(requireValue(args, ++i, a)); + case "--out-dir" -> outDir = Path.of(requireValue(args, ++i, a)); + case "--lock-list" -> lockList = Path.of(requireValue(args, ++i, a)); + case "--title" -> title = requireValue(args, ++i, a); + case "--require-maven" -> requireMaven = true; + case "--maven-name" -> mavenName = requireValue(args, ++i, a); + case "--npm-name" -> npmName = requireValue(args, ++i, a); + case "--merged-name" -> mergedName = requireValue(args, ++i, a); + case "--help", "-h" -> { + printUsage(out); + return 0; + } + default -> { + err.println("Unknown argument: " + a); + printUsage(err); + return 2; + } + } + } catch (IllegalArgumentException ex) { + err.println(ex.getMessage()); + printUsage(err); + return 2; + } + } + + if (root == null) { + err.println("--root is required"); + printUsage(err); + return 2; + } + + root = root.toAbsolutePath().normalize(); + if (outDir == null) { + outDir = root.resolve("target").resolve("generated-sources").resolve("license"); + } else { + outDir = outDir.toAbsolutePath().normalize(); + } + if (lockList == null) { + lockList = root.resolve("src").resolve("license").resolve("npm-package-locks.txt"); + } else { + lockList = lockList.toAbsolutePath().normalize(); + } + + try { + Path merged = + generateMergedInventory( + root, outDir, mavenName, npmName, mergedName, lockList, title, requireMaven); + long npmCount = + Files.isRegularFile(outDir.resolve(npmName)) + ? readProductionPackagesFromLockList(root, lockList).size() + : 0L; + out.println( + "Wrote " + + merged + + " (Maven present=" + + Files.isRegularFile(outDir.resolve(mavenName)) + + ", npm packages=" + + npmCount + + ")"); + return 0; + } catch (IllegalStateException | IllegalArgumentException ex) { + err.println("ERROR: " + ex.getMessage()); + return 1; + } catch (IOException ex) { + err.println("ERROR: " + ex.getMessage()); + return 1; + } + } + + private static void printUsage(PrintStream out) { + out.println( + "Usage: ThirdPartyLicenseInventory --root [--out-dir ] [--lock-list ]"); + out.println(" [--title ] [--require-maven] [--maven-name name] [--npm-name name]"); + out.println(" [--merged-name name]"); + } + + private static String requireValue(String[] args, int index, String flag) { + if (index >= args.length) { + throw new IllegalArgumentException("Missing value for " + flag); + } + return args[index]; + } + + private static void writeUtf8Lf(Path path, String content) throws IOException { + String normalized = content.replace("\r\n", "\n").replace('\r', '\n'); + if (!normalized.endsWith("\n")) { + normalized = normalized + "\n"; + } + Files.writeString(path, normalized, StandardCharsets.UTF_8); + } + + private static String labelFor(Path packageLockJson, Path sourceRoot) { + Path abs = packageLockJson.toAbsolutePath().normalize(); + if (sourceRoot != null) { + try { + return sourceRoot + .toAbsolutePath() + .normalize() + .relativize(abs) + .toString() + .replace('\\', '/'); + } catch (IllegalArgumentException ignored) { + // different roots + } + } + return abs.toString().replace('\\', '/'); + } + + /** + * Maps a package-lock {@code packages} key to an npm package name, or {@code null} if the key is + * not a {@code node_modules} install path. + */ + static String packageNameFromLockKey(String key) { + if (key == null || key.isBlank() || ".".equals(key)) { + return null; + } + String norm = key.replace('\\', '/'); + String marker = "node_modules/"; + int idx = norm.lastIndexOf(marker); + if (idx < 0) { + return null; + } + String name = norm.substring(idx + marker.length()); + return name.isBlank() ? null : name; + } + + private static boolean isTruthy(Object value) { + if (value instanceof Boolean b) { + return b; + } + if (value instanceof String s) { + return "true".equalsIgnoreCase(s); + } + return false; + } + + private static String stringOrEmpty(Object value) { + return value == null ? "" : String.valueOf(value).strip(); + } + + private static String licenseFromMeta(Object licenseNode) { + if (licenseNode == null) { + return "Unknown license"; + } + if (licenseNode instanceof String s) { + return s.isBlank() ? "Unknown license" : s; + } + if (licenseNode instanceof Map map) { + Object type = map.get("type"); + if (type == null) { + type = map.get("name"); + } + String s = stringOrEmpty(type); + return s.isBlank() ? "Unknown license" : s; + } + if (licenseNode instanceof List list) { + StringBuilder sb = new StringBuilder(); + for (Object o : list) { + if (sb.length() > 0) { + sb.append(" OR "); + } + sb.append(o); + } + return sb.length() == 0 ? "Unknown license" : sb.toString(); + } + return String.valueOf(licenseNode); + } + + /** + * Minimal JSON parser sufficient for npm package-lock documents. Not a general-purpose JSON + * library; kept dependency-free for this module. + */ + static final class MinimalJson { + private MinimalJson() {} + + static Object parse(String json) { + return new Parser(json).parseValue(); + } + + private static final class Parser { + private final String s; + private int i; + + Parser(String s) { + this.s = s; + } + + Object parseValue() { + skipWs(); + if (i >= s.length()) { + throw new IllegalArgumentException("Unexpected end of JSON"); + } + char c = s.charAt(i); + if (c == '{') { + return parseObject(); + } + if (c == '[') { + return parseArray(); + } + if (c == '"') { + return parseString(); + } + if (c == 't' || c == 'f') { + return parseBoolean(); + } + if (c == 'n') { + return parseNull(); + } + if (c == '-' || (c >= '0' && c <= '9')) { + return parseNumber(); + } + throw new IllegalArgumentException("Unexpected character at " + i + ": " + c); + } + + private Map parseObject() { + expect('{'); + skipWs(); + Map map = new LinkedHashMap<>(); + if (peek('}')) { + i++; + return map; + } + while (true) { + skipWs(); + String key = parseString(); + skipWs(); + expect(':'); + Object value = parseValue(); + map.put(key, value); + skipWs(); + if (peek('}')) { + i++; + return map; + } + expect(','); + } + } + + private List parseArray() { + expect('['); + skipWs(); + List list = new ArrayList<>(); + if (peek(']')) { + i++; + return list; + } + while (true) { + list.add(parseValue()); + skipWs(); + if (peek(']')) { + i++; + return list; + } + expect(','); + } + } + + private String parseString() { + expect('"'); + StringBuilder sb = new StringBuilder(); + while (i < s.length()) { + char c = s.charAt(i++); + if (c == '"') { + return sb.toString(); + } + if (c == '\\') { + if (i >= s.length()) { + throw new IllegalArgumentException("Unterminated escape"); + } + char e = s.charAt(i++); + sb.append( + switch (e) { + case '"', '\\', '/' -> e; + case 'b' -> '\b'; + case 'f' -> '\f'; + case 'n' -> '\n'; + case 'r' -> '\r'; + case 't' -> '\t'; + case 'u' -> { + if (i + 4 > s.length()) { + throw new IllegalArgumentException("Bad unicode escape"); + } + int code = Integer.parseInt(s.substring(i, i + 4), 16); + i += 4; + yield (char) code; + } + default -> throw new IllegalArgumentException("Bad escape: \\" + e); + }); + } else { + sb.append(c); + } + } + throw new IllegalArgumentException("Unterminated string"); + } + + private Boolean parseBoolean() { + if (s.startsWith("true", i)) { + i += 4; + return Boolean.TRUE; + } + if (s.startsWith("false", i)) { + i += 5; + return Boolean.FALSE; + } + throw new IllegalArgumentException("Invalid boolean at " + i); + } + + private Object parseNull() { + if (s.startsWith("null", i)) { + i += 4; + return null; + } + throw new IllegalArgumentException("Invalid null at " + i); + } + + private Number parseNumber() { + int start = i; + if (peek('-')) { + i++; + } + while (i < s.length() && Character.isDigit(s.charAt(i))) { + i++; + } + if (peek('.')) { + i++; + while (i < s.length() && Character.isDigit(s.charAt(i))) { + i++; + } + } + if (i < s.length() && (s.charAt(i) == 'e' || s.charAt(i) == 'E')) { + i++; + if (peek('+') || peek('-')) { + i++; + } + while (i < s.length() && Character.isDigit(s.charAt(i))) { + i++; + } + } + String num = s.substring(start, i); + if (num.contains(".") || num.contains("e") || num.contains("E")) { + return Double.valueOf(num); + } + try { + return Long.valueOf(num); + } catch (NumberFormatException ex) { + return Double.valueOf(num); + } + } + + private void skipWs() { + while (i < s.length() && Character.isWhitespace(s.charAt(i))) { + i++; + } + } + + private boolean peek(char c) { + return i < s.length() && s.charAt(i) == c; + } + + private void expect(char c) { + skipWs(); + if (i >= s.length() || s.charAt(i) != c) { + throw new IllegalArgumentException("Expected '" + c + "' at " + i); + } + i++; + } + } + } +} diff --git a/modules/intsof-common-utilities/src/test/java/com/intsof/common/utilities/license/ThirdPartyLicenseInventoryTest.java b/modules/intsof-common-utilities/src/test/java/com/intsof/common/utilities/license/ThirdPartyLicenseInventoryTest.java new file mode 100644 index 0000000000..2682057d3e --- /dev/null +++ b/modules/intsof-common-utilities/src/test/java/com/intsof/common/utilities/license/ThirdPartyLicenseInventoryTest.java @@ -0,0 +1,164 @@ +/* + * Copyright 2026 Intersoft Data Labs (https://intsof.com) + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.intsof.common.utilities.license; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertThrows; +import static org.junit.jupiter.api.Assertions.assertTrue; + +import com.intsof.common.utilities.license.ThirdPartyLicenseInventory.NpmPackage; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.List; +import java.util.Set; +import java.util.stream.Collectors; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.io.TempDir; + +class ThirdPartyLicenseInventoryTest { + + @TempDir Path tempDir; + + @Test + void packageNameFromLockKeyHandlesScopedAndNested() { + assertEquals("react", ThirdPartyLicenseInventory.packageNameFromLockKey("node_modules/react")); + assertEquals( + "@scope/pkg", ThirdPartyLicenseInventory.packageNameFromLockKey("node_modules/@scope/pkg")); + assertEquals( + "bar", + ThirdPartyLicenseInventory.packageNameFromLockKey("node_modules/foo/node_modules/bar")); + assertEquals(null, ThirdPartyLicenseInventory.packageNameFromLockKey("")); + assertEquals(null, ThirdPartyLicenseInventory.packageNameFromLockKey("../../../vendor/x")); + } + + @Test + void readProductionPackagesSkipsDevAndRequiresVersion() throws Exception { + Path lock = tempDir.resolve("package-lock.json"); + Files.writeString( + lock, + """ + { + "lockfileVersion": 3, + "packages": { + "": { "name": "app", "version": "1.0.0" }, + "node_modules/react": { "version": "19.2.8", "license": "MIT" }, + "node_modules/vitest": { "version": "4.1.0", "license": "MIT", "dev": true }, + "node_modules/@scope/pkg": { "version": "1.2.3", "license": "Apache-2.0" }, + "node_modules/noversion": { "license": "MIT" } + } + } + """, + StandardCharsets.UTF_8); + + List pkgs = + ThirdPartyLicenseInventory.readProductionPackagesFromLockFile(lock, tempDir); + Set names = pkgs.stream().map(NpmPackage::name).collect(Collectors.toSet()); + assertTrue(names.contains("react")); + assertTrue(names.contains("@scope/pkg")); + assertFalse(names.contains("vitest")); + assertFalse(names.contains("noversion")); + + NpmPackage react = + pkgs.stream().filter(p -> p.name().equals("react")).findFirst().orElseThrow(); + assertEquals("19.2.8", react.version()); + assertEquals("MIT", react.license()); + assertTrue(react.toInventoryLine().contains("npm:react:19.2.8")); + } + + @Test + void mergeContainsBothSections() { + String merged = + ThirdPartyLicenseInventory.mergeMavenAndNpm( + "Lists of 1 third-party dependencies.\n (MIT) foo", + "Lists of 1 third-party npm dependencies (production).\n (MIT) react", + "Demo inventory"); + assertTrue(merged.startsWith("Demo inventory")); + assertTrue(merged.contains("Maven third-party dependencies")); + assertTrue(merged.contains("npm third-party dependencies (production)")); + assertTrue(merged.contains("foo")); + assertTrue(merged.contains("react")); + } + + @Test + void generateMergedInventoryWritesFiles() throws Exception { + Path root = tempDir; + Path out = root.resolve("out"); + Files.createDirectories(out); + Files.writeString( + out.resolve(ThirdPartyLicenseInventory.DEFAULT_MAVEN_FILE_NAME), + "Lists of 1 third-party dependencies.\n (Apache License, Version 2.0) guava\n", + StandardCharsets.UTF_8); + + Path ui = root.resolve("ui"); + Files.createDirectories(ui); + Files.writeString( + ui.resolve("package-lock.json"), + """ + { + "lockfileVersion": 3, + "packages": { + "": {}, + "node_modules/jquery": { "version": "3.7.1", "license": "MIT" } + } + } + """, + StandardCharsets.UTF_8); + Path list = root.resolve("locks.txt"); + Files.writeString(list, "ui/package-lock.json\n", StandardCharsets.UTF_8); + + Path merged = + ThirdPartyLicenseInventory.generateMergedInventory( + root, + out, + ThirdPartyLicenseInventory.DEFAULT_MAVEN_FILE_NAME, + ThirdPartyLicenseInventory.DEFAULT_NPM_FILE_NAME, + ThirdPartyLicenseInventory.DEFAULT_MERGED_FILE_NAME, + list, + "Test product inventory", + true); + + String text = Files.readString(merged, StandardCharsets.UTF_8); + assertTrue(text.contains("guava")); + assertTrue(text.contains("jquery")); + assertTrue(text.contains("npm:jquery:3.7.1")); + assertTrue(Files.isRegularFile(out.resolve(ThirdPartyLicenseInventory.DEFAULT_NPM_FILE_NAME))); + } + + @Test + void requireMavenFailsWhenMissing() { + Path out = tempDir.resolve("out"); + Path list = tempDir.resolve("locks.txt"); + assertThrows( + IllegalStateException.class, + () -> + ThirdPartyLicenseInventory.generateMergedInventory( + tempDir, + out, + ThirdPartyLicenseInventory.DEFAULT_MAVEN_FILE_NAME, + ThirdPartyLicenseInventory.DEFAULT_NPM_FILE_NAME, + ThirdPartyLicenseInventory.DEFAULT_MERGED_FILE_NAME, + list, + null, + true)); + } + + @Test + void npmPackageRejectsBlankName() { + assertThrows(IllegalArgumentException.class, () -> new NpmPackage(" ", "1.0.0", "MIT", "src")); + } +} diff --git a/modules/perc-distribution-tree/pom.xml b/modules/perc-distribution-tree/pom.xml index ac986ac261..d0f30bf596 100644 --- a/modules/perc-distribution-tree/pom.xml +++ b/modules/perc-distribution-tree/pom.xml @@ -396,14 +396,46 @@ + + org.codehaus.mojo + exec-maven-plugin + + + merge-third-party-inventory + + java + + generate-resources + + com.intsof.common.utilities.license.ThirdPartyLicenseInventory + runtime + + --root + ${maven.multiModuleProjectDirectory} + --out-dir + ${maven.multiModuleProjectDirectory}/target/generated-sources/license + --lock-list + ${maven.multiModuleProjectDirectory}/src/license/npm-package-locks.txt + --title + Percussion CMS third-party dependency license inventory + --require-maven + + + + + org.apache.maven.plugins maven-resources-plugin @@ -413,6 +445,7 @@ generate-resources (not process-resources): antrun assembly is also wired early and can fail the process-resources phase (e.g. missing WebUI/target in standalone builds) before later process-resources executions run. Copy licenses first. + Declared after merge-third-party-inventory so THIRD-PARTY.txt exists. --> copy-license-inventory diff --git a/pom.xml b/pom.xml index b013f52d7d..f9f80eaee0 100644 --- a/pom.xml +++ b/pom.xml @@ -243,8 +243,6 @@ 42.7.12 UTF-8 UTF-8 - - python 2.5.2 2.1.0 6.5.3 @@ -3050,10 +3048,11 @@ org.codehaus.mojo @@ -3062,7 +3061,7 @@ false ${maven.multiModuleProjectDirectory}/target/generated-sources/license - + THIRD-PARTY-MAVEN.txt ${project.build.sourceEncoding} true @@ -3094,33 +3093,6 @@ - - - org.codehaus.mojo - exec-maven-plugin - ${maven.exec.plugin.version} - false - - - merge-third-party-inventory - - exec - - generate-resources - - - ${python.executable} - ${maven.multiModuleProjectDirectory} - - ${maven.multiModuleProjectDirectory}/scripts/generate-third-party-inventory.py - --root - ${maven.multiModuleProjectDirectory} - --require-maven - - - - - org.apache.maven.plugins diff --git a/scripts/README.md b/scripts/README.md index 6a2bfdecd4..24c8f64e3b 100644 --- a/scripts/README.md +++ b/scripts/README.md @@ -12,34 +12,17 @@ Out of scope for spec 994 (must NOT be touched): ## Scripts -### `generate-third-party-inventory.py` / `generate-third-party-inventory.bat` +### Third-party license inventory (Maven + npm merge) -Merge **Maven** and **npm** third-party license inventories into a single -`THIRD-PARTY.txt` (issue [#1689](https://github.com/intersoftdatalabs-in/percussioncms/issues/1689)). +**Not a Python script.** Merged inventory generation for issue #1689 lives in +`com.intsof.common:utilities` as +`com.intsof.common.utilities.license.ThirdPartyLicenseInventory` (generic Java API + +`main` for `exec-maven-plugin:java`). Product wiring: -- **Purpose**: `license-maven-plugin` only sees Maven GAVs. Product UI also ships npm - production deps from `package-lock.json`. This script writes the npm half and merges - both into one file for the installer. -- **Usage**: - - ```bash - # After Maven inventory exists: - ./mvnw license:aggregate-add-third-party - python3 scripts/generate-third-party-inventory.py --require-maven +- Root: `license-maven-plugin` → `THIRD-PARTY-MAVEN.txt` +- `perc-distribution-tree`: Java merge → `THIRD-PARTY.txt` + copy into assembly - # Windows: - mvnw.cmd license:aggregate-add-third-party - scripts\generate-third-party-inventory.bat --require-maven - ``` -- **Inputs**: - - `target/generated-sources/license/THIRD-PARTY-MAVEN.txt` (from license-maven-plugin) - - `src/license/npm-package-locks.txt` (list of product package-lock.json paths) -- **Outputs** (under `target/generated-sources/license/`): - - `THIRD-PARTY-NPM.txt` — npm production intermediate - - `THIRD-PARTY.txt` — **merged** inventory (shipped) -- **Tests**: `python3 -m pytest scripts/test_generate_third_party_inventory.py -v` - (or `python3 scripts/test_generate_third_party_inventory.py`) -- **Docs**: `src/license/README.md` +See `src/license/README.md` and `modules/intsof-common-utilities/README.md`. ### `prune-stale-worktrees.py` / `prune-stale-worktrees.bat` diff --git a/scripts/generate-third-party-inventory.bat b/scripts/generate-third-party-inventory.bat deleted file mode 100644 index cb110b9d02..0000000000 --- a/scripts/generate-third-party-inventory.bat +++ /dev/null @@ -1,9 +0,0 @@ -@echo off -REM Windows launcher for generate-third-party-inventory.py (issue #1689) -SETLOCAL -cd /d "%~dp0\.." -python scripts\generate-third-party-inventory.py %* -IF ERRORLEVEL 1 ( - python3 scripts\generate-third-party-inventory.py %* -) -ENDLOCAL diff --git a/scripts/generate-third-party-inventory.py b/scripts/generate-third-party-inventory.py deleted file mode 100644 index 18566eea3e..0000000000 --- a/scripts/generate-third-party-inventory.py +++ /dev/null @@ -1,284 +0,0 @@ -#!/usr/bin/env python3 -"""Merge Maven + npm third-party license inventories into a single THIRD-PARTY.txt. - -Issue #1689: org.codehaus.mojo:license-maven-plugin produces the Maven half; -this script reads product package-lock.json files for production npm deps and -writes one merged inventory consumed by the installer assembly. - -Cross-platform (Windows / Linux / macOS). Python 3.9+. Stdlib only. -""" - -from __future__ import annotations - -import argparse -import json -import sys -from dataclasses import dataclass -from pathlib import Path -from typing import Iterable - - -DEFAULT_MAVEN_NAME = "THIRD-PARTY-MAVEN.txt" -DEFAULT_NPM_INTERMEDIATE = "THIRD-PARTY-NPM.txt" -DEFAULT_MERGED_NAME = "THIRD-PARTY.txt" -DEFAULT_LOCK_LIST = "src/license/npm-package-locks.txt" - - -@dataclass(frozen=True, order=True) -class NpmPackage: - name: str - version: str - license: str - source: str # lockfile path (repo-relative) for provenance - - def format_line(self) -> str: - lic = self.license if self.license else "Unknown license" - ver = self.version if self.version else "unknown" - return f" ({lic}) {self.name} (npm:{self.name}:{ver} - {self.source})" - - -def repo_root_from_script() -> Path: - return Path(__file__).resolve().parent.parent - - -def read_lock_list(list_file: Path, root: Path) -> list[Path]: - if not list_file.is_file(): - return [] - locks: list[Path] = [] - for raw in list_file.read_text(encoding="utf-8").splitlines(): - line = raw.strip() - if not line or line.startswith("#"): - continue - # Portable: list entries use forward slashes; resolve via Path parts. - rel = Path(*line.replace("\\", "/").split("/")) - path = (root / rel).resolve() - locks.append(path) - return locks - - -def _package_name_from_lock_key(key: str) -> str | None: - """Map package-lock 'packages' key to npm package name. - - Keys look like: - node_modules/react - node_modules/@scope/pkg - node_modules/foo/node_modules/bar - """ - if not key or key == ".": - return None - # Use the last node_modules segment (nested installs). - marker = "node_modules/" - if marker not in key.replace("\\", "/"): - # Workspace / file: link entry (e.g. ../../../vendor/mkd-language) - # Prefer packages that are real registry installs under node_modules. - return None - norm = key.replace("\\", "/") - idx = norm.rfind(marker) - name = norm[idx + len(marker) :] - return name or None - - -def collect_npm_from_lock(lock_path: Path, root: Path) -> list[NpmPackage]: - if not lock_path.is_file(): - raise FileNotFoundError(f"package-lock.json not found: {lock_path}") - data = json.loads(lock_path.read_text(encoding="utf-8")) - packages = data.get("packages") - if not isinstance(packages, dict): - raise ValueError(f"Unsupported package-lock (missing packages map): {lock_path}") - - try: - rel_source = lock_path.resolve().relative_to(root.resolve()).as_posix() - except ValueError: - rel_source = str(lock_path) - - found: dict[tuple[str, str], NpmPackage] = {} - for key, meta in packages.items(): - if not isinstance(meta, dict): - continue - if meta.get("dev") is True or meta.get("devOptional") is True: - continue - name = _package_name_from_lock_key(str(key)) - if not name: - continue - version = str(meta.get("version") or "").strip() - if not version: - # file: linked packages sometimes omit version in the packages map - continue - license_raw = meta.get("license") - if isinstance(license_raw, dict): - # rare { "type": "MIT", "url": "..." } - license_str = str(license_raw.get("type") or license_raw.get("name") or "Unknown license") - elif isinstance(license_raw, list): - license_str = " OR ".join(str(x) for x in license_raw) - elif license_raw: - license_str = str(license_raw) - else: - # Optional fallback: node_modules package.json when present - license_str = _license_from_node_modules(lock_path.parent, name) or "Unknown license" - - pkg = NpmPackage(name=name, version=version, license=license_str, source=rel_source) - found[(name, version)] = pkg - - return sorted(found.values()) - - -def _license_from_node_modules(package_root: Path, name: str) -> str | None: - # Portable join of scoped package path segments. - parts = name.split("/") - pkg_json = package_root.joinpath("node_modules", *parts, "package.json") - if not pkg_json.is_file(): - return None - try: - meta = json.loads(pkg_json.read_text(encoding="utf-8")) - except (OSError, json.JSONDecodeError): - return None - lic = meta.get("license") - if isinstance(lic, dict): - return str(lic.get("type") or lic.get("name") or "") or None - if isinstance(lic, list): - return " OR ".join(str(x) for x in lic) or None - if lic: - return str(lic) - return None - - -def format_npm_section(packages: Iterable[NpmPackage]) -> str: - pkgs = list(packages) - lines = [ - f"Lists of {len(pkgs)} third-party npm dependencies (production).", - "", - ] - for p in pkgs: - lines.append(p.format_line()) - lines.append("") - return "\n".join(lines) - - -def merge_inventories(maven_text: str, npm_text: str) -> str: - maven_body = maven_text.strip() - npm_body = npm_text.strip() - parts = [ - "Percussion CMS third-party dependency license inventory", - "Generated at build time — do not hand-edit. See src/license/README.md (issue #1689).", - "", - "================================================================================", - "Maven third-party dependencies", - "================================================================================", - "", - maven_body if maven_body else "(no Maven inventory — run license:aggregate-add-third-party first)", - "", - "================================================================================", - "npm third-party dependencies (production)", - "================================================================================", - "", - npm_body if npm_body else "(no npm production dependencies found)", - "", - ] - return "\n".join(parts) - - -def generate( - root: Path, - out_dir: Path, - maven_name: str = DEFAULT_MAVEN_NAME, - npm_name: str = DEFAULT_NPM_INTERMEDIATE, - merged_name: str = DEFAULT_MERGED_NAME, - lock_list: Path | None = None, - require_maven: bool = False, -) -> Path: - out_dir.mkdir(parents=True, exist_ok=True) - maven_path = out_dir / maven_name - npm_path = out_dir / npm_name - merged_path = out_dir / merged_name - - if maven_path.is_file(): - maven_text = maven_path.read_text(encoding="utf-8") - else: - if require_maven: - raise FileNotFoundError( - f"Maven inventory missing: {maven_path}. " - "Run mvnw license:aggregate-add-third-party first." - ) - maven_text = "" - - list_file = lock_list if lock_list is not None else root / DEFAULT_LOCK_LIST - locks = read_lock_list(list_file, root) - npm_packages: dict[tuple[str, str], NpmPackage] = {} - missing_locks: list[Path] = [] - for lock in locks: - if not lock.is_file(): - missing_locks.append(lock) - continue - for pkg in collect_npm_from_lock(lock, root): - npm_packages[(pkg.name, pkg.version)] = pkg - - if missing_locks: - missing = ", ".join(str(p) for p in missing_locks) - print(f"WARNING: package-lock.json not found (skipped): {missing}", file=sys.stderr) - - npm_sorted = sorted(npm_packages.values()) - npm_text = format_npm_section(npm_sorted) - npm_path.write_text(npm_text, encoding="utf-8", newline="\n") - - merged = merge_inventories(maven_text, npm_text) - merged_path.write_text(merged, encoding="utf-8", newline="\n") - print( - f"Wrote {merged_path} " - f"(Maven present={maven_path.is_file()}, npm packages={len(npm_sorted)})" - ) - return merged_path - - -def main(argv: list[str] | None = None) -> int: - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument( - "--root", - type=Path, - default=None, - help="Repository root (default: parent of scripts/)", - ) - parser.add_argument( - "--out-dir", - type=Path, - default=None, - help="Output directory (default: /target/generated-sources/license)", - ) - parser.add_argument( - "--lock-list", - type=Path, - default=None, - help=f"File listing package-lock.json paths (default: /{DEFAULT_LOCK_LIST})", - ) - parser.add_argument( - "--require-maven", - action="store_true", - help="Fail if THIRD-PARTY-MAVEN.txt is missing", - ) - args = parser.parse_args(argv) - - root = (args.root or repo_root_from_script()).resolve() - out_dir = ( - args.out_dir.resolve() - if args.out_dir - else root / "target" / "generated-sources" / "license" - ) - lock_list = args.lock_list.resolve() if args.lock_list else root / DEFAULT_LOCK_LIST - - try: - generate( - root=root, - out_dir=out_dir, - lock_list=lock_list, - require_maven=args.require_maven, - ) - except FileNotFoundError as exc: - print(f"ERROR: {exc}", file=sys.stderr) - return 1 - except (OSError, ValueError, json.JSONDecodeError) as exc: - print(f"ERROR: {exc}", file=sys.stderr) - return 1 - return 0 - - -if __name__ == "__main__": - sys.exit(main()) diff --git a/scripts/test_generate_third_party_inventory.py b/scripts/test_generate_third_party_inventory.py deleted file mode 100644 index df613e2824..0000000000 --- a/scripts/test_generate_third_party_inventory.py +++ /dev/null @@ -1,117 +0,0 @@ -#!/usr/bin/env python3 -"""Unit tests for generate-third-party-inventory.py (issue #1689).""" - -from __future__ import annotations - -import importlib.util -import json -import tempfile -import unittest -from pathlib import Path - -# Load hyphenated sibling script as a module (stdlib only). -import sys - -SCRIPTS = Path(__file__).resolve().parent -_SPEC = importlib.util.spec_from_file_location( - "generate_third_party_inventory", - SCRIPTS / "generate-third-party-inventory.py", -) -assert _SPEC and _SPEC.loader -g = importlib.util.module_from_spec(_SPEC) -# dataclasses frozen/order needs the module registered before exec_module -sys.modules[_SPEC.name] = g -_SPEC.loader.exec_module(g) - - -class CollectNpmFromLockTest(unittest.TestCase): - def test_production_packages_include_react_exclude_dev(self) -> None: - with tempfile.TemporaryDirectory() as tmp: - root = Path(tmp) - lock_dir = root / "frontend" - lock_dir.mkdir() - lock = { - "lockfileVersion": 3, - "packages": { - "": {"name": "app", "version": "1.0.0"}, - "node_modules/react": { - "version": "19.2.8", - "license": "MIT", - }, - "node_modules/vitest": { - "version": "4.1.0", - "license": "MIT", - "dev": True, - }, - "node_modules/@scope/pkg": { - "version": "1.2.3", - "license": "Apache-2.0", - }, - }, - } - lock_path = lock_dir / "package-lock.json" - lock_path.write_text(json.dumps(lock), encoding="utf-8") - - pkgs = g.collect_npm_from_lock(lock_path, root) - names = {p.name for p in pkgs} - self.assertIn("react", names) - self.assertIn("@scope/pkg", names) - self.assertNotIn("vitest", names) - react = next(p for p in pkgs if p.name == "react") - self.assertEqual(react.version, "19.2.8") - self.assertEqual(react.license, "MIT") - self.assertIn("npm:react:19.2.8", react.format_line()) - - -class MergeInventoriesTest(unittest.TestCase): - def test_merge_contains_both_sections(self) -> None: - maven = "Lists of 1 third-party dependencies.\n (MIT) foo (g:a:1 - )" - npm = "Lists of 1 third-party npm dependencies (production).\n (MIT) react (npm:react:1 - x)" - merged = g.merge_inventories(maven, npm) - self.assertIn("Maven third-party dependencies", merged) - self.assertIn("npm third-party dependencies (production)", merged) - self.assertIn("foo", merged) - self.assertIn("react", merged) - self.assertIn("do not hand-edit", merged) - - -class GenerateEndToEndTest(unittest.TestCase): - def test_generate_writes_merged_file(self) -> None: - with tempfile.TemporaryDirectory() as tmp: - root = Path(tmp) - out = root / "out" - # Minimal maven half - out.mkdir() - (out / g.DEFAULT_MAVEN_NAME).write_text( - "Lists of 1 third-party dependencies.\n (Apache License, Version 2.0) guava\n", - encoding="utf-8", - ) - # Lock list + lock - lock_dir = root / "ui" - lock_dir.mkdir() - lock = { - "lockfileVersion": 3, - "packages": { - "": {}, - "node_modules/jquery": {"version": "3.7.1", "license": "MIT"}, - }, - } - (lock_dir / "package-lock.json").write_text(json.dumps(lock), encoding="utf-8") - list_file = root / "locks.txt" - list_file.write_text("ui/package-lock.json\n", encoding="utf-8") - - merged = g.generate( - root=root, - out_dir=out, - lock_list=list_file, - require_maven=True, - ) - text = merged.read_text(encoding="utf-8") - self.assertIn("guava", text) - self.assertIn("jquery", text) - self.assertIn("npm:jquery:3.7.1", text) - self.assertTrue((out / g.DEFAULT_NPM_INTERMEDIATE).is_file()) - - -if __name__ == "__main__": - unittest.main() diff --git a/src/license/README.md b/src/license/README.md index 091f9f147b..fcafaf96a5 100644 --- a/src/license/README.md +++ b/src/license/README.md @@ -6,52 +6,56 @@ The **versioned** third-party dependency / license inventory is **not** hand-edi It is produced at build time and written to a **single** merged file: -| Piece | How | -|-----------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------| -| Maven / Java dependencies | [`org.codehaus.mojo:license-maven-plugin`](https://www.mojohaus.org/license-maven-plugin/) `aggregate-add-third-party` → `THIRD-PARTY-MAVEN.txt` | -| npm production dependencies | `scripts/generate-third-party-inventory.py` reads product `package-lock.json` files listed in `npm-package-locks.txt` → `THIRD-PARTY-NPM.txt` | -| **Shipped inventory** | Same script **merges** both halves → **`THIRD-PARTY.txt`** | +| Piece | How | +|-------|-----| +| Maven / Java dependencies | [`org.codehaus.mojo:license-maven-plugin`](https://www.mojohaus.org/license-maven-plugin/) `aggregate-add-third-party` on the reactor root → `THIRD-PARTY-MAVEN.txt` | +| npm production dependencies + **merge** | [`ThirdPartyLicenseInventory`](../../modules/intsof-common-utilities/src/main/java/com/intsof/common/utilities/license/ThirdPartyLicenseInventory.java) in `com.intsof.common:utilities` (generic, product-agnostic) → `THIRD-PARTY-NPM.txt` + **`THIRD-PARTY.txt`** | +| Ship | `perc-distribution-tree` copies merged `THIRD-PARTY.txt` into the installer assembly root | Issue [#1689](https://github.com/intersoftdatalabs-in/percussioncms/issues/1689). -| Artifact | Location | -|----------------------------------|---------------------------------------------------------------------------------| -| Merged inventory (authoritative) | `${repo-root}/target/generated-sources/license/THIRD-PARTY.txt` | -| Maven intermediate | `…/THIRD-PARTY-MAVEN.txt` | -| npm intermediate | `…/THIRD-PARTY-NPM.txt` | -| Shipped with installer | `THIRD-PARTY.txt` at the root of the `perc-distribution-tree` assembly | -| Product notice (stable prose) | root `NOTICE.txt` — product copyright + pointer only | -| Startup / About blurb | `system` resource key `thirdPartyCopyright` — same pointer, **no version pins** | +| Artifact | Location | +|----------|----------| +| Merged inventory (authoritative) | `${repo-root}/target/generated-sources/license/THIRD-PARTY.txt` | +| Maven intermediate | `…/THIRD-PARTY-MAVEN.txt` | +| npm intermediate | `…/THIRD-PARTY-NPM.txt` | +| Shipped with installer | `THIRD-PARTY.txt` at the root of the `perc-distribution-tree` assembly | +| Product notice (stable prose) | root `NOTICE.txt` — product copyright + pointer only | +| Startup / About blurb | `system` resource key `thirdPartyCopyright` — same pointer, **no version pins** | Do **not** reintroduce hand-curated component lists or dependency version pins into -`NOTICE.txt` or `thirdPartyCopyright`. That was the drift failure mode this automation -replaces. +`NOTICE.txt` or `thirdPartyCopyright`. + +## Why Java (not a Python merge) + +The merge lives in **`com.intsof.common:utilities`** so the monorepo build stays on **JDK + Maven** only. The API is product-agnostic (paths and titles are caller-supplied) and covered by JUnit in that module. ## Manual generation -From the repository root (JDK 21 + Maven wrapper + Python 3.9+). +From the repository root (JDK 21 + Maven wrapper). -**Do not pass `-N` / `--non-recursive` to `license:aggregate-add-third-party`** — that only -loads the empty root POM and produces an empty Maven inventory. +**Do not pass `-N` to `license:aggregate-add-third-party`** — that only loads the empty root POM. ```bash # 1) Maven half (full reactor dependency graph) ./mvnw license:aggregate-add-third-party -# 2) npm half + merge into THIRD-PARTY.txt -python3 scripts/generate-third-party-inventory.py --require-maven -# Windows: -mvnw.cmd license:aggregate-add-third-party -scripts\generate-third-party-inventory.bat --require-maven +# 2) npm half + merge (uses the installed utilities jar) +./mvnw -pl modules/intsof-common-utilities install -DskipTests +./mvnw -pl modules/perc-distribution-tree process-resources \ + -Dmaven.antrun.skip=true -Dexec.skip=false + +# Or invoke the main class directly after utilities is installed: +java -cp modules/intsof-common-utilities/target/utilities-0.0.1.jar \ + com.intsof.common.utilities.license.ThirdPartyLicenseInventory \ + --root . --require-maven \ + --title "Percussion CMS third-party dependency license inventory" ``` -A full reactor build also runs both steps on the root `generate-resources` phase -(license plugin first, then the merge script). CLI -`license:aggregate-add-third-party` alone does **not** run the merge — always follow -with the Python script when generating by hand. +Windows: use `mvnw.cmd` and `;` / path separators as appropriate. -If `python` is not on `PATH`, set `-Dpython.executable=python3` on the Maven command -that runs the merge execution, or invoke `python3` on the script directly. +A full reactor build runs the Maven aggregate on the root `generate-resources` phase and +the Java merge on `perc-distribution-tree` `generate-resources` (after `utilities` is built). ## npm package locks @@ -59,20 +63,12 @@ Edit `npm-package-locks.txt` to add product-shipped frontend lockfiles. **Do not QA-only trees (`modules/perc-qa-automation`) or pure build tooling. The WebUI SPA is built from `WebUI/src/main/frontend` — that lockfile is listed once. -Do not also list `WebUI/package-lock.json` (duplicate production set). Production packages are those in the lockfile `packages` map that are **not** marked -`dev` / `devOptional`. Licenses come from the lockfile `license` field (with optional -`node_modules/.../package.json` fallback when present). +`dev` / `devOptional`. ## `THIRD-PARTY.properties` (this directory) -This file is **not** the inventory. It is an optional **missing-license map** for Maven -dependencies whose POM does not declare a license. Entries look like: - -```properties -groupId--artifactId--version=License Name -``` - -Only add rows for dependencies the plugin reports as missing a license. Never use this -file to re-list the full dependency set by hand. +Optional **missing-license map** for Maven dependencies whose POM does not declare a +license (format: `groupId--artifactId--version=License Name`). Not a hand-maintained +full inventory. From 3cd57f27544c8e1eb60774acf866a35dc53d8e48 Mon Sep 17 00:00:00 2001 From: Nate Chadwick <263952448+natechadwick-intsof@users.noreply.github.com> Date: Sun, 2 Aug 2026 10:19:11 -0400 Subject: [PATCH 5/6] fix(build): single exec-maven-plugin for license merge in distribution-tree Avoid duplicate org.codehaus.mojo:exec-maven-plugin declarations (Maven warning and unstable merge). Attach merge-third-party-inventory to the existing exec-maven-plugin next to setup-home / verify-jdbc-drivers. > Co-Authored by Grok Build using grok-4.5 with agent main. --- modules/perc-distribution-tree/pom.xml | 63 ++++++++++++-------------- src/license/README.md | 24 +++++----- 2 files changed, 41 insertions(+), 46 deletions(-) diff --git a/modules/perc-distribution-tree/pom.xml b/modules/perc-distribution-tree/pom.xml index d0f30bf596..52f5806b2f 100644 --- a/modules/perc-distribution-tree/pom.xml +++ b/modules/perc-distribution-tree/pom.xml @@ -399,43 +399,14 @@ - THIRD-PARTY.txt merged inventory (Maven + npm production) Maven half is produced on the reactor root (license-maven-plugin → - THIRD-PARTY-MAVEN.txt). This late module merges npm package-lock production - packages via com.intsof.common.utilities.license.ThirdPartyLicenseInventory - (utilities is already built earlier in the reactor), then copies the merged - file into the assembly. - - Standalone: from repo root first run + THIRD-PARTY-MAVEN.txt). The exec-maven-plugin execution + merge-third-party-inventory (below, with setup-home / verify-jdbc) merges + npm package-lock production packages via + com.intsof.common.utilities.license.ThirdPartyLicenseInventory, then this + copy runs. Standalone: from repo root first run mvnw.cmd license:aggregate-add-third-party (full reactor load required — do not use -N). --> - - org.codehaus.mojo - exec-maven-plugin - - - merge-third-party-inventory - - java - - generate-resources - - com.intsof.common.utilities.license.ThirdPartyLicenseInventory - runtime - - --root - ${maven.multiModuleProjectDirectory} - --out-dir - ${maven.multiModuleProjectDirectory}/target/generated-sources/license - --lock-list - ${maven.multiModuleProjectDirectory}/src/license/npm-package-locks.txt - --title - Percussion CMS third-party dependency license inventory - --require-maven - - - - - org.apache.maven.plugins maven-resources-plugin @@ -818,6 +789,30 @@ org.codehaus.mojo exec-maven-plugin + + + merge-third-party-inventory + + java + + generate-resources + + com.intsof.common.utilities.license.ThirdPartyLicenseInventory + runtime + + --root + ${maven.multiModuleProjectDirectory} + --out-dir + ${maven.multiModuleProjectDirectory}/target/generated-sources/license + --lock-list + ${maven.multiModuleProjectDirectory}/src/license/npm-package-locks.txt + --title + Percussion CMS third-party dependency license inventory + --require-maven + + + diff --git a/src/license/README.md b/src/license/README.md index fcafaf96a5..a5dd9cc9eb 100644 --- a/src/license/README.md +++ b/src/license/README.md @@ -6,22 +6,22 @@ The **versioned** third-party dependency / license inventory is **not** hand-edi It is produced at build time and written to a **single** merged file: -| Piece | How | -|-------|-----| -| Maven / Java dependencies | [`org.codehaus.mojo:license-maven-plugin`](https://www.mojohaus.org/license-maven-plugin/) `aggregate-add-third-party` on the reactor root → `THIRD-PARTY-MAVEN.txt` | +| Piece | How | +|-----------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| Maven / Java dependencies | [`org.codehaus.mojo:license-maven-plugin`](https://www.mojohaus.org/license-maven-plugin/) `aggregate-add-third-party` on the reactor root → `THIRD-PARTY-MAVEN.txt` | | npm production dependencies + **merge** | [`ThirdPartyLicenseInventory`](../../modules/intsof-common-utilities/src/main/java/com/intsof/common/utilities/license/ThirdPartyLicenseInventory.java) in `com.intsof.common:utilities` (generic, product-agnostic) → `THIRD-PARTY-NPM.txt` + **`THIRD-PARTY.txt`** | -| Ship | `perc-distribution-tree` copies merged `THIRD-PARTY.txt` into the installer assembly root | +| Ship | `perc-distribution-tree` copies merged `THIRD-PARTY.txt` into the installer assembly root | Issue [#1689](https://github.com/intersoftdatalabs-in/percussioncms/issues/1689). -| Artifact | Location | -|----------|----------| -| Merged inventory (authoritative) | `${repo-root}/target/generated-sources/license/THIRD-PARTY.txt` | -| Maven intermediate | `…/THIRD-PARTY-MAVEN.txt` | -| npm intermediate | `…/THIRD-PARTY-NPM.txt` | -| Shipped with installer | `THIRD-PARTY.txt` at the root of the `perc-distribution-tree` assembly | -| Product notice (stable prose) | root `NOTICE.txt` — product copyright + pointer only | -| Startup / About blurb | `system` resource key `thirdPartyCopyright` — same pointer, **no version pins** | +| Artifact | Location | +|----------------------------------|---------------------------------------------------------------------------------| +| Merged inventory (authoritative) | `${repo-root}/target/generated-sources/license/THIRD-PARTY.txt` | +| Maven intermediate | `…/THIRD-PARTY-MAVEN.txt` | +| npm intermediate | `…/THIRD-PARTY-NPM.txt` | +| Shipped with installer | `THIRD-PARTY.txt` at the root of the `perc-distribution-tree` assembly | +| Product notice (stable prose) | root `NOTICE.txt` — product copyright + pointer only | +| Startup / About blurb | `system` resource key `thirdPartyCopyright` — same pointer, **no version pins** | Do **not** reintroduce hand-curated component lists or dependency version pins into `NOTICE.txt` or `thirdPartyCopyright`. From 0b81eaa056baa826fe71742bd39c60cbaf1dcf07 Mon Sep 17 00:00:00 2001 From: Nate Chadwick <263952448+natechadwick-intsof@users.noreply.github.com> Date: Sun, 2 Aug 2026 11:37:11 -0400 Subject: [PATCH 6/6] fix(utilities): fail loud on missing npm lock sources for license inventory Address Kilo review on PR #1739: - collectProductionPackagesFromLockList records missing lock-list / locks - --require-maven requires complete Maven + npm sources (no silent empty npm) - GenerateResult carries npmPackageCount (no second lockfile pass in runMain) - Expand ThirdPartyLicenseInventoryTest (13 cases) - Tighten packaging test assertions (drop tautologies) > Co-Authored by Grok Build using grok-4.5 with agent main. --- .../license/ThirdPartyLicenseInventory.java | 176 +++++++++++++++--- .../ThirdPartyLicenseInventoryTest.java | 135 +++++++++++++- .../ThirdPartyInventoryPackagingTest.java | 14 +- 3 files changed, 282 insertions(+), 43 deletions(-) diff --git a/modules/intsof-common-utilities/src/main/java/com/intsof/common/utilities/license/ThirdPartyLicenseInventory.java b/modules/intsof-common-utilities/src/main/java/com/intsof/common/utilities/license/ThirdPartyLicenseInventory.java index be27a03ced..cdd545d385 100644 --- a/modules/intsof-common-utilities/src/main/java/com/intsof/common/utilities/license/ThirdPartyLicenseInventory.java +++ b/modules/intsof-common-utilities/src/main/java/com/intsof/common/utilities/license/ThirdPartyLicenseInventory.java @@ -198,6 +198,41 @@ public static List readProductionPackagesFromLockFile( return List.copyOf(list); } + /** + * Result of collecting production npm packages from a lock-list file. + * + * @param packages sorted union of production packages + * @param missingLockFiles absolute paths listed in the lock-list that are not regular files + * @param lockListFileMissing {@code true} when the lock-list path itself is not a regular file + */ + public record NpmCollectionResult( + List packages, List missingLockFiles, boolean lockListFileMissing) { + + /** Creates an immutable result. */ + public NpmCollectionResult { + packages = List.copyOf(Objects.requireNonNull(packages, "packages")); + missingLockFiles = List.copyOf(Objects.requireNonNull(missingLockFiles, "missingLockFiles")); + } + } + + /** + * Result of writing a merged inventory. + * + * @param mergedPath path to the merged {@code THIRD-PARTY.txt} (or equivalent) + * @param npmPackageCount number of production npm packages included + * @param mavenPresent whether the Maven inventory file was present + * @param missingLockFiles listed package-lock paths that were missing (empty when none) + */ + public record GenerateResult( + Path mergedPath, int npmPackageCount, boolean mavenPresent, List missingLockFiles) { + + /** Creates an immutable result. */ + public GenerateResult { + Objects.requireNonNull(mergedPath, "mergedPath"); + missingLockFiles = List.copyOf(Objects.requireNonNull(missingLockFiles, "missingLockFiles")); + } + } + /** * Reads every lockfile listed in {@code lockListFile} and unions production packages. * @@ -205,19 +240,30 @@ public static List readProductionPackagesFromLockFile( * and lines whose first non-whitespace character is {@code #} are ignored. Paths may use {@code * /} or {@code \} separators; they are resolved with {@link Path}. * + *

Missing listed lockfiles are recorded in {@link NpmCollectionResult#missingLockFiles()} — + * they are not silently ignored without a trace. Use {@link + * #requireCompleteNpmSources(NpmCollectionResult, Path)} to fail the build when sources are + * incomplete. + * * @param projectRoot root directory for resolving relative lock paths and source labels * @param lockListFile list of package-lock.json paths - * @return sorted union of production packages - * @throws IOException if the list or a listed lockfile cannot be read + * @return packages plus any missing listed lock paths + * @throws IOException if the list file exists but cannot be read, or a present lockfile cannot be + * parsed */ - public static List readProductionPackagesFromLockList( + public static NpmCollectionResult collectProductionPackagesFromLockList( Path projectRoot, Path lockListFile) throws IOException { Objects.requireNonNull(projectRoot, "projectRoot"); Objects.requireNonNull(lockListFile, "lockListFile"); + if (!Files.isRegularFile(lockListFile)) { + return new NpmCollectionResult(List.of(), List.of(), true); + } List locks = readLockList(projectRoot, lockListFile); Map byKey = new TreeMap<>(); + List missing = new ArrayList<>(); for (Path lock : locks) { if (!Files.isRegularFile(lock)) { + missing.add(lock); continue; } for (NpmPackage pkg : readProductionPackagesFromLockFile(lock, projectRoot)) { @@ -226,22 +272,66 @@ public static List readProductionPackagesFromLockList( } List list = new ArrayList<>(byKey.values()); Collections.sort(list); - return List.copyOf(list); + return new NpmCollectionResult(list, missing, false); + } + + /** + * Convenience wrapper: collects packages and fails if the lock-list file or any listed lockfile + * is missing. + * + * @param projectRoot project root + * @param lockListFile lock list + * @return sorted production packages + * @throws IOException on I/O failure + * @throws IllegalStateException if the lock list or any listed lockfile is missing + */ + public static List readProductionPackagesFromLockList( + Path projectRoot, Path lockListFile) throws IOException { + NpmCollectionResult result = collectProductionPackagesFromLockList(projectRoot, lockListFile); + requireCompleteNpmSources(result, lockListFile); + return result.packages(); + } + + /** + * Fails when npm sources are incomplete (missing lock-list file or missing listed lockfiles). + * + * @param result collection result + * @param lockListFile path used for the error message + * @throws IllegalStateException if sources are incomplete + */ + public static void requireCompleteNpmSources(NpmCollectionResult result, Path lockListFile) { + Objects.requireNonNull(result, "result"); + Objects.requireNonNull(lockListFile, "lockListFile"); + if (result.lockListFileMissing()) { + throw new IllegalStateException( + "npm package-lock list file is missing: " + + lockListFile + + ". Create the list (one package-lock.json path per line) or pass --lock-list."); + } + if (!result.missingLockFiles().isEmpty()) { + StringBuilder sb = new StringBuilder("Missing package-lock.json file(s) listed in "); + sb.append(lockListFile).append(':'); + for (Path p : result.missingLockFiles()) { + sb.append("\n - ").append(p); + } + throw new IllegalStateException(sb.toString()); + } } /** * Parses a lock-list file into absolute lockfile paths. * * @param projectRoot root for relative entries - * @param lockListFile list file - * @return absolute paths (missing files are still returned so callers can warn) + * @param lockListFile list file (must exist as a regular file) + * @return absolute paths in list order (may include paths that do not yet exist on disk) * @throws IOException if the list file cannot be read + * @throws IllegalStateException if the list file is not a regular file */ public static List readLockList(Path projectRoot, Path lockListFile) throws IOException { Objects.requireNonNull(projectRoot, "projectRoot"); Objects.requireNonNull(lockListFile, "lockListFile"); if (!Files.isRegularFile(lockListFile)) { - return List.of(); + throw new IllegalStateException("npm package-lock list file is missing: " + lockListFile); } List out = new ArrayList<>(); for (String raw : Files.readAllLines(lockListFile, StandardCharsets.UTF_8)) { @@ -323,20 +413,25 @@ public static String mergeMavenAndNpm( /** * Reads the Maven inventory and package-lock list, writes npm intermediate and merged outputs. * + *

When {@code requireCompleteSources} is {@code true} (typical for product builds), both the + * Maven inventory and the full npm lock-list (file present and every listed package-lock present) + * are required. When {@code false}, missing Maven inventory yields an empty Maven section; + * incomplete npm sources still fail only if the lock-list file is required by {@link + * #requireCompleteNpmSources} — callers should pass {@code true} for CI. + * * @param projectRoot project / repository root * @param outDir output directory (created if missing) * @param mavenFileName Maven inventory file name under {@code outDir} * @param npmFileName intermediate npm file name under {@code outDir} * @param mergedFileName merged file name under {@code outDir} - * @param lockListFile package-lock list file (absolute or relative paths per {@link - * #readLockList(Path, Path)}) + * @param lockListFile package-lock list file * @param documentTitle title line for the merged document; may be null - * @param requireMaven if true, fail when the Maven inventory file is missing - * @return path to the merged inventory + * @param requireCompleteSources if true, fail when Maven inventory or any npm source is missing + * @return generate result including package count (no second lockfile pass required) * @throws IOException on I/O failure - * @throws IllegalStateException if {@code requireMaven} and the Maven file is absent + * @throws IllegalStateException if required sources are incomplete */ - public static Path generateMergedInventory( + public static GenerateResult generateMergedInventory( Path projectRoot, Path outDir, String mavenFileName, @@ -344,7 +439,7 @@ public static Path generateMergedInventory( String mergedFileName, Path lockListFile, String documentTitle, - boolean requireMaven) + boolean requireCompleteSources) throws IOException { Objects.requireNonNull(projectRoot, "projectRoot"); Objects.requireNonNull(outDir, "outDir"); @@ -358,10 +453,11 @@ public static Path generateMergedInventory( Path npmPath = outDir.resolve(npmFileName); Path mergedPath = outDir.resolve(mergedFileName); + boolean mavenPresent = Files.isRegularFile(mavenPath); String mavenText; - if (Files.isRegularFile(mavenPath)) { + if (mavenPresent) { mavenText = Files.readString(mavenPath, StandardCharsets.UTF_8); - } else if (requireMaven) { + } else if (requireCompleteSources) { throw new IllegalStateException( "Maven inventory missing: " + mavenPath @@ -370,13 +466,19 @@ public static Path generateMergedInventory( mavenText = ""; } - List npmPackages = readProductionPackagesFromLockList(projectRoot, lockListFile); - String npmText = formatNpmSection(npmPackages); + NpmCollectionResult npmResult = + collectProductionPackagesFromLockList(projectRoot, lockListFile); + if (requireCompleteSources) { + requireCompleteNpmSources(npmResult, lockListFile); + } + + String npmText = formatNpmSection(npmResult.packages()); writeUtf8Lf(npmPath, npmText); String merged = mergeMavenAndNpm(mavenText, npmText, documentTitle); writeUtf8Lf(mergedPath, merged); - return mergedPath; + return new GenerateResult( + mergedPath, npmResult.packages().size(), mavenPresent, npmResult.missingLockFiles()); } /** @@ -392,7 +494,8 @@ public static Path generateMergedInventory( *

  • {@code --lock-list } — package-lock list (default: {@code * /src/license/npm-package-locks.txt}) *
  • {@code --title } — merged document title - *
  • {@code --require-maven} — fail if Maven inventory is missing + *
  • {@code --require-maven} — fail if Maven inventory or npm lock-list / listed + * package-lock files are missing (strict product-build mode) *
  • {@code --maven-name}, {@code --npm-name}, {@code --merged-name} — override file names * * @@ -418,7 +521,7 @@ static int runMain(String[] args, PrintStream out, PrintStream err) { Path outDir = null; Path lockList = null; String title = null; - boolean requireMaven = false; + boolean requireCompleteSources = false; String mavenName = DEFAULT_MAVEN_FILE_NAME; String npmName = DEFAULT_NPM_FILE_NAME; String mergedName = DEFAULT_MERGED_FILE_NAME; @@ -431,7 +534,7 @@ static int runMain(String[] args, PrintStream out, PrintStream err) { case "--out-dir" -> outDir = Path.of(requireValue(args, ++i, a)); case "--lock-list" -> lockList = Path.of(requireValue(args, ++i, a)); case "--title" -> title = requireValue(args, ++i, a); - case "--require-maven" -> requireMaven = true; + case "--require-maven" -> requireCompleteSources = true; case "--maven-name" -> mavenName = requireValue(args, ++i, a); case "--npm-name" -> npmName = requireValue(args, ++i, a); case "--merged-name" -> mergedName = requireValue(args, ++i, a); @@ -471,20 +574,29 @@ static int runMain(String[] args, PrintStream out, PrintStream err) { } try { - Path merged = + GenerateResult result = generateMergedInventory( - root, outDir, mavenName, npmName, mergedName, lockList, title, requireMaven); - long npmCount = - Files.isRegularFile(outDir.resolve(npmName)) - ? readProductionPackagesFromLockList(root, lockList).size() - : 0L; + root, + outDir, + mavenName, + npmName, + mergedName, + lockList, + title, + requireCompleteSources); + if (!requireCompleteSources && !result.missingLockFiles().isEmpty()) { + err.println("WARNING: missing package-lock.json file(s) (npm inventory incomplete):"); + for (Path p : result.missingLockFiles()) { + err.println(" - " + p); + } + } out.println( "Wrote " - + merged + + result.mergedPath() + " (Maven present=" - + Files.isRegularFile(outDir.resolve(mavenName)) + + result.mavenPresent() + ", npm packages=" - + npmCount + + result.npmPackageCount() + ")"); return 0; } catch (IllegalStateException | IllegalArgumentException ex) { @@ -501,6 +613,8 @@ private static void printUsage(PrintStream out) { "Usage: ThirdPartyLicenseInventory --root [--out-dir ] [--lock-list ]"); out.println(" [--title ] [--require-maven] [--maven-name name] [--npm-name name]"); out.println(" [--merged-name name]"); + out.println( + " --require-maven Fail if Maven inventory or npm lock-list / listed locks are missing"); } private static String requireValue(String[] args, int index, String flag) { diff --git a/modules/intsof-common-utilities/src/test/java/com/intsof/common/utilities/license/ThirdPartyLicenseInventoryTest.java b/modules/intsof-common-utilities/src/test/java/com/intsof/common/utilities/license/ThirdPartyLicenseInventoryTest.java index 2682057d3e..1dcfc5e125 100644 --- a/modules/intsof-common-utilities/src/test/java/com/intsof/common/utilities/license/ThirdPartyLicenseInventoryTest.java +++ b/modules/intsof-common-utilities/src/test/java/com/intsof/common/utilities/license/ThirdPartyLicenseInventoryTest.java @@ -20,7 +20,11 @@ import static org.junit.jupiter.api.Assertions.assertThrows; import static org.junit.jupiter.api.Assertions.assertTrue; +import com.intsof.common.utilities.license.ThirdPartyLicenseInventory.GenerateResult; +import com.intsof.common.utilities.license.ThirdPartyLicenseInventory.NpmCollectionResult; import com.intsof.common.utilities.license.ThirdPartyLicenseInventory.NpmPackage; +import java.io.ByteArrayOutputStream; +import java.io.PrintStream; import java.nio.charset.StandardCharsets; import java.nio.file.Files; import java.nio.file.Path; @@ -80,6 +84,84 @@ void readProductionPackagesSkipsDevAndRequiresVersion() throws Exception { assertTrue(react.toInventoryLine().contains("npm:react:19.2.8")); } + @Test + void readProductionPackagesFromLockFileThrowsWhenMissing() { + Path missing = tempDir.resolve("no-such-package-lock.json"); + assertThrows( + IllegalArgumentException.class, + () -> ThirdPartyLicenseInventory.readProductionPackagesFromLockFile(missing, tempDir)); + } + + @Test + void collectReportsMissingLockListAndMissingListedLocks() throws Exception { + Path missingList = tempDir.resolve("missing-list.txt"); + NpmCollectionResult absent = + ThirdPartyLicenseInventory.collectProductionPackagesFromLockList(tempDir, missingList); + assertTrue(absent.lockListFileMissing()); + assertTrue(absent.packages().isEmpty()); + + Path list = tempDir.resolve("locks.txt"); + Files.writeString(list, "ui/package-lock.json\n", StandardCharsets.UTF_8); + NpmCollectionResult missingLock = + ThirdPartyLicenseInventory.collectProductionPackagesFromLockList(tempDir, list); + assertFalse(missingLock.lockListFileMissing()); + assertEquals(1, missingLock.missingLockFiles().size()); + assertTrue(missingLock.packages().isEmpty()); + + assertThrows( + IllegalStateException.class, + () -> ThirdPartyLicenseInventory.requireCompleteNpmSources(missingLock, list)); + assertThrows( + IllegalStateException.class, + () -> ThirdPartyLicenseInventory.readProductionPackagesFromLockList(tempDir, list)); + } + + @Test + void readLockListThrowsWhenListFileMissing() { + assertThrows( + IllegalStateException.class, + () -> + ThirdPartyLicenseInventory.readLockList( + tempDir, tempDir.resolve("does-not-exist.txt"))); + } + + @Test + void npmPackageNormalizesBlankLicense() { + NpmPackage pkg = new NpmPackage("x", "1.0.0", " ", "src"); + assertEquals("Unknown license", pkg.license()); + } + + @Test + void licenseFromMetaViaLockSupportsMapAndListForms() throws Exception { + Path lock = tempDir.resolve("package-lock.json"); + Files.writeString( + lock, + """ + { + "lockfileVersion": 3, + "packages": { + "node_modules/a": { + "version": "1.0.0", + "license": { "type": "BSD-3-Clause" } + }, + "node_modules/b": { + "version": "2.0.0", + "license": ["MIT", "Apache-2.0"] + } + } + } + """, + StandardCharsets.UTF_8); + List pkgs = + ThirdPartyLicenseInventory.readProductionPackagesFromLockFile(lock, tempDir); + assertEquals( + "BSD-3-Clause", + pkgs.stream().filter(p -> p.name().equals("a")).findFirst().orElseThrow().license()); + assertEquals( + "MIT OR Apache-2.0", + pkgs.stream().filter(p -> p.name().equals("b")).findFirst().orElseThrow().license()); + } + @Test void mergeContainsBothSections() { String merged = @@ -95,7 +177,7 @@ void mergeContainsBothSections() { } @Test - void generateMergedInventoryWritesFiles() throws Exception { + void generateMergedInventoryWritesFilesAndReportsCount() throws Exception { Path root = tempDir; Path out = root.resolve("out"); Files.createDirectories(out); @@ -121,7 +203,7 @@ void generateMergedInventoryWritesFiles() throws Exception { Path list = root.resolve("locks.txt"); Files.writeString(list, "ui/package-lock.json\n", StandardCharsets.UTF_8); - Path merged = + GenerateResult result = ThirdPartyLicenseInventory.generateMergedInventory( root, out, @@ -132,15 +214,17 @@ void generateMergedInventoryWritesFiles() throws Exception { "Test product inventory", true); - String text = Files.readString(merged, StandardCharsets.UTF_8); + assertEquals(1, result.npmPackageCount()); + assertTrue(result.mavenPresent()); + assertTrue(result.missingLockFiles().isEmpty()); + String text = Files.readString(result.mergedPath(), StandardCharsets.UTF_8); assertTrue(text.contains("guava")); assertTrue(text.contains("jquery")); assertTrue(text.contains("npm:jquery:3.7.1")); - assertTrue(Files.isRegularFile(out.resolve(ThirdPartyLicenseInventory.DEFAULT_NPM_FILE_NAME))); } @Test - void requireMavenFailsWhenMissing() { + void generateMergedInventoryFailsWhenMavenMissingAndRequired() { Path out = tempDir.resolve("out"); Path list = tempDir.resolve("locks.txt"); assertThrows( @@ -157,6 +241,47 @@ void requireMavenFailsWhenMissing() { true)); } + @Test + void runMainStrictFailsOnMissingLockList() throws Exception { + Path out = tempDir.resolve("out"); + Files.createDirectories(out); + Files.writeString( + out.resolve(ThirdPartyLicenseInventory.DEFAULT_MAVEN_FILE_NAME), + "Lists of 0 third-party dependencies.\n", + StandardCharsets.UTF_8); + ByteArrayOutputStream errBuf = new ByteArrayOutputStream(); + ByteArrayOutputStream outBuf = new ByteArrayOutputStream(); + int code = + ThirdPartyLicenseInventory.runMain( + new String[] { + "--root", + tempDir.toString(), + "--out-dir", + out.toString(), + "--lock-list", + tempDir.resolve("no-list.txt").toString(), + "--require-maven" + }, + new PrintStream(outBuf, true, StandardCharsets.UTF_8), + new PrintStream(errBuf, true, StandardCharsets.UTF_8)); + assertEquals(1, code); + String err = errBuf.toString(StandardCharsets.UTF_8); + assertTrue(err.contains("ERROR:")); + assertTrue(err.contains("package-lock list") || err.contains("missing")); + } + + @Test + void runMainHelpExitsZero() { + ByteArrayOutputStream outBuf = new ByteArrayOutputStream(); + int code = + ThirdPartyLicenseInventory.runMain( + new String[] {"--help"}, + new PrintStream(outBuf, true, StandardCharsets.UTF_8), + new PrintStream(new ByteArrayOutputStream(), true, StandardCharsets.UTF_8)); + assertEquals(0, code); + assertTrue(outBuf.toString(StandardCharsets.UTF_8).contains("Usage:")); + } + @Test void npmPackageRejectsBlankName() { assertThrows(IllegalArgumentException.class, () -> new NpmPackage(" ", "1.0.0", "MIT", "src")); diff --git a/modules/perc-distribution-tree/src/test/java/com/percussion/distribution/install/ThirdPartyInventoryPackagingTest.java b/modules/perc-distribution-tree/src/test/java/com/percussion/distribution/install/ThirdPartyInventoryPackagingTest.java index c6cef7f2d2..90d2200323 100644 --- a/modules/perc-distribution-tree/src/test/java/com/percussion/distribution/install/ThirdPartyInventoryPackagingTest.java +++ b/modules/perc-distribution-tree/src/test/java/com/percussion/distribution/install/ThirdPartyInventoryPackagingTest.java @@ -67,16 +67,16 @@ void assemblyShipsGeneratedThirdPartyInventoryWhenAvailable() throws IOException String text = Files.readString(assemblyCopy, StandardCharsets.UTF_8); assertFalse(text.isBlank(), "Packaged THIRD-PARTY.txt must not be blank"); - assertTrue( - text.toLowerCase().contains("license") || text.contains("("), - "Packaged THIRD-PARTY.txt should look like a license inventory"); - // Merged inventory (Maven + npm) — issue #1689 + // Merged inventory shape (Maven + npm) — issue #1689 assertTrue( text.contains("Maven third-party dependencies"), - "Packaged THIRD-PARTY.txt should include the Maven inventory section"); + "Packaged THIRD-PARTY.txt should include the Maven inventory section header"); + assertTrue( + text.contains("npm third-party dependencies (production)"), + "Packaged THIRD-PARTY.txt should include the npm production section header"); assertTrue( - text.contains("npm third-party dependencies") || text.contains("npm:"), - "Packaged THIRD-PARTY.txt should include the npm production inventory"); + text.contains("npm:"), + "Packaged THIRD-PARTY.txt should list at least one npm:coordinate entry"); } @Test