Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions req/phase8-4-implementation-report.md
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,46 @@ URL-bound cache and verified all 57 URL/digest pairs plus three rebuilt JARs. Fo
evidence records this warm-cache check separately from the prior cold run. Both legal findings
remain open and `--require-clear` still exits 1.

The [license resolution investigation](phase8-4-native-license-resolution.md) follows
PR #100's merge on 2026-11-16. It retains Juffa's explicitly BSD-licensed forum samples,
records their differences from the shipped implementations, reviews the original Cephes
correspondence and prepares a provenance request. The owner subsequently submitted
[MLX issue #4902](https://github.com/ml-explore/mlx/issues/4902), which has no comments at this
check. The investigation and submission do not establish an additional
grant or close either legal finding. Validation confirms the comparison inputs and retained
file hashes and reproduces the unchanged audit result (57 URL/digest pairs, releaseReady false,
expected exit 1). The initial investigation changed only documentation and research evidence;
the 74-test
and native Gradle results above remain evidence for PR #100's implementation, not a new run.

PR #101 review corrections commit the dated `compare_literals.py` verifier and record its
digest, exact comment record shape/JSON serialization and function/literal extraction rules.
Fresh downloads reproduce the original two comment hashes and all five comparison results;
offline replay produces the identical report. The scope now explicitly excludes PyTorch-derived
CPU `erf` from the Juffa comparison. Five new regressions bring release selftests to 79, all
passing in 49.296 seconds. Native code/notices and both open legal findings are unchanged.

The recipe follow-up documents that exponent-only floating forms are excluded by the
decimal-point pattern, with no such forms in the seven recorded function inputs. Missing
research sources and unexpected evidence types now produce contextual CLI errors without
tracebacks; the live selftest dependency on the dated recipe path is explicitly documented.
All 80 release selftests passed (50.066 seconds), including six focused recipe tests.
Cached-input replay preserves the original seven records, five checks and both comment hashes.

The source-ID/HTTP follow-up validates all research source IDs before using them and catches
`http.client.HTTPException` during fetches. CLI regressions include null/non-string source
IDs and an injected `IncompleteRead` from response reading, with no partial cache/report.
The recorded latest command now exactly matches the offline invocation without
`--fetch-missing`. All 81 release selftests passed (68.461 seconds), including seven focused
recipe tests. Results and both legal findings remain unchanged; no live truncated transfer
is claimed by the fault-injection test.

The URL-type follow-up adds a narrow string check before `read_source` encodes a URL.
Actual cached-input CLI runs for both a numeric comment `url` and a numeric MLX comparison
`source` now return the prefixed validation error without a traceback or output report.
All 82 release selftests passed (50.286 seconds), including eight focused recipe tests.
Valid-input reproduction remains byte-identical; no native or legal finding changed.

## Review corrections (2026-11-15)

The minimum-host workflow now creates its Gradle home through `GITHUB_ENV` at step scope;
Expand Down
5 changes: 5 additions & 0 deletions req/phase8-4-native-license-audit.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,11 @@ libraries to the reviewed bundle.

## Open findings

The [resolution investigation](phase8-4-native-license-resolution.md) records primary-source
follow-up, comparisons with Juffa's BSD samples and the owner's submitted
[upstream provenance request #4902](https://github.com/ml-explore/mlx/issues/4902).
It supplies review material; both findings below remain open.

### NATIVE-LICENSE-1: Juffa `erf` / `erfinv` provenance

The pinned Metal `erf.h` explicitly says its error function is based on
Expand Down
191 changes: 191 additions & 0 deletions req/phase8-4-native-license-resolution.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,191 @@
# Phase 8.4 native license resolution review

Investigation date: 2026-11-16. Baseline: PR #100 merge
`ee807c99e89f06f63ef326e2bbb976b7cb247989`.

The next release step is an owner/legal decision on the two findings in the
[native audit](phase8-4-native-license-audit.md). This investigation found useful primary
sources, but did not establish an applicable additional grant for the shipped code.
Both findings and the license release gate remain open.

The owner submitted [MLX issue #4902](https://github.com/ml-explore/mlx/issues/4902) on
2026-11-16, covering both provenance questions. It is open with no comments at this check;
the next action is to review any upstream response against the exact code and license terms.
The [prepared request](plans/evidence/native-license-resolution-2026-11-16/upstream-request-draft.md)
is retained as a historical draft; the owner posted an adapted version. Native code, pins,
bindings, notices, artifact hashes and the publication flow are unchanged.

## Juffa implementations: the BSD samples differ

CPU `erf` is outside this Juffa comparison: its SIMD implementation cites PyTorch's
Abramowitz–Stegun approximation and is already covered by the packaged PyTorch notice.
The Juffa scope is Metal `erf` and Metal/CPU `erfinv`, rather than every function in `simd/math.h`.

Juffa published explicit BSD-2-Clause code in his NVIDIA forum posts for
[`erff`](https://forums.developer.nvidia.com/t/optimized-version-of-single-precision-error-function-erff/40977/1)
and [`erfinvf`](https://forums.developer.nvidia.com/t/a-very-compact-implementation-of-the-inverse-error-function-erfinvf/61961/1).
The captured code samples retain their complete copyright, conditions and disclaimer under
the [evidence directory](plans/evidence/native-license-resolution-2026-11-16/).
Only trailing whitespace is removed and a final LF is retained; the manifest records the
raw extracted-code hash separately from each normalized retained sample's hash.

| Material | Shipped MLX version | BSD forum sample | Result |
| --- | --- | --- | --- |
| Metal `erf` | Threshold `0.927734375`; first polynomial coefficient `-1.72853470e-5` | Threshold `1.0`; first coefficient `2.58907676e-5` | Different sample |
| Metal/CPU `erfinv` | Natural logarithm; threshold magnitude `6.125`; tail first coefficient `3.03697567e-10` | CUDA approximate base-two logarithm; threshold `-8.8359375`; first coefficient `1.50166858e-11` | Different sample and evaluation |

The [comparison record](plans/evidence/native-license-resolution-2026-11-16/code-comparison.json)
captures signed decimal floating literals from the named functions and identifies every input
by SHA-256. The Metal inverse function's literal sequence matches
[Stack Overflow answer 49743348](https://stackoverflow.com/a/49743348); the CPU inverse
function has the same literal multiset with its threshold placed later in the function.
The Metal error function's literals are a multiset subset of
[answer 35148199](https://stackoverflow.com/a/35148199), whose alternative exponential branch
adds another literal. These are bounded lexical observations, not proofs of equivalent
program behavior or of copyright/license scope. The existing audit's source attribution
and matching coefficients still support treating these as provenance questions.

Neither captured BSD sample has the shipped function's literal sequence. Their common author,
similar algorithms and grants allowing modifications do not themselves establish the rights
under which MLX incorporated these particular Stack Overflow-derived versions. The two specific
answer comment lists were also fetched completely; no licensing/permission keyword was found
in those comments. This does not establish the absence of a grant elsewhere.

The pinned file's [history](https://api.github.com/repos/ml-explore/mlx/commits?path=mlx/backend/metal/kernels/erf.h&per_page=100&sha=36969c094354086e15d2a327a9c50feb20b9cf11)
has four commits. The latest is the
[`expm1f` substitution](https://github.com/ml-explore/mlx/commit/634b148dd4203f80b8cc8db3e33452a91785fad8)
from [PR #3025](https://github.com/ml-explore/mlx/pull/3025). That PR describes a numerical
change and had no issue-thread comments when checked; it does not provide an additional
license grant. A public history entry named `copyright + ack` is also not a substitute
for the underlying permission record. The research manifest records this bounded history check.

## Cephes: original correspondence needs a scope decision

The original [2004 correspondence](https://lists.debian.org/debian-legal/2004/12/msg00295.html)
contains Moshier's response to a request about distributing Cephes in particular packages.
He distinguishes the terms of his own distribution from permission for another distribution
and supplies example BSD boilerplate. This is stronger context than a third-party notice
claiming BSD, but it does not explicitly identify MLX, fastermath, NEON math or a blanket
grant to every downstream recipient of the original Netlib code.

The [direct reply](https://lists.debian.org/debian-legal/2004/12/msg00297.html), by Brian
Thomas Sniffen rather than Moshier, discusses using excerpts actually distributed with that
boilerplate and porting modifications if needed. That is a contemporary interpretation,
not a fresh grant from the copyright holder. It does not establish the path by which the
Cephes-derived material reached the pinned MLX SIMD implementation.

The existing Netlib permission statement and immediate BSD/zlib notices remain packaged.
The owner/legal reviewer must decide whether those records already cover this distribution,
or obtain an applicable grant or a documented licensed provenance chain. This investigation
does not label the material prohibited, GPL-only or automatically BSD-licensed.

## Resolution choices

| Route | Concrete evidence needed before closing the finding |
| --- | --- |
| Ask upstream for an applicable grant/provenance record (recommended) | Identify exact functions/files, original copyright holder, granted terms and downstream scope; retain the response or immutable source record and package its required notices |
| Review the existing permissions | For Juffa, determine the applicable license/version, adaptation boundary and obligations for compiled MLX and Metal code; for Cephes, document why the original permission or a traced licensed distribution covers the actual derived material |
| Separately review a native implementation or source change | Establish a licensed replacement and validate numerical behavior, fixtures, both precision modes, kernel execution and the complete rebuilt artifact; repeat the audit and candidate verification |

The Juffa answers' recorded license is CC BY-SA 4.0. Its official
[legal code](https://creativecommons.org/licenses/by-sa/4.0/legalcode.en) distinguishes
attribution, modification notices, the adapter's license for adapted material and downstream
restrictions (sections 2 and 3). A link to that license alone does not establish that all
conditions for this compiled distribution have been met. The software-specific integration
and adaptation boundary require review; this document does not infer that all of jmlx becomes
CC-licensed. Creative Commons' [software FAQ](https://creativecommons.org/faq/#can-i-apply-a-creative-commons-license-to-software)
recommends software licenses for code and explains some differences from CC terms. Its
recommendation does not itself cancel the existing license or supply an alternative grant.

The first route preserves the reviewed native tuple and avoids choosing new licensing terms
or changing numerical implementations during release preparation. If the requested records
cannot be established, the owner can choose a reviewed compliance or replacement route.

## Evidence and validation

The [research manifest](plans/evidence/native-license-resolution-2026-11-16/research.json)
records source URLs, content projections and hashes. It identifies live pages as mutable;
their hashes capture what was reviewed, not an assurance of unchanged future responses.
Only the two explicitly BSD-licensed samples and derived comparison facts are retained here;
Stack Overflow answer bodies and correspondence HTML are referenced rather than recopied.
The original audit retains the Stack Overflow body hashes and licenses.

Comparison method: extract text starting at the function name (excluding the return type),
through its balanced closing brace, from these particular inputs. Strip C/C++ comments,
collect decimal floating literals, remove `f`/`F` suffixes and normalize with Python's
`decimal.Decimal`. An optional plus/minus sign is included only when directly attached to
the numeric token, as specified by `FLOAT_LITERAL_PATTERN` in the committed script.
A separated minus sign is not treated as part of the literal. The pattern requires a decimal
point, so exponent-only decimal floating forms such as `1e-5f` are also excluded; none occurs
in the seven recorded function inputs. The comparison does not parse arbitrary C++ and
excludes integer/hexadecimal literals. Code input hashes and all seven extracted function
hashes are recorded, so readers can check the exact variants being compared. The two BSD
samples are reference evidence only and are not built or added to the native artifacts.

The committed [comparison and projection script](plans/evidence/native-license-resolution-2026-11-16/compare_literals.py)
reproduces all seven function records, five checks and both comment-list hashes. From the
repository root with Python 3.11+:

```sh
python3 req/plans/evidence/native-license-resolution-2026-11-16/compare_literals.py \
--inputs build/releaseAll/license-resolution-inputs --fetch-missing \
--output build/releaseAll/license-resolution-reproduction.json
```

Release selftests import this dated recipe. Moving or archiving the evidence folder requires
retaining the recipe or updating the test's `RECIPE` path and recorded evidence/verifier paths
in the same change.

The script downloads missing inputs into a URL/digest-bound cache; omit `--fetch-missing`
to verify the cached responses offline. It checks the original audit's complete Stack Overflow
answer-body hashes before extracting their longest HTML-decoded code blocks, and validates
every function input/hash/literal record against `code-comparison.json`. The retained BSD
samples need no download. Changed projected content fails with a contextual error.
The latest offline command in `research.json` omits `--fetch-missing` and was executed with
its recorded input/output paths. `initialReproduction` retains the earlier first-download command.

For each comment response, preserve the API `items` array order. Project each item to exactly
`{"comment_id": item["comment_id"], "creation_date": item["creation_date"],
"body": item["body"], "ownerUserId": item.get("owner", {}).get("user_id")}`.
`body` remains the raw HTML string; the owner field is flattened to `ownerUserId`, and a
missing owner/user ID becomes JSON null. Serialize the complete list as
`json.dumps(records, sort_keys=True, separators=(",", ":"), ensure_ascii=True).encode("utf-8")`,
with no trailing newline. Both original hashes are preserved. The manifest records this exact
format and the script's digest. Partial comment pages are rejected rather than silently hashed.

The review rerun reproduced both original comment hashes and all five checks from fresh
downloads, then produced an identical report offline. Five focused regressions cover exact
serialization/order, incomplete responses, function/sign boundaries, HTML/body extraction and
URL-bound input integrity. All 79 release selftests passed (49.296 seconds); the research
manifest retains the verifier/test digests and reproduction result. No native runtime code,
notices, license findings or numerical implementations changed.

The limits/error-handling follow-up documents the decimal-point requirement and adds a CLI
regression for missing research source IDs and unexpected evidence value types. These failures
now report `license comparison: …` without a traceback. Cached-input replay preserves all
seven function records, five checks and both original comment hashes; the legal findings stay open.
All 80 release selftests passed (50.066 seconds), including six focused recipe tests.

The source-ID/HTTP follow-up validates every research source as an object with a string ID
before comparison or comment processing. The CLI also catches `http.client.HTTPException`,
including `IncompleteRead`. A fault-injection test raises `IncompleteRead` from response
reading and verifies the prefixed error and absence of a partial cache entry/report; it does
not claim to reproduce a live truncated transfer. The validation record lists the malformed
evidence cases actually tested rather than claiming to cover all possible evidence edits.
All 81 release selftests passed (68.461 seconds), including seven focused recipe tests.

The URL-type follow-up rejects non-string URLs in `read_source` before encoding, cache or
network access. Both a comment source's `url: 123` and a comparison record's `source: 123`
were exercised through the real CLI with the recorded cached inputs: each produces
`license comparison: source url must be a string` without a traceback or output report.
All 82 release selftests passed (50.286 seconds), including eight focused recipe tests.

Validation checks source hashes against the existing audit, the five recorded comparison
results and every retained sample/report digest. The unchanged audit command still produces
the same result as PR #100's follow-up evidence, including `releaseReady: false` and both
finding IDs; `--require-clear` exits 1. No new runtime test or legal clearance is claimed.

The owner posted the provenance request as `perNyfelt` at `2026-11-16T16:54:53Z`.
The assistant did not post an issue or comment. The request asks for provenance rather than
asserting an upstream license violation. A subsequent response must be reviewed against the
actual code and bundle before either finding is removed from the release gate.
4 changes: 4 additions & 0 deletions req/phase8-4-release-checklist.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,10 @@ BellSoft 21.0.12.1; the matching-toolchain audit reproduces every JAR exactly. T
The [2026-11-16 source audit](phase8-4-native-license-audit.md) expands the packaged
inventory to 26 entries. Two provenance findings (Juffa `erf`/`erfinv` and Cephes)
still prevent closing this gate; successful notice checks do not resolve them.
The [resolution review](phase8-4-native-license-resolution.md) compares alternative BSD
samples and original permission correspondence. The owner submitted
[MLX issue #4902](https://github.com/ml-explore/mlx/issues/4902); awaiting an upstream response
and review of any applicable grant or compliance decision.
- [ ] Verify current official Portal size limits/accounting against actual signed bundle.
- [x] Integrate released upstream plugin 2.6.0 and verify the clean historical 8.1 fixture.
[Compatibility evidence](phase8-4-plugin-2.6.0-compatibility.md) records the unpushed scratch
Expand Down
Loading
Loading