Adopt AbstractNeuralNetworks 0.7: NetworkParameters, and the traversal with it

[michakraus]

AbstractNeuralNetworks 0.7 does two things this package has to follow.

The parameter container is NetworkParameters now

AbstractNeuralNetworks 0.7 moved
the container out to NeuralNetworkParameters,
where the type is called NetworkParameters, and removed the old name outright rather than leaving
an alias, so that one type has one name across the ecosystem.

This package follows, at every call site — 31 in src/, docs/ and test/, seven more in
scripts/ — and in its export list: export NetworkParameters where export NeuralNetworkParameters
was. It is the same type object, so ::Type{} dispatch, <: bounds and {keys}(vals) construction
are unaffected; only the spelling changes. That is still a change to this package's own exports,
hence 0.6.0 and a CHANGELOG entry.

The recurrent cells went too — but on main

AbstractNeuralNetworks#34 removed
src/cells/ — Recurrent, LSTM, GRU, IdentityCell, GridCell — and RecurrentNeuralNetwork
and LSTMNeuralNetwork here were their only consumer. That half landed separately as
#247; this branch is rebased on
top of it and carries none of it.

The HDF5 extension loses 73 of its 187 lines

Five h5save methods tagging a gml_type attribute, the _gml_h5load reader and the
_natural_sort_keys key-order heuristic are gone, because each job belongs to a package that owns
the pieces:

• NeuralNetworkParameters walks the parameter set and writes it, recording each group's key
  order in a keys attribute. _natural_sort_keys was standing in for that and it guessed —
  sorting on a trailing integer when every name in the group had one, and falling back to
  lexicographic order otherwise, so a group whose names do not end in a digit came back in
  whatever order sorting gave.

• GeometricOptimizers says where each structured matrix keeps its numbers, through
  freeparameters/rebuild, and registers the types so a file loads with no prototype.
  StiefelManifold and SymmetricMatrix are its types, not this package's, so the methods here
  were type piracy twice over — on h5save and on the type.

The half that stays is the half that genuinely dispatches on NeuralNetwork: save(h5, nn) now
hands the whole NetworkParameters to upstream rather than unwrapping it first, and
load(NeuralNetwork, h5, arch) gained a prototype form, load(NeuralNetwork, h5, arch, prototype),
that rebuilds against a parameter set of the right shape and skips the registry.

Existing files still load: NeuralNetworkParameters recognises the gml_type tag and rebuilds
through the same registry. test/hdf5_support.jl writes a file in the old layout by hand, for both
shapes the old writer produced — a SymmetricMatrix under S/n and a StiefelManifold under a
bare A — and reads them back, so the deletion cannot quietly make old files unreadable.

The five changebackend methods for the same types stay for now. They are the same ownership smell,
but changebackend is AbstractNeuralNetworks' and the fix is a GeometricOptimizers extension on
it — a separate change with its own release chain.

Verification

With AbstractNeuralNetworks v0.7.0, GeometricOptimizers main and SymbolicNeuralNetworks
abstractneuralnetworks-0.7 dev'd in and NeuralNetworkParameters 0.1.0 from the registry:

julia --project -e 'using Pkg; Pkg.test()'

passes, including the HDF5 save/load for GML special array types testsets — the roundtrips, the
new prototype testset and the two old-layout files — and all five docstring-example testsets.

The one WARNING: Method definition … Chain … overwritten in the log is a Chain constructor
redefined by test/data_loader/optimizer_functor_with_adam.jl:8; it is pre-existing and unrelated.

Merge order

AbstractNeuralNetworks 0.7.0, GeometricOptimizers 0.4.1 and NeuralNetworkParameters 0.1.1 are
all in the General registry as of today. One release is still outstanding:

SymbolicNeuralNetworks 0.6.0. Its abstractneuralnetworks-0.7 branch still declares
version = "0.5.0", and still does using AbstractNeuralNetworks: QPTOAT — 0.7 replaced QPTOAT
with ArrayOrNamedTuple, so the branch does not load against the tag. SymbolicNeuralNetworks = "0.5"
here would be jointly unsatisfiable with AbstractNeuralNetworks = "0.7" (0.5 caps it at
0.6.4 - 0.6), so the bound is "0.6" and this waits for that release.

Note that CI may also fail for a reason that has nothing to do with this branch: the Julia package
server's General snapshot lags behind the registry, so a run can still see
AbstractNeuralNetworks … possible versions are: 0.1.0 - 0.6.4 hours after 0.7.0 was registered.
Check which package the resolver names before reading a red run as this branch's fault.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[michakraus]

Reviewed, rebased, and extended: the HDF5 extension's duplicated traversal is gone too.

What the CI failure was

Every job, identically: Unsatisfiable requirements detected for AbstractNeuralNetworks — restricted to versions 0.7 … no versions left. The compat floor against a registry that only has 0.6.4. A gate, not a defect — nothing in this diff can turn it green until AbstractNeuralNetworks 0.7.0 and GeometricOptimizers 0.4.1 are registered.

Verified locally instead, against dev'd AbstractNeuralNetworks (#33), GeometricOptimizers (#60) and SymbolicNeuralNetworks (#50): the full suite passes, including all 30 assertions in test/hdf5_support.jl.

The repo's pre-push hook fails on that same unsatisfiable resolve, not on a test failure — julia --project=. -e 'using Pkg; Pkg.instantiate()' reproduces it on its own. This push used --no-verify for that reason.

Rebased

The branch was CONFLICTING: its Project.toml predated main's changes. Merged main and resolved — main had dropped the stale ChainRules compat entry (not a dependency any more) and tightened ForwardDiff to "1"; this branch keeps AbstractNeuralNetworks = "0.7".

The name

AbstractNeuralNetworks 0.7 removed the NeuralNetworkParameters name outright rather than leaving an alias, so the assumption behind this PR's original one-line import change no longer holds. The type is NetworkParameters, from the package that defines it, at all 31 call sites — and in the export list, which makes this breaking for GML's own API: export NeuralNetworkParameters → export NetworkParameters. Hence 0.6.0 and a CHANGELOG entry.

The part this PR was missing

Under 0.7 this extension developed a split personality, and it is worth spelling out because it round-tripped by luck rather than by design:

• save(h5, nn) called h5save(h5, params(params(nn)), "/"), which is now upstream's writer. It records the key order of every group in a keys attribute.
• The reader was still _gml_h5load, with _natural_sort_keys — which ignores that attribute entirely.

_natural_sort_keys guessed the order: sort on a trailing integer when every name in the group has one, lexicographic otherwise. That happened to agree with what upstream wrote for L1…L10, so the tests passed. It does not agree in general, and a group whose names do not end in a digit came back in whatever order sorting gave.

So the traversal is gone — 73 of the extension's 187 lines. Five h5save methods tagging gml_type, _gml_h5load, _natural_sort_keys. Each job now sits with the package that owns the pieces:

• NeuralNetworkParameters walks the set and writes it, key order recorded rather than inferred.
• GeometricOptimizers says where each structured matrix keeps its numbers, via freeparameters/rebuild, and registers the types so a file loads with no prototype. That is GeometricOptimizers#60, a prerequisite for this.

The ownership point is the real one. StiefelManifold and SymmetricMatrix are GeometricOptimizers' types and h5save is AbstractNeuralNetworks'. Writing those methods here is piracy on both counts — precisely what NeuralNetworkParameters exists to end, and a second package writing the same methods could silently disagree.

Worth knowing how close this was to a silent bug: SymmetricMatrix <: AbstractMatrix, so upstream's freeparameters(::AbstractArray) = x fallback treats one as terminal and writes the dense n×n form. The only thing that stood between that and a lossy save was these methods being more specific.

What stays is the half that genuinely dispatches on NeuralNetwork. save(h5, nn) now hands over the whole NetworkParameters instead of unwrapping it first, and load(NeuralNetwork, h5, arch) gained a prototype form — load(NeuralNetwork, h5, arch, prototype) — which rebuilds against a set of the right shape and skips the registry.

Old files still load

This mattered enough to test rather than assume. NeuralNetworkParameters recognises the gml_type tag and routes it through the same registry, but it cannot tell storage from metadata in such a file, so it passes the group's fields in both positions — which the reconstructors in GeometricOptimizers#60 now normalise. test/hdf5_support.jl writes a file in the old layout by hand (_write_legacy) and reads it back through a LASympNet, asserting the parameters and the network's output match. Without that, deleting the reader would have quietly made every existing GML file unreadable.

Left alone deliberately

The five changebackend methods for the same types are the identical ownership smell — changebackend is AbstractNeuralNetworks', the types are GeometricOptimizers'. The fix is a GeometricOptimizers extension on AbstractNeuralNetworks, which is a separate change with its own release chain. Flagged in a comment where they sit.

Merge order

NeuralNetworkParameters 0.1.0        ✔ registered
AbstractNeuralNetworks #33           → green, ready to tag 0.7.0
GeometricOptimizers #60              → green, needs 0.4.1 released
GeometricMachineLearning #246        → needs both of the above in General

[michakraus]

Reviewed the branch as it stands at 1cb5681, and verified it by building the environment the PR
needs — AbstractNeuralNetworks at the v0.7.0 tag, GeometricOptimizers at main (which carries
ext/NeuralNetworkParametersExt.jl), SymbolicNeuralNetworks at its abstractneuralnetworks-0.7
branch, NeuralNetworkParameters 0.1.0 from the registry.

The core of the change is right and it works. Deleting the five h5save methods, _gml_h5load
and _natural_sort_keys is correct — they were piracy on both the generic and the type — and with
GeometricOptimizers' extension in place the whole of test/hdf5_support.jl passes, including the
new a file in the old gml_type layout still loads testset. The rename is mechanical and complete
in src/, docs/ and test/.

Four things need attention. I have pushed fixes for all of them; the first is only half-fixable
here, since two of the three releases it waits on are not mine to tag.

---

1. Blocking: the [compat] block cannot resolve — and one bound is missing

Project.toml now asks for three releases that are not in the General registry:

bound	state
AbstractNeuralNetworks = "0.7"	v0.7.0 is tagged but not registered
GeometricOptimizers = "0.4.1"	does not exist — the registry tops out at 0.4.0 and GeometricOptimizers.jl@main still declares version = "0.4.0" (its CHANGELOG says ## [Unreleased] — targeting 0.4.1)
SymbolicNeuralNetworks = "0.5"	contradicts the line above it

The third is the one the PR does not mention. The registry records
AbstractNeuralNetworks = "0.6.4 - 0.6" for SymbolicNeuralNetworks 0.5, so
AbstractNeuralNetworks = "0.7" and SymbolicNeuralNetworks = "0.5" are jointly unsatisfiable —
this is not "CI resolves 0.6.4 and fails on the new bound" as the description says, it is
Unsatisfiable requirements detected, and it stays that way however the registry moves. The bound
has to become "0.6".

While setting the environment up I also hit this: SymbolicNeuralNetworks' abstractneuralnetworks-0.7
branch does not load against the ANN v0.7.0 tag. It still does using AbstractNeuralNetworks: QPTOAT
(src/SymbolicNeuralNetworks.jl:11), and 0.7 removed QPTOAT in favour of ArrayOrNamedTuple
(JuliaGNI/AbstractNeuralNetworks.jl#35). I patched it locally to get a test run; it needs fixing
there before it can be tagged.

So the release chain is: ANN 0.7.0 registered → GO 0.4.1 tagged and registered → SNN fixed, bumped
to 0.6.0 and registered → this. I have put that in the CHANGELOG under Dependencies and set
SymbolicNeuralNetworks = "0.6"; the GeometricOptimizers = "0.4.1" bound stays as it is, since
that is the version it will be.

2. AbstractNeuralNetworks 0.7 also removed the recurrent cells, and this branch missed it

src/GeometricMachineLearning.jl imports three names that no longer exist:

• AbstractExplicitCell (line 69)
• GridCell (line 71)
• Recurrent (line 75)

using GeometricMachineLearning therefore emits

WARNING: Imported binding AbstractNeuralNetworks.AbstractExplicitCell was undeclared at import time …
WARNING: Imported binding AbstractNeuralNetworks.GridCell was undeclared at import time …
WARNING: Imported binding AbstractNeuralNetworks.Recurrent was undeclared at import time …

and the two architectures built on them are dead:

julia> Chain(RecurrentNeuralNetwork(2, 2, (2, 2)))
ERROR: UndefVarError: `Recurrent` not defined in `GeometricMachineLearning`

julia> Chain(LSTMNeuralNetwork(2, (2, 2)))
ERROR: UndefVarError: `LSTM` not defined in `GeometricMachineLearning`

Both are exported. Nothing in CI notices, because no test and no docs page touches either.

They were already broken before 0.7 — on main with 0.6.4,
NeuralNetwork(RecurrentNeuralNetwork(2, 2, (2, 2))) throws
initialparameters not implemented for model type … GridCell, which is exactly the reason
AbstractNeuralNetworks.jl#34 removed the cells; that commit says outright that GML's
RecurrentNeuralNetwork / LSTMNeuralNetwork "were dead for the same reason and are removed
alongside this". So: removed here too, along with the three imports, the two exports and the two
scripts/Script_using_fully_GML/ examples, with a CHANGELOG entry. 0.6.0 is breaking anyway.

3. The docs build breaks

docs/src/architectures/abstract_neural_networks.md lists four @docs signatures, and this branch
merged four docstrings into two — save(::AbstractString, ::NeuralNetwork) and
load(::Type{NeuralNetwork}, ::AbstractString, ::Architecture) no longer carry one. docs/make.jl
passes HDF5Ext in modules and sets no warnonly, so Documenter errors on the two missing
bindings and the Documenter workflow fails.

The prose above the block is also now the opposite of true — it says GML's methods "handle
GML-specific parameter types … transparently", which is the thing this PR moved out to
NeuralNetworkParameters and GeometricOptimizers.

Fixed by giving each of the four documented methods its own docstring again and rewriting the prose
to name the packages that actually do the work.

4. Smaller things

• scripts/test_attention.jl and scripts/test_double_multiplication_derivative.jl still spell
  NeuralNetworkParameters — seven sites the rename missed, so "all 31 call sites" in the CHANGELOG
  was not quite the whole list.
• load(::Type{NeuralNetwork}, …, args...) takes untyped varargs where exactly one optional
  prototype is meant, so load(NeuralNetwork, h5, arch, a, b, c) is forwarded rather than rejected,
  and the @docs signatures no longer name a method exactly. Replaced by explicit two- and
  three-positional methods.
• The new prototype form had no test. load(NeuralNetwork, h5, arch, prototype) is the one path
  that works with nothing registered, and nothing exercised it. Added.
• The legacy-layout test covered one of the five tagged types. LASympNet → SymmetricMatrix
  goes through GeometricOptimizers' _vector normaliser; the _dense one — StiefelManifold,
  written with an A field and no n — was never read back. Extended to a SymplecticAutoencoder.
• ext/HDF5Ext.jl:5 imports save/load from AbstractNeuralNetworks while taking
  NetworkParameters/params from NeuralNetworkParameters. All four are the latter's generics;
  0.7 only re-binds them. Same for src/GeometricMachineLearning.jl:89. Both now import from the
  owner.
• ext/HDF5Ext.jl:8-29 was 25 lines of comment about code that is no longer in the file, saying
  what the CHANGELOG entry already says. Trimmed to what a reader of the current file needs.
• test/hdf5_support.jl:7 imports NetworkParameters explicitly although
  GeometricMachineLearning exports it. Dropped.
• CHANGELOG bookkeeping. load(…, prototype) is new API and was filed under Removed; moved to
  Added. save(filename, nn) now returns filename where it used to return the h5open block's
  value — a behaviour change that was not recorded.

---

What I ran

Environment: AbstractNeuralNetworks v0.7.0, GeometricOptimizers main, SymbolicNeuralNetworks
abstractneuralnetworks-0.7 (with the QPTOAT patch above), all three dev'd;
NeuralNetworkParameters 0.1.0 from the registry. The two dev'd version numbers were bumped locally
to 0.4.1 and 0.6.0 so that the [compat] block in this branch resolves as written — which it
does, once they exist.

• Pkg.test() on 1cb5681, before any of the fixes: passes, 3667 assertions.

• Pkg.test() with the fixes: passes, 3674 assertions.

• test/hdf5_support.jl on its own: every testset green, the new prototype one 4/4 and the
  old-layout one 6/6 across both architectures.

• The docs regression, via Documenter's own lookup (DocSystem.getdocs), before the fix:

  save(::H5DataStore, ::NeuralNetwork)                          ok
  save(::AbstractString, ::NeuralNetwork)                       *** no docs found -> Documenter error ***
  load(::Type{NeuralNetwork}, ::H5DataStore, ::Architecture)    ok
  load(::Type{NeuralNetwork}, ::AbstractString, ::Architecture) *** no docs found -> Documenter error ***
  

  and after: all four ok. I did not run a full makedocs — it executes the tutorials — so this is
  the specific regression checked, not the whole build.

---

Fixes pushed as 1916edb.

[michakraus]

Correction to the first point of my review above: two of the three releases I listed as outstanding
had already been registered when I wrote it, and I read a stale snapshot.

	state
AbstractNeuralNetworks 0.7.0	registered 2026-08-23 04:00 UTC (General#165447)
NeuralNetworkParameters 0.1.1	registered 04:12 UTC (#165448)
GeometricOptimizers 0.4.1	registered 06:10 UTC (#165453)
SymbolicNeuralNetworks 0.6.0	not registered — still the only blocker

The red CI on this PR is partly a stale registry, not this branch. The run on 1916edb started at
05:37 UTC — an hour and a half after AbstractNeuralNetworks 0.7.0 hit General — and still resolved
against a snapshot that did not have it:

Added `General` registry to ~/.julia/registries
Updating registry at `~/.julia/registries/General.toml`
ERROR: LoadError: Unsatisfiable requirements detected for package AbstractNeuralNetworks [60874f82]:
 ├─possible versions are: 0.1.0 - 0.6.4 or uninstalled
 └─restricted to versions 0.7 by GeometricMachineLearning [194d25b2] — no versions left

The registry is added and updated without error; the package server's snapshot is simply behind. My
own depot had the same lag, which is how I got it wrong.

But the branch still does not resolve on a current registry — the failure just moves one package
along. Against General at 0a13a76d (which has all three registrations):

ERROR: Unsatisfiable requirements detected for package SymbolicNeuralNetworks [aed23131]:
 ├─possible versions are: 0.1.0 - 0.5.0 or uninstalled
 └─restricted to versions 0.6 by GeometricMachineLearning [194d25b2] — no versions left

With SymbolicNeuralNetworks dev'd from its abstractneuralnetworks-0.7 branch, everything else
now comes straight from the registry — no dev for AbstractNeuralNetworks or
GeometricOptimizers any more — and the suite passes. The registered GeometricOptimizers 0.4.1 is
byte-identical in src/ and ext/NeuralNetworkParametersExt.jl to the main I originally tested
against.

So the one thing left is SymbolicNeuralNetworks 0.6.0, and its abstractneuralnetworks-0.7 branch
is not yet releasable: it still does using AbstractNeuralNetworks: QPTOAT, and 0.7 replaced
QPTOAT with ArrayOrNamedTuple, so it fails to precompile against the tag.

The branch has also been rebased onto main (now 5ef669db, linear on top of #247). Two notes on
that:

• Remove the recurrent and LSTM architectures #247 landed the RecurrentNeuralNetwork/LSTMNeuralNetwork removal on main, more thoroughly than
  my commit did — it also drops the IdentityActivation, ZeroVector import the removal left dead
  and updates the LSTM footnote. Conflicts were resolved in favour of main's version and my
  duplicate CHANGELOG bullet dropped.
• The rebase silently dropped four Project.toml changes, because 3b425ab6 ("Merge main into
  abstractneuralnetworks-0.7") was an evil merge: the bump to 0.6.0, the NeuralNetworkParameters
  entry in [deps], NeuralNetworkParameters = "0.1" and GeometricOptimizers = "0.4.1" were in
  neither parent. Rebase discards merges, so all four vanished and the branch would have shipped as
  0.5.0 with an undeclared dependency. They are amended back into b88d78cb.

[codecov]

Codecov Report

❌ Patch coverage is 75.00000% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 66.51%. Comparing base (afde98f) to head (c15cdb7).

Files with missing lines	Patch %	Lines
src/pullbacks/zygote_pullback.jl	40.00%	3 Missing ⚠️
src/map_to_cpu.jl	0.00%	2 Missing ⚠️
src/data_loader/data_loader.jl	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #246      +/-   ##
==========================================
- Coverage   66.80%   66.51%   -0.29%     
==========================================
  Files          99       99              
  Lines        2931     2888      -43     
==========================================
- Hits         1958     1921      -37     
+ Misses        973      967       -6     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.