Type piracy introduced by the parametric Hamiltonian architectures (#207)

[michakraus]

The parametric generalized Hamiltonian neural networks (#207) need a handful of methods whose generic and every argument type belong to another package. They are marked with TODO: type piracy in the source and collected here.

The ones that had a local fix are already gone: Base.NamedTuple(::NeuralNetworkParameters) is just params, and the ParameterHandling.flatten and Symbolics.Arr methods went with their callers when #207 moved to NeuralNetworkParameters and SymbolicNeuralNetworks 0.5. What is left needs a release of an upstream package.

AbstractNeuralNetworks

method	file	note
applychain(::Tuple, ::Tuple{<:QPTOAT2, <:OptionalParameters}, ::Tuple)	src/architectures/generalized_hamiltonian_neural_network.jl	the parameter-dependent layers pass (state, system parameters) down the chain. ANN's applychain(layers, x, ps::Union{NamedTuple, NeuralNetworkParameters}) is already generic in x; widening the @generated method at chain.jl:39 the same way would remove the need
(::Chain)(qp, problem_params, ps) and three more Chain functors	same	alternatively a ParametricChain wrapper that GML owns
networkbackend(::LazyArrays.ApplyArray)	same	ANN already dispatches networkbackend on array types it does not own
h5save(::HDF5.Group, ::NeuralNetworkParameters, ::AbstractString)	ext/HDF5Ext.jl	ANN's own extension has h5save(::H5DataStore, ::NamedTuple, …) and save(::H5DataStore, ::NeuralNetworkParameters), but nothing for a parameter set nested at a path. Recorded as D10 in NeuralNetworkParameters' PLAN.md

SymbolicNeuralNetworks

method	file	note
Jacobian(f, ::AbstractSymbolicNeuralNetwork, ::Integer) and Jacobian(::AbstractSymbolicNeuralNetwork, ::Integer)	src/architectures/generalized_hamiltonian_neural_network.jl	Jacobian with respect to the first dim2 input variables only — the rest are the system parameters, which are not differentiated
four (::SymbolicPullback)(ps, model, ::Tuple{…, …, <:Union{NamedTuple, AbstractVector}}) call operators	src/pullbacks/symbolic_hnn_pullback.jl	these exist because build_nn_function takes one data argument, so the system parameters have to be concatenated onto the network input. A build_nn_function that takes more than one data argument would remove both the piracy and the concatenation

Related

• NeuralNetworkParameters.jl PLAN.md Phase 2 moves the parameter container upstream, which is what makes the h5save entry above someone's to own.
• GeometricOptimizers #16 tracks the same class of problem for the optimizer side.

🤖 Generated with Claude Code

[michakraus]

Re-checked against AbstractNeuralNetworks 0.7 and SymbolicNeuralNetworks 0.6. Six of the nine methods are gone; two groups are left, both genuinely upstream.

Fixed upstream, deleted in #207

method	why it went
applychain(::Tuple, ::Tuple{<:QPTOAT2, <:OptionalParameters}, ::Tuple)	ANN 0.7 leaves applychain's data argument untyped, and says why in a comment on the method: the old signature "forced downstream packages to commit type piracy to push anything else through a Chain". The generic method carries the (state, system parameters) tuple already.
networkbackend(::LazyArrays.ApplyArray)	LazyArrays is no longer a GML dependency at all (6a298708), so this could not have compiled.
h5save(::HDF5.Group, ::NetworkParameters, ::AbstractString)	NeuralNetworkParameters' own extension has the ::H5DataStore method and HDF5.Group is one. Worth noting this was not merely redundant: being more specific it shadowed the upstream method, which writes the kind attribute recording what to rebuild a structured leaf as. Deleting it fixes a silent loss of type information on save.

Base.NamedTuple(::NetworkParameters), Base.eltype(::Symbolics.Arr) and create_array(::Type{<:Symbolics.Arr}, …) were already gone. NNP now defines the first itself.

Removed without needing upstream

The four three-argument (::Chain)(qp, μ, ps) functors are now apply_parametric, a function GML owns. This became possible for the same reason the first deletion did: with applychain generic in its data argument, the (state, μ) tuple threads through the ordinary two-argument functor, so those methods were only a convenience entry point and never had to live on someone else's type.

Still open

• SymbolicNeuralNetworks.Jacobian(f, ::AbstractSymbolicNeuralNetwork, ::Integer) ×2 — the Jacobian restricted to the first dim2 input variables, the rest being system parameters that are not differentiated. SNN 0.6 still has only Jacobian(f, nn) and Jacobian(nn).
• the four three-tuple (::SymbolicPullback)(ps, model, (input, output, μ)) operators — SNN 0.6's is Tuple{<:ArrayOrNamedTuple, <:ArrayOrNamedTuple}. These exist because build_nn_function takes one data argument; one that took more would remove both them and the concatenation.

Both are marked with a TODO naming the home. Neither blocks #207.

🤖 Generated with Claude Code