Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,27 @@ breaking release).

## [Unreleased]

### Removed (breaking)

- **`RecurrentNeuralNetwork` and `LSTMNeuralNetwork`**, along with
`src/architectures/recurrent_neural_network.jl`, `src/architectures/LSTM_neural_network.jl` and the
driver scripts under `scripts/Script_using_fully_GML/{RNN,LSTM}/`.

Both were already unusable. `Chain(::RecurrentNeuralNetwork)` returned an
`AbstractNeuralNetworks.GridCell` rather than a `Chain`, and the cells it held still defined the
pre-0.6 `initialparameters(cell, backend, T; init, rng)` signature, so building a `NeuralNetwork`
from one errored out. Nothing in the test suite touched them. AbstractNeuralNetworks 0.7 removes
`src/cells/` — `Recurrent`, `LSTM`, `GRU`, `IdentityCell` and `GridCell` — so the imports have to go
regardless. The transformer-derived architectures are what GML maintains for time series; the docs
had flagged the LSTM implementation as likely to be deprecated since it was written.

`import AbstractNeuralNetworks: IdentityActivation, ZeroVector` went with them.
`Chain(::RecurrentNeuralNetwork)` was the only thing in `src/` that named `IdentityActivation`,
and `ZeroVector`'s last use moved to `legacy/` in `ec8e8fa5`. Neither was ever exported by GML,
and `IdentityActivation` is an `AbstractNeuralNetworks` export that the blanket `using` at the top
of the module already provides, so `GeometricMachineLearning.IdentityActivation` still resolves —
the line was redundant, not load-bearing.

### Fixed

- **Zygote 0.7 silently zeroed every gradient that flows through `assign_q_and_p`.** Its `rrule`
Expand Down
2 changes: 1 addition & 1 deletion docs/src/architectures/neural_network_integrators.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ where `sl` is short for *sequence length* and `pw` is short for *prediction wind

In `GeometricMachineLearning` all multi-step methods, as is the case with one-step methods, are explicit. There are essentially two ways to construct multi-step methods with neural networks: the older one is using recurrent neural networks such as long short-term memory cells (LSTMs) [hochreiter1997long](@cite) and the newer one is using transformer neural networks [vaswani2017attention](@cite). Both of these approaches have been successfully employed to learn multi-step methods (see [fresca2021comprehensive, lee2020model](@cite) for the former and [hemmasian2023reduced, solera2023beta, brantner2025volume](@cite) for the latter), but because the transformer architecture exhibits superior performance on modern hardware and can be imbued with geometric properties we almost always use a transformer-derived architecture when dealing with time series[^2].

[^2]: `GeometricMachineLearning` also has an LSTM implementation, but this may be deprecated in the future.
[^2]: `GeometricMachineLearning` used to ship an LSTM implementation. It has been removed.

Explicit multi-step methods derived from the transformer are always subtypes of the type [`TransformerIntegrator`](@ref) in `GeometricMachineLearning`. In `GeometricMachineLearning` the [standard transformer](@ref "Standard Transformer"), the [volume-preserving transformer](@ref "Volume-Preserving Transformer") and the [linear symplectic transformer](@ref "Linear Symplectic Transformer") are implemented.

Expand Down
68 changes: 0 additions & 68 deletions scripts/Script_using_fully_GML/LSTM/generate_data.jl

This file was deleted.

69 changes: 0 additions & 69 deletions scripts/Script_using_fully_GML/LSTM/training.jl

This file was deleted.

75 changes: 0 additions & 75 deletions scripts/Script_using_fully_GML/LSTM/validation.jl

This file was deleted.

68 changes: 0 additions & 68 deletions scripts/Script_using_fully_GML/RNN/generate_data.jl

This file was deleted.

75 changes: 0 additions & 75 deletions scripts/Script_using_fully_GML/RNN/training.jl

This file was deleted.

Loading
Loading