Code and frozen results for Forgotten Architectures #3 — Cascade-Correlation on Modern Compute. Paper: https://vulkgryph.com/research/nn-revival/papers/cascade-correlation.html Series: https://vulkgryph.com/research/nn-revival/
A controlled, honest re-run of Fahlman & Lebiere's Cascade-Correlation (1990) — a constructive learner that grows structure on demand, training and then freezing one hidden unit at a time via a correlation objective — in its native domain (two-spirals + a low-dim regression surface), with a labeled out-of-domain MNIST comparability arm. Uses the research-harness scaffolding, vendored in-repo. Predictions are pre-registered and frozen; three seeds; results frozen and SHA-256-hashed.
The mechanism reproduces faithfully — it solves two-spirals by growing ~22 frozen units — but the founding legend ("backprop can't do two-spirals") does not survive modern training: a fair two-layer MLP solves it at 99.65% held-out where Cascade-Correlation reaches ~79% (82% at a generous oracle early-stop), and beats it ~7× on regression and by 13–21 points on MNIST. This is overtaking, not refutation — the mainstream backprop path received three decades of cumulative tooling (Adam, initialization, depth) that dissolved the very problem Cascade-Correlation was built to route around. The residual value is the mechanism (adaptive capacity, cheap per-unit training, parameter-compactness), not accuracy; it overshoots useful depth and cannot back out, and grows quadratically. An honest earned-retirement verdict — one of three sorting outcomes across the series so far.
| arm | what it is |
|---|---|
cascor |
faithful Cascade-Correlation — candidate pool, correlation-trained, frozen cascade |
mlp (matched) |
fixed-size single-hidden-layer backprop baseline, matched unit count |
mlp (best-found) |
best MLP over an architecture × activation × learning-rate sweep — the fair comparator (a deep cascade's fair rival is a net allowed depth) |
Requires a recent Rust toolchain and Python 3 (standard library only), plus the research-harness crate (see Cargo.toml).
Native tasks (two-spirals + regression) need no external data. MNIST (comparability) requires the four standard IDX files in data/mnist/ (the runner requires real IDX and aborts rather than silently falling back).
Run the campaign:
cargo run --release --bin stage_a # native arms + harness wire-up (1-seed sanity)
cargo run --release --bin cascor_sweep # fair-shot CasCor hyperparameter sweep
cargo run --release --bin campaign # full 3-seed campaign (Q1–Q4)
cargo run --release --bin n4_rethinks # exploratory N4 rethinks (v1)
cargo run --release --bin n4_rethinks_v2 # exploratory N4 rethinks (leakage-clean CV)
Regenerate the tables from the frozen results:
python3 gen_tables.py
Every figure in the paper is generated from results/*_frozen/*.json by this script (using research-harness's harness_tables.py) — no result numbers are hand-typed.
- Frozen results live in
results/*_frozen/(JSON + Markdown). Their SHA-256 hashes are recorded in the paper and inresults/FROZEN_SHA256.txt. - Predictions were written in
SPEC.mdbefore the runs (frozen 2026-08-20) and are append-only — outcomes are added; predictions are never edited. - The faithful result (
full_campaign_frozen/,cascor_sweep_frozen/,stage_a_frozen/) is the anchor. The exploratory rethinks (rethinks_frozen/,rethinks_v2_frozen/) are labeled, not pre-registered, and frozen separately — they never contaminate the faithful set.
cd results && shasum -a 256 -c FROZEN_SHA256.txt
This studies Cascade-Correlation in its original form under modern controls, in its native domain, plus a labeled out-of-domain MNIST arm. Toy-scale, small baselines — not a SOTA claim. See the paper's Scope, Limitations, and "How to read this result" sections.
See vulkgryph.com.
Copyright 2026 Vulkgryph LLC. Code licensed under Apache-2.0 — see LICENSE and NOTICE.
Code and experiments produced with AI coding agents under the author's direction; figures are generated from the frozen results by gen_tables.py and audited by the author.