Skip to content

refactor!: one model contract, one authoritative config schema - #139

Merged
isayev merged 7 commits into
mainfrom
debt/wave2-model-contract-and-config
Aug 4, 2026
Merged

refactor!: one model contract, one authoritative config schema#139
isayev merged 7 commits into
mainfrom
debt/wave2-model-contract-and-config

Conversation

@isayev

@isayev isayev commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Wave 2: the two clusters that had to land together, because both change call sites in
auto3D.py.

One model contract

ModelAdapter moves from models/adapter.py into models/contract.py, 40 lines from
CustomNNP. That proximity is the point. The two Protocols deliberately take their
arguments in opposite orders — CustomNNP is forward(species, coords, charges) -> energies
and is the public custom-NNP contract; ModelAdapter is
forward(coords, species, charges) -> (energies, forces) and is internal. Confusing them is
the failure this layout prevents structurally rather than by comment.

REQUIRED_ATTRIBUTES is now derived from the Protocol's own annotations instead of being
maintained by hand beside it, so the two cannot drift. That makes those annotations
load-bearing for TorchScript archives, since contract.py skips the signature check for
them and is therefore the only gate — the code says so explicitly.

CustomNNP loses @runtime_checkable. It was never used, and it would not have helped:
isinstance against a Protocol checks only attribute presence, so it cannot see
Module.forward's stub, which the validator already handles by hand. isinstance now
raises TypeError and points at validate_custom_nnp. CustomNNP's signature is
unchanged
— it is the public contract.

batch_opt no longer imports model_factory at all. The adapter is injected, constructed
inside the worker so it never crosses a spawn boundary, and batch_opt/species.py is
deleted in favour of models/species.py — the species convention now lives next to the
models that define it.

The fp32 trap, and what was done about it

ANI2xAdapter.forward and CustomModelAdapter.forward both call coords.float(), and
ANI2xt additionally calls requires_grad_(True), which raises on the non-leaf tensor an
autograd Hessian supplies. So defining energy() as forward(...)[0] would have silently
downgraded an fp64 Hessian to fp32
— no error, just quietly worse thermochemistry. All
three adapters get their own dtype-preserving energy. AIMNet2Adapter.energy is written
out explicitly to record that its inherited default is safe there (fp64 is an upcast).
Four tests pin this; each was mutation-verified by replacing the override with the inherited
default and watching it go red.

Seven copy-pasted MLPs, and the checkpoint question answered rather than assumed

ANI2xt_no_rep.py's seven near-identical MLP blocks collapse to a WIDTHS table plus a
factory — 75 lines to 9. The obvious risk is checkpoint compatibility, so it was measured,
not reasoned about: a test loads the shipped ani2xt_no_repulsion.pt into the verbatim old
seven-block ModuleList and into the new one, then compares. All 56 tensors satisfy
torch.equal, with identical key sets.
nn.ModuleList state_dict keys are positional
indices and never derived from Python variable names, which is why. aev_dim is read off
the checkpoint rather than hardcoded. Mutating one WIDTHS entry turns the test red.

Incidentally, the old source declared S_network before F_network but placed F before S
in the ModuleList. Harmless only because F, S and Cl share widths.

One authoritative config schema

check_valid_configuration took ten keyword arguments and carried a third set of
defaults, including a literal opt_steps=2000 — which is exactly how a schema drifts away
from the one users actually configure. It now takes an Auto3DOptions, which is the single
source of truth, and the two byte-identical ten-kwarg marshalling blocks in auto3D.py and
workflow.py become one line each. Engine choices move to one ENGINE_CHOICES table.

A test walks CLIConfig.model_fields metadata and rejects any pydantic constraint on a
field whose bound lives in FIELD_BOUNDS. It works against Field(ge=),
Annotated[..., Ge()] and conint alike, because it inspects pydantic metadata rather than
source text. The duplicated constraint set that had to be backed out once cannot come back.

opt_steps had two different minimums; 10 wins

FIELD_BOUNDS declared ("ge", 1) while utils/validation.py hand-wrote >= 10 in two
places. 10 is correct: n_steps only tests all-converged on istep % 10 == 0, emits
progress on the same cadence, and guards its statistics with an explicit n >= 10. Below 10
there is no early exit, no progress and no reporting, and FIRE needs several steps to build
velocity — so opt_steps < 10 returned an unconverged structure labelled as optimized.
Loosening the bound to 1 would have accepted that; tightening only moves an existing refusal
earlier.

The test that pinned this defect was xfail(strict=True) and asserted only that the two
minimums agree, deliberately declining to pick a winner so that whoever fixed it would
choose. It is now rewritten to assert exactly one declared minimum exists, and AST-scans
validation.py for any comparison on opt_steps — text matching flagged the explanatory
comment.

The malformed-config exit code, and why no test caught it

An empty file, a non-mapping top level, or a YAML syntax error exited 1 "Unexpected
Error"
through auto3d <config>.yaml, but 2 ConfigurationError through
auto3d run -c. A script gating on exit 2 got different answers from the two entry points.
The legacy path now uses the same loader, and the startup banner moved after validation so
an unrunnable config is no longer announced as running.

The test that should have caught this claimed in its docstring to use "the exact
construction _run_legacy_yaml uses" — and then re-implemented it inline. That replica is
precisely why the missing shape guards were invisible: the test exercised its own copy, not
the code. It now calls the real function.

Documentation that had gone stale

config.py's docstring example still called optimizing positionally with a model, and the
migration guide still showed pad_from_mols taking a model name and to_model_species
taking an engine string.

The guide's warning about optimizing silently binding a stray positional argument to
progress_cb is replaced rather than edited, because that hazard no longer exists
everything after out_f is keyword-only, so such a call now raises TypeError instead of
handing n_steps a wrong-typed callback to swallow in except Exception: pass. The section
records what the hazard was, since it explains why the parameters are keyword-only.

Deliberately not done

  • B1's phase 3 — unifying the four Hessian model-calling conventions in ASE/thermo.py.
    Not approved, and the design's own author recommended handing the thermo migration to a
    later cluster. B1 took only the single species import line there.
  • to_model_species survives in models/species.py, documented as a residual with its
    delete condition, because migrating thermo.py's two call sites means dropping
    model_name from Calculator and mol2aimnet_input — a thermo.py restructuring. Its
    body is a two-line name dispatch onto to_ani2xt_species, so no table or error message is
    duplicated.

Verification

  • 1349 passed, 9 skipped, 1 xfailed, ruff clean.
  • The remaining xfail is the E_tot filter divergence, owned by a later cluster. The
    opt_steps pin is resolved and its marker removed — with strict=True, fixing the defect
    while leaving the pin would have turned the suite red.
  • Every fix mutation-verified: revert it, confirm the named test goes red, restore. Eleven
    mutations across the two clusters.
  • Tripwires written first and confirmed failing for the stated reason before each fix.
  • Eight-seed order-independence sweep on a quiescent tree.

Found while implementing

A third optimizing call site at auto3D.py:231 that the design missed. Leaving it
would have shipped a broken smiles2mols.

isayev added 7 commits August 4, 2026 00:56
…ions

These cover the code the next several refactors move, so they are hardened
first: a test that asserts nothing cannot tell you whether a refactor
preserved behavior, and hardening it afterwards proves nothing about the
move.

The substantive changes, rather than the mechanical ones:

test_parallel_embed's dynamic-conformer test used ethane, which is
degenerate and always collapses to a single conformer, so the assertion
held for a reason unrelated to what it claimed. It now uses hexane and
bounds the result against calculate_conformer_count.

test_SDF2chunks compared only chunk counts; it now reparses each chunk and
asserts the molecules match the source by name, order and atom count.

test_config's chunk-meta test built its expectation by hand from the same
literal it then compared against. It now derives from ChunkMeta's own
annotations and required keys.

Two tests are deleted rather than hardened. test_config_schema_exists
cannot fail once the module imports, and no field on Auto3DOptions has a
mutable default for test_immutable_default_list to guard.

Two xfail(strict=True) tests pin defects found while deriving the correct
assertions. Both fail today for the reason named in the marker:

- filter_unique_optimized raises KeyError on a record with no E_tot
  property, while the legacy filter_unique tolerates it. The two filters
  were given a shared duplicate criterion but still disagree on malformed
  input, and the stricter one is the survivor.
- opt_steps carries two minimums: FIELD_BOUNDS declares ("ge", 1) while
  validation.py hand-writes >= 10 in two places. The test asserts the two
  agree rather than picking a value, so whichever is authoritative can win.

Verified: 1299 passed, 9 skipped, 2 xfailed.
Three eager optional-dependency probes in __init__.py defeated the _LAZY_API
mechanism that exists to avoid exactly this. Probing for ANI2xt reached
batch_opt, which reached the utils barrel, which reached validation, which
imported torch and models.loading. So `import Auto3D` paid for the entire
package plus torch and rdkit before the caller had asked for anything.

Measured in a subprocess on one box:

  import Auto3D        1.35 s -> 0.031 s
  len(sys.modules)      1175  -> 154      (stdlib floor is 128)
  torch, rdkit loaded    yes  -> no
  Auto3D.* submodules     20  -> 0

The probes are deleted rather than made lazy: they are not public API, and
every real probe already exists at its use site. A comment records that and
says not to reintroduce one.

__version__ now comes from a private _detect_version(), and __dir__ reports
the lazy API, so `main` is discoverable and warnings, version, ANI2xt and
PackageNotFoundError stop leaking into the namespace.

__getattr__ deliberately does not cache. Caching would turn it into a
capturing binding, which is how a lazily imported reader ends up holding a
stub permanently while monkeypatch reports success; test_lazy_torchani_import
documents that failure appearing 182 tests downstream. A test asserts the
non-caching property.

The cost tests assert module count rather than wall-clock seconds, and run in
a subprocess since the parent pytest process has already imported everything.

Verified: five tripwires confirmed red first for the stated reasons, then
each fix mutation-verified by reverting it and confirming the named test goes
red. 1298 passed across three random seeds plus fixed order; ruff clean.
…tributes

Auto3D.ANI2xt, .warnings, .version and .PackageNotFoundError were never
public API, but removing them is still visible to anyone who reached for
them, so they belong in Breaking Changes rather than only in the perf note.
The same hardening as the previous commit, for the tests covering the code
the model-contract consolidation is about to move.

The assertions that were doing no work:

- test_create_unknown_model_raises_error caught bare Exception, so it could
  not tell an unresolvable registry name from any other failure. It now
  matches the message.
- Two pytest.raises(ModelLoadError) had no match=, so either would have
  passed on an unrelated ModelLoadError. Both now match on the argument
  order they exist to check.
- test_ensemble_opt_returns_convergence_info ignored the convergence info it
  is named for; it now asserts the mask and the oscillation counts.
- test_fire_independent_molecule_tracking asserted that per-molecule state
  existed, not that it was independent. Rewritten as a two-phase protocol:
  bootstrap with every molecule progressing, then make exactly one
  oscillate, and assert the exact a and dt values that follow.
- test_optimizer_handles_empty_file accepted any log output; it now asserts
  the empty-file message and that the missing-file message is absent, since
  those two guards were indistinguishable.

New coverage where a code path had none:

- The arity check's len(required) > 3 half was never exercised. Only the
  too-few-arguments half was.
- The alias vocabulary is now parametrized over the full synonym set in both
  correct and transposed order, 14 cases, calling validate_custom_nnp
  directly so nothing needs to pickle.
- forces = -grad is written separately in ANI2xtAdapter and ANI2xAdapter and
  was tested in neither. Both now assert -2*coords against a toy quadratic
  model, hermetically, by bypassing each adapter's real __init__.
- The "produced no conformers after clash relief" warning had no test. One
  drives the real embedding path with relieve_clash stubbed by atom count,
  and asserts the warning fires once, names the right molecule, and that a
  sibling species survives without being warned about.
- test_one_bad_molecule_does_not_remove_the_others parsed IDs with a stale
  split("_")[0] that made a Tier 1 test vacuous.

test_calc_spe_uses_model_factory and test_model_name2model_calculator_uses_factory
were slow-marked despite monkeypatching every model construction. Both move to
the fast tier; the module-level pytestmark becomes per-test decorators so the
15 real-NNP tests in those two files stay slow. Verified: exactly those two
are collected by -m "not slow".

Padding invariance gains an ANI2x case at 1e-3, slow-marked. It is not run
here: torchani is absent locally and no NNP may be loaded on this box, so CI
is what verifies it.

Verified: 1302 passed, 9 skipped, 2 xfailed.
Two independent clusters that had to land together because both touch
auto3D.py's call sites.

## One model contract

ModelAdapter moves from models/adapter.py into models/contract.py beside
CustomNNP, 40 lines apart, so the deliberate argument-order inversion between
them is structural rather than a comment: CustomNNP is
forward(species, coords, charges) -> energies and is the public custom-NNP
contract, while ModelAdapter is forward(coords, species, charges) ->
(energies, forces) and is internal. Confusing the two is the failure this
layout is meant to prevent.

REQUIRED_ATTRIBUTES is now derived from the Protocol's own annotations rather
than hand-maintained beside it, so the two cannot drift. That makes those
annotations load-bearing for TorchScript archives, since contract.py skips the
signature check for them and is the only gate; the code says so.

CustomNNP loses @runtime_checkable. It was never used, and it would not have
helped: isinstance against a Protocol only checks attribute presence, so it
cannot see Module.forward's stub, which the validator already handles by hand.
isinstance now raises TypeError pointing at validate_custom_nnp.

batch_opt no longer imports model_factory. The adapter is injected instead,
constructed inside the worker so it never crosses a spawn boundary, and
batch_opt/species.py is deleted in favour of models/species.py, which puts the
species convention next to the model that defines it.

Seven copy-pasted MLP definitions in ANI2xt_no_rep.py collapse to a WIDTHS
table and a factory, 75 lines to 9. Verified rather than assumed: a test loads
the shipped ani2xt_no_repulsion.pt into both the old seven-block ModuleList
and the new one and compares. All 56 tensors satisfy torch.equal with
identical key sets, because ModuleList state_dict keys are positional and
never derived from Python variable names. The old source declared S_network
before F_network but placed F before S in the list, which was harmless only
because F, S and Cl share widths.

Three adapters get their own dtype-preserving energy(). Defining it as
forward(...)[0] would have silently downgraded an fp64 Hessian to fp32,
because ANI2xAdapter and CustomModelAdapter both call coords.float() and
ANI2xt additionally calls requires_grad_(True), which raises on the non-leaf
tensor an autograd Hessian supplies. No error, just quietly worse
thermochemistry. Four tests pin it.

## One authoritative config schema

check_valid_configuration took ten keyword arguments and carried a third set
of defaults, including a literal opt_steps=2000. It now takes an
Auto3DOptions, which is the single source of truth, and the two
byte-identical ten-kwarg marshalling blocks in auto3D.py and workflow.py
become one line each. Engine choices move to one ENGINE_CHOICES table.

opt_steps had two different minimums: FIELD_BOUNDS declared 1 while
validation.py hand-wrote >= 10 twice. 10 is correct and wins. n_steps only
tests all-converged on istep % 10 == 0, emits progress on the same cadence,
and guards its statistics with an explicit n >= 10; below 10 there is no
early exit, no progress and no reporting, and FIRE needs several steps to
build velocity. So opt_steps < 10 returned an unconverged structure labelled
optimized. Loosening the bound to 1 would have accepted that; tightening only
moves an existing refusal earlier.

The legacy `auto3d cfg.yaml` path had three shape guards the `run -c` path
did not, so an empty file, a non-mapping top level or a syntax error exited 1
"Unexpected Error" instead of 2 ConfigurationError. It now goes through
load_yaml_config, and the banner is printed after validation so an unrunnable
config is never announced as running.

The test that should have caught this claimed in its docstring to use "the
exact construction _run_legacy_yaml uses" and then re-implemented it inline.
That replica is why the missing guards were invisible; it now calls the real
function.

A test walks CLIConfig.model_fields metadata and rejects any pydantic
constraint on a field whose bound lives in FIELD_BOUNDS, so the duplicated
constraint set that had to be backed out once cannot come back.

Verified: 1349 passed, 9 skipped, 1 xfailed, ruff clean. The remaining xfail
is the E_tot filter divergence, owned by a later cluster; the opt_steps pin is
resolved and its marker removed. Every fix mutation-verified by reverting it
and confirming the named test goes red.
Three places described code that no longer exists: config.py's docstring
example still called optimizing positionally with a model, and the migration
guide still showed pad_from_mols taking a model name and to_model_species
taking an engine string.

The migration guide's warning about optimizing binding a stray positional
argument to progress_cb is replaced rather than edited. That hazard is gone:
everything after out_f is keyword-only now, so the call raises TypeError
instead of silently accepting a wrong-typed callback that n_steps would then
swallow in `except Exception: pass`. The section records what the hazard was,
since it explains why the parameters are keyword-only.

Adds the wave-2 breaking changes to the CHANGELOG: the adapter injection and
its signature table, the opt_steps minimum with the reasoning for choosing 10,
check_valid_configuration's new parameter, and the legacy YAML exit-code
correction.
The wave-1 PR was squash-merged while this branch was stacked on it, so this
branch's base no longer existed in main's history. That made the PR
unmergeable and, less obviously, untestable: pull_request workflows run
against the merge ref, which GitHub cannot build for a conflicting PR, so CI
reported nothing at all rather than reporting a failure.

Three files conflicted, all resolved in favour of this branch, which is
strictly newer in each region:

- CHANGELOG.md: both sides appended at the same anchor. Wave-1's import-cost
  entry is retained from main and this branch's four entries follow it.
- tests/test_config.py: main still had the xfail(strict=True) pin asserting
  only that opt_steps' two minimums agree. This branch has the consolidated
  test that replaced it once the defect was fixed and the marker removed.
- tests/test_utils_validation.py: the _options() helper that builds the
  Auto3DOptions check_valid_configuration now takes.

Verified after resolution: both conflicted test modules parse, 1349 passed,
9 skipped, 1 xfailed, ruff clean.
@isayev
isayev merged commit f346086 into main Aug 4, 2026
8 checks passed
@isayev
isayev deleted the debt/wave2-model-contract-and-config branch August 4, 2026 12:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant