refactor!: one model contract, one authoritative config schema - #139
Merged
Conversation
…ions
These cover the code the next several refactors move, so they are hardened
first: a test that asserts nothing cannot tell you whether a refactor
preserved behavior, and hardening it afterwards proves nothing about the
move.
The substantive changes, rather than the mechanical ones:
test_parallel_embed's dynamic-conformer test used ethane, which is
degenerate and always collapses to a single conformer, so the assertion
held for a reason unrelated to what it claimed. It now uses hexane and
bounds the result against calculate_conformer_count.
test_SDF2chunks compared only chunk counts; it now reparses each chunk and
asserts the molecules match the source by name, order and atom count.
test_config's chunk-meta test built its expectation by hand from the same
literal it then compared against. It now derives from ChunkMeta's own
annotations and required keys.
Two tests are deleted rather than hardened. test_config_schema_exists
cannot fail once the module imports, and no field on Auto3DOptions has a
mutable default for test_immutable_default_list to guard.
Two xfail(strict=True) tests pin defects found while deriving the correct
assertions. Both fail today for the reason named in the marker:
- filter_unique_optimized raises KeyError on a record with no E_tot
property, while the legacy filter_unique tolerates it. The two filters
were given a shared duplicate criterion but still disagree on malformed
input, and the stricter one is the survivor.
- opt_steps carries two minimums: FIELD_BOUNDS declares ("ge", 1) while
validation.py hand-writes >= 10 in two places. The test asserts the two
agree rather than picking a value, so whichever is authoritative can win.
Verified: 1299 passed, 9 skipped, 2 xfailed.
Three eager optional-dependency probes in __init__.py defeated the _LAZY_API mechanism that exists to avoid exactly this. Probing for ANI2xt reached batch_opt, which reached the utils barrel, which reached validation, which imported torch and models.loading. So `import Auto3D` paid for the entire package plus torch and rdkit before the caller had asked for anything. Measured in a subprocess on one box: import Auto3D 1.35 s -> 0.031 s len(sys.modules) 1175 -> 154 (stdlib floor is 128) torch, rdkit loaded yes -> no Auto3D.* submodules 20 -> 0 The probes are deleted rather than made lazy: they are not public API, and every real probe already exists at its use site. A comment records that and says not to reintroduce one. __version__ now comes from a private _detect_version(), and __dir__ reports the lazy API, so `main` is discoverable and warnings, version, ANI2xt and PackageNotFoundError stop leaking into the namespace. __getattr__ deliberately does not cache. Caching would turn it into a capturing binding, which is how a lazily imported reader ends up holding a stub permanently while monkeypatch reports success; test_lazy_torchani_import documents that failure appearing 182 tests downstream. A test asserts the non-caching property. The cost tests assert module count rather than wall-clock seconds, and run in a subprocess since the parent pytest process has already imported everything. Verified: five tripwires confirmed red first for the stated reasons, then each fix mutation-verified by reverting it and confirming the named test goes red. 1298 passed across three random seeds plus fixed order; ruff clean.
…tributes Auto3D.ANI2xt, .warnings, .version and .PackageNotFoundError were never public API, but removing them is still visible to anyone who reached for them, so they belong in Breaking Changes rather than only in the perf note.
The same hardening as the previous commit, for the tests covering the code
the model-contract consolidation is about to move.
The assertions that were doing no work:
- test_create_unknown_model_raises_error caught bare Exception, so it could
not tell an unresolvable registry name from any other failure. It now
matches the message.
- Two pytest.raises(ModelLoadError) had no match=, so either would have
passed on an unrelated ModelLoadError. Both now match on the argument
order they exist to check.
- test_ensemble_opt_returns_convergence_info ignored the convergence info it
is named for; it now asserts the mask and the oscillation counts.
- test_fire_independent_molecule_tracking asserted that per-molecule state
existed, not that it was independent. Rewritten as a two-phase protocol:
bootstrap with every molecule progressing, then make exactly one
oscillate, and assert the exact a and dt values that follow.
- test_optimizer_handles_empty_file accepted any log output; it now asserts
the empty-file message and that the missing-file message is absent, since
those two guards were indistinguishable.
New coverage where a code path had none:
- The arity check's len(required) > 3 half was never exercised. Only the
too-few-arguments half was.
- The alias vocabulary is now parametrized over the full synonym set in both
correct and transposed order, 14 cases, calling validate_custom_nnp
directly so nothing needs to pickle.
- forces = -grad is written separately in ANI2xtAdapter and ANI2xAdapter and
was tested in neither. Both now assert -2*coords against a toy quadratic
model, hermetically, by bypassing each adapter's real __init__.
- The "produced no conformers after clash relief" warning had no test. One
drives the real embedding path with relieve_clash stubbed by atom count,
and asserts the warning fires once, names the right molecule, and that a
sibling species survives without being warned about.
- test_one_bad_molecule_does_not_remove_the_others parsed IDs with a stale
split("_")[0] that made a Tier 1 test vacuous.
test_calc_spe_uses_model_factory and test_model_name2model_calculator_uses_factory
were slow-marked despite monkeypatching every model construction. Both move to
the fast tier; the module-level pytestmark becomes per-test decorators so the
15 real-NNP tests in those two files stay slow. Verified: exactly those two
are collected by -m "not slow".
Padding invariance gains an ANI2x case at 1e-3, slow-marked. It is not run
here: torchani is absent locally and no NNP may be loaded on this box, so CI
is what verifies it.
Verified: 1302 passed, 9 skipped, 2 xfailed.
Two independent clusters that had to land together because both touch auto3D.py's call sites. ## One model contract ModelAdapter moves from models/adapter.py into models/contract.py beside CustomNNP, 40 lines apart, so the deliberate argument-order inversion between them is structural rather than a comment: CustomNNP is forward(species, coords, charges) -> energies and is the public custom-NNP contract, while ModelAdapter is forward(coords, species, charges) -> (energies, forces) and is internal. Confusing the two is the failure this layout is meant to prevent. REQUIRED_ATTRIBUTES is now derived from the Protocol's own annotations rather than hand-maintained beside it, so the two cannot drift. That makes those annotations load-bearing for TorchScript archives, since contract.py skips the signature check for them and is the only gate; the code says so. CustomNNP loses @runtime_checkable. It was never used, and it would not have helped: isinstance against a Protocol only checks attribute presence, so it cannot see Module.forward's stub, which the validator already handles by hand. isinstance now raises TypeError pointing at validate_custom_nnp. batch_opt no longer imports model_factory. The adapter is injected instead, constructed inside the worker so it never crosses a spawn boundary, and batch_opt/species.py is deleted in favour of models/species.py, which puts the species convention next to the model that defines it. Seven copy-pasted MLP definitions in ANI2xt_no_rep.py collapse to a WIDTHS table and a factory, 75 lines to 9. Verified rather than assumed: a test loads the shipped ani2xt_no_repulsion.pt into both the old seven-block ModuleList and the new one and compares. All 56 tensors satisfy torch.equal with identical key sets, because ModuleList state_dict keys are positional and never derived from Python variable names. The old source declared S_network before F_network but placed F before S in the list, which was harmless only because F, S and Cl share widths. Three adapters get their own dtype-preserving energy(). Defining it as forward(...)[0] would have silently downgraded an fp64 Hessian to fp32, because ANI2xAdapter and CustomModelAdapter both call coords.float() and ANI2xt additionally calls requires_grad_(True), which raises on the non-leaf tensor an autograd Hessian supplies. No error, just quietly worse thermochemistry. Four tests pin it. ## One authoritative config schema check_valid_configuration took ten keyword arguments and carried a third set of defaults, including a literal opt_steps=2000. It now takes an Auto3DOptions, which is the single source of truth, and the two byte-identical ten-kwarg marshalling blocks in auto3D.py and workflow.py become one line each. Engine choices move to one ENGINE_CHOICES table. opt_steps had two different minimums: FIELD_BOUNDS declared 1 while validation.py hand-wrote >= 10 twice. 10 is correct and wins. n_steps only tests all-converged on istep % 10 == 0, emits progress on the same cadence, and guards its statistics with an explicit n >= 10; below 10 there is no early exit, no progress and no reporting, and FIRE needs several steps to build velocity. So opt_steps < 10 returned an unconverged structure labelled optimized. Loosening the bound to 1 would have accepted that; tightening only moves an existing refusal earlier. The legacy `auto3d cfg.yaml` path had three shape guards the `run -c` path did not, so an empty file, a non-mapping top level or a syntax error exited 1 "Unexpected Error" instead of 2 ConfigurationError. It now goes through load_yaml_config, and the banner is printed after validation so an unrunnable config is never announced as running. The test that should have caught this claimed in its docstring to use "the exact construction _run_legacy_yaml uses" and then re-implemented it inline. That replica is why the missing guards were invisible; it now calls the real function. A test walks CLIConfig.model_fields metadata and rejects any pydantic constraint on a field whose bound lives in FIELD_BOUNDS, so the duplicated constraint set that had to be backed out once cannot come back. Verified: 1349 passed, 9 skipped, 1 xfailed, ruff clean. The remaining xfail is the E_tot filter divergence, owned by a later cluster; the opt_steps pin is resolved and its marker removed. Every fix mutation-verified by reverting it and confirming the named test goes red.
Three places described code that no longer exists: config.py's docstring example still called optimizing positionally with a model, and the migration guide still showed pad_from_mols taking a model name and to_model_species taking an engine string. The migration guide's warning about optimizing binding a stray positional argument to progress_cb is replaced rather than edited. That hazard is gone: everything after out_f is keyword-only now, so the call raises TypeError instead of silently accepting a wrong-typed callback that n_steps would then swallow in `except Exception: pass`. The section records what the hazard was, since it explains why the parameters are keyword-only. Adds the wave-2 breaking changes to the CHANGELOG: the adapter injection and its signature table, the opt_steps minimum with the reasoning for choosing 10, check_valid_configuration's new parameter, and the legacy YAML exit-code correction.
The wave-1 PR was squash-merged while this branch was stacked on it, so this branch's base no longer existed in main's history. That made the PR unmergeable and, less obviously, untestable: pull_request workflows run against the merge ref, which GitHub cannot build for a conflicting PR, so CI reported nothing at all rather than reporting a failure. Three files conflicted, all resolved in favour of this branch, which is strictly newer in each region: - CHANGELOG.md: both sides appended at the same anchor. Wave-1's import-cost entry is retained from main and this branch's four entries follow it. - tests/test_config.py: main still had the xfail(strict=True) pin asserting only that opt_steps' two minimums agree. This branch has the consolidated test that replaced it once the defect was fixed and the marker removed. - tests/test_utils_validation.py: the _options() helper that builds the Auto3DOptions check_valid_configuration now takes. Verified after resolution: both conflicted test modules parse, 1349 passed, 9 skipped, 1 xfailed, ruff clean.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Wave 2: the two clusters that had to land together, because both change call sites in
auto3D.py.One model contract
ModelAdaptermoves frommodels/adapter.pyintomodels/contract.py, 40 lines fromCustomNNP. That proximity is the point. The two Protocols deliberately take theirarguments in opposite orders —
CustomNNPisforward(species, coords, charges) -> energiesand is the public custom-NNP contract;
ModelAdapterisforward(coords, species, charges) -> (energies, forces)and is internal. Confusing them isthe failure this layout prevents structurally rather than by comment.
REQUIRED_ATTRIBUTESis now derived from the Protocol's own annotations instead of beingmaintained by hand beside it, so the two cannot drift. That makes those annotations
load-bearing for TorchScript archives, since
contract.pyskips the signature check forthem and is therefore the only gate — the code says so explicitly.
CustomNNPloses@runtime_checkable. It was never used, and it would not have helped:isinstanceagainst a Protocol checks only attribute presence, so it cannot seeModule.forward's stub, which the validator already handles by hand.isinstancenowraises
TypeErrorand points atvalidate_custom_nnp.CustomNNP's signature isunchanged — it is the public contract.
batch_optno longer importsmodel_factoryat all. The adapter is injected, constructedinside the worker so it never crosses a
spawnboundary, andbatch_opt/species.pyisdeleted in favour of
models/species.py— the species convention now lives next to themodels that define it.
The fp32 trap, and what was done about it
ANI2xAdapter.forwardandCustomModelAdapter.forwardboth callcoords.float(), andANI2xt additionally calls
requires_grad_(True), which raises on the non-leaf tensor anautograd Hessian supplies. So defining
energy()asforward(...)[0]would have silentlydowngraded an fp64 Hessian to fp32 — no error, just quietly worse thermochemistry. All
three adapters get their own dtype-preserving
energy.AIMNet2Adapter.energyis writtenout explicitly to record that its inherited default is safe there (fp64 is an upcast).
Four tests pin this; each was mutation-verified by replacing the override with the inherited
default and watching it go red.
Seven copy-pasted MLPs, and the checkpoint question answered rather than assumed
ANI2xt_no_rep.py's seven near-identical MLP blocks collapse to aWIDTHStable plus afactory — 75 lines to 9. The obvious risk is checkpoint compatibility, so it was measured,
not reasoned about: a test loads the shipped
ani2xt_no_repulsion.ptinto the verbatim oldseven-block
ModuleListand into the new one, then compares. All 56 tensors satisfytorch.equal, with identical key sets.nn.ModuleListstate_dict keys are positionalindices and never derived from Python variable names, which is why.
aev_dimis read offthe checkpoint rather than hardcoded. Mutating one
WIDTHSentry turns the test red.Incidentally, the old source declared
S_networkbeforeF_networkbut placed F before Sin the
ModuleList. Harmless only because F, S and Cl share widths.One authoritative config schema
check_valid_configurationtook ten keyword arguments and carried a third set ofdefaults, including a literal
opt_steps=2000— which is exactly how a schema drifts awayfrom the one users actually configure. It now takes an
Auto3DOptions, which is the singlesource of truth, and the two byte-identical ten-kwarg marshalling blocks in
auto3D.pyandworkflow.pybecome one line each. Engine choices move to oneENGINE_CHOICEStable.A test walks
CLIConfig.model_fieldsmetadata and rejects any pydantic constraint on afield whose bound lives in
FIELD_BOUNDS. It works againstField(ge=),Annotated[..., Ge()]andconintalike, because it inspects pydantic metadata rather thansource text. The duplicated constraint set that had to be backed out once cannot come back.
opt_stepshad two different minimums; 10 winsFIELD_BOUNDSdeclared("ge", 1)whileutils/validation.pyhand-wrote>= 10in twoplaces. 10 is correct:
n_stepsonly tests all-converged onistep % 10 == 0, emitsprogress on the same cadence, and guards its statistics with an explicit
n >= 10. Below 10there is no early exit, no progress and no reporting, and FIRE needs several steps to build
velocity — so
opt_steps < 10returned an unconverged structure labelled as optimized.Loosening the bound to 1 would have accepted that; tightening only moves an existing refusal
earlier.
The test that pinned this defect was
xfail(strict=True)and asserted only that the twominimums agree, deliberately declining to pick a winner so that whoever fixed it would
choose. It is now rewritten to assert exactly one declared minimum exists, and AST-scans
validation.pyfor any comparison onopt_steps— text matching flagged the explanatorycomment.
The malformed-config exit code, and why no test caught it
An empty file, a non-mapping top level, or a YAML syntax error exited 1 "Unexpected
Error" through
auto3d <config>.yaml, but 2ConfigurationErrorthroughauto3d run -c. A script gating on exit 2 got different answers from the two entry points.The legacy path now uses the same loader, and the startup banner moved after validation so
an unrunnable config is no longer announced as running.
The test that should have caught this claimed in its docstring to use "the exact
construction
_run_legacy_yamluses" — and then re-implemented it inline. That replica isprecisely why the missing shape guards were invisible: the test exercised its own copy, not
the code. It now calls the real function.
Documentation that had gone stale
config.py's docstring example still calledoptimizingpositionally with a model, and themigration guide still showed
pad_from_molstaking a model name andto_model_speciestaking an engine string.
The guide's warning about
optimizingsilently binding a stray positional argument toprogress_cbis replaced rather than edited, because that hazard no longer exists —everything after
out_fis keyword-only, so such a call now raisesTypeErrorinstead ofhanding
n_stepsa wrong-typed callback to swallow inexcept Exception: pass. The sectionrecords what the hazard was, since it explains why the parameters are keyword-only.
Deliberately not done
ASE/thermo.py.Not approved, and the design's own author recommended handing the thermo migration to a
later cluster. B1 took only the single species import line there.
to_model_speciessurvives inmodels/species.py, documented as a residual with itsdelete condition, because migrating
thermo.py's two call sites means droppingmodel_namefromCalculatorandmol2aimnet_input— athermo.pyrestructuring. Itsbody is a two-line name dispatch onto
to_ani2xt_species, so no table or error message isduplicated.
Verification
E_totfilter divergence, owned by a later cluster. Theopt_stepspin is resolved and its marker removed — withstrict=True, fixing the defectwhile leaving the pin would have turned the suite red.
mutations across the two clusters.
Found while implementing
A third
optimizingcall site atauto3D.py:231that the design missed. Leaving itwould have shipped a broken
smiles2mols.