Skip to content

Explicit setup-time coupling for CoupledSimulations (replaces couple_buffers) - #92

Merged
KnutAM merged 17 commits into
KnutAM:mainfrom
knutambot:cb/coupled-simulations-redesign
Sep 17, 2026
Merged

KnutAM merged 17 commits into
KnutAM:mainfrom
knutambot:cb/coupled-simulations-redesign

Conversation

@knutambot

@knutambot knutambot commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

Summary

Replaces the implicit, per-call coupled-buffer mechanism (couple_buffers, the keyword-only
CoupledSimulations partner bag, and the 3-arg work!(worker, sim, coupled)) with an
explicit, setup-time CoupledSimulations group. This fixes BUG-003 (stale partner time
increments under threading) at the root, and closes the whole class of bugs it belonged to
(stale/deepcopied partner buffer references in general, not just Δt), and also closes
BUG-017 (coupling setup silently under-validating domain/task compatibility).

  • CoupledSimulations((a = sima, b = simb); refs = (c = simc,)) builds a group once: every
    primary reads every other primary and every ref (a ref has no outgoing dependencies but is
    still accessible/workable). Setup validates matching grids, single/dictionary domain shape,
    supported buffer kinds (rejecting e.g. facet buffers with an actionable error, including for
    a sole primary with no partners), per-domain cell coverage, per-domain task-count
    compatibility (a threaded reader requires equal task counts, a sequential partner counting
    as 1 slot), reserved member names, and that no two members alias the same mutable
    item-buffer scratch (including scratch preserved by replace_material).
  • work!(worker, group.member_name) needs no extra arguments — the member handle already
    carries its resolved coupling, and scatters each partner's task-local buffers from its base
    before dispatch so a threaded reader always observes the partner's current state (this is
    the BUG-003 fix, generalized to the new design).
  • New CoupledCellBuffer holds direct, nonrecursive references into partners' own
    CellBuffers and Simulations — never a deepcopy, never rebuilt per cell or per work!
    call. Coupling is direct, not transitive: a partner's own coupling (if any) is not exposed.
  • A new AbstractSimulation{DB} type unifies Simulation and CoupledSimulation, with
    generic forwarding for the accessor API (get_dofhandler, get_state,
    set_time_increment!, update_states!, revert_states!, etc.) and a single, shared set of
    work!/work_domain_sequential!/work_domain_threaded!/scatter! methods dispatching on
    AbstractSimulation type-alias bounds — no more duplicated work-dispatch machinery between
    plain and coupled simulations.
  • CoupledSimulation (a handle to one primary member, e.g. group.a) forwards ordinary
    Simulation property access (.a, .aold, .db) via getproperty/propertynames, so it
    tab-completes and behaves like the underlying Simulation for everything except coupling
    and replacement.
  • FerriteAssembly.replace_material(group, member, f; domain=nothing) rebuilds the whole
    group from source simulations (rerunning validation and autodiff configuration
    construction); other members are reused by reference. Previously obtained handles keep
    their prior configuration. A member handle rejects replace_material directly, pointing the
    caller at the group-level call.
  • Migrated the phase-field fracture tutorial (the only in-repo coupled-buffer user) and the
    coupling test suite to the new API, and rewrote the coupling documentation
    (docs/src/DomainBuffers/Setup.md) including an explicit concurrency contract: coupled
    buffers reference partners' actual mutable storage, so work! calls sharing any of that
    storage (two members of the same group, a group member and its pre-group source
    Simulation, two groups sharing members by reference, or re-entrant calls) must not run
    concurrently — only ordinary sequential/staggered use is safe.

This is a breaking change: couple_buffers and the keyword-only CoupledSimulations/3-arg
work! are removed with no compatibility shim, per the agreed design.

Test plan

  • Pkg.test() — full suite green with both 1 and 4 threads (CoupledSimulations
    testset: 1724 assertions covering mutual/one-way/mixed groups; the BUG-003 Δt
    regression; autodiff-through-coupling numerical agreement vs. a hand-differentiated
    reference; replace_material through a group, including material-type changes;
    constant-propagation and tab-completion for the forwarded properties; and setup
    validation failures, including unsupported buffer kinds and incompatible task counts)
  • All literate tutorials/howtos run clean, including phasefield_fracture.jl
  • Full docs/make.jl HTML build completes without error
  • Reviewed with an independent Codex reviewer across the incremental development of this
    branch, and again as a complete, whole-branch review of the final diff against main
    (plan review + final-diff review); all accepted findings were fixed and re-verified

Co-Authored-By: Claude Sonnet 5 noreply@anthropic.com

🤖 Generated with Claude Code

https://claude.ai/code/session_01KySgpVgJ5fWR9AvPK1JQoU

…imulations

Remove couple_buffers, the keyword-only CoupledSimulations partner container,
and the 3-arg coupled work! interface. Replace with a setup-time
CoupledSimulations((primaries...); refs=(...)) group constructor that
validates and wires coupling once, producing CoupledMember handles whose
work! needs no extra arguments.

Fixes BUG-003 (stale partner Δt) at the root: coupled cell buffers now hold
direct, nonrecursive references into the partner's own mutable CellBuffer and
Simulation (never deepcopies), and work!(worker, ::CoupledMember, ...)
scatters each declared partner's task-local buffers from its base before
dispatch, so a threaded reader always observes the partner's current state
(time increment included) without re-working the partner.

Key pieces:
- New CoupledCellBuffer (src/ItemBuffers/CoupledCellBuffer.jl) wraps a
  reader's own CellBuffer plus per-partner (CellBuffer, Simulation) pairs;
  reinit_buffer! reinitializes the primary and every direct partner, with no
  recursion into partners' own coupling.
- New src/Coupling.jl: CoupledSimulations/CoupledMember construction,
  setup-time validation (matching grid, single/dictionary domain shape,
  supported buffer kinds, cell coverage, per-domain task-count compatibility,
  distinct mutable scratch across all members including replace_material'd
  buffers), task-count-aware partner binding for threaded readers, and
  group-level replace_material (whole-member or single named domain).
- CellBuffer/AutoDiffCellBuffer/FacetBuffer/work.jl: removed coupling fields,
  couple_buffers methods, and the coupled argument from reinit_buffer!/work!;
  AutoDiffCellBuffer and its residual/Jacobian construction now also accept
  CoupledCellBuffer, built fresh after coupling is wired.
- Migrated the phase-field fracture tutorial and coupling tests to the new
  API; added test/coupled_simulations.jl covering mutual/one-way/mixed
  groups, the BUG-003 Δt regression, autodiff-through-coupling numerical
  agreement against a hand-differentiated reference, replace_material
  through a group (including type changes), and setup-validation failures.

Validation: `Pkg.test()` passes in full (CoupledSimulations testset: 1515
assertions, 0 failures; whole suite green). All literate tutorials/howtos run
clean, including phasefield_fracture.jl (the only coupled-buffer user); full
`docs/make.jl` HTML build completes without error. Reviewed with Codex
(dual-review skill) at the plan stage (8 findings, all incorporated before
implementation: partner scatter-before-dispatch, strict threaded task-count
matching, per-domain execution records, facet/autodiff signature updates,
partner-buffer unwrapping, cross-member storage-identity checks, empty-domain
handling) and against the final diff (6 findings: scratch-array-identity
storage check surviving replace_material, empty multi-domain Dict typing,
positive-task-count validation independent of pairing, reserved member names,
documented concurrency contract, strengthened autodiff/replace_material
tests — all fixed and re-verified).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KySgpVgJ5fWR9AvPK1JQoU
@knutambot
knutambot force-pushed the cb/coupled-simulations-redesign branch from 6c8314b to d9f6ca3 Compare September 14, 2026 16:25
@codecov

codecov Bot commented Sep 14, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.79518% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 97.17%. Comparing base (6c7564c) to head (367a73c).

Files with missing lines Patch % Lines
src/work.jl 96.49% 2 Missing ⚠️
src/Coupling.jl 99.28% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #92      +/-   ##
==========================================
+ Coverage   96.94%   97.17%   +0.23%     
==========================================
  Files          30       32       +2     
  Lines        1210     1347     +137     
==========================================
+ Hits         1173     1309     +136     
- Misses         37       38       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

ClaudeBot and others added 15 commits September 14, 2026 13:02
…lation

CoupledCellBuffer previously carried a partner_sims::NamedTuple field so
reinit_buffer! could reinitialize each partner's CellBuffer. Since
CoupledCellBuffer is built once per task, this duplicated an identical copy
of partner_sims into every task-local buffer of a threaded coupled domain.

Rename CoupledMember to CoupledSimulation and give it a `partners` field
(NamedTuple for a single-domain member, Dict{String,<:NamedTuple} per domain
for a multi-domain member) holding the single, canonical copy of the
resolved partner Simulations. CoupledCellBuffer now only holds
partner_buffers (the CellBuffers themselves, still needed per task/per
domain for the correct threaded binding); reinit_buffer!(cb::CoupledCellBuffer,
sim::CoupledSimulation, cellnum) reads partner Simulations from `sim.partners`
at call time instead.

work! gained CoupledSimulation-specific work_domain_sequential!/
work_domain_threaded! methods (mirroring work.jl's plain-Simulation ones) so
that reinit_buffer! receives the full CoupledSimulation, not just the inner
plain Simulation; scatter-before-dispatch (the BUG-003 fix) is now driven
directly from `partners` instead of a separately-tracked container tuple.
Multi-domain members gained Base.iterate on CoupledSimulation, pairing each
per-domain Simulation with its own partner slice.

Pure internal-representation refactor: no public API or numerical-behavior
change. Reviewed with Codex (dual-review skill) at the plan stage (one
finding: the new reinit_buffer! method needed to live in Coupling.jl, after
CoupledSimulation is defined, not in CoupledCellBuffer.jl which is included
first — fixed) and against the final diff (no findings).

Pkg.test() passes in full (CoupledSimulations: 1515/1515). phasefield_fracture.jl
and the full docs/make.jl build both run clean.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KySgpVgJ5fWR9AvPK1JQoU
crate-ci/typos flagged the short local `pn` as a misspelling of `on` in
validate_domain_pair's task-count check. Renamed to reader_tasks/partner_tasks
(also just more readable than rn/pn). No behavior change.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KySgpVgJ5fWR9AvPK1JQoU
Coupling.jl duplicated all six work!/work_domain_sequential!/
work_domain_threaded! methods from work.jl, each body identical to the
plain-Simulation version except dispatching on CoupledSimulation instead.
CoupledSimulation already forwards every accessor these functions call
(get_itembuffer, getset, get_num_tasks, get_chunks, Base.iterate for
multi-domain) to its wrapped Simulation, so the duplication added nothing.

Replace the six duplicated methods with a single set in work.jl, dispatching
on Union type aliases (AnySingleDomainSim, AnyMultiDomainSim, etc.) that
cover both a plain Simulation and a CoupledSimulation wrapping the same
domain-buffer shape. Each work! method gains one line, _prepare_work!(sim),
a tiny extension point: a no-op for Any (defined in work.jl, so plain
Simulation calls are unaffected) with a single override in Coupling.jl,
_prepare_work!(csim::CoupledSimulation) = _scatter_all_partners!(csim),
replacing the scatter call that used to be hand-inlined into every duplicated
work! method.

This requires Coupling.jl (which defines CoupledSimulation) to be included
before work.jl (whose new Union aliases reference it); swapped their order in
FerriteAssembly.jl. Coupling.jl calls nothing from work.jl, so this has no
other effect.

Pure dispatch-layer refactor: no behavior or public API change. Reviewed with
Codex (dual-review skill) at the plan stage and against the final diff: no
findings either time.

Pkg.test() passes in full, unchanged pass counts (CoupledSimulations:
1515/1515). phasefield_fracture.jl and the full docs/make.jl build both run
clean.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KySgpVgJ5fWR9AvPK1JQoU
…r-dependent coupling machinery

The prior WIP commit (29d3e2e) introduced AbstractSimulation{DB} and moved
all of Coupling.jl's content into Simulation.jl, but didn't precompile:
Simulation.jl sits early in the include order (before CellBuffer.jl,
CoupledCellBuffer.jl, Autodiff/autodiff.jl), while the moved-in coupling
machinery (reinit_buffer! for CoupledCellBuffer, build_coupled_itembuffer,
validate_domain_pair, the CoupledSimulations group constructor, ...)
references CellBuffer/CoupledCellBuffer/AutoDiffCellBuffer, none of which
exist yet at that point. work.jl (untouched by the WIP) also broke, since it
still needed the concrete SingleDomainSim/MultiDomainSim/... aliases the WIP
had removed from Simulation.jl.

Clarified with the user: keep AbstractSimulation + CoupledSimulation's core
(struct, accessors, iteration, the partner-scatter _prepare_work! hook) in
Simulation.jl, since none of that needs cell-buffer types. Move only the
genuinely CellBuffer-dependent coupling machinery (reinit_buffer! for
CoupledCellBuffer, the CoupledSimulations struct/constructor/validation,
build_coupled_itembuffer/build_coupled_domain/build_coupled_simulation,
partner_domain_sim, the group-level replace_material) into a restored
Coupling.jl, included after Autodiff/autodiff.jl and before work.jl, same
position it held before the WIP.

This also lets work.jl's dispatch simplify beyond the previous round's
Union-based approach: since Simulation and CoupledSimulation share
AbstractSimulation{DB} with the same DB parameter, work! and
work_domain_sequential!/work_domain_threaded! now dispatch directly on
AbstractSingleDomainSim/AbstractMultiDomainSim/.../AbstractSimulation{<:AbstractDomainBuffer}
bounds - no Union{Simulation{<:X}, CoupledSimulation{<:Simulation{<:X}}}
plumbing needed at all. Also finished generalizing get_num_tasks/get_chunks/
get_itembuffer from ::Simulation-typed to ::AbstractSimulation-typed, which
the WIP had done for the other nine forwarding methods but not these three.

Fixed one bug caught by Codex's plan review: CoupledSimulation's per-domain
Base.iterate methods still matched on the old 2-type-param layout
(CoupledSimulation{<:Simulation{<:DomainBuffers}}); with the new DB-first
layout the correct bound is CoupledSimulation{<:DomainBuffers}, or a
multi-domain coupled primary's work! throws MethodError on iterate.

Pkg.test() passes in full, unchanged pass counts (CoupledSimulations:
1515/1515). phasefield_fracture.jl and the full docs/make.jl build both run
clean. Reviewed with Codex (dual-review skill) at the plan stage (1 finding,
fixed as above) and against the final diff (no findings).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KySgpVgJ5fWR9AvPK1JQoU
…ion commits

Reviewed 4 follow-up commits (2995d47, e5675de, 55577bd, 3bf72a1) that
simplify CoupledSimulation. e5675de (restyle multi-domain iteration) and
3bf72a1 (fix a map(...) do (b, s) bug: that syntax binds one tuple-destructured
argument, but map(f, coll1, coll2) calls the block with two separate
positional args per iteration) are correct as-is. Found and fixed two real
regressions in the other two, confirmed by direct REPL reproduction (Pkg.test()
and a docs build both passed despite the bugs, since the only code path that
exercises them - the fracture tutorial's `.a`/`.aold` dot access and its
solve() call - sits inside a markdown-only comment block, never executed by
`include()`):

- 2995d47 removed CoupledSimulation's custom Base.getproperty, which forwarded
  unknown property names (`.a`, `.aold`, `.db`) to the wrapped `.sim`. `.sim`/
  `.partners` still worked (real fields), but `g.a.a` now threw a FieldError -
  breaking the exact usage pattern the fracture tutorial documents and the
  CoupledSimulation docstring still (falsely) claimed to support. Restored the
  forwarding override.

- 55577bd's scatter!(sim::CoupledSimulation{<:ThreadedDomainBuffer}) calls
  map(scatter!, sim.partners), but only defined scatter! for
  Simulation{<:ThreadedDomainBuffer} - no fallback for a non-threaded partner.
  validate_domain_pair explicitly allows a threaded reader with exactly 1 task
  to pair with a sequential partner (a sequential partner counts as 1 slot);
  work! on such a group threw MethodError. Added a generic
  scatter!(::AbstractSimulation) = nothing fallback (the ThreadedDomainBuffer-
  bound methods stay and take precedence).

Also reworded Coupling.jl's now-stale "partner-scatter hook" comment (the
_prepare_work! hook these commits removed) and unified reinit_buffer!'s
getfield(sim, :sim)/sim.partners mix to plain dot-access, now that the
restored getproperty override makes that the safe, consistent form throughout
the file. Added regression tests for both bugs to
test/coupled_simulations.jl.

Reviewed with Codex (dual-review skill) at the plan stage (1 finding: a
proposed test assertion needed to compare `.db` against the *rebuilt* coupled
buffer, not the original source `Simulation`'s `.db` - fixed) and against the
final diff (no findings).

Pkg.test() passes in full (CoupledSimulations: 1571/1571, up from 1515 with
the new regression tests). phasefield_fracture.jl and the full docs/make.jl
build both run clean.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KySgpVgJ5fWR9AvPK1JQoU
Add Base.propertynames(csim::CoupledSimulation) so the forwarded Simulation
properties (.a, .aold, .db) appear alongside the real fields (.sim, .partners)
during tab-completion, mirroring what CoupledSimulations already had.

Add test coverage:
- is_concrete_inferred helper using Base.return_types on a closure with the
  literal dot-access baked in, since @inferred cannot check constant
  propagation through property-name dispatch (it only accepts call
  expressions and infers from argument runtime types, not the compile-time
  constant property symbol).
- Constprop assertions for .a/.aold/.db/.sim/.partners on CoupledSimulation
  and .a on the CoupledSimulations group, confirming each resolves to a
  single concrete type rather than a Union across getproperty branches.
- propertynames assertions confirming forwarded properties tab-complete.

Regression tests for the two bugs fixed in 5e34dcf (lost property forwarding,
missing scatter! fallback) already existed from that commit; no additional
tests needed there.

Validation: Pkg.test() 1659/1659 passing (up from 1571 before these test
additions), phasefield_fracture.jl tutorial runs cleanly, docs/make.jl build
succeeds.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KySgpVgJ5fWR9AvPK1JQoU
The Spell Check CI job flagged \`pn\` as a likely typo for \`on\`. Renamed to
\`propnames\`.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KySgpVgJ5fWR9AvPK1JQoU
…ion gaps

A complete review of the whole cb/coupled-simulations-redesign branch (main...HEAD),
not just the latest commits, using the dual-review skill with Codex as independent
reviewer (2 rounds: plan review, final-diff review).

Fixes found:
- src/Coupling.jl: unwrap_cb had no fallback method, so coupling an unsupported
  buffer kind (e.g. FacetBuffer) threw a raw MethodError from storage-identity
  validation (which runs before the actionable validate_domain_pair check, and even
  for a sole primary with no partners) instead of an ArgumentError. Added a fallback
  that throws an actionable ArgumentError.
- test/coupled_simulations.jl:
  - The 3-member mutual-coupling test only checked construction, never called
    work!, so a positional mix-up between two partners' buffers/simulations could
    pass undetected. Now works g.a (sequentially and threaded) with distinguishable
    per-partner values for both partners.
  - The allocation-scaling smoke test only checked one 4-cell mesh against a fixed
    ceiling, which couldn't detect a small per-cell allocation. Added a dedicated
    testset comparing a 2x2 vs 20x20 grid against an uncoupled baseline, asserting
    exactly zero coupling-specific overhead (caught, in review, that the first
    version of this fix itself allocated a slice per cell, defeating its purpose).
  - Added facet-buffer coupling rejection tests (sole primary and partner).

Also checked off BUG-017 in identified_bugs.md ("Coupling setup incompletely
validates domain/task compatibility"): this branch's setup-time validation (domain
key matching, task-count compatibility with actionable errors) is exactly the fix
that bug called for.

Test results: Pkg.test() all green with 1 and 4 threads (CoupledSimulations:
1724/1724, up from 1659). phasefield_fracture.jl tutorial and full docs/make.jl
build (every tutorial/how-to) both run clean, no errors.
Codecov flagged 13 missed lines on this branch (project coverage 96.94% -> 96.43%,
patch 94.77%), all in coupling code added/restructured by this PR: src/Coupling.jl
(getproperty/propertynames on the CoupledSimulations group, the empty multi-domain
reader branch of build_coupled_simulation, and the domain-selector path of
replace_material), src/ItemBuffers/CoupledCellBuffer.jl (dof_range forwarding and
_replace_material_with), and src/work.jl (the can_thread/skip_this_domain generic
trait-default fallbacks, which every worker in the package overrides everywhere else).

Added regression tests for each (test/coupled_simulations.jl, test/assemblers.jl), via
the dual-review skill with Codex as independent reviewer:
- CoupledSimulations group getproperty (unknown-member ArgumentError) and propertynames.
- replace_material(group, member, f; domain=...) success and non-dict-member error paths.
- CoupledCellBuffer's dof_range forwarding and _replace_material_with (calling
  replace_material directly on a member's `.db`, framed as an internal-plumbing check,
  not a supported user workflow, since the result isn't re-workable via work! without the
  partner Simulations that live on the owning CoupledSimulation).
- A primary member with a genuinely empty multi-domain Dict (construction-only edge case).
- can_thread/skip_this_domain defaults for a worker with no trait overrides.

Left Coupling.jl:88 (_is_autodiff(::AutoDiffCellBuffer)) untested: reasoned, and Codex's
plan review confirmed via a runtime probe, that it is already dispatched to by the
existing sequential-autodiff coupling test - Codecov's miss there looks like a
coverage-tool artifact for a trivial one-line method, not an untested path.

Codex plan review caught that the initial domain-selector fixture mixed a multi-domain
primary with a single-domain ref, which fails coupling's shape validation before
reaching either intended branch; fixed by using two matching multi-domain members for
the success path and a separate plain group for the error path. It also flagged that
two comments overstated support for edge cases the tests don't actually exercise via
work!; reworded both to state only what's verified. Final-diff review: no findings.

Test results: Pkg.test() all green (CoupledSimulations 1784/1784, up from 1724;
Assemblers 23/23, up from 22; full suite passes). docs/make.jl builds clean.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Claude-Session: https://claude.ai/code/session_01KySgpVgJ5fWR9AvPK1JQoU
Comment thread docs/src/literate_tutorials/phasefield_fracture.jl Outdated
Comment thread docs/src/literate_tutorials/phasefield_fracture.jl Outdated
Co-authored-by: Knut Andreas Meyer <knutam@gmail.com>
@KnutAM
KnutAM merged commit d30c676 into KnutAM:main Sep 17, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants