feat: v1 semantic convention

[FBumann]

The strict v1 semantic convention for linopy — predictable coordinate alignment and NaN handling.

This is the master PR for the new semantic convention in linopy. It starts with our Design & transitioning goals, which is carried out in our New Semantics spec. Both files are tracked in this branch. WHat you read is the current state.

Both might change until this PR is merged

Scope

The convention ships behind linopy.options["semantics"] — v1 opt-in, legacy the default. This PR carries the design, spec, tests and implementation; documentation notebooks follow separately.

Testing

All tests in linopy will be executed for both semantics.
Differing behaviour will be tested using pytest.markers.

This will increase ci time temporarily until v1 is released.

Defered to a follow up PR

Docs: Documentation, Migration guide, Release notes

[FBumann]

Coming next, in order:

• Slice F — §11 auxiliary-coordinate conflicts. Raise on non-dimension-coord conflict during alignment (covers Auxiliary non-dimension coordinates leak into expressions and break alignment #295). Small.
• Slice G — §13 reductions audit. Most reductions already use skipna=True; need a focused audit + tests for sum / mean / groupby / resample / coarsen and the objective. Likely mostly tests.
• Slice P — linopy/piecewise.py and linopy/sos_reformulation.py. Internal callers build expressions by .isel(...)-slicing along a piece dim and comparing the two slices (delta_hi <= delta_lo); the slices share the dim with different coords, which v1 §8 rejects. Until this lands, test_piecewise_constraints.py + test_piecewise_feasibility.py carry a module-level pytestmark = pytest.mark.legacy and two SOS2 tests are method-marked legacy. Fixing piecewise removes those marks.

Then a final pass on docs (user-facing migration / rollout — deferred so far).

[MaykThewessen]

Thanks @FBumann — that ordering works for me.

Plan:

Hold alignment(join="override") ctx mgr and the short-circuits until #717 merges (and revisit short-circuits post legacy removal).
For alignment_trace: try a pure-stdlib docs snippet first, leaning on the existing LinopySemanticsWarning channel — no new public API needed. Sketch below. If reviewers think it deserves to live in linopy/semantics.py after all (e.g. for stacklevel correctness across the call sites), happy to promote it then.
Proposed docs snippet (drop into the migration guide, not the package):

import warnings
from collections import Counter
from contextlib import contextmanager

from linopy.semantics import LinopySemanticsWarning


@contextmanager
def alignment_trace(*, summary: bool = True):
    """Capture every LinopySemanticsWarning raised inside the block.

    Use during migration to surface where legacy alignment/NaN behaviour
    is being relied on, without touching the codebase under test.
    """
    with warnings.catch_warnings(record=True) as caught:
        warnings.simplefilter("always", LinopySemanticsWarning)
        yield caught

    if summary:
        hits = Counter(
            (w.filename, w.lineno, str(w.message).splitlines()[0])
            for w in caught
            if issubclass(w.category, LinopySemanticsWarning)
        )
        for (fname, lineno, msg), n in hits.most_common():
            print(f"{n:>4}× {fname}:{lineno}  {msg}")

Usage in migration docs:

with alignment_trace() as warns:
    m = build_model()      # legacy semantics still default
    m.solve()

→ ranked list of call sites still relying on legacy NaN/align behaviour

Lets users sweep a real PyPSA-Eur / linopy workflow once, get a ranked punch-list of sites to fix, then flip linopy.options["semantics"] = "v1". Zero surface area in the library.

I'll open the docs PR after #717 lands so the import paths and warning class are stable.

[FBumann]

Blocked by #732

[FBumann]

For reference: Needed changes in flixopt to pass under the new v1 convention. With those changes, the code works under both v1 and legacy

[FabianHofmann]

@FBumann while working on #732 and trying to align the multiindex handling with the new convention, a gap in the convention revealed: The alignment on coords and dims does not only count for two linopy objects but for also for operations with non-linopy objects. IE. when performing a arithmetic operation between a linopy object and a non-linopy array-object, all expectations and conventions should be applied on the non-linopy object too. likely worth defining the object scope.

[FBumann]

@FBumann while working on #732 and trying to align the multiindex handling with the new convention, a gap in the convention revealed: The alignment on coords and dims does not only count for two linopy objects but for also for operations with non-linopy objects. IE. when performing a arithmetic operation between a linopy object and a non-linopy array-object, all expectations and conventions should be applied on the non-linopy object too. likely worth defining the object scope.

@FabianHofmann I think this should currently hold, as linopy owns the arithmetics. Did you find a case where this doesnt?

[FabianHofmann]

@FBumann while working on #732 and trying to align the multiindex handling with the new convention, a gap in the convention revealed: The alignment on coords and dims does not only count for two linopy objects but for also for operations with non-linopy objects. IE. when performing a arithmetic operation between a linopy object and a non-linopy array-object, all expectations and conventions should be applied on the non-linopy object too. likely worth defining the object scope.

@FabianHofmann I think this should currently hold, as linopy owns the arithmetics. Did you find a case where this doesnt?

it's true, just explicitly naming that in the conventions would be beneficial. other objects have to first go via as_dataarray and then arithmetics are performed. that is as_dataarray must be fully aligned with the convention in order to get object-agnostic behavior, ie. operations between expr and arr must behave exactly the same as operations between expr and arr_expr where arr_expr is a linear expression which only holds constants of the same shape as arr. this needs to hold for commutativity, associativity, distributivity.

[FBumann]

Note

Status update (Claude Code, prompted by @FBumann) — the branch is now current with master and the alignment layer is integrated with the semantics system.

Pushed in two commits:

04c7cb4 — Merge master (brings in #732, #742, #743, #716)

The conflicts were small because this branch never touched common.py — the v1 logic lives in its own files. Resolution rule: keep this branch's v1/legacy dispatch structure, use master's broadcast_to_coords(..., strict=False) calls inside it. Test adaptations: the unnamed-MultiIndex coords entry in test_legacy_violations (required to be named since #732), and the masked-addend tails of test_nterm / test_variable_names split into legacy/v1 pairs (§6 divergence).

f036c24 — Alignment ↔ semantics integration

Integration point	Resolution
#738 (warning channels)	the MI-projection deprecation now goes through warn_legacy() / LinopySemanticsWarning; EvolvingAPIWarning stays piecewise-only
Scenario B, v1 side	under options["semantics"] = "v1", implicit MI-level projection raises (§8/§11) with guidance to project explicitly
Tests	MI deprecation tests marked @pytest.mark.legacy; each has a @pytest.mark.v1 counterpart asserting the raise
Bonus fix	as_expression no longer swallows conversion errors ("Cannot convert to LinearExpression: ")

Verification: full suite under both semantics — 6446 passed, 532 skipped, mypy + pre-commit clean.

Remaining for this PR (as far as we can tell): the docs/migration-guide follow-up you listed as deferred, and your review of the merge resolutions in expressions.py / variables.py (the v1/legacy operator dispatch — worth a close look since both sides changed there).

[FBumann]

Claude Code:

One scope question surfaced while writing the §11 stacked-MultiIndex spec: whether v1 should carry first-class MultiIndex support at all, or store MI input as flat dims + auxiliary level coords (which §11 already governs, with none of the projection machinery).

Filed as #744 with the complexity inventory, a verified flat+aux demo against this branch, and three options. It's your call — but the decision window is before this PR merges, so flagging it now rather than after.