Skip to content

Cross module callbacks - #70

Open
saidctb wants to merge 23 commits into
mainfrom
cross-module-callbacks
Open

saidctb wants to merge 23 commits into
mainfrom
cross-module-callbacks

Conversation

@saidctb

@saidctb saidctb commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

No description provided.

saidctb and others added 4 commits September 14, 2026 05:37
Wrapping PRIMA surfaced four defects in the callback path.

A dummy procedure's interface name kept the casefolded key used to match
it, so a generated .pyi annotated `procedure(OBJ)` as `obj` while importing
`OBJ` and could not be rebuilt from its own contract. The parser now keeps
the declared spelling and normalizes case at each comparison.

An assumed-shape array in a callback prototype emitted the plan's runtime
extent marker as Fortran text, `dimension(::Strided)`. The bridge now
lowers a runtime extent to an assumed-shape dummy and measures the
contiguous call-local copy from it. An array result has no caller
descriptor to measure and reports that directly instead.

An abstract interface imported from another file did not resolve during
single-file conversion, so `generate --pyi a.f90 b.f90` degraded the dummy
to an opaque placeholder. Resolution now matches multi-file builds, and an
interface that no supplied source declares is reported by name.

An `intent(out)` primitive scalar was projected as an independent value.
Python has no writable scalar, so the write was silently discarded and the
native caller read uninitialized memory. Such a dummy now reaches Python as
rank-zero storage, and the value spelling is a policy error naming the
replacement. A prototype describes a native callback interface, so it still
mirrors the native argument list; `@native_call` projection remains
available in the contract for a return-oriented callable. Plan validation
covers the storage projection, which the scalar rule previously skipped.

Callback parameters now document the exact callable they expect, generated
from the same completed prototype the trampoline is built from, so the
documented signature cannot drift from the real ABI.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DN6oB7jC7wZXgFmuec4B7Q
Fortran permits a dummy with no declared INTENT to be both read and
modified, but a primitive scalar callback dummy without one was projected
as an independent value and copied in only, so a write by the Python
callable was discarded.

Semantic normalization now records rank-zero storage for such a dummy, and
the callback transfer direction follows that completed storage rather than
re-deriving copy-in from the absent intent. The declaration itself is
unchanged: no intent is synthesized into the semantic origin or the
generated Fortran interface, so the contract records the absence by
carrying no direction wrapper.

    real(8), intent(in) :: f       ->  f: In(Addr(Float64))   copy-in
    real(8), intent(out) :: f      ->  f: Out(Float64[()])    copy-out
    real(8), intent(inout) :: f    ->  f: InOut(Float64[()])  copy-in/out
    real(8) :: f                   ->  f: Float64[()]         copy-in/out

--assume-intent-in-scalars continues to elect which default an undeclared
intent receives, narrowing that last row to the input-only projection
without giving the dummy a direction it never declared.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DN6oB7jC7wZXgFmuec4B7Q
The callback key rules kept a sentence stating that every primitive scalar
callback argument arrives as an independent NumPy scalar value, which
contradicted the writable-storage rule documented directly below it. One
rule now covers both projections.

The undeclared-intent regression asserted the generated contract text and
then rebuilt from the Fortran source, so nothing proved the bare
Float64[()] spelling survived being read back. It now builds through that
generated contract and runs the callback, covering source, contract,
policy, codegen and runtime in one pass; the shared helper takes an
optional fixture package so a round trip needs no checked-in contract.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DN6oB7jC7wZXgFmuec4B7Q
An abstract interface imported from another module was converted in the
scope of the module that imported it. An interface body is written where it
is declared, so a derived type it names belongs to the declaring module; a
consumer that imported only the interface attributed that type to itself
and failed with no completed wrapper type definition for a type it never
declares. Interface lookup now carries the declaring module, the prototype's
dummies convert in that module's context, and a type local to the declaring
module records it as the origin.

Resolution also stopped at module-level imports. A `use` inside a single
procedure, a standalone procedure's own imports, and an interface
re-exported through another module now all resolve, following a chain of any
length to the module that declares it. File, project and per-file CLI
conversion share one resolver instead of each carrying its own lookup, and
contract reconciliation follows a re-export so a prototype imported from a
module that only republishes it still binds to its declaration.

A contract now also imports a prototype it references but never declares,
which a procedure-local `use` previously left as a free name.

Callback docstrings state each array argument's rank and extents, taken from
the completed transfer plan, and every generated docstring spells a runtime
extent the way the contract spells it rather than exposing the internal
marker.

Regression coverage: imported interfaces owning derived types, the three
resolution routes, multi-file `generate --pyi` through parse and build
including a renamed import, rank-two assumed-shape callbacks, writable
scalar storage on the bridge-free direct bind(C) route, and the
`--assume-intent-in-scalars` override end to end.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DN6oB7jC7wZXgFmuec4B7Q
@codecov

codecov Bot commented Sep 15, 2026

Copy link
Copy Markdown

saidctb and others added 19 commits September 15, 2026 02:42
Three gaps remained in how an imported callback interface carries its
source facts.

Declaring-module ownership was recorded only while iterating dummies, so a
function interface returning a type its own module declares attributed that
type to the consuming module and failed to build. The helper now takes a
semantic type and its declaration rather than an argument, and both the
dummies and the result use it. Binding a prototype reference from a
contract records the same origin, so the generated `.pyi` builds too.

A renamed import kept only the local spelling, so the reference named an
interface the declaring module never defines and the contract imported a
name that does not exist there. The resolver now carries the local spelling
beside the declaring signature, through any number of re-export hops, and
the reference records both. A reference differing from the declaration only
in case is the same interface, so it is spelled canonically rather than
binding a second name.

Following a re-export ignored Fortran accessibility, so a module that
imported an interface privately still appeared to publish it. Reaching
names from another module now applies that module's own visibility rules,
at every hop; a module still sees its own private interfaces.

Not addressed here: resolving a cross-module derived type through the
runtime namespace. A wrapper looks the type up on the module owning the
function rather than the one declaring the type, which also affects an
ordinary function returning an imported type and predates this branch. The
callback-result regression therefore asserts the build, not a call.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DN6oB7jC7wZXgFmuec4B7Q
Two bridge diagnostics interpolated extent expressions straight from the
plan, so rejecting a strided callback array result reported extents
['::Strided'] -- the explicit step the IR stores -- rather than the
shorthand the author wrote.

`Strided` is a public contract name, so `T[::Strided]` and `T[::]` are two
spellings of one contract while `T[:]` is the distinct contiguous one. The
shorthand now has a single owner beside the marker set it belongs to, and
the docstring builder reads it from there instead of keeping a private copy
under a name that implied the explicit form was internal.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DN6oB7jC7wZXgFmuec4B7Q
Carrying the declaring module's classes into prototype binding pushed
reconcile_external_type_refs to complexity 21, over the staged limit of 20.
The prototype branch moves to its own function, which also lets the module
name candidates reuse the helper the re-export index already uses.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DN6oB7jC7wZXgFmuec4B7Q
`T[::]` already spells a strided axis and `T[:]` a contiguous one, so the
explicit `T[::Strided]` and `T[0:n:Strided]` forms were a second way to
write contracts that already had one. The docs described `Strided` as a
compatibility spelling for an older form and told authors to use the short
one; it is now gone rather than carried.

The step position spelled nothing else, so a value there is refused with a
message naming the spelling to use instead. Without that check the removed
form would still have parsed: its text happens to match the marker the IR
carries for a strided axis, so dropping the contract name alone left it
working for anyone who did not import the name.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DN6oB7jC7wZXgFmuec4B7Q
The IR named a strided axis after a contract name that no longer exists,
so every layer that showed one to a reader translated it back: the `.pyi`
printer, the docstring builder and two bridge diagnostics each converted
the token to `::`. Producing `::` directly removes the translation and the
mismatch behind it.

The axis mode had been read from the word itself, so that rule moves beside
the marker set it belongs to and states what actually marks a strided axis:
a trailing empty step, with bounds (`lower:upper:`) or without (`::`).

Six sites re-declared the runtime marker sets as literals; they now read
the shared ones. The absence assertions in the callback planning tests went
with the token -- `::` is Fortran's declaration separator, so its absence
from generated source says nothing, and the positive spellings beside them
already prove the lowering.

`prik semantics` output changes with the IR, so its two expected payloads
are regenerated. Contracts, docstrings and generated sources are byte for
byte unchanged, having already printed the contract spelling.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DN6oB7jC7wZXgFmuec4B7Q
A reference reached through renaming re-exports followed the module
provenance back to the declaration but kept the alias it was last bound to,
so the metadata claimed the declaring module defines a name it never does:
name MID against origin_module A, where A declares OBJ.

The declaration names the symbol, so _bind_prototype_reference takes it from
the resolved prototype instead of from a caller that may only hold an
intermediate alias. The one caller that already passed the declaring name is
unaffected, and the caller that could not know it no longer has to.

A rename and a same-name re-export were each covered; their combination was
not, which is where this sat. Both routes are now covered: the contract
chain through reconciliation, and a Fortran chain generated to contracts,
built and called.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DN6oB7jC7wZXgFmuec4B7Q
Fortran lets a scope build a generic interface from several blocks, each
contributing specifics. PRIMA does this under preprocessor guards, adding
kind-specific procedures only for the precisions a build supports, so
`huge_value` arrives as two blocks that gfortran accepts and the parser
rejected as a duplicate declaration.

Blocks naming one generic in one scope now merge into a single interface
carrying every entry in declaration order, keyed by module so two modules
in a file keep their own. Abstract and unnamed blocks are never generics
and are untouched, and the duplicate check still holds for every other unit
kind.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DN6oB7jC7wZXgFmuec4B7Q
A local interface block repeating a use-associated generic name extends
that generic; it does not replace it. PRIK resolved only the specifics a
module declared, so an extending module published a generic missing
everything it inherited and rejected calls gfortran accepts.

The importing scope now resolves the specifics that reached it through the
import as well as its own, following the import chain. Accumulation stays
one-directional, as Fortran requires: the declaring module gains nothing
from a module that extends it later. An inherited specific joins the
importing module privately, since the import bound the generic name and not
the specific's own, so it is reachable only through the generic.

Two identities had been inferred from an overload's first specific, which
only holds while one module owns them all. A generic now records the scope
that declares it, so an extended generic is published by the extending
module rather than the one it inherited from, and a module generic
addresses each candidate by the scope owning that procedure so an inherited
one stays findable. Class-bound overloads are addressed by their class as
before, which owns every candidate.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DN6oB7jC7wZXgFmuec4B7Q
A module that names an imported procedure in a `public` statement means it
to be part of its own interface, but PRIK dropped the module entirely: a
facade that only re-exports reached Python as nothing at all, so callers
had to reach past it into the modules it was hiding.

The name is published without repeating the declaration. A re-export names
an existing wrapper rather than adding one, so the plan carries an alias
binding the name to the callable its declaring namespace already exposes.
One wrapper is generated, the contract keeps spelling the re-export as the
import it already was, and `facade.proc is home.proc` holds.

Naming the entity is what states the intent. A name public only because the
module default is public carries no such statement, and mirroring that would
republish everything a module happens to import under every namespace that
imports it, so those are unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DN6oB7jC7wZXgFmuec4B7Q
A declaration expression naming an imported specification function recorded
the import spelling as the function's native scope, so a relative sibling
import left the scope as `.extent_helpers` where Fortran names the module
`extent_helpers`. Imported type identities were already normalised; this
applies the same rule to declaration callables, and fixes the namespace
branch beside it, which split on the leading dot and produced an empty name.

Assertions across the Fortran, C and round-trip suites pinned the previous
absolute spelling and now expect the relative one. The C frontend emits
sibling header imports through the same printer, so those move with it.

Found by running the full suite, which the relative-import change had not
been through: five Fortran and three C failures, one of them this defect and
the rest pinned spellings.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DN6oB7jC7wZXgFmuec4B7Q
A `use` statement replaced any earlier import of the same module instead of
adding to it, so a scope naming one module across several statements kept
only the last. PRIMA splits iso_fortran_env across three lines, so
`DP => REAL64` was dropped and `real(RP)` could not be resolved from source:
the kind reached the compiler probe as a project name the probe cannot see.
With every import kept, the existing project symbol table resolves `RP` to
`REAL64` and `IK` to `kind(0)`, which the probe evaluates as the intrinsic
expressions they are.

A bare `use` imports everything, so it absorbs any list beside it rather
than being narrowed by one.

Ordering a procedure's outputs also compared an unplaced position against
placed ones and raised a comparison error. An output with no position is
what the check exists to catch, so it is reported as an unsupported wrapper
policy instead.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DN6oB7jC7wZXgFmuec4B7Q
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DN6oB7jC7wZXgFmuec4B7Q
A derived type may build one type-bound generic from several `generic ::`
statements, each contributing specifics. The parser recorded one binding
per statement, so a type declaring

    generic :: area => area_int
    generic :: area => area_real

carried two bindings both named `area`. Only the first reached dispatch,
and calling the generic with the argument types of any later statement
raised `no matching overload` at runtime. The single-statement spelling
worked, so whether a call resolved depended on how the source was written.

The generated contract hid this: its printer renders same-named overload
sets as consecutive `@overload` defs, which is what Python wants, so both
spellings produced byte-identical `.pyi` text and the loss surfaced only
in the built extension.

Merge the statements where the module-level generic interface blocks are
already merged. The key ignores case and internal spacing so a defined
operator merges across `operator(+)` and `operator (+)`. Attributes come
from the first statement: the standard requires every statement for one
binding to declare the same accessibility.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DN6oB7jC7wZXgFmuec4B7Q
An interface body types its own dummies, and the kind it names may come
from a `use` written inside that body -- no module variable or module
procedure declares it. The variable-context walk visited a module's
variables, procedures and derived types but never its interfaces, so
those dummies contributed no target-probe requirement and the conversion
later raised on a storage fact nothing had measured.

The two input routes disagreed as a result: `generate --pyi` failed with
`Unsupported Fortran semantic type for variable 'nf':
integer(kind=kind(0))` on sources that `build_fortran_extension`
accepted, because a wrapper build's larger parsed set happened to raise
the same requirement elsewhere.

Walk the interfaces a file or module declares, and report the variables
of the bodies they hold.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DN6oB7jC7wZXgFmuec4B7Q
An overload declaration names a specific and restates its public
signature. The projection that shapes that signature lives on the
specific -- a declaration carrying `native_call` is rejected outright --
so the declaration can only spell what the projection leaves visible.
The comparison read the native form instead, and rejected two shapes a
generated contract routinely holds.

An output argument projected into a result kept the write-through its
argument passing states. Whether the call writes through a dummy is not
part of a result type, and the comparison already read ownership from
the declaration for that reason; its storage mutability now follows.

A native scalar descriptor result kept its descriptor topology, which
only a `native_call` result wrapper can name. The contract printer
already strips it when emitting such a result as a nullable value, and
the comparison now expects what the printer writes. Reading that
annotation back needed the `| None` unwrapped as well, which until now
happened only for a slot some projection marked nullable.

The effect was a contract the same tool refused to read back: a generic
over `intent(out)` allocatable arguments, such as an allocation helper,
failed on `safealloc` against its first specific.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DN6oB7jC7wZXgFmuec4B7Q
A source-derived contract declares its entities under Python names, so a
Fortran entity kept in capitals is declared lower case with its source
spelling recorded beside it. Imports were written straight from the
parser's `use` mapping instead, leaving a contract that defines `ik`
imported as `IK` -- a name nothing defines, which failed when the package
was loaded back.

A prototype is the exception. It keeps the spelling its own contract
declares, because an annotation naming it is written the same way, so an
import binding one keeps that spelling too.

Which names those are is a fact about the contracts that declare them,
not the one reading them: a module re-exporting a prototype references it
nowhere in its own body. The stub emitter already holds every module it
renders, so it collects the prototypes they declare and tells each module
before any of them writes an import. Either spelling in a renamed import
identifies a prototype -- the source names what the dependency declares,
the target what the importer calls it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DN6oB7jC7wZXgFmuec4B7Q
A contract is written to be edited, and the name a declaration states is
what Python should call the entity. `SourceName` was read the other way
round: the source spelling replaced the declared name, so renaming a
variable exported the native spelling and dropped the edit entirely. It
records the native entity now, exactly as `bind` does for a callable,
and the declared name stands. A source name inside `Final[...]` reaches
its declaration as well, where the reader looked only through a bare
`Annotated` and dropped it.

Generated contracts were caught by this too. A Fortran entity Python
cannot spell is declared under a name that it can -- `lambda` becomes
`lambda_` -- and reading that back installed the unusable spelling, so
the declaration the contract stated was unreachable.

A class may state a native type through `bind`, which was refused
outright, leaving a derived type locked to a name its Fortran type also
answers to. Policy already read `native_name or name`, so only the
refusal and the reference lookup had to change: an imported reference
names a type the way its declaring contract writes it, and resolving it
searches that module alone, never a type of the same name elsewhere.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DN6oB7jC7wZXgFmuec4B7Q
Fortran names entities without regard to case, so writing a capitalized
`IK` as `ik` renames nothing -- the generated Fortran reaches it either
way. Every such declaration nevertheless carried a `SourceName` or
`@bind` stating the capitals back, which said nothing the declaration
did not already say. Across one real library's contracts that was 60
annotations, none of them load-bearing.

The naming policy already drew this line: `normalize_public_name`
reports `needs_fix` against the casefolded source, so a pure case change
is deliberately not a fix. The printer compared the spellings exactly
instead and never consulted it.

A name Python cannot hold as written keeps its original: a keyword, a
character an identifier cannot carry, a name a collision moved aside.
So does every name from a source language that is case-sensitive, where
the spellings are still compared as written.

A renamed class now states its native type, so the rename survives
regeneration. A C struct keeps its own representation rules, which spell
`struct node` without a decorator.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DN6oB7jC7wZXgFmuec4B7Q
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant