Prototype resolution-dependent DSL compiler boundary for #958

[trippyogi]

Summary

This is a draft architecture prototype for #958.

I wanted to test one question before implementing the registration API: how should Rubydex handle a DSL-shaped call when the compiler cannot be selected correctly until the receiver has semantic identity?

The prototype keeps normal OperationBuilder behavior authoritative, records eligible calls as deferred compiler candidates, resolves the receiver through normal constant resolution and aliases, and then emits ordinary existing Operations from a test Struct.new compiler.

I stopped before making those generated operations authoritative. That execution boundary overlaps with the ordered IR / Chunk / VM direction discussed in #798 and the incremental work in #960.

flowchart LR
    A[RubyOperationBuilder] --> B[Known Operation]
    A --> C[Deferred compiler call]
    C --> D[Resolver: owner + aliases]
    D --> E[Existing Operations]
    B --> F[Semantic graph]
    E -. authoritative execution is open .-> F

Loading

Current prototype scope:

constant assignment
+ explicit constant receiver
+ method `new`
+ no block

Example:

Foo = Candidate.new(:name)

The normal DefineConstant(Foo) still wins, so graph semantics do not change. The deferred call is only an observation used to test compiler selection and generated IR.

This only exercises the OperationBuilder path and temporarily changes its result from Vec<Operation> to Vec<CompiledItem> to preserve ordering. I'm not proposing that public Rust API shape as settled.

What this shows

Compiler selection belongs after semantic resolution

Receiver text is not enough.

The tests cover direct and root-qualified Struct, one/multi-hop aliases, lexical shadows, wrong declaration kinds, reopened owners, multiple resolved alias targets, ambiguity removal, class/module/class transitions, and alias cycles.

The reliable boundary is the resolved DeclarationId after normal constant resolution and alias normalization.

The existing worklist model can carry the experiment

Deferred candidates use the existing NameDependent, Unit, and pending_work machinery.

Dependencies are limited to the receiver plus the alias path actually traversed:

Struct                     -> 1 edge
Alias -> Struct            -> 2 edges
AliasA -> AliasB -> Struct -> 3 edges

Unrelated aliases/references do not wake the candidate.

The prototype duplicates a small amount of alias traversal to capture dependency provenance. I would expect a production version to expose that from the canonical resolver, especially with #960 in flight.

Existing Operations can model the tested Struct effect

For:

Foo = Struct.new(:name)

the test compiler emits existing operations equivalent to:

ReferenceConstant(Struct)
EnterClass(Foo, superclass=Struct, is_lexical_scope=false)
DefineAttribute(accessor, name)
ExitScope

A test-only substitution at the original stream position proves that this can affect normal resolution:

class Struct; end
Foo = Struct.new(:name)
class Bar < Foo; end

Foo becomes a class with superclass Struct, owns the generated attribute, and Bar < Foo resolves normally.

Using the dependency chain from #958, the same in-stream oracle also resolves an ordinary SOME_REF inside Bar < Foo through the DSL-generated Foo < Struct chain to Struct::SOME_REF.

The same oracle works for Outer::Foo / Outer::Bar while keeping the generated class non-lexical.

This is an IR expressiveness oracle, not a production-complete implementation of Struct.new.

Where I stopped

The unresolved part is how selected compiler output becomes authoritative.

OperationApplier consumes an ordered stream with transient state such as owner, lexical scope, visibility, preceding references, and instruction position. #1024 now preserves exact lexical constant-resolution context from a namespace Definition / NameId, but generic DSL execution can still depend on additional stream state.

The narrow Struct expansion can replay pre-merge because it establishes most of the state it needs itself. I don't think that is enough evidence to make out-of-band replay the general model.

Applying generated semantics after LocalGraph has merged would also require graph surgery and reintroduce the replacement/ownership problems explored by earlier approaches.

The prototype also exposes an ordering limit:

class Struct; end
Struct = 123
Foo = Struct.new(:name)

Rubydex still has a class-shaped Struct declaration, so the prototype selects the Struct compiler. I left source-order runtime-value tracking out of #958; it appears tied to ordered IR execution rather than compiler matching.

Cost

I benchmarked the OperationBuilder backend against the same baseline on synthetic corpora plus ruby-lsp, Rails, Tapioca, and Discourse.

Real-world candidate counts for this narrow shape were low:

Corpus	Candidates
ruby-lsp	1
tapioca	5
discourse	94
rails	121

Real-world RSS deltas were roughly 1ΓÇô2%, with ~0.6% on candidate-free corpora. Stress tests showed cost scaling with candidate count and actual alias depth.

Windows CPU timings were noisy, so I would not draw performance conclusions from those yet.

Questions

Before taking this further, I'd like guidance on two things:

1. Where should a resolution-dependent compiler invocation live?
   The prototype uses an ordered CompiledItem::DeferredCall. Would this fit better as an IR instruction interpreted by the future VM / Chunk model from Add operation-based indexing pipeline #798?

2. What ordered execution state should authoritative expansion retain?
   Should execution pause/resume at the original stream position when semantic dependencies are unresolved, or should owner/visibility/reference/block state become explicitly replayable?

Those answers also affect fallback-vs-compiler ownership, reverse/unload behavior, compiler-owner bootstrap, and how dependency provenance should integrate with #960.

Deliberately out of scope

This draft does not add the public compiler registry, Ruby/C/FFI registration, Ruby callbacks, block or implicit-receiver DSLs, authoritative replay, reverse operations, or additional DSL compilers.

I want to settle the execution boundary before adding those layers.

Validation

Current branch is rebased on main.

• cargo fmt --check ΓÇö pass
• cargo clippy --lib --all-features ΓÇö pass
• cargo test ΓÇö 1188 library + 9 CLI + 2 doctests passed
• focused operation::deferred tests ΓÇö 33 passed

This draft is for architecture review, not as a complete fix for #958.