Skip to content

fix(jit): a REPL fn calling a prior REPL binding no longer emits invalid IR (both backends) - #340

Merged
Ch4s3 merged 11 commits into
mainfrom
claude/wizardly-cray-333762
Aug 25, 2026
Merged

fix(jit): a REPL fn calling a prior REPL binding no longer emits invalid IR (both backends)#340
Ch4s3 merged 11 commits into
mainfrom
claude/wizardly-cray-333762

Conversation

@Ch4s3

@Ch4s3 Ch4s3 commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Symptom

march(1)> fn f(x) do x + 1 end
val f = <fn>
march(2)> fn g(x) do f(x) end
jit error: clang failed (...): invalid redefinition of function 'f'
march(3)> g(41)
I cannot find `g`.

Same failure with a prior let-bound lambda as the callee, on both the clang and ORC backends. Interpreter mode (MARCH_REPL_INTERP=1) was fine. Filed as the "Follow-up noticed, NOT fixed here" section of specs/progress/2026-08-24-orc-repl-segfault-fn-def-after-let-lambda.md.

Root cause

Fn-defining REPL fragments expose prior bindings via module-level slot loaders (Llvm_repl.emit_slot_loader_fns emits define ptr @f()), unlike expression fragments, which bridge slots into var_slot allocas. The EApp emission path knew about neither top_fns nor var_slot entries for f, so the call fell into the unknown-function fallback, which emitted declare ptr @f(i64) for the very symbol the loader defines in the same module — invalid IR, whole fragment (including g) lost.

Note the collision partner is that fallback declare, not partition_fns/extern_fnsf is absent from the second fragment's TIR entirely. Even without the collision, a direct declare+call would be wrong twice over: the erased call-site signature (ptr @f(i64)) mismatches the real fragment-0 define (i64 @f(i64)), and a direct extern call pins the callee version compiled at g's definition time instead of following the slot (the semantics 2026-08-24-repl-jit-fn-redefinition-silently-ignored.md says to prefer).

Fix

Route calls to slot-loader names through the loader + closure dispatch:

  • Llvm_ctx.repl_slot_fns — new ctx table; emit_slot_loader_fns records each bare name it emits a loader for. Only REPL fn-fragment emission ever populates it, so non-REPL compilation is untouched.
  • emit_atom: new AVar arm — a reference to a slot-loader name emits call <ty> @<name>(), materialising the slot's current value (the closure, for fn/lambda bindings). Placed above the runtime-prefix/builtin first-class arms so a REPL binding shadows a same-named builtin, exactly as the expression-fragment var_slot bridge does.
  • EApp: new arm mirroring the existing var_slot→ECallPtr redirect.
  • ECallPtr no-var-slot catch-all: excludes slot-loader names so they reach the generic closure-dispatch arm.

No declare is emitted, and the call goes through the closure's $clo_wrap (uniform tagged-ptr ABI — correct for odd/even Ints, Floats, Strings), reading the slot at call time.

Testing

  • 4 new subprocess session tests in test/test_jit.ml ({clang,orc} × {fn-calls-prior-fn, fn-calls-let-lambda}), written first and failing 4/4 before the fix; 8/8 green after. The harness was generalised to take a backend (clang sessions don't require libLLVM to run).
  • Manual sessions on both backends: chained slot calls (k → g → f), odd/even Int results, String concat through two slot fns, Float chain, recursion + slot call, prior-let capture, let-bound-lambda callee, helper-lambda callee — all correct.
  • Full suite: compiler 932, eval 273, stdlib 877, stdlib_march 61, snapshots 33 — green. run_codegen: 587 run with 21 pre-existing failures that are ORC-by-default fallout from the included branch (in-process sessions sharing one LLJIT dylib collide on _Eq$Int.eq in repl_0, before any slot loader exists; all 21 pass with MARCH_JIT_BACKEND=clang). A separate task is under way for that.

Notes

  • This branch includes a merge of claude/jit-repl-interpreted-perf-68373c (ORC default + internal-linkage slot loaders + pre-warm); origin's copy of that branch ref was stale, so those commits ride along here.
  • Complementary to fix(repl-jit): fn redefinition rebinds instead of being silently ignored #339 (fn redefinition rebinds): that PR makes redefinition rebind the closure slot; this one makes calls read through the slot — together they give interpreter-parity late binding. Whichever merges second should add a session test: fn ffn g calls f → redefine fg(x) picks up the new body.

Ch4s3 added 11 commits August 24, 2026 15:08
… not double-free the module (ORC REPL segfault on fn after let-bound lambda)
…lid IR (both backends)

A fn-defining fragment exposes prior bindings via prev-slot loader fns
(define @<name>()), but the EApp path knew nothing about them: the call
fell into the unknown-function fallback, which declared the very symbol
the loader defines in the same module — invalid redefinition, whole
fragment lost ("I cannot find `g`").

Route such calls through the loader + closure dispatch instead: a new
Llvm_ctx.repl_slot_fns table (populated by emit_slot_loader_fns) drives
an emit_atom arm that materialises the slot's current closure via the
loader, an EApp->ECallPtr redirect, and an exclusion in the ECallPtr
no-var-slot catch-all. No declare is emitted, and the call follows the
slot's current value instead of pinning the callee version compiled at
definition time (see the redefinition-semantics preference in
specs/progress/2026-08-24-repl-jit-fn-redefinition-silently-ignored.md).

4 new subprocess session tests in test/test_jit.ml (clang+orc x
fn-calls-prior-fn / fn-calls-let-lambda), failing-first.
…333762

# Conflicts:
#	CHANGELOG.md
#	test/dune
#	test/test_jit.ml
With ORC as the default (libLLVM present), run_codegen's 21 in-process
repl_jit_cross_line / repl_jit_regression tests failed: each test creates
its own Repl_jit session, but the ORC backend keeps one process-global
LLJIT, so the second session's repl_0 re-defines prelude-synthesized
symbols in the shared JITDylib ("duplicate definition of symbol
'_Eq$Int.eq'").

Pin set_backend_for_tests `Clang in setup_jit_runtime — the gate every
such test goes through — so they keep exercising the clang + dlopen
pipeline they were written against. ORC stays covered end-to-end by
test_jit.ml's subprocess sessions. The multi-session-per-process ORC
limitation is filed as
specs/todos/2026-08-24-orc-multi-session-per-process-duplicate-symbols.md.
CI red on both OSes: ensure_home did a non-recursive mkdir of dune's
HOME=%{project_root}/_build/jit_home, which expands to a RELATIVE path
whose parent does not exist from the test cwd — ENOENT, 6 of 13 tests.
Use the same per-pid tmp session_home the repl_session harness already
uses (now shared by both), passing it to the child REPL explicitly.
@Ch4s3
Ch4s3 merged commit 5ecb01f into main Aug 25, 2026
47 of 50 checks passed
@Ch4s3
Ch4s3 deleted the claude/wizardly-cray-333762 branch August 25, 2026 00:44
Ch4s3 added a commit that referenced this pull request Aug 25, 2026
…urface

Kept both harnesses in test_jit.ml (check_session + redefinition trio) under
one suite; dune rule keeps MARCH_BIN + both source trees. Validated:
cross_line 12/12, regression 15/15, test_jit 13/13, parity 16/16.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant