TypedPython: recursion with CPython's RecursionError (#57)

docs/SPEC.md

@@ -437,7 +437,11 @@ All in `PM/commonTest`, so they run wherever the interpreter loads.
   through the typed IR (`typedpython/ir.py`) to C against the CPython C API and builds an extension with
   the platform C compiler. Compiled functions return what CPython returns (IR contract: checked i64
   with deopt to the interpreted function, Python floor division/modulo, exact int/float comparison,
-  CPython's exception types and messages; tracebacks are the one accepted difference). `Status: planned`.
+  CPython's exception types and messages; tracebacks are the one accepted difference). Call cycles
+  (direct and mutual recursion) are compiled: every compiled function counts its frame (`tp_enter_call` /
+  `tp_leave_call`, added to the Python frame depth and compared with `sys.getrecursionlimit()`), so recursion
+  past the limit raises `RecursionError('maximum recursion depth exceeded')` where CPython does, and a C-stack
+  guard keeps the process alive at any limit (#57). `Status: planned`.
 - **N-9** TypedPython memory safety is proved on the IR before C is generated (#41; maintainer decision
   2026-10-03). Each property, and what happens when it cannot be proved:
 

docs/design/typedpython.md

@@ -169,6 +169,22 @@ Gradle 태스크는 입력·출력을 정확히 선언해 up-to-date 판정과 
 - **LPython** — 업스트림 활동이 LFortran 쪽으로 옮겨간 것으로 보인다.
 - **Rust 생성** — 위.
 
+### 4.3.3 재귀 (SPEC N-8, #57)
+
+호출 사이클(직접·상호 재귀)은 컴파일한다. 컴파일된 프레임에는 파이썬 프레임이 없으므로 모든 컴파일
+함수의 진입에서 `tp_enter_call()`, 모든 출구(ok·error·deopt)에서 `tp_leave_call()` 을 부른다. 진입은
+스레드별 컴파일 프레임 수에 현재 파이썬 프레임 깊이(`PyThreadState_GetFrame`/`PyFrame_GetBack`, 공개 API 만,
+최상위 프레임 동일성으로 캐시)를 더해 `Py_GetRecursionLimit()` 와 비교하고, 넘으면 CPython 과 같은
+`RecursionError("maximum recursion depth exceeded")` 를 낸다. 이어서 `Py_EnterRecursiveCall` 이 C 스택을
+지킨다. **측정(3.13.0, 3.14.7, 기본 한도 1000):** 인터프리터와 컴파일 재귀 모두 같은 깊이(997)에서
+실패하고, 인터프리터 프레임 k 개 위에서 컴파일 재귀를 시작해도 `최대 n + (추가 프레임 수)` 가 일정해
+오프셋이 없다. 오프셋이 생기는 곳은 둘이다. (1) 한도를 C 스택이 먼저 닫을 만큼 올리면(예: 100만) C 가드의
+메시지가 나온다 — 3.13 은 `maximum recursion depth exceeded in compiled code`, 3.14 는
+`Stack overflow (used 8144 kB) in compiled code` — 인터프리터는 같은 한도에서 파이썬 한도 메시지를 낸다.
+(2) 카운터는 모듈별(헤더가 `static`)이라, 한 컴파일 모듈의 두 프레임 사이에 **다른** 컴파일 모듈의
+프레임이 끼면 그 프레임은 세지 않는다. 테스트는 두 경우 모두 "둘 다 예외를 내고 죽지 않는다" 만 단언하고
+깊이를 기록한다(`test_recursion.py`).
+
 ### 4.4 Kotlin 경계의 직접 호출
 
 이 플랫폼에서만 가능한 이점이다.

python-multiplatform-ksp/src/main/python/typedpython/cgen.py

@@ -22,6 +22,10 @@
   guard, or the impl's 1 in a pure function) increments `__typedpython_deopts__` and calls
   `__typedpython_interpreted__[f]` with the original arguments.
 
+Depth (issue #57): every impl function calls `tp_enter_call()` before it owns anything (a refusal
+returns -1 directly: RecursionError set, nothing counted) and `tp_leave_call()` in the one exit
+block, so ok, error and deopt exits all leave exactly once.
+
 Ownership rule (ir.py "Safety"), implemented by the `_own_*` emit helpers below and nowhere else:
   1. every OBJ local — including each OBJ parameter, which the impl copies with a new reference at
      entry — owns one strong reference, or is NULL;
@@ -728,10 +732,8 @@ def ev_Call(self, e: ir.Call) -> str | None:
             args.append(f"&{t}")
         self.emit("if (tp_s == 0) {")
         self.depth += 1
-        # C-to-C recursion has no Python frame: guard the C stack as CPython guards its own.
-        self.emit('if (Py_EnterRecursiveCall(" in compiled code")) { tp_rc = -1; goto tp_exit; }')
+        # Depth is counted by the callee itself (tp_enter_call at its entry, issue #57).
         self.emit(f"tp_s = {_ident('tp_impl', e.function)}(tp_module{''.join(', ' + x for x in args)});")
-        self.emit("Py_LeaveRecursiveCall();")
         self.depth -= 1
         self.emit("}")
         if e.redo:
@@ -1096,10 +1098,14 @@ def generate(self) -> str:
         out.append("    (void)tp_st; (void)tp_dict; (void)tp_s;"
                    + "".join(f" (void){_ident('l', n)};" for n in self.types if n not in params)
                    + "".join(f" (void){_ident('l', n)};" for n in self.local_arrays))
+        # Depth guard (issue #57): nothing is owned yet, so a refusal returns directly and counts
+        # nothing; every other exit goes through tp_exit, which leaves exactly once.
+        out.append("    if (tp_enter_call() != 0) return -1;")
         out += prologue
         out += self.lines
         out.append("tp_exit:")
         out += self._own_release_all()
+        out.append("    tp_leave_call();")
         out.append("    return tp_rc;")
         out.append("}")
         out.append("")

python-multiplatform-ksp/src/main/python/typedpython/frontend.py

@@ -146,12 +146,6 @@ def lower(path: Path) -> ir.Module:
         except Skip as e:
             skipped[node.name] = e.reason
 
-    for name, cycle in _call_cycles(signatures).items():
-        skipped[name] = (f"line {signatures[name].node.lineno}: recursion "
-                         f"({' -> '.join(cycle)}) is not compiled: C has no recursion limit "
-                         "where CPython raises RecursionError")
-        del signatures[name]
-
     kinds = _inferred_kinds(path, source, tree, [s.node for s in signatures.values()])
 
     # Optimistic start: every candidate lowered, pure, not deopting; then iterate to a fixpoint
@@ -182,30 +176,6 @@ def lower(path: Path) -> ir.Module:
     return ir.Module(path.stem, tuple(functions), skipped)
 
 
-def _call_cycles(signatures: dict[str, _Signature]) -> dict[str, list[str]]:
-    """Candidates on a call cycle through calls by name to other candidates (syntactic, so a
-    call that would end up as CallObject is counted too: conservative)."""
-    calls = {
-        name: {n.func.id for n in ast.walk(s.node) if isinstance(n, ast.Call)
-               and isinstance(n.func, ast.Name) and n.func.id in signatures}
-        for name, s in signatures.items()
-    }
-    cycles: dict[str, list[str]] = {}
-    for start in signatures:
-        stack = [(start, [start])]
-        seen: set[str] = set()
-        while stack and start not in cycles:
-            name, path = stack.pop()
-            for callee in sorted(calls[name]):
-                if callee == start:
-                    cycles[start] = path + [start]
-                    break
-                if callee not in seen:
-                    seen.add(callee)
-                    stack.append((callee, path + [callee]))
-    return cycles
-
-
 def _module_info(tree: ast.Module) -> _ModuleInfo:
     bindings: dict[str, list[tuple[int, str]]] = {}
 

python-multiplatform-ksp/src/main/python/typedpython/ir.py

@@ -240,8 +240,9 @@ class Call(Expr):
     """A direct call of another compiled function in the same module (C to C). If the callee may
     deopt, the caller must be pure (the deopt propagates and the outermost compiled call is
     redone); otherwise the front end calls it as an object (`CallObject(Global(name), ...)`).
-    Recursion (any call cycle) is rejected by the verifier: compiled C has no recursion limit where
-    CPython raises RecursionError."""
+    A call cycle (direct or mutual recursion) is legal: every compiled function counts its frame at
+    entry (runtime `tp_enter_call`), so a recursion that CPython stops with RecursionError is stopped
+    with the same error, and the C stack cannot overflow."""
 
     function: str
     args: tuple[Expr, ...]

python-multiplatform-ksp/src/main/python/typedpython/runtime/API.md

@@ -84,3 +84,16 @@ Same structs as above with `list == NULL` and `dirty == NULL`. 0 ok, -1 MemoryEr
     void tp_f64_array_free(tp_f64_array *a);
     /* tp_*_array_slot works unchanged on local arrays (same IndexError messages as list). */
     PyObject *tp_tuple(PyObject *const *items, Py_ssize_t n);           /* new tuple; steals nothing */
+
+## Call depth (every compiled impl function: enter at entry, leave at every exit; issue #57)
+    int  tp_enter_call(void);   /* 0 ok; -1 RecursionError set, nothing counted (the caller must NOT call leave) */
+    void tp_leave_call(void);   /* undoes one successful tp_enter_call; exactly once per success, on ok, error and deopt exits alike */
+`tp_enter_call` makes compiled recursion fail where interpreted recursion fails. It counts, per thread,
+the compiled frames in flight (they have no Python frame), adds the current Python frame depth
+(`PyThreadState_GetFrame` / `PyFrame_GetBack`, public API only; cached while the top frame is the same
+object, so a pure compiled recursion pays O(1) per call) and compares the total with
+`Py_GetRecursionLimit()`. Over it: `RecursionError("maximum recursion depth exceeded")`, the text
+CPython's own Python-frame check uses. It then calls `Py_EnterRecursiveCall(" in compiled code")`, the
+C-stack guard: a raised recursion limit can never let compiled C overflow the stack. Counters are
+per module (the header is `static`): frames of a *different* compiled module that sit between two
+frames of this one are not counted.

python-multiplatform-ksp/src/main/python/typedpython/runtime/tp_runtime.h

@@ -1008,4 +1008,72 @@ static inline void tp_release(PyObject **slot)
     Py_CLEAR(*slot);
 }
 
+/* ------------------------------------------------------------------------------------------
+ * Call depth (issue #57): compiled frames have no Python frame, so a recursion made of them would
+ * neither raise RecursionError nor stop before the C stack ends. tp_enter_call / tp_leave_call
+ * bracket every compiled impl function. See API.md "Call depth".
+ * ------------------------------------------------------------------------------------------ */
+#if defined(_MSC_VER)
+#define TP_THREAD_LOCAL __declspec(thread)
+#else
+#define TP_THREAD_LOCAL __thread
+#endif
+
+typedef struct {
+    Py_ssize_t count;       /* compiled frames in flight on this thread (this module) */
+    PyObject *frame;        /* strong ref: the top Python frame when `pydepth` was measured */
+    Py_ssize_t pydepth;     /* Python frames below and including `frame` */
+} tp_depth_state;
+
+static TP_THREAD_LOCAL tp_depth_state tp_depth = {0, NULL, 0};
+
+/* Python frame depth of the running thread: public API only. The walk is O(depth), so it is
+ * cached by the identity of the top frame (kept alive while compiled frames are in flight). */
+static inline Py_ssize_t tp_python_depth(void)
+{
+    PyFrameObject *top = PyThreadState_GetFrame(PyThreadState_Get());    /* new ref or NULL */
+    if (top == NULL) {
+        return 0;
+    }
+    if (tp_depth.frame == (PyObject *)top) {
+        Py_DECREF(top);
+        return tp_depth.pydepth;
+    }
+    Py_ssize_t depth = 0;
+    PyFrameObject *cur = top;
+    Py_INCREF(cur);
+    while (cur != NULL) {
+        depth++;
+        PyFrameObject *back = PyFrame_GetBack(cur);
+        Py_DECREF(cur);
+        cur = back;
+    }
+    PyObject *old = tp_depth.frame;
+    tp_depth.frame = (PyObject *)top;       /* takes the reference from GetFrame */
+    tp_depth.pydepth = depth;
+    Py_XDECREF(old);
+    return depth;
+}
+
+static inline int tp_enter_call(void)
+{
+    if (tp_python_depth() + tp_depth.count + 1 > Py_GetRecursionLimit()) {
+        PyErr_SetString(PyExc_RecursionError, "maximum recursion depth exceeded");
+        return -1;
+    }
+    if (Py_EnterRecursiveCall(" in compiled code")) {
+        return -1;
+    }
+    tp_depth.count++;
+    return 0;
+}
+
+static inline void tp_leave_call(void)
+{
+    Py_LeaveRecursiveCall();
+    if (--tp_depth.count == 0) {
+        Py_CLEAR(tp_depth.frame);
+    }
+}
+
 #endif /* TP_RUNTIME_H */

python-multiplatform-ksp/src/main/python/typedpython/verify.py

@@ -31,8 +31,6 @@
   verify/unknown-node         a node (or enum field) that `ir` does not define
   verify/call-unresolved      a Call names a function that is not compiled in this module
   verify/call-unverified      a Call names a function that the verifier rejected
-  verify/recursion            the function is on a cycle of Calls (C stack depth is unbounded;
-                              CPython would raise RecursionError instead)
   verify/condition-only       a CompareObj outside a condition (If/While cond, or an And/Or
                               operand that is itself in a condition): its rich-comparison result
                               need not be a bool, so only its truth value may be used
@@ -146,7 +144,6 @@
 UNKNOWN = "verify/unknown-node"
 UNRESOLVED = "verify/call-unresolved"
 UNVERIFIED = "verify/call-unverified"
-RECURSION = "verify/recursion"
 STRUCTURE = "verify/structure"
 CONDITION_ONLY = "verify/condition-only"
 PROVEN = "verify/proven-overflow"
@@ -1018,45 +1015,6 @@ def _exact(op: BinOpKind, a: tuple[int, int], b: tuple[int, int]) -> tuple[int,
     return min(products), max(products)
 
 
-def _cycles(graph: dict[str, set[str]]) -> set[str]:
-    """Names on a cycle of `graph` (Tarjan's strongly connected components)."""
-    index: dict[str, int] = {}
-    low: dict[str, int] = {}
-    stack: list[str] = []
-    on: set[str] = set()
-    result: set[str] = set()
-    counter = [0]
-
-    def visit(v: str) -> None:
-        index[v] = low[v] = counter[0]
-        counter[0] += 1
-        stack.append(v)
-        on.add(v)
-        for w in graph.get(v, ()):
-            if w not in graph:
-                continue
-            if w not in index:
-                visit(w)
-                low[v] = min(low[v], low[w])
-            elif w in on:
-                low[v] = min(low[v], index[w])
-        if low[v] == index[v]:
-            comp = []
-            while True:
-                w = stack.pop()
-                on.discard(w)
-                comp.append(w)
-                if w == v:
-                    break
-            if len(comp) > 1 or v in graph.get(v, ()):
-                result.update(comp)
-
-    for v in graph:
-        if v not in index:
-            visit(v)
-    return result
-
-
 def verify(module: Module) -> tuple[Module, list[Diagnostic]]:
     """Prove each function of `module`; return the module of proved functions (with proven bounds
     marked) and every diagnostic. Rejected functions move to `skipped` with the reason."""
@@ -1112,9 +1070,11 @@ def reject(fn_name: str, line: int, rule: str, message: str) -> None:
             reject(name, g.source_line, ENTRY_OPEN,
                    f"has entry globals but is not closed: {open_why[name]}")
 
-    for name in sorted(_cycles(callees)):
-        reject(name, sigs[name].source_line, RECURSION,
-               f"'{name}' is on a cycle of calls; compiled recursion has no depth limit")
+    # A call cycle (direct or mutual recursion) is legal: every compiled impl function counts its
+    # frame at entry (tp_enter_call), so recursion ends in RecursionError where CPython raises it
+    # and the C stack cannot overflow (issue #57). Purity and may_deopt flow through the cycle as
+    # through any Call: each flag is proved on the function's own body against its callees'
+    # declared flags, so a cycle through a may_deopt callee is legal only in a pure function.
 
     # A call into a function that is not compiled cannot be a C call: propagate to a fixpoint.
     changed = True

python-multiplatform-ksp/src/test/python/test_frontend.py

@@ -783,9 +783,10 @@ def shrink(lo: float, x: float, hi: float) -> float:
     "@compiled\ndef f(n: int) -> int:\n    return g(n)\n\n"
     "@compiled\ndef g(n: int) -> int:\n    return f(n)\n",
 ])
-def test_recursion_is_skipped(tmp_path, source):
+def test_recursion_is_lowered_not_skipped(tmp_path, source):
+    # #57: a call cycle is compiled; the run-time depth guard replaces the old refusal.
     m = lower(tmp_path, source)
-    assert m.functions == () and "recurs" in m.skipped["f"]
+    assert {f.name for f in m.functions} >= {"f"} and not any("recurs" in r for r in m.skipped.values())
 
 
 def test_int_modulo_cannot_be_recorded_as_proven_so_it_deopts(tmp_path):