fix(debugger): resolve cell-local names at the pdb prompt - #10552
fix(debugger): resolve cell-local names at the pdb prompt#10552NoiceHax wants to merge 2 commits into
Conversation
marimo mangles a cell's underscore-prefixed variables so they stay private to the cell (`_b` is stored as `_cell_<cell_id>_b`), so the names as written never existed in the frame pdb evaluates against and `p _b` failed with a NameError while `p a` worked. MarimoPdb now rewrites cell-local names in debugger input back to their mangled counterparts before handing the source to pdb, but only when the name isn't already defined in the frame and its mangled counterpart is -- so unmangled underscore imports keep shadowing, and non-cell frames are untouched. Closes marimo-team#10269
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
All contributors have signed the CLA ✍️ ✅ |
|
I have read the CLA Document and I hereby sign the CLA |
|
@mscolnick I have started the AI code review. It will take a few minutes to complete. |
There was a problem hiding this comment.
All reported issues were addressed across 2 files
Architecture diagram
sequenceDiagram
participant User as User at pdb prompt
participant Pdb as MarimoPdb
participant Parser as AST Parser
participant Mangler as _CellLocalMangler
participant Frame as Cell Frame (f_locals/f_globals)
participant CPdb as Pdb (super)
Note over User,CPdb: Debugger expression/statement evaluation flow
User->>Pdb: p _b (or _b = _b + 1)
alt Expression (p/pp/display)
Pdb->>Pdb: _getval/_getval_except(line)
Pdb->>Pdb: _mangle_cell_locals(source, frame)
alt Non-cell frame or invalid Python
Pdb->>Pdb: Return source untouched
else Cell frame
Pdb->>Parser: ast.parse(source)
Parser-->>Pdb: AST tree
Pdb->>Mangler: visit(tree) with cell_id + frame
Mangler->>Frame: Check if name defined (f_locals/f_globals)
alt Name is cell-local and not already defined
Mangler->>Frame: Check mangled name exists
alt Mangled name exists
Mangler->>Mangler: Rewrite to mangled name (e.g. _b → _cell_0_b)
else No mangled name
Mangler->>Mangler: Leave name untouched
end
end
Mangler-->>Pdb: Transformed AST
Pdb->>Pdb: ast.unparse(tree) if mangled
end
Pdb->>CPdb: _getval(mangled_source)
CPdb->>Frame: Evaluate expression
Frame-->>CPdb: Value (resolves mangled name)
CPdb-->>Pdb: Result
Pdb-->>User: Print value
else Statement (assignment)
Pdb->>Pdb: default(line)
Pdb->>Pdb: Strip leading "!" if present
Pdb->>Pdb: _mangle_cell_locals(source, frame)
alt Cell frame and valid Python
Pdb->>Parser: ast.parse(source)
Parser-->>Pdb: AST tree
Pdb->>Mangler: visit(tree)
Mangler->>Frame: Check names
alt _b = _b + 1 (mangled exists)
Mangler->>Mangler: Rewrite to mangled name
else _c = 1 (new local)
Mangler->>Mangler: Leave unmangled
end
end
Pdb->>CPdb: default(mangled_source)
CPdb->>Frame: Execute statement
Frame-->>CPdb: Assignment written to mangled name (_cell_0_b)
end
Note over Frame: Cell locals stored mangled (_cell_<cell_id>_name)
Note over Pdb: Also uses cell_id_from_filename() to<br/>detect whether frame belongs to a cell
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
A lambda parameter, comprehension target or function local that starts with an underscore belongs to the expression typed at the prompt, not to the cell, so `p (lambda _b: _b + 1)(5)` was rewriting the body reference to the cell's mangled name while the parameter kept its own. Track the scopes the source introduces and skip names they bind. This mirrors the compiler, which only mangles locals resolving against the cell's top-level scope.
|
Fixed the cubic finding. The mangler now tracks scopes the typed source introduces (lambda, def, comprehensions) and leaves names those scopes bind alone, so |
There was a problem hiding this comment.
Pull request overview
This PR updates MarimoPdb so that expressions/statements typed at the pdb prompt can reference marimo cell-local underscore variables (stored as _cell_<cell_id>_<name>) by rewriting input source before evaluation when stopped in a cell frame.
Changes:
- Add an AST-based mangling pass that rewrites cell-local underscore names in debugger input to their mangled counterparts when appropriate.
- Hook the mangling into
pdbevaluation paths (default,_getval,_getval_except) so it applies to statements,p/pp, anddisplay. - Add runtime tests covering expression evaluation, statement assignment, shadowing, non-cell frames, and nested binding constructs.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
marimo/_runtime/marimo_pdb.py |
Adds AST name-rewriting logic and wires it into pdb evaluation entry points. |
tests/_runtime/test_marimo_pdb.py |
Adds tests exercising name mangling behavior at the debugger prompt. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| self._visit_scope( | ||
| bound=_names_bound_by([node.args, *node.body]), | ||
| outer=[*node.decorator_list, *_defaults(node.args)], | ||
| inner=node.body, | ||
| ) |
| Python fixes a scope's locals for the whole body up front, so a reference | ||
| can precede the binding. Bindings from scopes nested deeper still are | ||
| folded in too: over-approximating here only leaves a name untouched, which | ||
| is the safe direction. | ||
| """ |
| Names bound by a scope the typed source introduces itself (a `lambda`, | ||
| `def` or comprehension) belong to that scope, not to the cell, and are | ||
| left alone. That matches how cell code is compiled: marimo only mangles | ||
| locals that resolve against the cell's top-level scope. | ||
| """ |
|
Pulled the updated branch — the lambda/comprehension scope tracking works as described: The Capture names bind through def f(v):
match v:
case _x:
return _x
elif isinstance(child, (ast.MatchAs, ast.MatchStar)) and child.name is not None:
names.add(child.name)
elif isinstance(child, ast.MatchMapping) and child.rest is not None:
names.add(child.rest)Same shape applies to Worth a test alongside the lambda/comprehension ones so the next scope form doesn't slip through the same way. |
dmadisetti
left a comment
There was a problem hiding this comment.
Nice! I think we can simplify the visitor stuff a bunch by just using ours, but this is a good addition. Thank you!
| return [d for d in (*args.defaults, *args.kw_defaults) if d is not None] | ||
|
|
||
|
|
||
| def _names_bound_by(nodes: Iterable[ast.AST]) -> set[str]: |
There was a problem hiding this comment.
I think this information should be available on the cell level
| return names | ||
|
|
||
|
|
||
| class _CellLocalMangler(ast.NodeTransformer): |
There was a problem hiding this comment.
If we did want to do this, I think we would want to break it out into its own module.
There was a problem hiding this comment.
Ahh I see. I think we could maybe just use our standard visitor, which does mangling as well. Worried about bifurcation. I don't think we need to check against what's in memory (so no changes needed, don't need to pass the frame). Just always do the rewrite with the correct mangle prefix
| if cell_id is None: | ||
| return source | ||
| try: | ||
| tree = ast.parse(source) |
There was a problem hiding this comment.
Our helper ast_parse since ast.parse has warning side effects
| frame = getattr(self, "curframe", None) | ||
| if frame is None: | ||
| return source | ||
| cell_id = cell_id_from_filename(frame.f_code.co_filename) |
There was a problem hiding this comment.
This may break in certain cases- line cache isn't always super reliable. safe_get_context() is likely better and extraction from there
marimo renames a cell's underscore variables so they stay private to the cell, so
_bis really stored as_cell_<cell_id>_b. The name as written never exists in the frame pdb stops in, which is whyp _bfails with a NameError whilep aworks.MarimoPdb now parses what you type and rewrites cell-local names to the mangled ones before pdb evaluates it. A name is only rewritten if it isn't already defined in the frame and the mangled version is, so an unmangled underscore import still shadows the cell-local. Non-cell frames and input that isn't valid Python go through untouched. Names bound by the typed source itself are skipped as well, otherwise
p (lambda _b: _b + 1)(5)would rewrite the body but not the parameter.Worth poking at:
p,pp,display, and plain statements like_b = _b + 1, which should land on the cell's variable. New tests are in tests/_runtime/test_marimo_pdb.py.Closes #10269