Skip to content

Lexer assertion tok->parenstack[level] == '{' fails on 3.15 for a nested f-string whose format spec ends in a backslash #162140

Open

Description

@Nitjsefnie

Crash report

On debug builds of 3.15 (checked v3.15.1 and v3.15.2), this input fails a C assertion in the lexer, through both tokenize and compile():

import io, tokenize

src = 'f"{\n(\nf"{:\\\n""'
for tok in tokenize.generate_tokens(io.StringIO(src).readline):
    print(tok)
# compile(src, "<s>", "exec") aborts the same way
python: Parser/lexer/lexer.c:1150: int tok_get_normal_mode(struct tok_state *, tokenizer_mode *, struct token *): Assertion `tok->parenstack[level] == '{'' failed.

The input is an unclosed replacement field, an unclosed (, then a nested f-string whose format spec ends in a backslash continuation. Without the backslash, or without the (, there is no assertion failure.

Release builds do not abort, but they report the wrong line: 3.15.1 tokenize raises TokenError: ("f-string: expecting '}' to close '{' on line 2", (4, 2)), while line 2 holds the ( and the { is on line 1. compile() raises SyntaxError: f-string: valid expression required before ':'.

main is unaffected: a debug build raises TokenError: ("f-string: expecting '}' to close '{' on line 1", (4, 2)).

This looks like a regression from the 3.15 backport of gh-157778 (#158047, first in 3.15.0rc3). The backport computes

int level = tok->level - the_current_tok->curly_bracket_depth
            + the_current_tok->curly_bracket_expr_start_depth;

while the main commit (GH-157779) uses state->paren_level + state->replacement_depth - 1 on main's refactored f-string state. For this input the 3.15 formula selects the ( entry of parenstack, which fails the assertion on debug builds and names the wrong line on release builds. The 3.15 branch head still has this code.

CPython versions tested on:

3.15, CPython main branch

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.15.2 (tags/v3.15.2:5e58eaad7c, Nov 16 2026, 23:50:48) [Clang 17.0.0] (pydebug build with AddressSanitizer)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions