Compiling code that mixes: deeply nested brackets and invalid escapes in f-strings causes debug assert/abort

[stestagg]

Crash report

What happened?

This code:

compile(
    '(x""f"\\{0}"<' * 176,
    "<seed>",
    "exec",
)

Gives:

Fatal Python error: _Py_CheckFunctionResult: a function returned a result with an exception set
MemoryError: Parser stack overflowed - Python source too complex to parse

The above exception was the direct cause of the following exception:

SystemError: <class 'SyntaxWarning'> returned a result with an exception set

What's going on

The stack when the stack overflow happens is:

 * thread #1, name = 'python3', stop reason = breakpoint 1.261
 0x0000aaaaaad91f44 python3`_Pypegen_stack_overflow(p=<unavailable>) at pegen_errors.c:414:5 [inlined]
 0x0000aaaaaad91f44 python3`fstring_replacement_field_rule(p=0x0000fffff7c92190) at parser.c:16721:9
 0x0000aaaaaad90c28 python3`fstring_middle_rule(p=0x0000fffff7c92190) at parser.c:16673:46
 0x0000aaaaaad90ad0 python3`_loop0_79_rule(p=0x0000fffff7c92190) at parser.c:33624:35
 0x0000aaaaaad90a60 python3`fstring_rule(p=0x0000fffff7c92190) at parser.c:17008:18
 0x0000aaaaaad9067c python3`_tmp_156_rule(p=0x0000fffff7c92190) at parser.c:38246:28
 0x0000aaaaaad8e1c4 python3`_loop1_82_rule(p=0x0000fffff7c92190) at parser.c:33825:29
 0x0000aaaaaad6cea8 python3`invalid_string_tstring_concat_rule(p=0x0000fffff7c92190) at parser.c:28361:18

And the stack around when the assert happens is:

* thread #1, name = 'python3', stop reason = signal SIGABRT
 0x0000aaaaab420c14 python3`fatal_error_exit(status=<unavailable>) at pylifecycle.c:3517:9
 0x0000aaaaab420464 python3`fatal_error(fd=2, header=<unavailable>, prefix=<unavailable>, msg=<unavailable>, status=-1) at pylifecycle.c:3749:5
 0x0000aaaaab41f76c python3`_Py_FatalErrorFunc(func=<unavailable>, msg=<unavailable>) at pylifecycle.c:3765:5
 0x0000aaaaaaea8ce4 python3`_Py_CheckFunctionResult(tstate=0x0000aaaaab819ff0, callable=0x0000aaaaab788c28, result=<unavailable>, where=0x0000000000000000) at call.c:65:13
 0x0000aaaaab1b9548 python3`PyObject_CallOneArg(func=0x0000aaaaab788c28, arg=0x0000fffff79f8ee0) at call.c:395:12 [inlined]
 0x0000aaaaab1b9518 python3`warn_explicit(tstate=0x0000aaaaab819ff0, category=0x0000aaaaab788c28, message=0x0000fffff79f8ee0, filename=<unavailable>, lineno=1, module=0x0000000000000000, registry=0x0000000000000000, sourceline=0x0000000000000000, source=0x0000000000000000) at _warnings.c:806:19
 0x0000aaaaab1bec70 python3`PyErr_WarnExplicitObject(category=0x0000aaaaab788c28, message=0x0000fffff79f8ee0, filename=0x0000fffff7a67740, lineno=1, module=0x0000000000000000, registry=0x0000000000000000) at _warnings.c:1439:11
 0x0000aaaaaae35c58 python3`_PyTokenizer_warn_invalid_escape_sequence(tok=0x0000aaaaabd6b690, first_invalid_escape_char=123) at helpers.c:131:9
 0x0000aaaaaae28c60 python3`tok_get_fstring_mode(tok=0x0000aaaaabd6b690, current_tok=0x0000aaaaabd6c228, token=0x0000fffffff78b58) at lexer.c:1581:25
 0x0000aaaaaae28998 python3`tok_get(tok=0x0000aaaaabd6b690, token=<unavailable>) at lexer.c:1622:16 [inlined]
 0x0000aaaaaae2880c python3`_PyTokenizer_Get(tok=0x0000aaaaabd6b690, token=0x0000fffffff78b58) at lexer.c:1629:18
 0x0000aaaaaad27c3c python3`_PyPegen_fill_token(p=0x0000fffff7c92190) at pegen.c:249:16
 0x0000aaaaaad911c8 python3`_PyPegen_expect_token(p=0x0000fffff7c92190, type=61) at pegen.c:408:13 [inlined]
 0x0000aaaaaad911bc python3`fstring_rule(p=0x0000fffff7c92190) at parser.c:17010:18
 0x0000aaaaaad9067c python3`_tmp_156_rule(p=0x0000fffff7c92190) at parser.c:38246:28
 0x0000aaaaaad8e1c4 python3`_loop1_82_rule(p=0x0000fffff7c92190) at parser.c:33825:29
 0x0000aaaaaad6cea8 python3`invalid_string_tstring_concat_rule(p=0x0000fffff7c92190) at parser.c:28361:18

So, the fstring_rule is doing this:

        if (
            (a = _PyPegen_expect_token(p, FSTRING_START))  // token='FSTRING_START'
            &&
            (b = _loop0_79_rule(p))  // fstring_middle*
            &&
            (c = _PyPegen_expect_token(p, FSTRING_END))  // token='FSTRING_END'
        )

And that looks correct, the && chains should only continue while each function call returns non-NULL.

BUT: _loop0_79_rule detects errors in the fstring-middle function, (and logs it) but doesn't return NULL:

        while (
            (fstring_middle_var = fstring_middle_rule(p))  // fstring_middle
        )
        {
            _res = fstring_middle_var;
            if (_n == _children_capacity) {
                _children_capacity *= 2;
                void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *));
                if (!_new_children) {
                    PyMem_Free(_children);
                    p->error_indicator = 1;
                    PyErr_NoMemory();
                    p->level--;
                    return NULL;
                }
                _children = _new_children;
            }
            _children[_n++] = _res;
            _mark = p->mark;
        }
        p->mark = _mark;
        D(fprintf(stderr, "%*c%s _loop0_79[%d-%d]: %s failed!\n", p->level, ' ',
                  p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "fstring_middle"));
    }

instead copying the partial children set into the output sequence and returning it. (with the parser error and exception still correctly set!)

So then the fstring_rule enters the final part of that chained if, and calls _PyPegen_expect_token(p, FSTRING_END) to find the end, and that call generates the assert() when it tries to raise a SyntaxWarning, and triggers the _Py_CheckFunctionResult check

Looking at a similar function: _loop1_82: (fstring | string)
we have this after the while loop:

    if (_n == 0 || p->error_indicator) {
        PyMem_Free(_children);
        p->level--;
        return NULL;
    }

So maybe we need a similar check without the _n == 0? on loop0_79?

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.16.0a0 (heads/main-dirty:cd9994e, Jul 9 2026, 13:38:10) [Clang 22.1.6 ]

Linked PRs

• gh-153437: Make pegen output error detection checks after loop body for '*' rules. #153441

[terryjreedy]

Same on fresh Windows build, with this traceback:

Current thread 0x0000971c (most recent call first):
  File "<python-input-0>", line 1 in <module>
  File "f:\dev\3x\Lib\_pyrepl\console.py", line 235 in runcode
  File "f:\dev\3x\Lib\_pyrepl\console.py", line 292 in runsource
  File "f:\dev\3x\Lib\code.py", line 324 in push
  File "f:\dev\3x\Lib\_pyrepl\simple_interact.py", line 150 in run_multiline_interactive_console
  File "f:\dev\3x\Lib\_pyrepl\main.py", line 58 in interactive_console

If one reduces 176 to 175, the last copy is marked as having a SyntaxError due to the missing ) and a new prompt given. If one adds the needed ) to the input code, making it '(x""f"\\{0}"<)', the first x"" is marked as a SyntaxError and a new prompt given.

[stestagg]

@terryjreedy thanks for checking, glad it survived the platform change too!

[StanFromIreland]

Is it a duplicate of #150001?

[stestagg]

@StanFromIreland damn, yes thank you, good spot, I forgot to check before raising this one.

[StanFromIreland]

Duplicate of #150001