Skip to content

Windows REPL (_pyrepl) crashes with UnicodeEncodeError when pasting multiline code containing emoji #153361

Description

@Tetta-Watanabe1

Bug report

Bug description:

Environment

  • Python: 3.14.4
  • OS: Windows 11 25H2
  • Launch method: py
  • Terminal: PowerShell and Command Prompt (reproduced in both)

Description

The REPL raises an unhandled UnicodeEncodeError before executing the pasted code.

The traceback ends with:

UnicodeEncodeError: 'utf-8' codec can't encode characters in position 7-8: surrogates not allowed

The traceback originates from _pyrepl:

Lib\_pyrepl\readline.py
Lib\_pyrepl\reader.py
Lib\_pyrepl\utils.py
Lib\tokenize.py

Steps to reproduce

  1. Start the REPL:
py
  1. Paste the following code as a single multiline paste:
print("😢")
print("💫")
print("✨")
  1. The REPL crashes with UnicodeEncodeError.

Expected behavior

The pasted code should execute normally and print the emoji.

Actual behavior

The REPL crashes before execution with a UnicodeEncodeError originating from _pyrepl.

Additional observations

  • Running the exact same code from a .py file works correctly.
  • A single-line input such as:
print("✨")

works correctly.

  • The problem only occurs when pasting multiline code.
  • The issue reproduces when copying from Windows Notepad, so it does not appear to be specific to ChatGPT or a browser.
  • The issue reproduces in both PowerShell and Command Prompt.
  • The issue reproduces every time on my system.

Full traceback

Traceback (most recent call last):
File "C:\Users\tetuw\AppData\Local\Python\pythoncore-3.14-64\Lib_pyrepl\readline.py", line 395, in multiline_input
return reader.readline()
~~~~~~~~~~~~~~~^^
File "C:\Users\tetuw\AppData\Local\Python\pythoncore-3.14-64\Lib_pyrepl\reader.py", line 758, in readline
self.handle1()
~~~~~~~~~~~~^^
File "C:\Users\tetuw\AppData\Local\Python\pythoncore-3.14-64\Lib_pyrepl\reader.py", line 741, in handle1
self.do_cmd(cmd)
~~~~~~~~~~~^^^^^
File "C:\Users\tetuw\AppData\Local\Python\pythoncore-3.14-64\Lib_pyrepl\reader.py", line 671, in do_cmd
self.refresh()
~~~~~~~~~~~~^^
File "C:\Users\tetuw\AppData\Local\Python\pythoncore-3.14-64\Lib_pyrepl\reader.py", line 648, in refresh
self.screen = self.calc_screen()
~~~~~~~~~~~~~~~~^^
File "C:\Users\tetuw\AppData\Local\Python\pythoncore-3.14-64\Lib_pyrepl\completing_reader.py", line 261, in calc_screen
screen = super().calc_screen()
File "C:\Users\tetuw\AppData\Local\Python\pythoncore-3.14-64\Lib_pyrepl\reader.py", line 315, in calc_screen
colors = list(gen_colors(self.get_unicode()))
File "C:\Users\tetuw\AppData\Local\Python\pythoncore-3.14-64\Lib_pyrepl\utils.py", line 115, in gen_colors
for color in gen_colors_from_token_stream(gen, line_lengths):
~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
File "C:\Users\tetuw\AppData\Local\Python\pythoncore-3.14-64\Lib_pyrepl\utils.py", line 175, in gen_colors_from_token_stream
for prev_token, token, next_token in token_window:
^^^^^^^^^^^^
File "C:\Users\tetuw\AppData\Local\Python\pythoncore-3.14-64\Lib_pyrepl\utils.py", line 382, in prev_next_window
window = deque((None, next(iterator)), maxlen=3)
~~~~^^^^^^^^^^
File "C:\Users\tetuw\AppData\Local\Python\pythoncore-3.14-64\Lib\tokenize.py", line 588, in _generate_tokens_from_c_tokenizer
for info in it:
^^
UnicodeEncodeError: 'utf-8' codec can't encode characters in position 7-8: surrogates not allowed

CPython versions tested on:

3.14

Operating systems tested on:

Windows

Metadata

Metadata

Assignees

No one assigned

    Labels

    type-bugAn unexpected behavior, bug, or error

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions