Merge remote-tracking branch 'origin/main' into gh-152754-os-race-mutex

Author: nascheme

Doc/library/codecs.rst

@@ -1087,6 +1087,18 @@ On Windows, ``cpXXX`` codecs are available for all code pages.
 But only codecs listed in the following table are guaranteed to exist on
 other platforms.
 
+On platforms that provide the C library's :manpage:`iconv(3)` function
+(such as those using the GNU C Library),
+every encoding known to ``iconv`` for which Python has no built-in codec
+is available as well.
+Such an encoding is looked up by its ``iconv`` name (for example ``cp1133``).
+Prefixing the name with ``iconv:`` forces the use of the ``iconv``-based codec
+even when a built-in codec of the same name exists (for example ``iconv:latin1``),
+which is mostly useful for testing.
+
+.. versionchanged:: next
+   Added support for encodings provided by the C library's ``iconv``.
+
 .. impl-detail::
 
    Some common encodings can bypass the codecs lookup machinery to

Doc/library/imaplib.rst

@@ -192,6 +192,19 @@ In general, pass arguments unquoted and let the module quote them as needed.
 An argument that is already enclosed in double quotes is left unchanged,
 so that code which quotes arguments itself keeps working.
 
+Mailbox names are encoded as modified UTF-7 (:rfc:`3501`, section 5.1.3),
+so a mailbox name containing non-ASCII characters can be passed as an
+ordinary :class:`str`.
+A :class:`str` that is already valid modified UTF-7 is left unchanged,
+so that a name obtained from :meth:`~IMAP4.list` (raw ``bytes`` decoded to
+text) round-trips; pass :class:`bytes` to send the exact bytes with no
+encoding.
+When ``UTF8=ACCEPT`` is enabled (see :meth:`~IMAP4.enable`), mailbox names
+are sent as UTF-8 instead.
+
+.. versionchanged:: next
+   Non-ASCII mailbox names are automatically encoded as modified UTF-7.
+
 Most commands return a tuple: ``(type, [data, ...])`` where *type* is usually
 ``'OK'`` or ``'NO'``, and *data* is either the text from the command response,
 or mandated results from the command. Each *data* is either a ``bytes``, or a

Doc/library/locale.rst

@@ -342,6 +342,10 @@ The :mod:`!locale` module defines the following exception and functions:
    .. versionchanged:: 3.14
       The function now temporarily sets the ``LC_CTYPE`` locale in some cases.
 
+   .. versionchanged:: next
+      On glibc, the ``LC_TIME`` items (except ``ERA``) are now decoded
+      independently of the ``LC_CTYPE`` encoding.
+
 
 .. function:: getdefaultlocale([envvars])
 

Doc/library/tkinter.rst

@@ -2066,7 +2066,7 @@ Base and mixin classes
 
    .. method:: winfo_exists()
 
-      Return ``1`` if the widget exists, ``0`` otherwise.
+      Return ``True`` if the widget exists, ``False`` otherwise.
 
    .. method:: winfo_fpixels(number)
 
@@ -2115,7 +2115,7 @@ Base and mixin classes
 
    .. method:: winfo_ismapped()
 
-      Return ``1`` if the widget is currently mapped, ``0`` otherwise.
+      Return ``True`` if the widget is currently mapped, ``False`` otherwise.
 
    .. method:: winfo_manager()
 
@@ -2246,8 +2246,8 @@ Base and mixin classes
 
    .. method:: winfo_viewable()
 
-      Return ``1`` if the widget and all of its ancestors up through the
-      nearest toplevel window are mapped, ``0`` otherwise.
+      Return ``True`` if the widget and all of its ancestors up through the
+      nearest toplevel window are mapped, ``False`` otherwise.
 
    .. method:: winfo_visual()
 
@@ -5737,7 +5737,7 @@ Widget classes
    .. method:: edit_modified(arg=None)
 
       If *arg* is omitted, return the current state of the modified flag as
-      ``0`` or ``1``; the flag is set automatically whenever the text is
+      a :class:`bool`; the flag is set automatically whenever the text is
       inserted or deleted.
       Otherwise set the flag to the boolean *arg*.
 

Doc/library/tkinter.ttk.rst

@@ -1372,7 +1372,7 @@ ttk.Treeview
       Without arguments, returns a tuple of all detached items,
       but not their descendants (see :meth:`detached_all`).
       With *item*, returns whether *item* is detached; since Tk 9.1, also
-      returns true if an ancestor of *item* is detached.
+      returns ``True`` if an ancestor of *item* is detached.
 
       Requires Tk 9.0 or newer.
 

Doc/library/urllib.parse.rst

@@ -649,8 +649,9 @@ task isn't already covered by the URL parsing functions above.
 
    The optional *encoding* and *errors* parameters specify how to deal with
    non-ASCII characters, as accepted by the :meth:`str.encode` method.
-   *encoding* defaults to ``'utf-8'``.
-   *errors* defaults to ``'strict'``, meaning unsupported characters raise a
+   Although these parameters default to ``None`` in the function signature,
+   when processing :class:`str` inputs, *encoding* effectively defaults to ``'utf-8'``
+   and *errors* to ``'strict'``, meaning unsupported characters raise a
    :class:`UnicodeEncodeError`.
    *encoding* and *errors* must not be supplied if *string* is a
    :class:`bytes`, or a :class:`TypeError` is raised.

Doc/whatsnew/3.16.rst

@@ -86,6 +86,16 @@ New modules
 Improved modules
 ================
 
+codecs
+------
+
+* On platforms that provide the C library's :manpage:`iconv(3)` function,
+  every encoding known to ``iconv`` for which Python has no built-in codec
+  is now available (for example ``cp1133``).
+  Prefixing an encoding name with ``iconv:`` forces the ``iconv``-based codec
+  even when a built-in codec of the same name exists.
+  (Contributed by Serhiy Storchaka in :gh:`152997`.)
+
 curses
 ------
 
@@ -252,6 +262,11 @@ imaplib
   user names and passwords.
   (Contributed by Przemysław Buczkowski and Serhiy Storchaka in :gh:`89869`.)
 
+* Non-ASCII mailbox names are now automatically encoded as modified UTF-7
+  (:rfc:`3501`, section 5.1.3), so international mailbox names can be passed
+  as ordinary :class:`str`.
+  (Contributed by Serhiy Storchaka in :gh:`49555`.)
+
 
 ipaddress
 ---------

Include/internal/pycore_pylifecycle.h

@@ -108,9 +108,6 @@ extern int _Py_LegacyLocaleDetected(int warn);
 // Export for 'readline' shared extension
 PyAPI_FUNC(char*) _Py_SetLocaleFromEnv(int category);
 
-// Export for special main.c string compiling with source tracebacks
-int _PyRun_SimpleStringFlagsWithName(const char *command, const char* name, PyCompilerFlags *flags);
-
 
 /* interpreter config */
 

Include/internal/pycore_pythonrun.h

@@ -8,23 +8,18 @@ extern "C" {
 #  error "this header requires Py_BUILD_CORE define"
 #endif
 
-extern int _PyRun_SimpleFileObject(
+extern PyObject* _PyRun_SimpleFile(
     FILE *fp,
     PyObject *filename,
     int closeit,
     PyCompilerFlags *flags);
 
-extern int _PyRun_AnyFileObject(
+extern PyObject* _PyRun_AnyFile(
     FILE *fp,
     PyObject *filename,
     int closeit,
     PyCompilerFlags *flags);
 
-extern int _PyRun_InteractiveLoopObject(
-    FILE *fp,
-    PyObject *filename,
-    PyCompilerFlags *flags);
-
 extern int _PyObject_SupportedAsScript(PyObject *);
 extern const char* _Py_SourceAsString(
     PyObject *cmd,
@@ -39,6 +34,12 @@ extern PyObject * _Py_CompileStringObjectWithModule(
     PyCompilerFlags *flags, int optimize,
     PyObject *module);
 
+// Export for special main.c string compiling with source tracebacks
+extern PyObject* _PyRun_SimpleString(
+    const char *command,
+    PyObject* name,
+    PyCompilerFlags *flags);
+
 
 /* Stack size, in "pointers". This must be large enough, so
  * no two calls to check recursion depth are more than this far

Include/internal/pycore_unicodeobject.h

@@ -182,6 +182,22 @@ extern int _PyUnicodeWriter_FormatV(
     const char *format,
     va_list vargs);
 
+/* --- iconv Codec -------------------------------------------------------- */
+
+#ifdef HAVE_ICONV
+extern PyObject* _PyUnicode_DecodeIconv(
+    const char *encoding,       /* iconv encoding name */
+    const char *string,         /* encoded string */
+    Py_ssize_t length,          /* size of string */
+    const char *errors,         /* error handling */
+    Py_ssize_t *consumed);      /* bytes consumed, or NULL for non-stateful */
+
+extern PyObject* _PyUnicode_EncodeIconv(
+    const char *encoding,       /* iconv encoding name */
+    PyObject *unicode,          /* Unicode object */
+    const char *errors);        /* error handling */
+#endif
+
 /* --- UTF-7 Codecs ------------------------------------------------------- */
 
 extern PyObject* _PyUnicode_EncodeUTF7(