Merged main into regmachine branch

.github/workflows/build.yml

@@ -235,7 +235,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        openssl_ver: [1.1.1s, 3.0.7]
+        openssl_ver: [1.1.1s, 3.0.7, 3.1.0-beta1]
     env:
       OPENSSL_VER: ${{ matrix.openssl_ver }}
       MULTISSL_DIR: ${{ github.workspace }}/multissl

.github/workflows/doc.yml

@@ -50,18 +50,8 @@ jobs:
       run: make -C Doc/ venv
     - name: 'Check documentation'
       run: make -C Doc/ check
-    - name: 'Upload NEWS'
-      uses: actions/upload-artifact@v3
-      with:
-        name: NEWS
-        path: Doc/build/NEWS
     - name: 'Build HTML documentation'
       run: make -C Doc/ SPHINXOPTS="-q" SPHINXERRORHANDLING="-W --keep-going" html
-    - name: 'Upload docs'
-      uses: actions/upload-artifact@v3
-      with:
-        name: doc-html
-        path: Doc/build/html
 
   # Run "doctest" on HEAD as new syntax doesn't exist in the latest stable release
   doctest:

Doc/c-api/arg.rst

@@ -34,24 +34,39 @@ These formats allow accessing an object as a contiguous chunk of memory.
 You don't have to provide raw storage for the returned unicode or bytes
 area.
 
-In general, when a format sets a pointer to a buffer, the buffer is
-managed by the corresponding Python object, and the buffer shares
-the lifetime of this object.  You won't have to release any memory yourself.
-The only exceptions are ``es``, ``es#``, ``et`` and ``et#``.
-
-However, when a :c:type:`Py_buffer` structure gets filled, the underlying
-buffer is locked so that the caller can subsequently use the buffer even
-inside a :c:type:`Py_BEGIN_ALLOW_THREADS` block without the risk of mutable data
-being resized or destroyed.  As a result, **you have to call**
-:c:func:`PyBuffer_Release` after you have finished processing the data (or
-in any early abort case).
-
 Unless otherwise stated, buffers are not NUL-terminated.
 
-Some formats require a read-only :term:`bytes-like object`, and set a
-pointer instead of a buffer structure.  They work by checking that
-the object's :c:member:`PyBufferProcs.bf_releasebuffer` field is ``NULL``,
-which disallows mutable objects such as :class:`bytearray`.
+There are three ways strings and buffers can be converted to C:
+
+*  Formats such as ``y*`` and ``s*`` fill a :c:type:`Py_buffer` structure.
+   This locks the underlying buffer so that the caller can subsequently use
+   the buffer even inside a :c:type:`Py_BEGIN_ALLOW_THREADS`
+   block without the risk of mutable data being resized or destroyed.
+   As a result, **you have to call** :c:func:`PyBuffer_Release` after you have
+   finished processing the data (or in any early abort case).
+
+*  The ``es``, ``es#``, ``et`` and ``et#`` formats allocate the result buffer.
+   **You have to call** :c:func:`PyMem_Free` after you have finished
+   processing the data (or in any early abort case).
+
+*  .. _c-arg-borrowed-buffer:
+
+   Other formats take a :class:`str` or a read-only :term:`bytes-like object`,
+   such as :class:`bytes`, and provide a ``const char *`` pointer to
+   its buffer.
+   In this case the buffer is "borrowed": it is managed by the corresponding
+   Python object, and shares the lifetime of this object.
+   You won't have to release any memory yourself.
+
+   To ensure that the underlying buffer may be safely borrowed, the object's
+   :c:member:`PyBufferProcs.bf_releasebuffer` field must be ``NULL``.
+   This disallows common mutable objects such as :class:`bytearray`,
+   but also some read-only objects such as :class:`memoryview` of
+   :class:`bytes`.
+
+   Besides this ``bf_releasebuffer`` requirement, there is no check to verify
+   whether the input object is immutable (e.g. whether it would honor a request
+   for a writable buffer, or whether another thread can mutate the data).
 
 .. note::
 
@@ -89,7 +104,7 @@ which disallows mutable objects such as :class:`bytearray`.
    Unicode objects are converted to C strings using ``'utf-8'`` encoding.
 
 ``s#`` (:class:`str`, read-only :term:`bytes-like object`) [const char \*, :c:type:`Py_ssize_t`]
-   Like ``s*``, except that it doesn't accept mutable objects.
+   Like ``s*``, except that it provides a :ref:`borrowed buffer <c-arg-borrowed-buffer>`.
    The result is stored into two C variables,
    the first one a pointer to a C string, the second one its length.
    The string may contain embedded null bytes. Unicode objects are converted
@@ -108,8 +123,9 @@ which disallows mutable objects such as :class:`bytearray`.
    pointer is set to ``NULL``.
 
 ``y`` (read-only :term:`bytes-like object`) [const char \*]
-   This format converts a bytes-like object to a C pointer to a character
-   string; it does not accept Unicode objects.  The bytes buffer must not
+   This format converts a bytes-like object to a C pointer to a
+   :ref:`borrowed <c-arg-borrowed-buffer>` character string;
+   it does not accept Unicode objects.  The bytes buffer must not
    contain embedded null bytes; if it does, a :exc:`ValueError`
    exception is raised.
 

Doc/faq/programming.rst

@@ -113,6 +113,8 @@ Yes.  The coding style required for standard library modules is documented as
 Core Language
 =============
 
+.. _faq-unboundlocalerror:
+
 Why am I getting an UnboundLocalError when the variable has a value?
 --------------------------------------------------------------------
 

Doc/howto/annotations.rst

@@ -57,6 +57,12 @@ Accessing The Annotations Dict Of An Object In Python 3.10 And Newer
   newer is to call :func:`getattr` with three arguments,
   for example ``getattr(o, '__annotations__', None)``.
 
+  Before Python 3.10, accessing ``__annotations__`` on a class that
+  defines no annotations but that has a parent class with
+  annotations would return the parent's ``__annotations__``.
+  In Python 3.10 and newer, the child class's annotations
+  will be an empty dict instead.
+
 
 Accessing The Annotations Dict Of An Object In Python 3.9 And Older
 ===================================================================

Doc/library/argparse.rst

@@ -765,7 +765,7 @@ The add_argument() method
 
    * type_ - The type to which the command-line argument should be converted.
 
-   * choices_ - A container of the allowable values for the argument.
+   * choices_ - A sequence of the allowable values for the argument.
 
    * required_ - Whether or not the command-line option may be omitted
      (optionals only).
@@ -1209,7 +1209,7 @@ choices
 ^^^^^^^
 
 Some command-line arguments should be selected from a restricted set of values.
-These can be handled by passing a container object as the *choices* keyword
+These can be handled by passing a sequence object as the *choices* keyword
 argument to :meth:`~ArgumentParser.add_argument`.  When the command line is
 parsed, argument values will be checked, and an error message will be displayed
 if the argument was not one of the acceptable values::
@@ -1223,9 +1223,9 @@ if the argument was not one of the acceptable values::
    game.py: error: argument move: invalid choice: 'fire' (choose from 'rock',
    'paper', 'scissors')
 
-Note that inclusion in the *choices* container is checked after any type_
+Note that inclusion in the *choices* sequence is checked after any type_
 conversions have been performed, so the type of the objects in the *choices*
-container should match the type_ specified::
+sequence should match the type_ specified::
 
    >>> parser = argparse.ArgumentParser(prog='doors.py')
    >>> parser.add_argument('door', type=int, choices=range(1, 4))
@@ -1235,8 +1235,8 @@ container should match the type_ specified::
    usage: doors.py [-h] {1,2,3}
    doors.py: error: argument door: invalid choice: 4 (choose from 1, 2, 3)
 
-Any container can be passed as the *choices* value, so :class:`list` objects,
-:class:`set` objects, and custom containers are all supported.
+Any sequence can be passed as the *choices* value, so :class:`list` objects,
+:class:`tuple` objects, and custom sequences are all supported.
 
 Use of :class:`enum.Enum` is not recommended because it is difficult to
 control its appearance in usage, help, and error messages.

Doc/library/compileall.rst

@@ -199,7 +199,7 @@ Public functions
 
    The *stripdir*, *prependdir* and *limit_sl_dest* arguments correspond to
    the ``-s``, ``-p`` and ``-e`` options described above.
-   They may be specified as ``str``, ``bytes`` or :py:class:`os.PathLike`.
+   They may be specified as ``str`` or :py:class:`os.PathLike`.
 
    If *hardlink_dupes* is true and two ``.pyc`` files with different optimization
    level have the same content, use hard links to consolidate duplicate files.
@@ -269,7 +269,7 @@ Public functions
 
    The *stripdir*, *prependdir* and *limit_sl_dest* arguments correspond to
    the ``-s``, ``-p`` and ``-e`` options described above.
-   They may be specified as ``str``, ``bytes`` or :py:class:`os.PathLike`.
+   They may be specified as ``str`` or :py:class:`os.PathLike`.
 
    If *hardlink_dupes* is true and two ``.pyc`` files with different optimization
    level have the same content, use hard links to consolidate duplicate files.

Doc/library/contextvars.rst

@@ -144,6 +144,11 @@ Manual Context Management
    To get a copy of the current context use the
    :func:`~contextvars.copy_context` function.
 
+   Every thread will have a different top-level :class:`~contextvars.Context`
+   object. This means that a :class:`ContextVar` object behaves in a similar
+   fashion to :func:`threading.local()` when values are assigned in different
+   threads.
+
    Context implements the :class:`collections.abc.Mapping` interface.
 
    .. method:: run(callable, *args, **kwargs)

Doc/library/functions.rst

@@ -650,20 +650,23 @@ are always available.  They are listed here in alphabetical order.
    sign may be ``'+'`` or ``'-'``; a ``'+'`` sign has no effect on the value
    produced.  The argument may also be a string representing a NaN
    (not-a-number), or positive or negative infinity.  More precisely, the
-   input must conform to the following grammar after leading and trailing
-   whitespace characters are removed:
+   input must conform to the ``floatvalue`` production rule in the following
+   grammar, after leading and trailing whitespace characters are removed:
 
    .. productionlist:: float
       sign: "+" | "-"
       infinity: "Infinity" | "inf"
       nan: "nan"
-      numeric_value: `floatnumber` | `infinity` | `nan`
-      numeric_string: [`sign`] `numeric_value`
+      digitpart: `digit` (["_"] `digit`)*
+      number: [`digitpart`] "." `digitpart` | `digitpart` ["."]
+      exponent: ("e" | "E") ["+" | "-"] `digitpart`
+      floatnumber: number [`exponent`]
+      floatvalue: [`sign`] (`floatnumber` | `infinity` | `nan`)
 
-   Here ``floatnumber`` is the form of a Python floating-point literal,
-   described in :ref:`floating`.  Case is not significant, so, for example,
-   "inf", "Inf", "INFINITY", and "iNfINity" are all acceptable spellings for
-   positive infinity.
+   Here ``digit`` is a Unicode decimal digit (character in the Unicode general
+   category ``Nd``). Case is not significant, so, for example, "inf", "Inf",
+   "INFINITY", and "iNfINity" are all acceptable spellings for positive
+   infinity.
 
    Otherwise, if the argument is an integer or a floating point number, a
    floating point number with the same value (within Python's floating point
@@ -1733,6 +1736,10 @@ are always available.  They are listed here in alphabetical order.
    .. versionchanged:: 3.8
       The *start* parameter can be specified as a keyword argument.
 
+   .. versionchanged:: 3.12 Summation of floats switched to an algorithm
+      that gives higher accuracy on most builds.
+
+
 .. class:: super()
            super(type, object_or_type=None)
 

Doc/library/imaplib.rst

@@ -187,7 +187,7 @@ IMAP4 Objects
 -------------
 
 All IMAP4rev1 commands are represented by methods of the same name, either
-upper-case or lower-case.
+uppercase or lowercase.
 
 All arguments to commands are converted to strings, except for ``AUTHENTICATE``,
 and the last argument to ``APPEND`` which is passed as an IMAP4 literal.  If

Doc/library/inspect.rst

@@ -343,15 +343,36 @@ attributes (see :ref:`import-mod-attrs` for module attributes):
 
 .. function:: iscoroutinefunction(object)
 
-   Return ``True`` if the object is a :term:`coroutine function`
-   (a function defined with an :keyword:`async def` syntax).
+   Return ``True`` if the object is a :term:`coroutine function` (a function
+   defined with an :keyword:`async def` syntax), a :func:`functools.partial`
+   wrapping a :term:`coroutine function`, or a sync function marked with
+   :func:`markcoroutinefunction`.
 
    .. versionadded:: 3.5
 
    .. versionchanged:: 3.8
       Functions wrapped in :func:`functools.partial` now return ``True`` if the
       wrapped function is a :term:`coroutine function`.
 
+   .. versionchanged:: 3.12
+      Sync functions marked with :func:`markcoroutinefunction` now return
+      ``True``.
+
+
+.. function:: markcoroutinefunction(func)
+
+   Decorator to mark a callable as a :term:`coroutine function` if it would not
+   otherwise be detected by :func:`iscoroutinefunction`.
+
+   This may be of use for sync functions that return a :term:`coroutine`, if
+   the function is passed to an API that requires :func:`iscoroutinefunction`.
+
+   When possible, using an :keyword:`async def` function is preferred. Also
+   acceptable is calling the function and testing the return with
+   :func:`iscoroutine`.
+
+   .. versionadded:: 3.12
+
 
 .. function:: iscoroutine(object)
 

Doc/library/itertools.rst

@@ -788,6 +788,11 @@ which incur interpreter overhead.
 
 .. testcode::
 
+   import collections
+   import math
+   import operator
+   import random
+
    def take(n, iterable):
        "Return first n items of the iterable as a list"
        return list(islice(iterable, n))
@@ -834,7 +839,8 @@ which incur interpreter overhead.
        return chain.from_iterable(repeat(tuple(iterable), n))
 
    def dotproduct(vec1, vec2):
-       return sum(map(operator.mul, vec1, vec2))
+       "Compute a sum of products."
+       return sum(starmap(operator.mul, zip(vec1, vec2, strict=True)))
 
    def convolve(signal, kernel):
        # See:  https://betterexplained.com/articles/intuitive-convolution/
@@ -846,7 +852,7 @@ which incur interpreter overhead.
        window = collections.deque([0], maxlen=n) * n
        for x in chain(signal, repeat(0, n-1)):
            window.append(x)
-           yield sum(map(operator.mul, kernel, window))
+           yield dotproduct(kernel, window)
 
    def polynomial_from_roots(roots):
        """Compute a polynomial's coefficients from its roots.
@@ -891,6 +897,21 @@ which incur interpreter overhead.
        data[2] = 1
        return iter_index(data, 1) if n > 2 else iter([])
 
+   def factor(n):
+       "Prime factors of n."
+       # factor(97) --> 97
+       # factor(98) --> 2 7 7
+       # factor(99) --> 3 3 11
+       for prime in sieve(n+1):
+           while True:
+               quotient, remainder = divmod(n, prime)
+               if remainder:
+                   break
+               yield prime
+               n = quotient
+               if n == 1:
+                   return
+
    def flatten(list_of_lists):
        "Flatten one level of nesting"
        return chain.from_iterable(list_of_lists)
@@ -1133,11 +1154,6 @@ which incur interpreter overhead.
 
     Now, we test all of the itertool recipes
 
-    >>> import operator
-    >>> import collections
-    >>> import math
-    >>> import random
-
     >>> take(10, count())
     [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
 
@@ -1250,6 +1266,35 @@ which incur interpreter overhead.
     >>> set(sieve(10_000)).isdisjoint(carmichael)
     True
 
+    list(factor(0))
+    []
+    list(factor(1))
+    []
+    list(factor(2))
+    [2]
+    list(factor(3))
+    [3]
+    list(factor(4))
+    [2, 2]
+    list(factor(5))
+    [5]
+    list(factor(6))
+    [2, 3]
+    list(factor(7))
+    [7]
+    list(factor(8))
+    [2, 2, 2]
+    list(factor(9))
+    [3, 3]
+    list(factor(10))
+    [2, 5]
+    all(math.prod(factor(n)) == n for n in range(1, 1000))
+    True
+    all(set(factor(n)) <= set(sieve(n+1)) for n in range(1, 1000))
+    True
+    all(list(factor(n)) == sorted(factor(n)) for n in range(1, 1000))
+    True
+
     >>> list(flatten([('a', 'b'), (), ('c', 'd', 'e'), ('f',), ('g', 'h', 'i')]))
     ['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i']
 

Doc/library/lzma.rst

@@ -147,7 +147,7 @@ Compressing and decompressing data in memory
       This format is more limited than ``.xz`` -- it does not support integrity
       checks or multiple filters.
 
-   * :const:`FORMAT_RAW`: A raw data stream, not using sequences format.
+   * :const:`FORMAT_RAW`: A raw data stream, not using any container format.
       This format specifier does not support integrity checks, and requires that
       you always specify a custom filter chain (for both compression and
       decompression). Additionally, data compressed in this manner cannot be