diff --git a/Doc/library/urllib.parse.rst b/Doc/library/urllib.parse.rst index 26e159306c81070..c8d6069b9fccce1 100644 --- a/Doc/library/urllib.parse.rst +++ b/Doc/library/urllib.parse.rst @@ -44,7 +44,7 @@ which was introduced in :rfc:`1808`. However, this term has been obsoleted by :rfc:`3986`, which introduced the term ``authority`` as its replacement. The use of ``netloc`` is continued for backward compatibility. -URL Parsing +URL parsing ----------- The URL parsing functions focus on splitting a URL string into its components, @@ -130,7 +130,8 @@ or on combining URL components into a URL string. If the *allow_fragments* argument is false, fragment identifiers are not recognized. Instead, they are parsed as part of the path - or query component, and :attr:`fragment` is set to ``None`` or the empty + or query component, and :attr:`~SplitResult.fragment` is set to + ``None`` or the empty string (depending on the value of *missing_as_none*) in the return value. The return value is a :term:`named tuple`, which means that its items can @@ -139,37 +140,40 @@ or on combining URL components into a URL string. +------------------+-------+-------------------------+-------------------------------+ | Attribute | Index | Value | Value if not present | +==================+=======+=========================+===============================+ - | :attr:`scheme` | 0 | URL scheme specifier | *scheme* parameter or | + | |split-scheme| | 0 | URL scheme specifier | *scheme* parameter or | | | | | empty string [1]_ | +------------------+-------+-------------------------+-------------------------------+ - | :attr:`netloc` | 1 | Network location part | ``None`` or empty string [1]_ | + | |split-netloc| | 1 | Network location part | ``None`` or empty string [1]_ | +------------------+-------+-------------------------+-------------------------------+ - | :attr:`path` | 2 | Hierarchical path | empty string | + | |split-path| | 2 | Hierarchical path | empty string | +------------------+-------+-------------------------+-------------------------------+ - | :attr:`query` | 3 | Query component | ``None`` or empty string [1]_ | + | |split-query| | 3 | Query component | ``None`` or empty string [1]_ | +------------------+-------+-------------------------+-------------------------------+ - | :attr:`fragment` | 4 | Fragment identifier | ``None`` or empty string [1]_ | + | |split-frag| | 4 | Fragment identifier | ``None`` or empty string [1]_ | +------------------+-------+-------------------------+-------------------------------+ - | :attr:`username` | | User name | ``None`` | + | |split-username| | | User name | ``None`` | +------------------+-------+-------------------------+-------------------------------+ - | :attr:`password` | | Password | ``None`` | + | |split-password| | | Password | ``None`` | +------------------+-------+-------------------------+-------------------------------+ - | :attr:`hostname` | | Host name (lower case) | ``None`` | + | |split-hostname| | | Host name (lower case) | ``None`` | +------------------+-------+-------------------------+-------------------------------+ - | :attr:`port` | | Port number as integer, | ``None`` | + | |split-port| | | Port number as integer, | ``None`` | | | | if present | | +------------------+-------+-------------------------+-------------------------------+ .. [1] Depending on the value of the *missing_as_none* argument. - Reading the :attr:`port` attribute will raise a :exc:`ValueError` if + Reading the :attr:`~SplitResult.port` attribute will raise a + :exc:`ValueError` if an invalid port is specified in the URL. See section :ref:`urlparse-result-object` for more information on the result object. - Unmatched square brackets in the :attr:`netloc` attribute will raise a + Unmatched square brackets in the :attr:`~SplitResult.netloc` + attribute will raise a :exc:`ValueError`. - Characters in the :attr:`netloc` attribute that decompose under NFKC + Characters in the :attr:`~SplitResult.netloc` attribute that + decompose under NFKC normalization (as used by the IDNA encoding) into any of ``/``, ``?``, ``#``, ``@``, or ``:`` will raise a :exc:`ValueError`. If the URL is decomposed before parsing, no error will be raised. @@ -179,8 +183,10 @@ or on combining URL components into a URL string. ``\r`` and tab ``\t`` characters are removed from the URL at any position. As is the case with all named tuples, the subclass has a few additional methods - and attributes that are particularly useful. One such method is :meth:`_replace`. - The :meth:`_replace` method will return a new :class:`SplitResult` object + and attributes that are particularly useful. One such method is + :meth:`~SplitResult._replace`. + The :meth:`~SplitResult._replace` method will return a new + :class:`SplitResult` object replacing specified fields with new values. .. doctest:: @@ -436,9 +442,9 @@ or on combining URL components into a URL string. +------------------+-------+-------------------------+-------------------------------+ | Attribute | Index | Value | Value if not present | +==================+=======+=========================+===============================+ - | :attr:`url` | 0 | URL with no fragment | empty string | + | |defrag-url| | 0 | URL with no fragment | empty string | +------------------+-------+-------------------------+-------------------------------+ - | :attr:`fragment` | 1 | Fragment identifier | ``None`` or empty string [3]_ | + | |defrag-frag| | 1 | Fragment identifier | ``None`` or empty string [3]_ | +------------------+-------+-------------------------+-------------------------------+ .. [3] Depending on the value of the *missing_as_none* argument. @@ -491,7 +497,7 @@ to be very cautious about making API behavior changes. .. _parsing-ascii-encoded-bytes: -Parsing ASCII Encoded Bytes +Parsing ASCII encoded bytes --------------------------- The URL parsing functions were originally designed to operate on character @@ -511,14 +517,15 @@ byte values will trigger :exc:`UnicodeDecodeError`. To support easier conversion of result objects between :class:`str` and :class:`bytes`, all return values from URL parsing functions provide -either an :meth:`encode` method (when the result contains :class:`str` -data) or a :meth:`decode` method (when the result contains :class:`bytes` +either an :meth:`~DefragResult.encode` method (when the result contains +:class:`str` data) or a :meth:`~DefragResultBytes.decode` method (when +the result contains :class:`bytes` data). The signatures of these methods match those of the corresponding :class:`str` and :class:`bytes` methods (except that the default encoding is ``'ascii'`` rather than ``'utf-8'``). Each produces a value of a corresponding type that contains either :class:`bytes` data (for -:meth:`encode` methods) or :class:`str` data (for -:meth:`decode` methods). +:meth:`~DefragResult.encode` methods) or :class:`str` data (for +:meth:`~DefragResultBytes.decode` methods). Applications that need to operate on potentially improperly quoted URLs that may contain non-ASCII data will need to do their own decoding from @@ -535,14 +542,31 @@ individual URL quoting functions. .. _urlparse-result-object: -Structured Parse Results +Structured parse results ------------------------ The result objects from the :func:`urlsplit`, :func:`urlparse` and :func:`urldefrag` functions are subclasses of the :class:`tuple` type. These subclasses add the attributes listed in the documentation for those functions, the encoding and decoding support described in the -previous section, as well as an additional method: +previous section, as well as additional methods: + +.. |split-scheme| replace:: :attr:`~SplitResult.scheme` +.. |split-netloc| replace:: :attr:`~SplitResult.netloc` +.. |split-path| replace:: :attr:`~SplitResult.path` +.. |split-query| replace:: :attr:`~SplitResult.query` +.. |split-frag| replace:: :attr:`~SplitResult.fragment` +.. |split-username| replace:: :attr:`~SplitResult.username` +.. |split-password| replace:: :attr:`~SplitResult.password` +.. |split-hostname| replace:: :attr:`~SplitResult.hostname` +.. |split-port| replace:: :attr:`~SplitResult.port` +.. |defrag-url| replace:: :attr:`~DefragResult.url` +.. |defrag-frag| replace:: :attr:`~DefragResult.fragment` + +.. method:: SplitResult._replace(**kwargs) + + Return a new structured parse result replacing specified fields with new + values. .. method:: urllib.parse.SplitResult.geturl() @@ -578,22 +602,75 @@ results when operating on :class:`str` objects: .. class:: DefragResult(url, fragment) Concrete class for :func:`urldefrag` results containing :class:`str` - data. The :meth:`encode` method returns a :class:`DefragResultBytes` - instance. + data. + + .. attribute:: url + + URL with no fragment. + + .. attribute:: fragment + + Fragment identifier. + + .. method:: encode(encoding='ascii', errors='strict') + + Return a :class:`DefragResultBytes` instance. .. versionadded:: 3.2 .. class:: ParseResult(scheme, netloc, path, params, query, fragment) Concrete class for :func:`urlparse` results containing :class:`str` - data. The :meth:`encode` method returns a :class:`ParseResultBytes` - instance. + data. + + .. method:: encode(encoding='ascii', errors='strict') + + Return a :class:`ParseResultBytes` instance. .. class:: SplitResult(scheme, netloc, path, query, fragment) Concrete class for :func:`urlsplit` results containing :class:`str` - data. The :meth:`encode` method returns a :class:`SplitResultBytes` - instance. + data. + + .. attribute:: scheme + + URL scheme specifier. + + .. attribute:: netloc + + Network location part. + + .. attribute:: path + + Hierarchical path. + + .. attribute:: query + + Query component. + + .. attribute:: fragment + + Fragment identifier. + + .. attribute:: username + + User name. + + .. attribute:: password + + Password. + + .. attribute:: hostname + + Host name, lower case. + + .. attribute:: port + + Port number as integer, if present. + + .. method:: encode(encoding='ascii', errors='strict') + + Return a :class:`SplitResultBytes` instance. The following classes provide the implementations of the parse results when @@ -602,29 +679,38 @@ operating on :class:`bytes` or :class:`bytearray` objects: .. class:: DefragResultBytes(url, fragment) Concrete class for :func:`urldefrag` results containing :class:`bytes` - data. The :meth:`decode` method returns a :class:`DefragResult` - instance. + data. + + .. method:: decode(encoding='ascii', errors='strict') + + Return a :class:`DefragResult` instance. .. versionadded:: 3.2 .. class:: ParseResultBytes(scheme, netloc, path, params, query, fragment) Concrete class for :func:`urlparse` results containing :class:`bytes` - data. The :meth:`decode` method returns a :class:`ParseResult` - instance. + data. + + .. method:: decode(encoding='ascii', errors='strict') + + Return a :class:`ParseResult` instance. .. versionadded:: 3.2 .. class:: SplitResultBytes(scheme, netloc, path, query, fragment) Concrete class for :func:`urlsplit` results containing :class:`bytes` - data. The :meth:`decode` method returns a :class:`SplitResult` - instance. + data. + + .. method:: decode(encoding='ascii', errors='strict') + + Return a :class:`SplitResult` instance. .. versionadded:: 3.2 -URL Quoting +URL quoting ----------- The URL quoting functions focus on taking program data and making it safe diff --git a/Doc/library/urllib.request.rst b/Doc/library/urllib.request.rst index 95e4d2627c8b23b..062cb5c0d7424cc 100644 --- a/Doc/library/urllib.request.rst +++ b/Doc/library/urllib.request.rst @@ -138,8 +138,9 @@ The :mod:`!urllib.request` module defines the following functions: If the Python installation has SSL support (i.e., if the :mod:`ssl` module can be imported), :class:`HTTPSHandler` will also be added. - A :class:`BaseHandler` subclass may also change its :attr:`handler_order` - attribute to modify its position in the handlers list. + A :class:`BaseHandler` subclass may also change its + :attr:`~BaseHandler.handler_order` attribute to modify its position in the + handlers list. .. function:: pathname2url(path, *, add_scheme=False) @@ -358,10 +359,12 @@ The following classes are provided: To disable autodetected proxy pass an empty dictionary. - The :envvar:`no_proxy` environment variable can be used to specify hosts - which shouldn't be reached via proxy; if set, it should be a comma-separated - list of hostname suffixes, optionally with ``:port`` appended, for example - ``cern.ch,ncsa.uiuc.edu,some.host:8080``. + .. envvar:: no_proxy + + This variable can be used to specify hosts which shouldn't be reached via + proxy; if set, it should be a comma-separated list of hostname suffixes, + optionally with ``:port`` appended, for example + ``cern.ch,ncsa.uiuc.edu,some.host:8080``. .. note:: @@ -514,7 +517,7 @@ The following classes are provided: .. _request-objects: -Request Objects +Request objects --------------- The following methods describe :class:`Request`'s public interface, @@ -656,7 +659,7 @@ request. .. _opener-director-objects: -OpenerDirector Objects +OpenerDirector objects ---------------------- :class:`OpenerDirector` instances have the following methods: @@ -667,9 +670,9 @@ OpenerDirector Objects *handler* should be an instance of :class:`BaseHandler`. The following methods are searched, and added to the possible chains (note that HTTP errors are a special case). Note that, in the following, *protocol* should be replaced - with the actual protocol to handle, for example :meth:`http_response` would + with the actual protocol to handle, for example :meth:`!http_response` would be the HTTP protocol response handler. Also *type* should be replaced with - the actual HTTP code, for example :meth:`http_error_404` would handle HTTP + the actual HTTP code, for example :meth:`!http_error_404` would handle HTTP 404 errors. * :meth:`!_open` --- signal that the handler knows how to open *protocol* @@ -695,10 +698,10 @@ OpenerDirector Objects See |protocol_response|_ for more information. -.. |protocol_open| replace:: :meth:`BaseHandler._open` -.. |http_error_nnn| replace:: :meth:`BaseHandler.http_error_\` -.. |protocol_request| replace:: :meth:`BaseHandler._request` -.. |protocol_response| replace:: :meth:`BaseHandler._response` +.. |protocol_open| replace:: :meth:`!BaseHandler._open` +.. |http_error_nnn| replace:: :meth:`!BaseHandler.http_error_\` +.. |protocol_request| replace:: :meth:`!BaseHandler._request` +.. |protocol_response| replace:: :meth:`!BaseHandler._response` .. method:: OpenerDirector.open(url, data=None[, timeout]) @@ -751,7 +754,7 @@ sorting the handler instances. .. _base-handler-objects: -BaseHandler Objects +BaseHandler objects ------------------- :class:`BaseHandler` objects provide a couple of methods that are directly @@ -784,6 +787,11 @@ The following attribute and methods should only be used by classes derived from protocol, or handle errors. +.. attribute:: BaseHandler.handler_order + + The order in which the handler is called within an opener chain. + + .. method:: BaseHandler.default_open(req) This method is *not* defined in :class:`BaseHandler`, but subclasses should @@ -881,7 +889,7 @@ The following attribute and methods should only be used by classes derived from .. _http-redirect-handler: -HTTPRedirectHandler Objects +HTTPRedirectHandler objects --------------------------- .. note:: @@ -948,7 +956,7 @@ HTTPRedirectHandler Objects .. _http-cookie-processor: -HTTPCookieProcessor Objects +HTTPCookieProcessor objects --------------------------- :class:`HTTPCookieProcessor` instances have one attribute: @@ -960,7 +968,7 @@ HTTPCookieProcessor Objects .. _proxy-handler: -ProxyHandler Objects +ProxyHandler objects -------------------- @@ -976,7 +984,7 @@ ProxyHandler Objects .. _http-password-mgr: -HTTPPasswordMgr Objects +HTTPPasswordMgr objects ----------------------- These methods are available on :class:`HTTPPasswordMgr` and @@ -1002,7 +1010,7 @@ These methods are available on :class:`HTTPPasswordMgr` and .. _http-password-mgr-with-prior-auth: -HTTPPasswordMgrWithPriorAuth Objects +HTTPPasswordMgrWithPriorAuth objects ------------------------------------ This password manager extends :class:`HTTPPasswordMgrWithDefaultRealm` to support @@ -1038,7 +1046,7 @@ tracking URIs for which authentication credentials should always be sent. .. _abstract-basic-auth-handler: -AbstractBasicAuthHandler Objects +AbstractBasicAuthHandler objects -------------------------------- @@ -1058,7 +1066,7 @@ AbstractBasicAuthHandler Objects .. _http-basic-auth-handler: -HTTPBasicAuthHandler Objects +HTTPBasicAuthHandler objects ---------------------------- @@ -1069,7 +1077,7 @@ HTTPBasicAuthHandler Objects .. _proxy-basic-auth-handler: -ProxyBasicAuthHandler Objects +ProxyBasicAuthHandler objects ----------------------------- @@ -1080,7 +1088,7 @@ ProxyBasicAuthHandler Objects .. _abstract-digest-auth-handler: -AbstractDigestAuthHandler Objects +AbstractDigestAuthHandler objects --------------------------------- @@ -1094,7 +1102,7 @@ AbstractDigestAuthHandler Objects .. _http-digest-auth-handler: -HTTPDigestAuthHandler Objects +HTTPDigestAuthHandler objects ----------------------------- @@ -1105,7 +1113,7 @@ HTTPDigestAuthHandler Objects .. _proxy-digest-auth-handler: -ProxyDigestAuthHandler Objects +ProxyDigestAuthHandler objects ------------------------------ @@ -1116,7 +1124,7 @@ ProxyDigestAuthHandler Objects .. _http-handler-objects: -HTTPHandler Objects +HTTPHandler objects ------------------- @@ -1128,7 +1136,7 @@ HTTPHandler Objects .. _https-handler-objects: -HTTPSHandler Objects +HTTPSHandler objects -------------------- @@ -1140,7 +1148,7 @@ HTTPSHandler Objects .. _file-handler-objects: -FileHandler Objects +FileHandler objects ------------------- @@ -1156,7 +1164,7 @@ FileHandler Objects .. _data-handler-objects: -DataHandler Objects +DataHandler objects ------------------- .. method:: DataHandler.data_open(req) @@ -1171,7 +1179,7 @@ DataHandler Objects .. _ftp-handler-objects: -FTPHandler Objects +FTPHandler objects ------------------ @@ -1183,7 +1191,7 @@ FTPHandler Objects .. _cacheftp-handler-objects: -CacheFTPHandler Objects +CacheFTPHandler objects ----------------------- :class:`CacheFTPHandler` objects are :class:`FTPHandler` objects with the @@ -1202,7 +1210,7 @@ following additional methods: .. _unknown-handler-objects: -UnknownHandler Objects +UnknownHandler objects ---------------------- @@ -1213,7 +1221,7 @@ UnknownHandler Objects .. _http-error-processor-objects: -HTTPErrorProcessor Objects +HTTPErrorProcessor objects -------------------------- .. method:: HTTPErrorProcessor.http_response(request, response) @@ -1490,7 +1498,7 @@ some point in the future. installed by :func:`install_opener`. -:mod:`!urllib.request` Restrictions +:mod:`!urllib.request` restrictions ----------------------------------- .. index:: diff --git a/Doc/tools/.nitignore b/Doc/tools/.nitignore index c5d474895966119..8f66c0cf1285d57 100644 --- a/Doc/tools/.nitignore +++ b/Doc/tools/.nitignore @@ -23,8 +23,6 @@ Doc/library/socket.rst Doc/library/ssl.rst Doc/library/termios.rst Doc/library/test.rst -Doc/library/urllib.parse.rst -Doc/library/urllib.request.rst Doc/library/wsgiref.rst Doc/library/xml.dom.minidom.rst Doc/library/xml.dom.pulldom.rst