From e8383839022201ac628c208f425efbcd5989b6e0 Mon Sep 17 00:00:00 2001 From: Baptiste Mispelon Date: Sat, 24 Oct 2026 00:23:56 +0200 Subject: [PATCH] [3.14] gh-83156: Add dedicated anchor for regexp syntax elements (GH-17470) (cherry picked from commit 321925c92c9e67864cafa1566840927f6f07e90a) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Baptiste Mispelon Co-authored-by: Muse Code (Meta Muse Spark) Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com> Co-authored-by: Jelle Zijlstra Co-authored-by: Alex Waygood Co-authored-by: Lukasz Langa Co-authored-by: Łukasz Langa --- Doc/library/re.rst | 942 ++++++++++++++++++++++++++------------------- 1 file changed, 555 insertions(+), 387 deletions(-) diff --git a/Doc/library/re.rst b/Doc/library/re.rst index 456d74292ef4a7e..44e1960e2ce293f 100644 --- a/Doc/library/re.rst +++ b/Doc/library/re.rst @@ -1,3 +1,5 @@ +:tocdepth: 3 + :mod:`!re` --- Regular expression operations ============================================ @@ -94,584 +96,746 @@ repetition to an inner repetition, parentheses may be used. For example, the expression ``(?:a{6})*`` matches any multiple of six ``'a'`` characters. +Special characters +^^^^^^^^^^^^^^^^^^ + The special characters are: .. index:: single: . (dot); in regular expressions + :name: re-syntax-dot ``.`` - (Dot.) In the default mode, this matches any character except a newline. If - the :const:`DOTALL` flag has been specified, this matches any character - including a newline. ``(?s:.)`` matches any character regardless of flags. +""""" + +(Dot.) In the default mode, this matches any character except a newline. If +the :const:`DOTALL` flag has been specified, this matches any character +including a newline. ``(?s:.)`` matches any character regardless of flags. .. index:: single: ^ (caret); in regular expressions + :name: re-syntax-caret ``^`` - (Caret.) Matches the start of the string, and in :const:`MULTILINE` mode also - matches immediately after each newline. +""""" + +(Caret.) Matches the start of the string, and in :const:`MULTILINE` mode also +matches immediately after each newline. + .. index:: single: $ (dollar); in regular expressions + :name: re-syntax-dollar ``$`` - Matches the end of the string or just before the newline at the end of the - string, and in :const:`MULTILINE` mode also matches before a newline. ``foo`` - matches both 'foo' and 'foobar', while the regular expression ``foo$`` matches - only 'foo'. More interestingly, searching for ``foo.$`` in ``'foo1\nfoo2\n'`` - matches 'foo2' normally, but 'foo1' in :const:`MULTILINE` mode; searching for - a single ``$`` in ``'foo\n'`` will find two (empty) matches: one just before - the newline, and one at the end of the string. +""""" + +Matches the end of the string or just before the newline at the end of the +string, and in :const:`MULTILINE` mode also matches before a newline. ``foo`` +matches both 'foo' and 'foobar', while the regular expression ``foo$`` matches +only 'foo'. More interestingly, searching for ``foo.$`` in ``'foo1\nfoo2\n'`` +matches 'foo2' normally, but 'foo1' in :const:`MULTILINE` mode; searching for +a single ``$`` in ``'foo\n'`` will find two (empty) matches: one just before +the newline, and one at the end of the string. + .. index:: single: * (asterisk); in regular expressions + :name: re-syntax-asterisk ``*`` - Causes the resulting RE to match 0 or more repetitions of the preceding RE, as - many repetitions as are possible. ``ab*`` will match 'a', 'ab', or 'a' followed - by any number of 'b's. +""""" + +Causes the resulting RE to match 0 or more repetitions of the preceding RE, as +many repetitions as are possible. ``ab*`` will match 'a', 'ab', or 'a' followed +by any number of 'b's. + .. index:: single: + (plus); in regular expressions + :name: re-syntax-plus ``+`` - Causes the resulting RE to match 1 or more repetitions of the preceding RE. - ``ab+`` will match 'a' followed by any non-zero number of 'b's; it will not - match just 'a'. +""""" + +Causes the resulting RE to match 1 or more repetitions of the preceding RE. +``ab+`` will match 'a' followed by any non-zero number of 'b's; it will not +match just 'a'. + .. index:: single: ? (question mark); in regular expressions + :name: re-syntax-question-mark ``?`` - Causes the resulting RE to match 0 or 1 repetitions of the preceding RE. - ``ab?`` will match either 'a' or 'ab'. +""""" + +Causes the resulting RE to match 0 or 1 repetitions of the preceding RE. +``ab?`` will match either 'a' or 'ab'. + .. index:: single: *?; in regular expressions single: +?; in regular expressions single: ??; in regular expressions + :name: re-syntax-non-greedy-quantifiers ``*?``, ``+?``, ``??`` - The ``'*'``, ``'+'``, and ``'?'`` quantifiers are all :dfn:`greedy`; they match - as much text as possible. Sometimes this behaviour isn't desired; if the RE - ``<.*>`` is matched against ``' b '``, it will match the entire - string, and not just ``''``. Adding ``?`` after the quantifier makes it - perform the match in :dfn:`non-greedy` or :dfn:`minimal` fashion; as *few* - characters as possible will be matched. Using the RE ``<.*?>`` will match - only ``''``. +"""""""""""""""""""""" + +The ``'*'``, ``'+'``, and ``'?'`` quantifiers are all :dfn:`greedy`; they match +as much text as possible. Sometimes this behaviour isn't desired; if the RE +``<.*>`` is matched against ``' b '``, it will match the entire +string, and not just ``''``. Adding ``?`` after the quantifier makes it +perform the match in :dfn:`non-greedy` or :dfn:`minimal` fashion; as *few* +characters as possible will be matched. Using the RE ``<.*?>`` will match +only ``''``. + .. index:: single: *+; in regular expressions single: ++; in regular expressions single: ?+; in regular expressions + :name: re-syntax-possessive-quantifiers ``*+``, ``++``, ``?+`` - Like the ``'*'``, ``'+'``, and ``'?'`` quantifiers, those where ``'+'`` is - appended also match as many times as possible. - However, unlike the true greedy quantifiers, these do not allow - back-tracking when the expression following it fails to match. - These are known as :dfn:`possessive` quantifiers. - For example, ``a*a`` will match ``'aaaa'`` because the ``a*`` will match - all 4 ``'a'``\ s, but, when the final ``'a'`` is encountered, the - expression is backtracked so that in the end the ``a*`` ends up matching - 3 ``'a'``\ s total, and the fourth ``'a'`` is matched by the final ``'a'``. - However, when ``a*+a`` is used to match ``'aaaa'``, the ``a*+`` will - match all 4 ``'a'``, but when the final ``'a'`` fails to find any more - characters to match, the expression cannot be backtracked and will thus - fail to match. - ``x*+``, ``x++`` and ``x?+`` are equivalent to ``(?>x*)``, ``(?>x+)`` - and ``(?>x?)`` correspondingly. - - .. versionadded:: 3.11 +"""""""""""""""""""""" + +Like the ``'*'``, ``'+'``, and ``'?'`` quantifiers, those where ``'+'`` is +appended also match as many times as possible. +However, unlike the true greedy quantifiers, these do not allow +back-tracking when the expression following it fails to match. +These are known as :dfn:`possessive` quantifiers. +For example, ``a*a`` will match ``'aaaa'`` because the ``a*`` will match +all 4 ``'a'``\ s, but, when the final ``'a'`` is encountered, the +expression is backtracked so that in the end the ``a*`` ends up matching +3 ``'a'``\ s total, and the fourth ``'a'`` is matched by the final ``'a'``. +However, when ``a*+a`` is used to match ``'aaaa'``, the ``a*+`` will +match all 4 ``'a'``, but when the final ``'a'`` fails to find any more +characters to match, the expression cannot be backtracked and will thus +fail to match. +``x*+``, ``x++`` and ``x?+`` are equivalent to ``(?>x*)``, ``(?>x+)`` +and ``(?>x?)`` correspondingly. + +.. versionadded:: 3.11 .. index:: single: {} (curly brackets); in regular expressions + :name: re-syntax-curly-brackets ``{m}`` - Specifies that exactly *m* copies of the previous RE should be matched; fewer - matches cause the entire RE not to match. For example, ``a{6}`` will match - exactly six ``'a'`` characters, but not five. +""""""" + +Specifies that exactly *m* copies of the previous RE should be matched; fewer +matches cause the entire RE not to match. For example, ``a{6}`` will match +exactly six ``'a'`` characters, but not five. + ``{m,n}`` - Causes the resulting RE to match from *m* to *n* repetitions of the preceding - RE, attempting to match as many repetitions as possible. For example, - ``a{3,5}`` will match from 3 to 5 ``'a'`` characters. Omitting *m* specifies a - lower bound of zero, and omitting *n* specifies an infinite upper bound. As an - example, ``a{4,}b`` will match ``'aaaab'`` or a thousand ``'a'`` characters - followed by a ``'b'``, but not ``'aaab'``. The comma may not be omitted or the - modifier would be confused with the previously described form. +""""""""" + +Causes the resulting RE to match from *m* to *n* repetitions of the preceding +RE, attempting to match as many repetitions as possible. For example, +``a{3,5}`` will match from 3 to 5 ``'a'`` characters. Omitting *m* specifies a +lower bound of zero, and omitting *n* specifies an infinite upper bound. As an +example, ``a{4,}b`` will match ``'aaaab'`` or a thousand ``'a'`` characters +followed by a ``'b'``, but not ``'aaab'``. The comma may not be omitted or the +modifier would be confused with the previously described form. + ``{m,n}?`` - Causes the resulting RE to match from *m* to *n* repetitions of the preceding - RE, attempting to match as *few* repetitions as possible. This is the - non-greedy version of the previous quantifier. For example, on the - 6-character string ``'aaaaaa'``, ``a{3,5}`` will match 5 ``'a'`` characters, - while ``a{3,5}?`` will only match 3 characters. +"""""""""" + +Causes the resulting RE to match from *m* to *n* repetitions of the preceding +RE, attempting to match as *few* repetitions as possible. This is the +non-greedy version of the previous quantifier. For example, on the +6-character string ``'aaaaaa'``, ``a{3,5}`` will match 5 ``'a'`` characters, +while ``a{3,5}?`` will only match 3 characters. + ``{m,n}+`` - Causes the resulting RE to match from *m* to *n* repetitions of the - preceding RE, attempting to match as many repetitions as possible - *without* establishing any backtracking points. - This is the possessive version of the quantifier above. - For example, on the 6-character string ``'aaaaaa'``, ``a{3,5}+aa`` - attempt to match 5 ``'a'`` characters, then, requiring 2 more ``'a'``\ s, - will need more characters than available and thus fail, while - ``a{3,5}aa`` will match with ``a{3,5}`` capturing 5, then 4 ``'a'``\ s - by backtracking and then the final 2 ``'a'``\ s are matched by the final - ``aa`` in the pattern. - ``x{m,n}+`` is equivalent to ``(?>x{m,n})``. +"""""""""" + +Causes the resulting RE to match from *m* to *n* repetitions of the +preceding RE, attempting to match as many repetitions as possible +*without* establishing any backtracking points. +This is the possessive version of the quantifier above. +For example, on the 6-character string ``'aaaaaa'``, ``a{3,5}+aa`` +attempt to match 5 ``'a'`` characters, then, requiring 2 more ``'a'``\ s, +will need more characters than available and thus fail, while +``a{3,5}aa`` will match with ``a{3,5}`` capturing 5, then 4 ``'a'``\ s +by backtracking and then the final 2 ``'a'``\ s are matched by the final +``aa`` in the pattern. +``x{m,n}+`` is equivalent to ``(?>x{m,n})``. + +.. versionadded:: 3.11 - .. versionadded:: 3.11 .. index:: single: \ (backslash); in regular expressions + :name: re-syntax-backslash ``\`` - Either escapes special characters (permitting you to match characters like - ``'*'``, ``'?'``, and so forth), or signals a special sequence; special - sequences are discussed below. - - If you're not using a raw string to express the pattern, remember that Python - also uses the backslash as an escape sequence in string literals; if the escape - sequence isn't recognized by Python's parser, the backslash and subsequent - character are included in the resulting string. However, if Python would - recognize the resulting sequence, the backslash should be repeated twice. This - is complicated and hard to understand, so it's highly recommended that you use - raw strings for all but the simplest expressions. +""""" + +Either escapes special characters (permitting you to match characters like +``'*'``, ``'?'``, and so forth), or signals a special sequence; special +sequences are discussed below. + +If you're not using a raw string to express the pattern, remember that Python +also uses the backslash as an escape sequence in string literals; if the escape +sequence isn't recognized by Python's parser, the backslash and subsequent +character are included in the resulting string. However, if Python would +recognize the resulting sequence, the backslash should be repeated twice. This +is complicated and hard to understand, so it's highly recommended that you use +raw strings for all but the simplest expressions. + .. index:: single: [] (square brackets); in regular expressions + :name: re-syntax-square-brackets ``[]`` - Used to indicate a set of characters. In a set: - - * Characters can be listed individually, e.g. ``[amk]`` will match ``'a'``, - ``'m'``, or ``'k'``. - - .. index:: single: - (minus); in regular expressions - - * Ranges of characters can be indicated by giving two characters and separating - them by a ``'-'``, for example ``[a-z]`` will match any lowercase ASCII letter, - ``[0-5][0-9]`` will match all the two-digits numbers from ``00`` to ``59``, and - ``[0-9A-Fa-f]`` will match any hexadecimal digit. If ``-`` is escaped (e.g. - ``[a\-z]``) or if it's placed as the first or last character - (e.g. ``[-a]`` or ``[a-]``), it will match a literal ``'-'``. - - * Special characters except backslash lose their special meaning inside sets. - For example, - ``[(+*)]`` will match any of the literal characters ``'('``, ``'+'``, - ``'*'``, or ``')'``. - - .. index:: single: \ (backslash); in regular expressions - - * Backslash either escapes characters which have special meaning in a set - such as ``'-'``, ``']'``, ``'^'`` and ``'\\'`` itself or signals - a special sequence which represents a single character such as - ``\xa0`` or ``\n`` or a character class such as ``\w`` or ``\S`` - (defined below). - Note that ``\b`` represents a single "backspace" character, - not a word boundary as outside a set, and numeric escapes - such as ``\1`` are always octal escapes, not group references. - Special sequences which do not match a single character such as ``\A`` - and ``\z`` are not allowed. - - .. index:: single: ^ (caret); in regular expressions - - * Characters that are not within a range can be matched by :dfn:`complementing` - the set. If the first character of the set is ``'^'``, all the characters - that are *not* in the set will be matched. For example, ``[^5]`` will match - any character except ``'5'``, and ``[^^]`` will match any character except - ``'^'``. ``^`` has no special meaning if it's not the first character in - the set. - - * To match a literal ``']'`` inside a set, precede it with a backslash, or - place it at the beginning of the set. For example, both ``[()[\]{}]`` and - ``[]()[{}]`` will match a right bracket, as well as left bracket, braces, - and parentheses. - - .. .. index:: single: --; in regular expressions - .. .. index:: single: &&; in regular expressions - .. .. index:: single: ~~; in regular expressions - .. .. index:: single: ||; in regular expressions - - * Support of nested sets and set operations as in `Unicode Technical - Standard #18`_ might be added in the future. This would change the - syntax, so to facilitate this change a :exc:`FutureWarning` will be raised - in ambiguous cases for the time being. - That includes sets starting with a literal ``'['`` or containing literal - character sequences ``'--'``, ``'&&'``, ``'~~'``, and ``'||'``. To - avoid a warning escape them with a backslash. - - .. _Unicode Technical Standard #18: https://unicode.org/reports/tr18/ +"""""" - .. versionchanged:: 3.7 - :exc:`FutureWarning` is raised if a character set contains constructs - that will change semantically in the future. +Used to indicate a set of characters. In a set: + +* Characters can be listed individually, e.g. ``[amk]`` will match ``'a'``, + ``'m'``, or ``'k'``. + +.. index:: single: - (minus); in regular expressions + :name: re-syntax-square-brackets-minus + +* Ranges of characters can be indicated by giving two characters and separating + them by a ``'-'``, for example ``[a-z]`` will match any lowercase ASCII letter, + ``[0-5][0-9]`` will match all the two-digits numbers from ``00`` to ``59``, and + ``[0-9A-Fa-f]`` will match any hexadecimal digit. If ``-`` is escaped (e.g. + ``[a\-z]``) or if it's placed as the first or last character + (e.g. ``[-a]`` or ``[a-]``), it will match a literal ``'-'``. + +* Special characters except backslash lose their special meaning inside sets. + For example, + ``[(+*)]`` will match any of the literal characters ``'('``, ``'+'``, + ``'*'``, or ``')'``. + +.. index:: single: \ (backslash); in regular expressions + :name: re-syntax-square-brackets-backslash + +* Backslash either escapes characters which have special meaning in a set + such as ``'-'``, ``']'``, ``'^'`` and ``'\\'`` itself or signals + a special sequence which represents a single character such as + ``\xa0`` or ``\n`` or a character class such as ``\w`` or ``\S`` + (defined below). + Note that ``\b`` represents a single "backspace" character, + not a word boundary as outside a set, and numeric escapes + such as ``\1`` are always octal escapes, not group references. + Special sequences which do not match a single character such as ``\A`` + and ``\z`` are not allowed. + +.. index:: single: ^ (caret); in regular expressions + :name: re-syntax-square-brackets-caret + +* Characters that are not within a range can be matched by :dfn:`complementing` + the set. If the first character of the set is ``'^'``, all the characters + that are *not* in the set will be matched. For example, ``[^5]`` will match + any character except ``'5'``, and ``[^^]`` will match any character except + ``'^'``. ``^`` has no special meaning if it's not the first character in + the set. + +* To match a literal ``']'`` inside a set, precede it with a backslash, or + place it at the beginning of the set. For example, both ``[()[\]{}]`` and + ``[]()[{}]`` will match a right bracket, as well as left bracket, braces, + and parentheses. + +.. .. index:: single: --; in regular expressions +.. .. index:: single: &&; in regular expressions +.. .. index:: single: ~~; in regular expressions +.. .. index:: single: ||; in regular expressions + +* Support of nested sets and set operations as in `Unicode Technical + Standard #18`_ might be added in the future. This would change the + syntax, so to facilitate this change a :exc:`FutureWarning` will be raised + in ambiguous cases for the time being. + That includes sets starting with a literal ``'['`` or containing literal + character sequences ``'--'``, ``'&&'``, ``'~~'``, and ``'||'``. To + avoid a warning escape them with a backslash. + +.. _Unicode Technical Standard #18: https://unicode.org/reports/tr18/ + +.. versionchanged:: 3.7 + :exc:`FutureWarning` is raised if a character set contains constructs + that will change semantically in the future. .. index:: single: | (vertical bar); in regular expressions + :name: re-syntax-vertical-bar ``|`` - ``A|B``, where *A* and *B* can be arbitrary REs, creates a regular expression that - will match either *A* or *B*. An arbitrary number of REs can be separated by the - ``'|'`` in this way. This can be used inside groups (see below) as well. As - the target string is scanned, REs separated by ``'|'`` are tried from left to - right. When one pattern completely matches, that branch is accepted. This means - that once *A* matches, *B* will not be tested further, even if it would - produce a longer overall match. In other words, the ``'|'`` operator is never - greedy. To match a literal ``'|'``, use ``\|``, or enclose it inside a - character class, as in ``[|]``. +""""" + +``A|B``, where *A* and *B* can be arbitrary REs, creates a regular expression that +will match either *A* or *B*. An arbitrary number of REs can be separated by the +``'|'`` in this way. This can be used inside groups (see below) as well. As +the target string is scanned, REs separated by ``'|'`` are tried from left to +right. When one pattern completely matches, that branch is accepted. This means +that once *A* matches, *B* will not be tested further, even if it would +produce a longer overall match. In other words, the ``'|'`` operator is never +greedy. To match a literal ``'|'``, use ``\|``, or enclose it inside a +character class, as in ``[|]``. + .. index:: single: () (parentheses); in regular expressions + :name: re-syntax-parentheses ``(...)`` - Matches whatever regular expression is inside the parentheses, and indicates the - start and end of a group; the contents of a group can be retrieved after a match - has been performed, and can be matched later in the string with the ``\number`` - special sequence, described below. To match the literals ``'('`` or ``')'``, - use ``\(`` or ``\)``, or enclose them inside a character class: ``[(]``, ``[)]``. +""""""""" + +Matches whatever regular expression is inside the parentheses, and indicates the +start and end of a group; the contents of a group can be retrieved after a match +has been performed, and can be matched later in the string with the ``\number`` +special sequence, described below. To match the literals ``'('`` or ``')'``, +use ``\(`` or ``\)``, or enclose them inside a character class: ``[(]``, ``[)]``. + .. index:: single: (?; in regular expressions + :name: re-syntax-extension ``(?...)`` - This is an extension notation (a ``'?'`` following a ``'('`` is not meaningful - otherwise). The first character after the ``'?'`` determines what the meaning - and further syntax of the construct is. Extensions usually do not create a new - group; ``(?P...)`` is the only exception to this rule. Following are the - currently supported extensions. +"""""""""" + +This is an extension notation (a ``'?'`` following a ``'('`` is not meaningful +otherwise). The first character after the ``'?'`` determines what the meaning +and further syntax of the construct is. Extensions usually do not create a new +group; ``(?P...)`` is the only exception to this rule. Following are the +currently supported extensions. + + +.. _re-syntax-inline-flags: ``(?aiLmsux)`` - (One or more letters from the set - ``'a'``, ``'i'``, ``'L'``, ``'m'``, ``'s'``, ``'u'``, ``'x'``.) - The group matches the empty string; - the letters set the corresponding flags for the entire regular expression: - - * :const:`re.A` (ASCII-only matching) - * :const:`re.I` (ignore case) - * :const:`re.L` (locale dependent) - * :const:`re.M` (multi-line) - * :const:`re.S` (dot matches all) - * :const:`re.U` (Unicode matching) - * :const:`re.X` (verbose) - - (The flags are described in :ref:`contents-of-module-re`.) - This is useful if you wish to include the flags as part of the - regular expression, instead of passing a *flag* argument to the - :func:`re.compile` function. - Flags should be used first in the expression string. - - .. versionchanged:: 3.11 - This construction can only be used at the start of the expression. +"""""""""""""" + +(One or more letters from the set +``'a'``, ``'i'``, ``'L'``, ``'m'``, ``'s'``, ``'u'``, ``'x'``.) +The group matches the empty string; +the letters set the corresponding flags for the entire regular expression: + +* :const:`re.A` (ASCII-only matching) +* :const:`re.I` (ignore case) +* :const:`re.L` (locale dependent) +* :const:`re.M` (multi-line) +* :const:`re.S` (dot matches all) +* :const:`re.U` (Unicode matching) +* :const:`re.X` (verbose) + +(The flags are described in :ref:`contents-of-module-re`.) +This is useful if you wish to include the flags as part of the +regular expression, instead of passing a *flag* argument to the +:func:`re.compile` function. +Flags should be used first in the expression string. + +.. versionchanged:: 3.11 + This construction can only be used at the start of the expression. + .. index:: single: (?:; in regular expressions + :name: re-syntax-non-capturing ``(?:...)`` - A non-capturing version of regular parentheses. Matches whatever regular - expression is inside the parentheses, but the substring matched by the group - *cannot* be retrieved after performing a match or referenced later in the - pattern. +""""""""""" + +A non-capturing version of regular parentheses. Matches whatever regular +expression is inside the parentheses, but the substring matched by the group +*cannot* be retrieved after performing a match or referenced later in the +pattern. + + +.. _re-syntax-inline-flags-group: ``(?aiLmsux-imsx:...)`` - (Zero or more letters from the set - ``'a'``, ``'i'``, ``'L'``, ``'m'``, ``'s'``, ``'u'``, ``'x'``, - optionally followed by ``'-'`` followed by - one or more letters from the ``'i'``, ``'m'``, ``'s'``, ``'x'``.) - The letters set or remove the corresponding flags for the part of the expression: - - * :const:`re.A` (ASCII-only matching) - * :const:`re.I` (ignore case) - * :const:`re.L` (locale dependent) - * :const:`re.M` (multi-line) - * :const:`re.S` (dot matches all) - * :const:`re.U` (Unicode matching) - * :const:`re.X` (verbose) - - (The flags are described in :ref:`contents-of-module-re`.) - - The letters ``'a'``, ``'L'`` and ``'u'`` are mutually exclusive when used - as inline flags, so they can't be combined or follow ``'-'``. Instead, - when one of them appears in an inline group, it overrides the matching mode - in the enclosing group. In Unicode patterns ``(?a:...)`` switches to - ASCII-only matching, and ``(?u:...)`` switches to Unicode matching - (default). In bytes patterns ``(?L:...)`` switches to locale dependent - matching, and ``(?a:...)`` switches to ASCII-only matching (default). - This override is only in effect for the narrow inline group, and the - original matching mode is restored outside of the group. +""""""""""""""""""""""" + +(Zero or more letters from the set +``'a'``, ``'i'``, ``'L'``, ``'m'``, ``'s'``, ``'u'``, ``'x'``, +optionally followed by ``'-'`` followed by +one or more letters from the ``'i'``, ``'m'``, ``'s'``, ``'x'``.) +The letters set or remove the corresponding flags for the part of the expression: + +* :const:`re.A` (ASCII-only matching) +* :const:`re.I` (ignore case) +* :const:`re.L` (locale dependent) +* :const:`re.M` (multi-line) +* :const:`re.S` (dot matches all) +* :const:`re.U` (Unicode matching) +* :const:`re.X` (verbose) + +(The flags are described in :ref:`contents-of-module-re`.) + +The letters ``'a'``, ``'L'`` and ``'u'`` are mutually exclusive when used +as inline flags, so they can't be combined or follow ``'-'``. Instead, +when one of them appears in an inline group, it overrides the matching mode +in the enclosing group. In Unicode patterns ``(?a:...)`` switches to +ASCII-only matching, and ``(?u:...)`` switches to Unicode matching +(default). In bytes patterns ``(?L:...)`` switches to locale dependent +matching, and ``(?a:...)`` switches to ASCII-only matching (default). +This override is only in effect for the narrow inline group, and the +original matching mode is restored outside of the group. + +.. versionadded:: 3.6 - .. versionadded:: 3.6 +.. versionchanged:: 3.7 + The letters ``'a'``, ``'L'`` and ``'u'`` also can be used in a group. - .. versionchanged:: 3.7 - The letters ``'a'``, ``'L'`` and ``'u'`` also can be used in a group. + +.. index:: single: (?>; in regular expressions + :name: re-syntax-atomic-group ``(?>...)`` - Attempts to match ``...`` as if it was a separate regular expression, and - if successful, continues to match the rest of the pattern following it. - If the subsequent pattern fails to match, the stack can only be unwound - to a point *before* the ``(?>...)`` because once exited, the expression, - known as an :dfn:`atomic group`, has thrown away all stack points within - itself. - Thus, ``(?>.*).`` would never match anything because first the ``.*`` - would match all characters possible, then, having nothing left to match, - the final ``.`` would fail to match. - Since there are no stack points saved in the Atomic Group, and there is - no stack point before it, the entire expression would thus fail to match. +""""""""""" + +Attempts to match ``...`` as if it was a separate regular expression, and +if successful, continues to match the rest of the pattern following it. +If the subsequent pattern fails to match, the stack can only be unwound +to a point *before* the ``(?>...)`` because once exited, the expression, +known as an :dfn:`atomic group`, has thrown away all stack points within +itself. +Thus, ``(?>.*).`` would never match anything because first the ``.*`` +would match all characters possible, then, having nothing left to match, +the final ``.`` would fail to match. +Since there are no stack points saved in the Atomic Group, and there is +no stack point before it, the entire expression would thus fail to match. + +.. versionadded:: 3.11 - .. versionadded:: 3.11 .. index:: single: (?P<; in regular expressions + :name: re-syntax-named-capture ``(?P...)`` - Similar to regular parentheses, but the substring matched by the group is - accessible via the symbolic group name *name*. Group names must be valid - Python identifiers, and in :class:`bytes` patterns they can only contain - bytes in the ASCII range. Each group name must be defined only once within - a regular expression. A symbolic group is also a numbered group, just as if - the group were not named. - - Named groups can be referenced in three contexts. If the pattern is - ``(?P['"]).*?(?P=quote)`` (i.e. matching a string quoted with either - single or double quotes): - - +---------------------------------------+----------------------------------+ - | Context of reference to group "quote" | Ways to reference it | - +=======================================+==================================+ - | in the same pattern itself | * ``(?P=quote)`` (as shown) | - | | * ``\1`` | - +---------------------------------------+----------------------------------+ - | when processing match object *m* | * ``m.group('quote')`` | - | | * ``m.end('quote')`` (etc.) | - +---------------------------------------+----------------------------------+ - | in a string passed to the *repl* | * ``\g`` | - | argument of ``re.sub()`` | * ``\g<1>`` | - | | * ``\1`` | - +---------------------------------------+----------------------------------+ - - .. versionchanged:: 3.12 - In :class:`bytes` patterns, group *name* can only contain bytes - in the ASCII range (``b'\x00'``-``b'\x7f'``). +""""""""""""""""" + +Similar to regular parentheses, but the substring matched by the group is +accessible via the symbolic group name *name*. Group names must be valid +Python identifiers, and in :class:`bytes` patterns they can only contain +bytes in the ASCII range. Each group name must be defined only once within +a regular expression. A symbolic group is also a numbered group, just as if +the group were not named. + +Named groups can be referenced in three contexts. If the pattern is +``(?P['"]).*?(?P=quote)`` (i.e. matching a string quoted with either +single or double quotes): + ++---------------------------------------+----------------------------------+ +| Context of reference to group "quote" | Ways to reference it | ++=======================================+==================================+ +| in the same pattern itself | * ``(?P=quote)`` (as shown) | +| | * ``\1`` | ++---------------------------------------+----------------------------------+ +| when processing match object *m* | * ``m.group('quote')`` | +| | * ``m.end('quote')`` (etc.) | ++---------------------------------------+----------------------------------+ +| in a string passed to the *repl* | * ``\g`` | +| argument of ``re.sub()`` | * ``\g<1>`` | +| | * ``\1`` | ++---------------------------------------+----------------------------------+ + +.. versionchanged:: 3.12 + In :class:`bytes` patterns, group *name* can only contain bytes + in the ASCII range (``b'\x00'``-``b'\x7f'``). .. index:: single: (?P=; in regular expressions + :name: re-syntax-named-backreference ``(?P=name)`` - A backreference to a named group; it matches whatever text was matched by the - earlier group named *name*. +""""""""""""" + +A backreference to a named group; it matches whatever text was matched by the +earlier group named *name*. + .. index:: single: (?#; in regular expressions + :name: re-syntax-comment ``(?#...)`` - A comment; the contents of the parentheses are simply ignored. +""""""""""" + +A comment; the contents of the parentheses are simply ignored. + .. index:: single: (?=; in regular expressions + :name: re-syntax-lookahead ``(?=...)`` - Matches if ``...`` matches next, but doesn't consume any of the string. This is - called a :dfn:`lookahead assertion`. For example, ``Isaac (?=Asimov)`` will match - ``'Isaac '`` only if it's followed by ``'Asimov'``. +""""""""""" + +Matches if ``...`` matches next, but doesn't consume any of the string. This is +called a :dfn:`lookahead assertion`. For example, ``Isaac (?=Asimov)`` will match +``'Isaac '`` only if it's followed by ``'Asimov'``. + .. index:: single: (?!; in regular expressions + :name: re-syntax-negative-lookahead ``(?!...)`` - Matches if ``...`` doesn't match next. This is a :dfn:`negative lookahead assertion`. - For example, ``Isaac (?!Asimov)`` will match ``'Isaac '`` only if it's *not* - followed by ``'Asimov'``. +""""""""""" + +Matches if ``...`` doesn't match next. This is a :dfn:`negative lookahead assertion`. +For example, ``Isaac (?!Asimov)`` will match ``'Isaac '`` only if it's *not* +followed by ``'Asimov'``. + .. index:: single: (?<=; in regular expressions + :name: re-syntax-lookbehind ``(?<=...)`` - Matches if the current position in the string is preceded by a match for ``...`` - that ends at the current position. This is called a :dfn:`positive lookbehind - assertion`. ``(?<=abc)def`` will find a match in ``'abcdef'``, since the - lookbehind will back up 3 characters and check if the contained pattern matches. - The contained pattern must only match strings of some fixed length, meaning that - ``abc`` or ``a|b`` are allowed, but ``a*`` and ``a{3,4}`` are not. Note that - patterns which start with positive lookbehind assertions will not match at the - beginning of the string being searched; you will most likely want to use the - :func:`search` function rather than the :func:`match` function: - - >>> import re - >>> m = re.search('(?<=abc)def', 'abcdef') - >>> m.group(0) - 'def' - - This example looks for a word following a hyphen: - - >>> m = re.search(r'(?<=-)\w+', 'spam-egg') - >>> m.group(0) - 'egg' +"""""""""""" + +Matches if the current position in the string is preceded by a match for ``...`` +that ends at the current position. This is called a :dfn:`positive lookbehind +assertion`. ``(?<=abc)def`` will find a match in ``'abcdef'``, since the +lookbehind will back up 3 characters and check if the contained pattern matches. +The contained pattern must only match strings of some fixed length, meaning that +``abc`` or ``a|b`` are allowed, but ``a*`` and ``a{3,4}`` are not. Note that +patterns which start with positive lookbehind assertions will not match at the +beginning of the string being searched; you will most likely want to use the +:func:`search` function rather than the :func:`match` function: + + >>> import re + >>> m = re.search('(?<=abc)def', 'abcdef') + >>> m.group(0) + 'def' + +This example looks for a word following a hyphen: + + >>> m = re.search(r'(?<=-)\w+', 'spam-egg') + >>> m.group(0) + 'egg' + +.. versionchanged:: 3.5 + Added support for group references of fixed length. - .. versionchanged:: 3.5 - Added support for group references of fixed length. .. index:: single: (?|$)`` is a poor email matching pattern, which - matches ``''`` as well as ``'user@host.com'``, but does not - match ``''`` in their entirety - (:func:`re.search` finds only ``'user@host.com'`` in the former). +"""""""""""""""""""""""""""""""""""""" - .. versionchanged:: 3.12 - Group *id* can only contain ASCII digits. - In :class:`bytes` patterns, group *name* can only contain bytes - in the ASCII range (``b'\x00'``-``b'\x7f'``). +Will try to match with ``yes-pattern`` if the group with given *id* or +*name* exists, and with ``no-pattern`` if it doesn't. ``no-pattern`` is +optional and can be omitted. For example, +``(<)?(\w+@\w+(?:\.\w+)+)(?(1)>|$)`` is a poor email matching pattern, which +matches ``''`` as well as ``'user@host.com'``, but does not +match ``''`` in their entirety +(:func:`re.search` finds only ``'user@host.com'`` in the former). + +.. versionchanged:: 3.12 + Group *id* can only contain ASCII digits. + In :class:`bytes` patterns, group *name* can only contain bytes + in the ASCII range (``b'\x00'``-``b'\x7f'``). .. _re-special-sequences: +Special sequences +^^^^^^^^^^^^^^^^^ + The special sequences consist of ``'\'`` and a character from the list below. If the ordinary character is not an ASCII digit or an ASCII letter, then the resulting RE will match the second character. For example, ``\$`` matches the character ``'$'``. + .. index:: single: \ (backslash); in regular expressions + :name: re-syntax-special-group-reference ``\number`` - Matches the contents of the group of the same number. Groups are numbered - starting from 1. For example, ``(.+) \1`` matches ``'the the'`` or ``'55 55'``, - but not ``'thethe'`` (note the space after the group). This special sequence - can only be used to match one of the first 99 groups. If the first digit of - *number* is 0, or *number* is 3 octal digits long, it will not be interpreted as - a group match, but as the character with octal value *number*. Inside the - ``'['`` and ``']'`` of a character class, all numeric escapes are treated as - characters. +""""""""""" + +Matches the contents of the group of the same number. Groups are numbered +starting from 1. For example, ``(.+) \1`` matches ``'the the'`` or ``'55 55'``, +but not ``'thethe'`` (note the space after the group). This special sequence +can only be used to match one of the first 99 groups. If the first digit of +*number* is 0, or *number* is 3 octal digits long, it will not be interpreted as +a group match, but as the character with octal value *number*. Inside the +``'['`` and ``']'`` of a character class, all numeric escapes are treated as +characters. + .. index:: single: \A; in regular expressions + :name: re-syntax-special-start ``\A`` - Matches only at the start of the string. +"""""" + +Matches only at the start of the string. + .. index:: single: \b; in regular expressions + :name: re-syntax-special-word-boundary ``\b`` - Matches the empty string, but only at the beginning or end of a word. - A word is defined as a sequence of word characters. - Note that formally, ``\b`` is defined as the boundary - between a ``\w`` and a ``\W`` character (or vice versa), - or between ``\w`` and the beginning or end of the string. - This means that ``r'\bat\b'`` matches ``'at'``, ``'at.'``, ``'(at)'``, - and ``'as at ay'`` but not ``'attempt'`` or ``'atlas'``. - - The default word characters in Unicode (str) patterns - are Unicode alphanumerics and the underscore, - but this can be changed by using the :py:const:`~re.ASCII` flag. - Word boundaries are determined by the current locale - if the :py:const:`~re.LOCALE` flag is used. +"""""" - .. note:: +Matches the empty string, but only at the beginning or end of a word. +A word is defined as a sequence of word characters. +Note that formally, ``\b`` is defined as the boundary +between a ``\w`` and a ``\W`` character (or vice versa), +or between ``\w`` and the beginning or end of the string. +This means that ``r'\bat\b'`` matches ``'at'``, ``'at.'``, ``'(at)'``, +and ``'as at ay'`` but not ``'attempt'`` or ``'atlas'``. - Inside a character range, ``\b`` represents the backspace character, - for compatibility with Python's string literals. +The default word characters in Unicode (str) patterns +are Unicode alphanumerics and the underscore, +but this can be changed by using the :py:const:`~re.ASCII` flag. +Word boundaries are determined by the current locale +if the :py:const:`~re.LOCALE` flag is used. + +.. note:: + + Inside a character range, ``\b`` represents the backspace character, + for compatibility with Python's string literals. .. index:: single: \B; in regular expressions + :name: re-syntax-special-inside-word ``\B`` - Matches the empty string, - but only when it is *not* at the beginning or end of a word. - This means that ``r'at\B'`` matches ``'athens'``, ``'atom'``, - ``'attorney'``, but not ``'at'``, ``'at.'``, or ``'at!'``. - ``\B`` is the opposite of ``\b``, - so word characters in Unicode (str) patterns - are Unicode alphanumerics or the underscore, - although this can be changed by using the :py:const:`~re.ASCII` flag. - Word boundaries are determined by the current locale - if the :py:const:`~re.LOCALE` flag is used. - - .. versionchanged:: 3.14 - ``\B`` now matches empty input string. +"""""" + +Matches the empty string, +but only when it is *not* at the beginning or end of a word. +This means that ``r'at\B'`` matches ``'athens'``, ``'atom'``, +``'attorney'``, but not ``'at'``, ``'at.'``, or ``'at!'``. +``\B`` is the opposite of ``\b``, +so word characters in Unicode (str) patterns +are Unicode alphanumerics or the underscore, +although this can be changed by using the :py:const:`~re.ASCII` flag. +Word boundaries are determined by the current locale +if the :py:const:`~re.LOCALE` flag is used. + +.. versionchanged:: 3.14 + ``\B`` now matches empty input string. .. index:: single: \d; in regular expressions + :name: re-syntax-special-digits ``\d`` - For Unicode (str) patterns: - Matches any Unicode decimal digit - (that is, any character in Unicode character category `[Nd]`__). - This includes ``[0-9]``, and also many other digit characters. +"""""" + +For Unicode (str) patterns: + Matches any Unicode decimal digit + (that is, any character in Unicode character category `[Nd]`__). + This includes ``[0-9]``, and also many other digit characters. - Matches ``[0-9]`` if the :py:const:`~re.ASCII` flag is used. + Matches ``[0-9]`` if the :py:const:`~re.ASCII` flag is used. - __ https://www.unicode.org/versions/Unicode15.0.0/ch04.pdf#G134153 + __ https://www.unicode.org/versions/Unicode15.0.0/ch04.pdf#G134153 - For 8-bit (bytes) patterns: - Matches any decimal digit in the ASCII character set; - this is equivalent to ``[0-9]``. +For 8-bit (bytes) patterns: + Matches any decimal digit in the ASCII character set; + this is equivalent to ``[0-9]``. .. index:: single: \D; in regular expressions + :name: re-syntax-special-non-digits ``\D`` - Matches any character which is not a decimal digit. - This is the opposite of ``\d``. +"""""" - Matches ``[^0-9]`` if the :py:const:`~re.ASCII` flag is used. +Matches any character which is not a decimal digit. +This is the opposite of ``\d``. + +Matches ``[^0-9]`` if the :py:const:`~re.ASCII` flag is used. .. index:: single: \s; in regular expressions + :name: re-syntax-special-whitespace ``\s`` - For Unicode (str) patterns: - Matches Unicode whitespace characters (as defined by :py:meth:`str.isspace`). - This includes ``[ \t\n\r\f\v]``, and also many other characters, for example the - non-breaking spaces mandated by typography rules in many languages. +"""""" + +For Unicode (str) patterns: + Matches Unicode whitespace characters (as defined by :py:meth:`str.isspace`). + This includes ``[ \t\n\r\f\v]``, and also many other characters, for example the + non-breaking spaces mandated by typography rules in many languages. - Matches ``[ \t\n\r\f\v]`` if the :py:const:`~re.ASCII` flag is used. + Matches ``[ \t\n\r\f\v]`` if the :py:const:`~re.ASCII` flag is used. + +For 8-bit (bytes) patterns: + Matches characters considered whitespace in the ASCII character set; + this is equivalent to ``[ \t\n\r\f\v]``. - For 8-bit (bytes) patterns: - Matches characters considered whitespace in the ASCII character set; - this is equivalent to ``[ \t\n\r\f\v]``. .. index:: single: \S; in regular expressions + :name: re-syntax-special-non-whitespace ``\S`` - Matches any character which is not a whitespace character. This is - the opposite of ``\s``. +"""""" + +Matches any character which is not a whitespace character. This is +the opposite of ``\s``. - Matches ``[^ \t\n\r\f\v]`` if the :py:const:`~re.ASCII` flag is used. +Matches ``[^ \t\n\r\f\v]`` if the :py:const:`~re.ASCII` flag is used. .. index:: single: \w; in regular expressions + :name: re-syntax-special-word ``\w`` - For Unicode (str) patterns: - Matches Unicode word characters; - this includes all Unicode alphanumeric characters - (as defined by :py:meth:`str.isalnum`), - as well as the underscore (``_``). +"""""" - Matches ``[a-zA-Z0-9_]`` if the :py:const:`~re.ASCII` flag is used. +For Unicode (str) patterns: + Matches Unicode word characters; + this includes all Unicode alphanumeric characters + (as defined by :py:meth:`str.isalnum`), + as well as the underscore (``_``). - For 8-bit (bytes) patterns: - Matches characters considered alphanumeric in the ASCII character set; - this is equivalent to ``[a-zA-Z0-9_]``. - If the :py:const:`~re.LOCALE` flag is used, - matches characters considered alphanumeric in the current locale and the underscore. + Matches ``[a-zA-Z0-9_]`` if the :py:const:`~re.ASCII` flag is used. + +For 8-bit (bytes) patterns: + Matches characters considered alphanumeric in the ASCII character set; + this is equivalent to ``[a-zA-Z0-9_]``. + If the :py:const:`~re.LOCALE` flag is used, + matches characters considered alphanumeric in the current locale and the underscore. .. index:: single: \W; in regular expressions + :name: re-syntax-special-non-word ``\W`` - Matches any character which is not a word character. - This is the opposite of ``\w``. - By default, matches non-underscore (``_``) characters - for which :py:meth:`str.isalnum` returns ``False``. +"""""" - Matches ``[^a-zA-Z0-9_]`` if the :py:const:`~re.ASCII` flag is used. +Matches any character which is not a word character. +This is the opposite of ``\w``. +By default, matches non-underscore (``_``) characters +for which :py:meth:`str.isalnum` returns ``False``. + +Matches ``[^a-zA-Z0-9_]`` if the :py:const:`~re.ASCII` flag is used. + +If the :py:const:`~re.LOCALE` flag is used, +matches characters which are neither alphanumeric in the current locale +nor the underscore. - If the :py:const:`~re.LOCALE` flag is used, - matches characters which are neither alphanumeric in the current locale - nor the underscore. .. index:: single: \z; in regular expressions single: \Z; in regular expressions + :name: re-syntax-special-end ``\z`` - Matches only at the end of the string. +"""""" + +Matches only at the end of the string. + +.. versionadded:: 3.14 - .. versionadded:: 3.14 ``\Z`` - The same as ``\z``. For compatibility with old Python versions. +"""""" + +The same as ``\z``. For compatibility with old Python versions. .. index:: single: \a; in regular expressions @@ -686,6 +850,10 @@ character ``'$'``. single: \v; in regular expressions single: \x; in regular expressions single: \\; in regular expressions + :name: re-syntax-standard-escapes + +Standard escapes +^^^^^^^^^^^^^^^^ Most of the :ref:`escape sequences ` supported by Python string literals are also accepted by the regular expression parser::