X Tutup
Skip to content

Commit fbb490f

Browse files
serhiy-storchakaserhiy-storchaka
authored
bpo-32308: Replace empty matches adjacent to a previous non-empty match in re.sub(). (python#4846)
1 parent 0cc99c8 commit fbb490f

File tree

6 files changed

+35
-25
lines changed

6 files changed

+35
-25
lines changed

‎Doc/howto/regex.rst‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1140,12 +1140,12 @@ new string value and the number of replacements that were performed::
11401140
>>> p.subn('colour', 'no colours at all')
11411141
('no colours at all', 0)
11421142

1143-
Empty matches are replaced only when they're not adjacent to a previous match.
1143+
Empty matches are replaced only when they're not adjacent to a previous empty match.
11441144
::
11451145

11461146
>>> p = re.compile('x*')
11471147
>>> p.sub('-', 'abxd')
1148-
'-a-b-d-'
1148+
'-a-b--d-'
11491149

11501150
If *replacement* is a string, any backslash escapes in it are processed. That
11511151
is, ``\n`` is converted to a single newline character, ``\r`` is converted to a

‎Doc/library/re.rst‎

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -708,12 +708,15 @@ form.
708708
That way, separator components are always found at the same relative
709709
indices within the result list.
710710

711-
The pattern can match empty strings. ::
711+
Empty matches for the pattern split the string only when not adjacent
712+
to a previous empty match.
712713

713714
>>> re.split(r'\b', 'Words, words, words.')
714715
['', 'Words', ', ', 'words', ', ', 'words', '.']
716+
>>> re.split(r'\W*', '...words...')
717+
['', '', 'w', 'o', 'r', 'd', 's', '', '']
715718
>>> re.split(r'(\W*)', '...words...')
716-
['', '...', 'w', '', 'o', '', 'r', '', 'd', '', 's', '...', '']
719+
['', '...', '', '', 'w', '', 'o', '', 'r', '', 'd', '', 's', '...', '', '', '']
717720

718721
.. versionchanged:: 3.1
719722
Added the optional flags argument.
@@ -778,8 +781,8 @@ form.
778781
The optional argument *count* is the maximum number of pattern occurrences to be
779782
replaced; *count* must be a non-negative integer. If omitted or zero, all
780783
occurrences will be replaced. Empty matches for the pattern are replaced only
781-
when not adjacent to a previous match, so ``sub('x*', '-', 'abc')`` returns
782-
``'-a-b-c-'``.
784+
when not adjacent to a previous empty match, so ``sub('x*', '-', 'abxd')`` returns
785+
``'-a-b--d-'``.
783786

784787
In string-type *repl* arguments, in addition to the character escapes and
785788
backreferences described above,
@@ -805,6 +808,9 @@ form.
805808
Unknown escapes in *repl* consisting of ``'\'`` and an ASCII letter
806809
now are errors.
807810

811+
Empty matches for the pattern are replaced when adjacent to a previous
812+
non-empty match.
813+
808814

809815
.. function:: subn(pattern, repl, string, count=0, flags=0)
810816

‎Doc/whatsnew/3.7.rst‎

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -881,8 +881,9 @@ Changes in the Python API
881881
* The result of splitting a string on a :mod:`regular expression <re>`
882882
that could match an empty string has been changed. For example
883883
splitting on ``r'\s*'`` will now split not only on whitespaces as it
884-
did previously, but also between any pair of non-whitespace
885-
characters. The previous behavior can be restored by changing the pattern
884+
did previously, but also on empty strings before all non-whitespace
885+
characters and just before the end of the string.
886+
The previous behavior can be restored by changing the pattern
886887
to ``r'\s+'``. A :exc:`FutureWarning` was emitted for such patterns since
887888
Python 3.5.
888889

@@ -893,7 +894,13 @@ Changes in the Python API
893894
positions 2--3. To match only blank lines, the pattern should be rewritten
894895
as ``r'(?m)^[^\S\n]*$'``.
895896

896-
(Contributed by Serhiy Storchaka in :issue:`25054`.)
897+
:func:`re.sub()` now replaces empty matches adjacent to a previous
898+
non-empty match. For example ``re.sub('x*', '-', 'abxd')`` returns now
899+
``'-a-b--d-'`` instead of ``'-a-b--d-'`` (the first minus between 'b' and
900+
'd' replaces 'x', and the second minus replaces an empty string between
901+
'x' and 'd').
902+
903+
(Contributed by Serhiy Storchaka in :issue:`25054` and :issue:`32308`.)
897904

898905
* :class:`tracemalloc.Traceback` frames are now sorted from oldest to most
899906
recent to be more consistent with :mod:`traceback`.

‎Lib/test/test_re.py‎

Lines changed: 9 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -213,11 +213,6 @@ def test_bug_114660(self):
213213
self.assertEqual(re.sub(r'(\S)\s+(\S)', r'\1 \2', 'hello there'),
214214
'hello there')
215215

216-
def test_bug_462270(self):
217-
# Test for empty sub() behaviour, see SF bug #462270
218-
self.assertEqual(re.sub('x*', '-', 'abxd'), '-a-b-d-')
219-
self.assertEqual(re.sub('x+', '-', 'abxd'), 'ab-d')
220-
221216
def test_symbolic_groups(self):
222217
re.compile(r'(?P<a>x)(?P=a)(?(a)y)')
223218
re.compile(r'(?P<a1>x)(?P=a1)(?(a1)y)')
@@ -331,10 +326,10 @@ def test_re_split(self):
331326
['', 'a', '', '', 'c'])
332327

333328
for sep, expected in [
334-
(':*', ['', 'a', 'b', 'c', '']),
335-
('(?::*)', ['', 'a', 'b', 'c', '']),
336-
('(:*)', ['', ':', 'a', ':', 'b', '::', 'c', '', '']),
337-
('(:)*', ['', ':', 'a', ':', 'b', ':', 'c', None, '']),
329+
(':*', ['', '', 'a', '', 'b', '', 'c', '']),
330+
('(?::*)', ['', '', 'a', '', 'b', '', 'c', '']),
331+
('(:*)', ['', ':', '', '', 'a', ':', '', '', 'b', '::', '', '', 'c', '', '']),
332+
('(:)*', ['', ':', '', None, 'a', ':', '', None, 'b', ':', '', None, 'c', None, '']),
338333
]:
339334
with self.subTest(sep=sep):
340335
self.assertTypedEqual(re.split(sep, ':a:b::c'), expected)
@@ -357,7 +352,7 @@ def test_qualified_re_split(self):
357352
self.assertEqual(re.split("(:+)", ":a:b::c", maxsplit=2),
358353
['', ':', 'a', ':', 'b::c'])
359354
self.assertEqual(re.split("(:*)", ":a:b::c", maxsplit=2),
360-
['', ':', 'a', ':', 'b::c'])
355+
['', ':', '', '', 'a:b::c'])
361356

362357
def test_re_findall(self):
363358
self.assertEqual(re.findall(":+", "abc"), [])
@@ -1753,13 +1748,13 @@ def test_match_repr(self):
17531748
def test_zerowidth(self):
17541749
# Issues 852532, 1647489, 3262, 25054.
17551750
self.assertEqual(re.split(r"\b", "a::bc"), ['', 'a', '::', 'bc', ''])
1756-
self.assertEqual(re.split(r"\b|:+", "a::bc"), ['', 'a', '', 'bc', ''])
1757-
self.assertEqual(re.split(r"(?<!\w)(?=\w)|:+", "a::bc"), ['', 'a', 'bc'])
1751+
self.assertEqual(re.split(r"\b|:+", "a::bc"), ['', 'a', '', '', 'bc', ''])
1752+
self.assertEqual(re.split(r"(?<!\w)(?=\w)|:+", "a::bc"), ['', 'a', '', 'bc'])
17581753
self.assertEqual(re.split(r"(?<=\w)(?!\w)|:+", "a::bc"), ['a', '', 'bc', ''])
17591754

17601755
self.assertEqual(re.sub(r"\b", "-", "a::bc"), '-a-::-bc-')
1761-
self.assertEqual(re.sub(r"\b|:+", "-", "a::bc"), '-a--bc-')
1762-
self.assertEqual(re.sub(r"(\b|:+)", r"[\1]", "a::bc"), '[]a[][::]bc[]')
1756+
self.assertEqual(re.sub(r"\b|:+", "-", "a::bc"), '-a---bc-')
1757+
self.assertEqual(re.sub(r"(\b|:+)", r"[\1]", "a::bc"), '[]a[][::][]bc[]')
17631758

17641759
self.assertEqual(re.findall(r"\b|:+", "a::bc"), ['', '', '::', '', ''])
17651760
self.assertEqual(re.findall(r"\b|\w+", "a::bc"),

‎Misc/NEWS.d/next/Library/2017-12-13-20-31-30.bpo-32308.CUbsb2.rst‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
:func:`re.sub()` now replaces empty matches adjacent to a previous non-empty
2+
match.

‎Modules/_sre.c‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -955,7 +955,7 @@ _sre_SRE_Pattern_split_impl(PatternObject *self, PyObject *string,
955955
}
956956

957957
n = n + 1;
958-
state.must_advance = 1;
958+
state.must_advance = (state.ptr == state.start);
959959
last = state.start = state.ptr;
960960

961961
}
@@ -1109,7 +1109,7 @@ pattern_subx(PatternObject* self, PyObject* ptemplate, PyObject* string,
11091109

11101110
i = e;
11111111
n = n + 1;
1112-
state.must_advance = 1;
1112+
state.must_advance = (state.ptr == state.start);
11131113
state.start = state.ptr;
11141114
}
11151115

0 commit comments

Comments
 (0)
X Tutup