Regex \B doesn't match empty string

Bug report

Bug description:

>>> import re
>>> list(re.finditer(r'\b', 'e'))
[<re.Match object; span=(0, 0), match=''>, <re.Match object; span=(1, 1), match=''>]
>>> list(re.finditer(r'\B', 'e'))
[]
>>> list(re.finditer(r'\b', '%'))
[]
>>> list(re.finditer(r'\B', '%'))
[<re.Match object; span=(0, 0), match=''>, <re.Match object; span=(1, 1), match=''>]
>>> list(re.finditer(r'\b', ''))
[]
>>> list(re.finditer(r'\B', ''))
[]

Apparently the empty string neither is nor isn't a word boundary. Is that supposed to happen? \B matches the empty string in every other language I can think of.

Online reproducer: https://godbolt.org/z/8q6fehss7

CPython versions tested on:

3.11, 3.12

Operating systems tested on:

Linux

Linked PRs

The current behavior (\B not matching "") appears to be intentional:

cpython/Lib/test/test_re.py

Lines 896 to 898 in aba42c0

    
           # However, an empty string contains no word boundaries, and also no 
        
           # non-boundaries. 
        
           self.assertIsNone(re.search(r"\B", ""))

Strictly speaking, re is working as designed here, even if that design is arguably "incorrect" with respect to other regex implementations.

Changing this behavior would technically be a feature request (not a bug), and would likely require a deprecation period. I'd recommend opening a thread on https://discuss.python.org/c/ideas/6 to see if there's community interest in changing this.

For reference, JavaScript treats the empty string as containing a non-word boundary:

>> /\b/.test("")
false
>> /\B/.test("")
true

Another Perl reference:

$ perl -E 'say "Boundary" if "" =~ /\b/; say "Non-boundary" if "" =~ /\B/;'
Non-boundary

Seems that it is taking Perl as a baseline to some extent, as indicated in test_re:

cpython/Lib/test/test_re.py

Lines 896 to 901 in 9017b95

    
           # However, an empty string contains no word boundaries, and also no 
        
           # non-boundaries. 
        
           self.assertIsNone(re.search(r"\B", "")) 
        
           # This one is questionable and different from the perlre behaviour, 
        
           # but describes current behavior. 
        
           self.assertIsNone(re.search(r"\b", ""))

However the comments really confuse me since the behavior for \b atm is same as Perl's but it's still claiming different from the perlre behaviour. Or have I missed some changes on the behavior of \b later on? If that was the case, I think it's also reasonable to align the behavior of \B to Perl's.

@serhiy-storchaka

However the comments really confuse me

I think both comments apply to the \B case, as that is the one different from Perl. Splitting the comment that way is a bit confusing if one does not know Perl, which most do not now. @serhiy-storchaka can comment, but I don't think we can really change this now.

This test was added in 5a045b9 (bpo-10713/gh-54922). It was not an assertion for the intended behavior, it was added to ensure that the current behavior would not change. Strictly speaking, the current behavior contradicts the documentation that says that \B is the opposite of \b.

Of course, it may be that the documentation is wrong. But taking into account that the current behavior differs from the behavior of many (if not all) other RE engines, that the code is most likely a copying error (it was not properly tested), that we already did several breaking changes related to zero-width matches in the past, and that it affects only very specific cases, I think that we can and should change this behavior.

It is preferable to emit a FutureWarning first, but I do not know how difficult to do this without producing false positives. If it is too difficult or impossible, we will have no other way as to change the behavior without warning.

We've been reluctant to make any changes to the re module behavior in the past as people's existing code depends on all of its behaviors per Hyrum's Law.

The best that we can do is to document the bug, if there is a bug, and explain that it's kept for backward compatibility.

The linked PR is now just a documentation note.

gh-124130: Notes on empty string corner case of category `\B` (#124133)

pythongh-124130: Notes on empty string corner case of category `\B` (p…

[3.13] gh-124130: Notes on empty string corner case of category `\B` (G…

gh-124130: Increase test coverage for \b and \B in regular expressions (

pythongh-124130: Increase test coverage for \b and \B in regular expr…

[3.12] gh-124130: Increase test coverage for \b and \B in regular exp…

[3.13] gh-124130: Increase test coverage for \b and \B in regular exp…

I tested the recent version of:

python  main branch
regex   2024.11.6
openjdk 21.0.5
.net    9.0
rust    1.81.0 (regex v1.11.1)
go      1.23.3
js      (firefox 132.0.2/chromium 131.0.6778.70)
ruby    3.3.6
php     8.3.13
perl    5.40.0
pcre2   10.43
boost   1.83.0

\B can't match "" only in Python:

matching occurrence:

       \b ""   \B ""
python   0       0
OTHERS   0       1

       \b "a"  \B "a"
ALL      2       0

       \b "="  \B "="
ALL      0       2

Searching \B on GitHub did not find valuable information.
It would be great if Regex101 website (the link above this comment) could provide some examples of how users using \B.

pythongh-124130: Fix a bug in matching regular expression \B in empty…

#127007 changes the behavior of \B for an empty string.

gh-124130: Fix a bug in matching regular expression \B in empty string (

pythongh-124130: Fix a bug in matching regular expression \B in empty…

Navigation Menu

Repository navigation

Regex \B doesn't match empty string #124130

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

6 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

	# However, an empty string contains no word boundaries, and also no
	# non-boundaries.
	self.assertIsNone(re.search(r"\B", ""))

Regex \B doesn't match empty string #124130

Description

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

Activity

brianschubert commented on Sep 16, 2024

y5c4l3 commented on Sep 16, 2024

terryjreedy commented on Sep 16, 2024

serhiy-storchaka commented on Sep 17, 2024

gpshead commented on Sep 17, 2024

vstinner commented on Sep 20, 2024

ZeroIntensity commented on Sep 21, 2024

6 remaining items

wjssz commented on Nov 19, 2024

serhiy-storchaka commented on Nov 19, 2024

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions