Repository navigation
Conversation
glob.translate() dropped the star indices from fnmatch._translate(), so every star in a segment became a plain backtracking group and matching took exponential time in pathlib's match, full_match and glob. Each interior star and the fixed part after it are now wrapped in an atomic group, as fnmatch already does. Pairs whose fixed part can match a separator are left unchanged, so the set of matching names stays the same.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
closes #159717
glob.translate()kept only the parts list fromfnmatch._translate()and dropped the star indices, so every*in a segment became a plain backtracking[^/]*. A failing match then took exponential time, which madePurePath.match,PurePath.full_matchandPath.globhang on short patterns such as'*a' * 8 + '*b'.This change builds each segment from both the parts and the star indices and wraps every interior
STAR fixedpair in an atomic group(?>[^/]*?fixed), the same rewritefnmatch._join_translated_parts()uses. A pair is left as it was when its fixed part can match a separator. That happens only with a character class such as[!x]or[+-0], and there the atomic group could change the result: today*[!x]*xmatchesa/xbecause the class matches the/, and this change keeps that.Tests:
test_glob.test_translate_slowand the pathlibtest_full_match_slowandtest_glob_sloware counterparts oftest_slow_fnmatchthat use the pattern from the issue against a 100 character name. Before this change none of them finished within a 30 second timeout; with it they take a few milliseconds.test_translate_negated_class_matches_seppins the current result for*[!x]*x.test_translate*tests are updated for the*.*segment, which now translates to(?>[^/]*?\.)[^/]*.I also compared the old and new translation on 20,000 random patterns built from stars, literals, separators and character classes, with every combination of
recursive,include_hiddenandseps, against 30 random names each, and every result was the same.