[pathlib] Option so that OSError does not block glob in pathlib library

[matt32106]

BPO	46050
Nosy	@matt32106

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-12-11.16:25:35.541>
labels = ['3.8', 'type-feature', 'library']
title = '[pathlib] Option so that OSError does not block glob in pathlib library'
updated_at = <Date 2021-12-11.16:25:35.541>
user = 'https://github.com/matt32106'

bugs.python.org fields:

activity = <Date 2021-12-11.16:25:35.541>
actor = 'matt32106'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2021-12-11.16:25:35.541>
creator = 'matt32106'
dependencies = []
files = []
hgrepos = []
issue_num = 46050
keywords = []
message_count = 1.0
messages = ['408314']
nosy_count = 1.0
nosy_names = ['matt32106']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue46050'
versions = ['Python 3.8']

Linked PRs

• GH-90208: Suppress OSError exceptions from pathlib.Path.glob() #104141

[matt32106]

Hi there,
ISSUE DESCRIPTION
when I browse starting from the linux root ('/')
path = pathlib.Path('/')
_glob = '**/*'
for p in path.glob(_glob):

The program stops on my machine because of OSError.
File "/usr/lib/python3.8/pathlib.py", line 535, in _select_from
entries = list(scandir_it)

OSError: [Errno 22] Invalid argument: '/proc/5916/task/5916/net'

---

Entering post mortem debugging...

---

/usr/lib/python3.8/pathlib.py(535)_select_from()
533 try:
534 with scandir(parent_path) as scandir_it:
--> 535 entries = list(scandir_it)
536 for entry in entries:
537 if self.dironly:

I also another case is if a cloud drive is disconnected.

QUICK SOLUTION
I solved both issues for me by adding an adding an except OSError: return at the end of the two function below:

class _WildcardSelector(_Selector):
    def _select_from(self, parent_path, is_dir, exists, scandir):
        (...)
        except OSError:
            return

class _RecursiveWildcardSelector(_Selector):
    def _iterate_directories(self, parent_path, is_dir, scandir):
        (...)
        except OSError:
            return

FEATURE REQUEST
I don't know the consequences of those 2 modifications so perhaps they could follow an option when calling glob with an optional parameter ignoreOSerror = false by default?!

[barneygale]

IMO it should ignore all OSError exceptions that are generated below the top-level path. PR incoming...

[barneygale]

Fixed in #104141 / 94f30c7