Skip to content

gh-116608: Bring back importlib.resources functional API #116609

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
encukou merged 12 commits into from
Apr 5, 2024
Merged

gh-116608: Bring back importlib.resources functional API #116609

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
bdc3209
gh-116608: Bring back importlib.resources functional API
encukou Mar 7, 2024
9b065b1
Docs fix
encukou Mar 11, 2024
265ae91
Require explicit encoding arg for ambiguous cases
encukou Mar 11, 2024
9176619
tests: Expect OSError -- we can get PermissionError instead of IsADir…
encukou Mar 12, 2024
33466da
Merge in the main branch
encukou Mar 20, 2024
56603fe
Allow separators in path segments
encukou Mar 20, 2024
4d276e0
Merge in the main branch
encukou Mar 21, 2024
a520a78
Fix issues & nitbicks found in backport
encukou Mar 21, 2024
e788bf3
PEP-8 formatting
encukou Mar 21, 2024
09c7bf7
Docs/ReST fix
encukou Mar 21, 2024
e15331c
Add a leading underscore to the module name
encukou Apr 2, 2024
de5b0d5
Merge in the main branch
encukou Apr 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Allow separators in path segments
  • Loading branch information
@encukou
encukou committed Mar 20, 2024
commit 56603feb76a07afe1946b61312f566095ec1f47f
37 changes: 19 additions & 18 deletions Doc/library/importlib.resources.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -115,19 +115,20 @@ For all the following functions:

- *path_names* are components of a resource's path name, relative to
the anchor.
The individual components may not contain path separators.
For example, to get the text of resource named ``info.txt``, use::

importlib.resources.read_text(my_module, "info.txt")

To get the contents of ``pics/painting.png`` as bytes, use::
Like :meth:`Traversable.joinpath <importlib.resources.abc.Traversable>`,
The individual components should use forward slashes `/` as path separators.
The following are equivalent::

importlib.resources.read_binary(my_module, "pics/painting.png")
importlib.resources.read_binary(my_module, "pics", "painting.png")

For backward compatibility reasons, functions that read text require
an explicit *encoding* argument if multiple *path_names* are given.

So, to get the text of ``info/chapter1.txt``, use::
For example, to get the text of ``info/chapter1.txt``, use::

importlib.resources.read_text(my_module, "info", "chapter1.txt",
encoding='utf-8')
Expand All @@ -142,9 +143,9 @@ For all the following functions:
This function returns a :class:`~typing.BinaryIO` object,
that is, a binary stream open for reading.

For a single path name *name*, this function is roughly equivalent to::
This function is roughly equivalent to::

files(anchor).joinpath(name).open('rb')
files(anchor).joinpath(*path_names).open('rb')

.. versionchanged:: 3.13
Multiple *path_names* are accepted.
Expand All @@ -166,9 +167,9 @@ For all the following functions:
This function returns a :class:`~typing.TextIO` object,
that is, a text stream open for reading.

For a single path name *name*, this function is roughly equivalent to::
This function is roughly equivalent to::

files(anchor).joinpath(name).open('r', encoding=encoding)
files(anchor).joinpath(*path_names).open('r', encoding=encoding)

.. versionchanged:: 3.13
Multiple *path_names* are accepted.
Expand All @@ -182,9 +183,9 @@ For all the following functions:
See :ref:`the introduction <importlib_resources_functional>` for
details on *anchor* and *path_names*.

For a single path name *name*, this function is roughly equivalent to::
This function is roughly equivalent to::

files(anchor).joinpath(name).read_bytes()
files(anchor).joinpath(*path_names).read_bytes()

.. versionchanged:: 3.13
Multiple *path_names* are accepted.
Expand All @@ -203,9 +204,9 @@ For all the following functions:
explicitly if there are multiple *path_names*.
This limitation is scheduled to be removed in Python 3.15.

For a single path name *name*, this function is roughly equivalent to::
This function is roughly equivalent to::

files(anchor).joinpath(name).read_text(encoding=encoding)
files(anchor).joinpath(*path_names).read_text(encoding=encoding)

.. versionchanged:: 3.13
Multiple *path_names* are accepted.
Expand All @@ -230,9 +231,9 @@ For all the following functions:
See :ref:`the introduction <importlib_resources_functional>` for
details on *anchor* and *path_names*.

For a single path name *name*, this function is roughly equivalent to::
This function is roughly equivalent to::

as_file(files(anchor).joinpath(name))
as_file(files(anchor).joinpath(*path_names))

.. versionchanged:: 3.13
Multiple *path_names* are accepted.
Expand All @@ -247,9 +248,9 @@ For all the following functions:
See :ref:`the introduction <importlib_resources_functional>` for
details on *anchor* and *path_names*.

For a single path name *name*, this function is roughly equivalent to::
This function is roughly equivalent to::

files(anchor).joinpath(name).is_file()
files(anchor).joinpath(*path_names).is_file()

.. versionchanged:: 3.13
Multiple *path_names* are accepted.
Expand All @@ -265,9 +266,9 @@ For all the following functions:
See :ref:`the introduction <importlib_resources_functional>` for
details on *anchor* and *path_names*.

For a single path name *name*, this function is roughly equivalent to::
This function is roughly equivalent to::

for resource in files(anchor).joinpath(name).iterdir():
for resource in files(anchor).joinpath(*path_names).iterdir():
yield resource.name

.. deprecated:: 3.11
Expand Down
11 changes: 1 addition & 10 deletions Lib/importlib/resources/functional.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -85,13 +85,4 @@ def _get_encoding_arg(path_names, encoding):
def _get_resource(anchor, path_names):
if anchor is None:
raise TypeError("anchor must be module or string, got None")
traversable = files(anchor)
for name in path_names:
str_path = str(name)
parent, file_name = os.path.split(str_path)
if parent:
raise ValueError(
'path name elements must not contain path separators, '
f'got {name!r}')
traversable = traversable.joinpath(file_name)
return traversable
return files(anchor).joinpath(*path_names)
65 changes: 43 additions & 22 deletions Lib/test/test_importlib/resources/test_functional.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,17 @@ class ModuleAnchorMixin:


class FunctionalAPIBase():
def _gen_resourcetxt_path_parts(self):
"""Yield various names of a text file in anchor02, each in a subTest
"""
for path_parts in (
('subdirectory', 'subsubdir', 'resource.txt'),
('subdirectory/subsubdir/resource.txt',),
('subdirectory/subsubdir', 'resource.txt'),
):
with self.subTest(path_parts=path_parts):
yield path_parts

def test_read_text(self):
self.assertEqual(
importlib.resources.read_text(self.anchor01, 'utf-8.file'),
Expand All @@ -33,6 +44,13 @@ def test_read_text(self):
),
'a resource',
)
for path_parts in self._gen_resourcetxt_path_parts():
self.assertEqual(
importlib.resources.read_text(
self.anchor02, *path_parts, encoding='utf-8',
),
'a resource',
)
# Use generic OSError, since e.g. attempting to read a directory can
# fail with PermissionError rather than IsADirectoryError
with self.assertRaises(OSError):
Expand Down Expand Up @@ -62,21 +80,21 @@ def test_read_binary(self):
importlib.resources.read_binary(self.anchor01, 'utf-8.file'),
b'Hello, UTF-8 world!\n',
)
self.assertEqual(
importlib.resources.read_binary(
self.anchor02, 'subdirectory', 'subsubdir', 'resource.txt',
),
b'a resource',
)
for path_parts in self._gen_resourcetxt_path_parts():
self.assertEqual(
importlib.resources.read_binary(self.anchor02, *path_parts),
b'a resource',
)

def test_open_text(self):
with importlib.resources.open_text(self.anchor01, 'utf-8.file') as f:
self.assertEqual(f.read(), 'Hello, UTF-8 world!\n')
with importlib.resources.open_text(
self.anchor02, 'subdirectory', 'subsubdir', 'resource.txt',
encoding='utf-8',
) as f:
self.assertEqual(f.read(), 'a resource')
for path_parts in self._gen_resourcetxt_path_parts():
with importlib.resources.open_text(
self.anchor02, *path_parts,
encoding='utf-8',
) as f:
self.assertEqual(f.read(), 'a resource')
# Use generic OSError, since e.g. attempting to read a directory can
# fail with PermissionError rather than IsADirectoryError
with self.assertRaises(OSError):
Expand Down Expand Up @@ -104,10 +122,11 @@ def test_open_text(self):
def test_open_binary(self):
with importlib.resources.open_binary(self.anchor01, 'utf-8.file') as f:
self.assertEqual(f.read(), b'Hello, UTF-8 world!\n')
with importlib.resources.open_binary(
self.anchor02, 'subdirectory', 'subsubdir', 'resource.txt',
) as f:
self.assertEqual(f.read(), b'a resource')
for path_parts in self._gen_resourcetxt_path_parts():
with importlib.resources.open_binary(
self.anchor02, *path_parts,
) as f:
self.assertEqual(f.read(), b'a resource')

def test_path(self):
with importlib.resources.path(self.anchor01, 'utf-8.file') as path:
Expand All @@ -123,6 +142,8 @@ def test_is_resource(self):
self.assertFalse(is_resource(self.anchor01, 'no_such_file'))
self.assertFalse(is_resource(self.anchor01))
self.assertFalse(is_resource(self.anchor01, 'subdirectory'))
for path_parts in self._gen_resourcetxt_path_parts():
self.assertTrue(is_resource(self.anchor02, *path_parts))

def test_contents(self):
is_resource = importlib.resources.is_resource
Expand All @@ -133,9 +154,14 @@ def test_contents(self):
{'utf-8.file', 'utf-16.file', 'binary.file', 'subdirectory'},
)
with (self.assertRaises(OSError),
check_warnings((".*contents.*", DeprecationWarning)),
):
check_warnings((".*contents.*", DeprecationWarning)),
):
importlib.resources.contents(self.anchor01, 'utf-8.file')
for path_parts in self._gen_resourcetxt_path_parts():
with (self.assertRaises(OSError),
check_warnings((".*contents.*", DeprecationWarning)),
):
importlib.resources.contents(self.anchor01, *path_parts)
with check_warnings((".*contents.*", DeprecationWarning)):
c = importlib.resources.contents(self.anchor01, 'subdirectory')
self.assertGreaterEqual(
Expand All @@ -155,11 +181,6 @@ def test_common_errors(self):
importlib.resources.contents,
):
with self.subTest(func=func):
# Rejecting path separators
with self.assertRaises(ValueError):
func(self.anchor02, os.path.join(
'subdirectory', 'subsubdir', 'resource.txt',
))
# Rejecting None anchor
with self.assertRaises(TypeError):
func(None)
Expand Down