Allow path objects for data-files

[jaraco]

• Allow path objects
• Need to include 'three' in the input.
• Consolidate str and Path handling.
• Expand convert_path to also accept pathlib.Path objects.
• Prefer simply 'pathlib' for import.
• Extract a singledispatchmethod _copy for handling the copy of each data file.
• Use explicit registration for compatibility with older Pythons.
• Prefer os.PathLike in convert_path

[jaraco]

It's not apparent from the tracebacks, because pytest is currently not doing assertion rewriting (pytest-dev/pytest#12044), but the tests are catching an emergent failure with this approach.

When one uses os.fspath(pathlib.WindowsPath('/foo')), it emits '\\foo'. Even str(pathlib.WindowsPath('/foo')) does the same.

That suggests that it's not possible in general to accept pathlib objects because users may use pathlib.Path on Windows, implying pathlib.WindowsPath, which will mangle the forward slashes to backslashes.

It does look like it may be possible to extract the Path._raw_path to get the raw underlying string, but that's a private variable and not accessible. It does also look like it might be possible to convert a WindowsPath to a PurePosixPath and that will give the forward slashes:

>>> pathlib.PurePosixPath(pathlib.Path('/home'))
PurePosixPath('/home')

Yucky, but maybe works.

[jaraco]

Nope. Because casting to a PurePosixPath also strips the trailing slash:

>>> pathlib.PurePosixPath(pathlib.Path('/home/'))
PurePosixPath('/home')

So suppresses the check if a trailing slash was supplied.

The mere act of constructing a PurePosixPath strips any trailing slashes:

>>> pathlib.PurePosixPath('/home/')
PurePosixPath('/home')

This makes me wonder - how important is it to fail when a trailing slash has been supplied? If pathlib is silently stripping trailing slashes, maybe convert_path should do so too.

The more I think about it, the more I think that's the right approach, since users supplying pathlib.Path('foo/') will also get the stripping behavior.

Such a simple change so fraught with peril.

[jaraco]

It's more complicated than I could have imagined. Now the tests are failing with:

_________________________ TestUtil.test_convert_path __________________________

self = <distutils.tests.test_util.TestUtil object at 0x000001BF8BCC6520>

    def test_convert_path(self):
        # linux/mac
        os.sep = '/'
        os.path.join = posixpath.join
    
        assert convert_path('/home/to/my/stuff') == '/home/to/my/stuff'
>       assert convert_path(pathlib.Path('/home/to/my/stuff')) == '/home/to/my/stuff'
E       AssertionError: assert '\\/home/to/my/stuff' == '/home/to/my/stuff'
E         
E         - /home/to/my/stuff
E         + \/home/to/my/stuff
E         ? +

distutils/tests/test_util.py:74: AssertionError

On Windows only.

But curiously, I don't encounter that behavior when I test it myself.

[jaraco]

It seems that the behavior of one or more of os.fspath, pathlib.Path, or pathlib.PurePosixPath changed between Python 3.8 and 3.12:

 ~ @# py -3.12 -c "import pathlib, os; print(os.fspath(pathlib.PurePosixPath(pathlib.Path('/home/stuff'))))"
/home/stuff
 ~ @# py -3.8 -c "import pathlib, os; print(os.fspath(pathlib.PurePosixPath(pathlib.Path('/home/stuff'))))"
\/home/stuff

It's the construction of the PurePosixPath on Windows:

 ~ @# py -3.8 -c "import pathlib, os; print(pathlib.Path('/home/stuff'))"
\home\stuff
 ~ @# py -3.8 -c "import pathlib, os; print(pathlib.PurePosixPath(pathlib.Path('/home/stuff')))"
\/home/stuff

And that behavior remains through Python 3.11.

[jaraco]

I'm going to have to sleep on this one. It's a tricky one.

[jaraco]

I set out to try to understand when the behavior changed for PurePosixPath, and I suspect it was python/cpython#101362. I was thinking I might grab a backport of pathlib, such as pathlib2, but I can see it hasn't seen a release since well before the fix in Python 3.12. While reading that issue, however, I was reminded of the .as_posix() function, which may prove more reliable than os.fspath(pathlib.PurePosixPath(...)).

[jaraco]

With this latest change, tests are still failing... because ntpath.join('C:', 'Users') -> 'C:Users', which doesn't exist, so make_native('C:/Users') produces that invalid path and causes tests with absolute paths in them to fail. What's weird and annoying is that I don't know how this could have worked before.

Oh! I see. Tests like test_simple_run are generating cmd.data_files=[temp_path] where temp_path is an absolute path on Windows (with drive letter and backslash separators). In main, when that temp_path passes through convert_path, it's returned unchanged because 'C:\\Users'.split('/') is simply ['C:\\Users'] and so joins to itself. The tests are improperly exercising the wrong things.

• it's passing absolute paths, which are meant to be disallowed, but since it includes the C: prefix, it's not detected as absolute.
• it's using the wrong separator characters (it's supposed to be separated by posixpath.sep).

[jaraco]

After what I've learned, I'm pretty confident that the convert_path function can be drastically simplified based on pathlib.