strange order changes using pkgutils namespace packages

[finswimmer]

Hello,

at poetry we are isort to sort our imports.

A contributor now stumble upon a strange behavior of isort. The PR is located here: python-poetry/poetry#4985

In our __init__.py in the root folder we have this:

__path__ = __import__("pkgutil").extend_path(__path__, __name__)  # type: ignore[has-type]

The PR changed it to:

__path__ = __import__("pkgutil").extend_path(
    __path__,  # type: ignore[has-type]
    __name__,
)

See: https://github.com/python-poetry/poetry/pull/4985/files#diff-796398bc03bf8ac0368aba22f7416c9948776a4a666e779700f08bd532c27249

Now all imports of poetry.core (which is separate namespace package) are sorted to a different place, e.g.:

from poetry.core.toml.file import TOMLFile
from tomlkit.toml_document import TOMLDocument

becomes

from tomlkit.toml_document import TOMLDocument

from poetry.core.toml.file import TOMLFile

To be honest I'm not sure which sorting is the expected one. But I would expect, that the line breaks in the above code, shouldn't change any sorts by isort.

Any ideas what's going on?

Thanks a lot.

fin swimmer

[timothycrosley]

Hi!

So sorry you ran into this corner case! Older versions of Python didn't have implicit namespace package support, which allows namespaces to be created in a way that's statically checkable with code parsers, however there was still fairly strict requirements on what command is called to make it a namespace package for these older versions, which is what isort checks (since it can't look at the env etc.)

This is fairly simple logic which lives here:

isort/isort/place.py

Line 113 in c2ca2e2

def _is_namespace_package(path: Path, src_extensions: FrozenSet[str]) -> bool:

, and certainly should be improved to at least allow extra newlines etc.

You can add the namespace to namespace_packages in your isort config, to force isort to pick it up correctly, if this mechanism fails. The enabled of this auto detection method itself is toggled via https://pycqa.github.io/isort/docs/configuration/options.html#auto-identify-namespace-packages.

I hope this is helpful!

Thanks,

~Timothy

[finswimmer]

Thanks a lot @timothycrosley for your answer and explanation. Adding poetry to the list of namespace_packages helps 👍

Feel free to close the issue or leave it open as a reminder for a possible improvement.

fin swimmer