pytest.ini pythonpath doesn't work with local early-loaded (-p) plugin

[a3f]

I am using pytest 7.2.1 on NixOS 23.05 and want to early-load a plugin, so I can do some custom command line parsing using pytest_load_initial_conftests. Here's my setup:

 >conftest.py     echo 'pytest_plugins = [ "my_plugin" ]'
 >my_plugin.py    echo 'print("Plugin loaded")'
 >pytest.ini      echo '[pytest]' 
>>pytest.ini      echo 'pythonpath = .' 

Normal non-early loading of plugin works:

$ pytest -q
no tests ran in 0.00s
Plugin loaded

Early-loading plugin with manual PYTHONPATH override also works:

$ PYTHONPATH="$PYTHONPATH:." pytest -p my_plugin -q
Plugin loaded

no tests ran in 0.00s

But relying on pytest.ini pythonpath for early-loading fails:

$ pytest -p my_plugin
Traceback (most recent call last):
  File "/nix/store/apjpa7zk1p61ri44n1a4mykdyqpbdvw1-python3.10-pytest-7.2.1/lib/python3.10/site-packages/_pytest/config/__init__.py", line 774, in import_plugin
    __import__(importspec)
ModuleNotFoundError: No module named 'my_plugin'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/nix/store/apjpa7zk1p61ri44n1a4mykdyqpbdvw1-python3.10-pytest-7.2.1/bin/.pytest-wrapped", line 9, in <module>
    sys.exit(console_main())
  File "/nix/store/apjpa7zk1p61ri44n1a4mykdyqpbdvw1-python3.10-pytest-7.2.1/lib/python3.10/site-packages/_pytest/config/__init__.py", line 190, in console_main
    code = main()
  File "/nix/store/apjpa7zk1p61ri44n1a4mykdyqpbdvw1-python3.10-pytest-7.2.1/lib/python3.10/site-packages/_pytest/config/__init__.py", line 148, in main
    config = _prepareconfig(args, plugins)
  File "/nix/store/apjpa7zk1p61ri44n1a4mykdyqpbdvw1-python3.10-pytest-7.2.1/lib/python3.10/site-packages/_pytest/config/__init__.py", line 329, in _prepareconfig
    config = pluginmanager.hook.pytest_cmdline_parse(
  File "/nix/store/r5wq2w8xy457h1bwx3fyb8lvlaim3gkx-python3.10-pluggy-1.0.0/lib/python3.10/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/nix/store/r5wq2w8xy457h1bwx3fyb8lvlaim3gkx-python3.10-pluggy-1.0.0/lib/python3.10/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/nix/store/r5wq2w8xy457h1bwx3fyb8lvlaim3gkx-python3.10-pluggy-1.0.0/lib/python3.10/site-packages/pluggy/_callers.py", line 55, in _multicall
    gen.send(outcome)
  File "/nix/store/apjpa7zk1p61ri44n1a4mykdyqpbdvw1-python3.10-pytest-7.2.1/lib/python3.10/site-packages/_pytest/helpconfig.py", line 103, in pytest_cmdline_parse
    config: Config = outcome.get_result()
  File "/nix/store/r5wq2w8xy457h1bwx3fyb8lvlaim3gkx-python3.10-pluggy-1.0.0/lib/python3.10/site-packages/pluggy/_result.py", line 60, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/nix/store/r5wq2w8xy457h1bwx3fyb8lvlaim3gkx-python3.10-pluggy-1.0.0/lib/python3.10/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/nix/store/apjpa7zk1p61ri44n1a4mykdyqpbdvw1-python3.10-pytest-7.2.1/lib/python3.10/site-packages/_pytest/config/__init__.py", line 1058, in pytest_cmdline_parse
    self.parse(args)
  File "/nix/store/apjpa7zk1p61ri44n1a4mykdyqpbdvw1-python3.10-pytest-7.2.1/lib/python3.10/site-packages/_pytest/config/__init__.py", line 1346, in parse
    self._preparse(args, addopts=addopts)
  File "/nix/store/apjpa7zk1p61ri44n1a4mykdyqpbdvw1-python3.10-pytest-7.2.1/lib/python3.10/site-packages/_pytest/config/__init__.py", line 1225, in _preparse
    self.pluginmanager.consider_preparse(args, exclude_only=False)
  File "/nix/store/apjpa7zk1p61ri44n1a4mykdyqpbdvw1-python3.10-pytest-7.2.1/lib/python3.10/site-packages/_pytest/config/__init__.py", line 702, in consider_preparse
    self.consider_pluginarg(parg)
  File "/nix/store/apjpa7zk1p61ri44n1a4mykdyqpbdvw1-python3.10-pytest-7.2.1/lib/python3.10/site-packages/_pytest/config/__init__.py", line 728, in consider_pluginarg
    self.import_plugin(arg, consider_entry_points=True)
  File "/nix/store/apjpa7zk1p61ri44n1a4mykdyqpbdvw1-python3.10-pytest-7.2.1/lib/python3.10/site-packages/_pytest/config/__init__.py", line 776, in import_plugin
    raise ImportError(
  File "/nix/store/apjpa7zk1p61ri44n1a4mykdyqpbdvw1-python3.10-pytest-7.2.1/lib/python3.10/site-packages/_pytest/config/__init__.py", line 774, in import_plugin
    __import__(importspec)
ImportError: Error importing plugin "my_plugin": No module named 'my_plugin'

I haven't seen anything in the docs to indicate that pythonpath is not applicable to -p. If that's intended behavior, it would be nice to amend the docs to reflect that. Also, I am curious if there's another way to get pytest -p my_plugin to just work? I'd like to add -p my_plugin to my pytest.ini eventually, so users just need to type in pytest and not have to worry about paths.

[a3f]

I haven't seen anything in the docs to indicate that pythonpath is not applicable to -p

Rereading the docs, it says: Directories remain in path for the duration of the test session and the preparse precedes that. Still, I wonder if this should be made to work. After all I can use addopts to inject an option and it is surprising, at least to me, that pythonpath defined in the same pytest.ini doesn't have an effect.

[bluetech]

You are right @a3f, currently pythonpath is implemented as an internal pytest plugin, it tries to run as early as possible, however -p my_plugin loading happens before that, necessarily.

I think that if pythonpath is to affect early plugin loading, it can't be implemented in a plugin but needs to be implemented in Config directly. As is, I think the complexity is not quite worth it.

Agreed about mentioning this gotcha in the docs.

I am using pytest 7.2.1 on NixOS 23.05 and want to early-load a plugin, so I can do some custom command line parsing using pytest_load_initial_conftests.

Can you describe a bit more what the plugin does? Maybe we can suggest a different way to achieve it.

[a3f]

Thanks for the confirmation.

I wanted to support following command line syntax:

pytest [pytest-options] path/to/tests -- [extra-options and arguments]

Or alternatively:

pytest [pytest-options] path/to/tests --qemu [extra-options and arguments]

These extra-options and arguments would be passed as-is to Qemu, which the pytest tests are running against. My idea was to have an early plugin hook command line parsing to facilitate this.

[bluetech]

The --qemu one looks like something that should be achievable just through pytest_addoption, at least if you're OK with requiring that it comes last. Something like this:

import argparse

import pytest


def pytest_addoption(parser: pytest.Parser) -> None:
    parser.addoption("--qemu", nargs=argparse.REMAINDER)


# Use the option
def pytest_configure(config: pytest.Config) -> None:
    qemu_opts = config.getoption("qemu") or []

[a3f]

@bluetech Nice, Thanks a lot! This works fine for me. :)