Add type stubs to the Python module

[TimothyEDawson]

Changes proposed in this pull request

• Add Python stub (.pyi) files to the Cython interface which cover the public API for the Python code, including runtime-generated attributes.
• Include the stub files in the installed Python package.
• Add testing infrastructure to verify consistent typing and adequate coverage.

There was some discussion about the approach here, but in short I consider this to be a stepping stone toward adding static typing to the Cython interface. Some advantage of starting with stub files include:

• The full public API is made immediately available for developers who utilize Cantera as a dependency in their Python projects (like me!).
• Work on type hints is fully segregated from the actual Python code as many details are worked out and the testing infrastructure is set up.
• It is substantially easier to add typing for the Python API without contending with Cython syntax.
• It makes dynamically-generated parts of the API, such as pass-through attributes from a Solution to a Quantity or SolutionArray, trivial to type-hint, and may (?) be the correct way to do so.

There are some disadvantages, with the largest perhaps being the substantial increase in the amount of code to be maintained going forward. My hope is that the majority of the stub file content is gradually replaced with type hints within the source code, and I'm aware that I already have some redundant type hints in here.

If applicable, fill in the issue number this pull request is fixing

Potentially closes Cantera/enhancements#85.

Checklist

• The pull request includes a clear description of this code change
• Commit messages have short titles and reference relevant issues
• Build passes (scons build & scons test) and unit tests address code coverage
• Style & formatting of contributed code follows contributing guidelines
• The pull request is ready for review

[TimothyEDawson]

Some notes:

I covered a good chunk of the public API already, but I am aware of some holes here and there which I plan to fill in as time permits. I have deliberately omitted the various file conversion scripts (e.g. ck2yaml.py), but am not opposed to including them here.

I still need to figure out a good way to add unit testing for the type hints. I expect that would look like liberal use of typing.assert_type, as used in scipy-stubs, but to get and maintain thorough coverage may require some thought on infrastructure.

Several decisions were a little impulsive and I'm sure there are many improvements and changes needed. Some examples:

• I have defined numerous TypeAliases for the sake of convenience, many of which should probably start with an underscore so they are not included as part of the public API.
• Any is used in several places where it may be possible to replace it with an Unpacked TypedDict or other option.
• I started to add __all__ in some areas when that should probably be a separate pull request to add it to the source files.
• __init__.pyi is quite cumbersome. I attempted to turn all of the implicit imports into explicit imports, then removed the exported symbols which originate from third-party libraries, but I'm sure there is a more elegant way to do this. I think that thorough use of __all__ might make the explicit imports fully redundant.
• NumPy typing always feels like there are too many ways to do things, so I went with a very minimal approach of assuming all ndarrays are arbitrarily-sized np.typing.NDArray[np.float64] which I call Array (should probably be _ArrayFloat64 or something), and anything which is specifically going to be coerced into an ndarray is typed as np.typing.ArrayLike. I'm aware some packages like optype and NumType may provide more descriptive and flexible type hints, but I don't want to add any external dependencies unless absolutely necessary.
• I did not use backwards-compatibility stuff like from typing_extensions import because I was under the impression those are largely intended for source code, but I need to look into it more to see if those imports should be modified.
• The stubs simply assume you have the necessary optional dependencies to work, such as Pandas. I'm not certain whether that's the correct way to handle them.

And I'm sure there's more, I just wanted to start getting feedback now rather than keep postponing this pull request forever!

[ischoegl]

Thanks for your efforts on this, @TimothyEDawson - this looks really promising! I am, however, somewhat concerned about maintaining parallel signatures in pyx and pyi files: for any change or addition, things need to be edited in two (usually large!) parallel files, which, imho, is not ideal.

Based on what you suggest here, I ran a couple of quick tests: specifically, I was interested in whether type hints are preserved if they are added directly in the pyx file. I ran some rudimentary tests on interfaces/cython/cantera/_utils.pyx (i.e., one of the shorter files), without adding AnyMap.

Based on successful tests, it appears that adding type hints directly to pyx may be more maintainable than keeping separate pyi files. However, this assumes that things can be combined, i.e., where static types are known, add it to pyx, whereas dynamic properties and methods may have to be added via pyi.

You can find my (very rudimentary) test here: https://github.com/ischoegl/cantera/tree/test-type-hints (just one commit) ... I simply copied over what you had in the pyi file.

[TimothyEDawson]

@ischoegl yes, we're on the same page. That was the same concern I highlighted in the pull request text.

There's no inherent issues with .pyx files as they're generally treated the same as .py files as far as type hints go, that was not part of the motivation for this. There's an order of resolution to type hints as defined here: https://typing.python.org/en/latest/spec/distributing.html#partial-stub-packages Because stub files take precedence over inline type hints, the path forward would essentially be to delete type hints from the stub files as they are added to the source code files.

I am open to whichever approach the Cantera developers wish to take. The two options which seem best to me are:

• Merge in a full set of stubs with this pull request, then gradually work toward moving type hints into the inline code via subsequent pull requests.
• Work towards inlining the type hints as much as possible within this pull request, stopping when it's either complete (no more .pyi files) or it gets too tricky (arbitrary).

I lean toward the first option because it gives users a working set of type hints while we work out details for the inline version. It also means I don't even need to touch any .pyx files. Maintainability is a concern, but mostly just for API changes, and any new functionality which is properly type-hinted inline won't need to be added to the .pyi files at all.

At the same time, realistically it will be trivial to move the majority of these type hints inline right now. I expect there will be some edge cases which might take a while to figure out what to do, but if that's the route we want to go, I'd be happy to try it.