Compile everything with meson

[tobiasdiez]

Add meson configuration to compile sagelib with meson. Basic developer docs are added as well (meant as background info, not real installation instructions and thus under "developer").

I had to add a few empty __init__.py files due to relative imports that didn't worked otherwise or due to cython otherwise creating wrong module names (which then fails on runtime).

Also had to remove some code that relied on Cython env variables because I was not able to pass it Python version as an integer (cython has -E but that results in a str variable). For the same reason, I deleted old unused code for cygwin support.

📝 Checklist

• The title is concise, informative, and self-explanatory.
• The description explains in detail what this PR is about.
• I have linked a relevant issue or discussion.
• I have created tests covering the changes.
• I have updated the documentation accordingly.

⌛ Dependencies

• Readd init files #36753

[mkoeppe]

Note that the sage.categories namespace package is not shipped by one distribution, but rather split into two distribution packages - sagemath-objects and sagemath-categories.

[tobiasdiez]

Note that the sage.categories namespace package is not shipped by one distribution, but rather split into two distribution packages - sagemath-objects and sagemath-categories.

I don't see how this is relevant here. I'm trying to build the full sagelib with meson. Just started with a small part to make reviewing easier

[mkoeppe]

Just a heads up that the monolithic build is going to go away very soon. So it would be unwise to design a new system for the monolithic library.

[dimpase]

Here is how I'm trying to test this. I create a standard venv, activate it, and pip install few things there, like scipy, pytest, etc.

Then there I run pip install .. It errors out very quickly with

Processing /mnt/opt/Sage/sage-dev
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [24 lines of output]
      + meson setup /mnt/opt/Sage/sage-dev /mnt/opt/Sage/sage-dev/.mesonpy-xave0p3t -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md --native-file=/mnt/opt/Sage/sage-dev/.mesonpy-xave0p3t/meson-python-native-file.ini
      The Meson build system
      Version: 1.4.1
      Source dir: /mnt/opt/Sage/sage-dev
      Build dir: /mnt/opt/Sage/sage-dev/.mesonpy-xave0p3t
      Build type: native build
      Project name: SageMath
      Project version: 10.4.beta8
      C compiler for the host machine: ccache cc (gcc 13.2.1 "cc (Gentoo 13.2.1_p20240210 p14) 13.2.1 20240210")
      C linker for the host machine: cc ld.bfd 2.42
      C++ compiler for the host machine: ccache c++ (gcc 13.2.1 "c++ (Gentoo 13.2.1_p20240210 p14) 13.2.1 20240210")
      C++ linker for the host machine: c++ ld.bfd 2.42
      Cython compiler for the host machine: cython (cython 3.0.10)
      Host machine cpu family: x86_64
      Host machine cpu: x86_64
      Program python found: YES (/mnt/opt/Sage/sage-dev/a/bin/python3)
      Found pkg-config: YES (/usr/bin/pkg-config) 2.2.0
      Run-time dependency python found: YES 3.11
      Program python (pytest) found: NO
      
      ../src/meson.build:7:12: ERROR: Command `/mnt/opt/Sage/sage-dev/a/bin/python3 -c 'import numpy
      print(numpy.get_include())'` failed with status 1.
      
      A full log can be found at /mnt/opt/Sage/sage-dev/.mesonpy-xave0p3t/meson-logs/meson-log.txt
      [end of output]

which seems to indicate that it refuses to use packages etc from the venv (as it won't find pytest, and won't find numpy).

If instead I invoke pip install . from src/ then it fails, after a while, with
a weird error with jinja2 (which I do have in the venv I am using).

Building wheels for collected packages: sagemath-standard
  Building wheel for sagemath-standard (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for sagemath-standard (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [32 lines of output]
      ************************************************************************
      Traceback (most recent call last):
        File "/mnt/opt/Sage/sage-dev/a/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/mnt/opt/Sage/sage-dev/a/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/mnt/opt/Sage/sage-dev/a/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
          return _build_backend().build_wheel(wheel_directory, config_settings,
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-4osujpv5/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 410, in build_wheel
          return self._build_with_temp_dir(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-4osujpv5/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 395, in _build_with_temp_dir
          self.run_setup()
        File "/tmp/pip-build-env-4osujpv5/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 311, in run_setup
          exec(code, locals())
        File "<string>", line 74, in <module>
        File "/mnt/opt/Sage/sage-dev/src/sage_setup/autogen/__init__.py", line 3, in <module>
          from . import interpreters
        File "/mnt/opt/Sage/sage-dev/src/sage_setup/autogen/interpreters/__init__.py", line 117, in <module>
          from .generator import AUTOGEN_WARN, InterpreterGenerator
        File "/mnt/opt/Sage/sage-dev/src/sage_setup/autogen/interpreters/generator.py", line 19, in <module>
          from .memory import string_of_addr
        File "/mnt/opt/Sage/sage-dev/src/sage_setup/autogen/interpreters/memory.py", line 16, in <module>
          from .utils import je, reindent_lines as ri
        File "/mnt/opt/Sage/sage-dev/src/sage_setup/autogen/interpreters/utils.py", line 19, in <module>
          from jinja2 import Environment
      ModuleNotFoundError: No module named 'jinja2'
      ************************************************************************
      Error building the Sage library
      ************************************************************************
      [end of output]

that is, again somewhere the python packages path gets poisoned, I suppose.

[mkoeppe]

It does not matter what you have in your venv. Build isolation.