[Feature]: bind python and c++ through tools other than pybind11

[youkaichao]

🚀 The feature, motivation and pitch

As vLLM goes into a fast release schedule (currently one release every two weeks), we will quickly hit the project-wide limit of pypi (around 5GB per project). One solution, as pointed out in pypi/support#3792 (comment) , is to build one wheel for all python versions (Python 3.8+).

I have figured out the procedure pypi/support#3792 (comment) , but pybind11 does not support this Python Limited API protocol.

One possible solution is to replace pybind11 with some other tools, so that the binding procedure can be used with Python Limited API.

Possible solutions:

• Nanobind (seems to support it starting from Python 3.12 only: Support the stable ABI for all supported CPython versions wjakob/nanobind#561 )
• register ops through pytorch directly https://pytorch.org/tutorials/advanced/torch_script_custom_ops.html

Alternatives

No response

Additional context

No response

[youkaichao]

cc @simon-mo