Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fails to build on Python 3.11 - longintrepr.h: No such file or directory #6600

Closed
1 task done
gaborbernat opened this issue Feb 8, 2022 · 75 comments
Closed
1 task done
Labels

Comments

@gaborbernat
Copy link

gaborbernat commented Feb 8, 2022

Describe the bug

Because of python/cpython#28968, cython fixed it in cython/cython#4428 and is released with 0.29.5

To Reproduce

pip install aiohttp

Expected behavior

To build the wheel.

Logs/tracebacks

Failed to build aiohttp yarl
  error: subprocess-exited-with-error

  × Building wheel for aiohttp (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [31 lines of output]
      *********************
      * Accelerated build *
      *********************
      running bdist_wheel
      running build
      running build_py
      running egg_info
      writing aiohttp.egg-info/PKG-INFO
      writing dependency_links to aiohttp.egg-info/dependency_links.txt
      writing requirements to aiohttp.egg-info/requires.txt
      writing top-level names to aiohttp.egg-info/top_level.txt
      reading manifest file 'aiohttp.egg-info/SOURCES.txt'
      reading manifest template 'MANIFEST.in'
      warning: no files found matching 'aiohttp' anywhere in distribution
      warning: no previously-included files matching '*.pyc' found anywhere in distribution
      warning: no previously-included files matching '*.pyd' found anywhere in distribution
      warning: no previously-included files matching '*.so' found anywhere in distribution
      warning: no previously-included files matching '*.lib' found anywhere in distribution
      warning: no previously-included files matching '*.dll' found anywhere in distribution
      warning: no previously-included files matching '*.a' found anywhere in distribution
      warning: no previously-included files matching '*.obj' found anywhere in distribution
      warning: no previously-included files found matching 'aiohttp/*.html'
      no previously-included directories found matching 'docs/_build'
      adding license file 'LICENSE.txt'
      running build_ext
      building 'aiohttp._websocket' extension
      aiohttp/_websocket.c:198:12: fatal error: longintrepr.h: No such file or directory
        198 |   #include "longintrepr.h"
            |            ^~~~~~~~~~~~~~~
      compilation terminated.
      error: command '/usr/bin/gcc' failed with exit code 1
      [end of output]

Python Version

3.11

aiohttp Version

latest

multidict Version

n/a

yarl Version

latest

OS

Any

Related component

Server, Client

Additional context

No response

Code of Conduct

  • I agree to follow the aio-libs Code of Conduct
@gaborbernat gaborbernat added the bug label Feb 8, 2022
@webknjaz
Copy link
Member

webknjaz commented Feb 15, 2022

Thanks! It usually doesn't make sense for us to be working on the next interpreter support until rc are out because the build deps like Cython can't keep up with the ABI changes anyway.

@gaborbernat
Copy link
Author

Perhaps RC is a bit too late, IMHO beta should be ok, no? 😊 (this blocks us from testing tox)

@webknjaz
Copy link
Member

I don't think it's up to us really. We can enable testing in the CI (I would be actually in favor if that was realistic) but Cython having to go through upgrades first is traditionally a blocker that is out of our control. This usually results in this effort being rather pointless until the ABI is stable and Cython supports the respective changes. Also, Cython still isn't able to produce abi3 wheels which I suspect would greatly improve the situation for everyone.

@webknjaz
Copy link
Member

webknjaz commented Feb 15, 2022

@gaborbernat you say that it's fixed in Cython that is released already but the problem is building wheels from the published sdist? Are you asking for the release of a new version with C-files generated with newer Cython? If that's an action item right now, I think that it's doable.

@gaborbernat
Copy link
Author

Are you asking for the release of a new version with C-files generated with newer Cython? If that's an action item right now, I think that it's doable.

I think this is the case 🤔

@sagar1025
Copy link

Are you asking for the release of a new version with C-files generated with newer Cython? If that's an action item right now, I think that it's doable.

I think this is the case 🤔

Seems that way.
cython/cython#4428

achimnol added a commit to lablup/backend.ai that referenced this issue May 27, 2022
* Without the upper bound, it tries to use Python 3.11 and aiohttp
  fails to build. (aio-libs/aiohttp#6600)
achimnol added a commit to lablup/backend.ai that referenced this issue May 31, 2022
* We use Pants (www.pantsbuild.org) as our build toolchain.
  - Details will follow in subsequent document updates.
* Add a custom setup generator plugin for Pants
  - It single-sources the version number from VERSION.
  - It takes the description, license, name from kwargs.
  - It validates the package name to start with "backend.ai-".
  - It takes the long-description from package-specific README.
* Add a custom platform-specific dependency selector for Pants
  - It provides `platform_resources()` target with a per-platform
    dependency map. (Thanks to Andreas Stenius)
* Move mypy/pytest configs to root `pyproject.toml`
  - flake8 does not support pyproject.toml yet (PyCQA/flake8#234),
    so keep it at `.flake8`
  - Explicitly add `setuptools` as requirement of flake8 because flake8
    uses `pkg_resources` to detect its own plugin entrypoints.
* Implement entrypoint scan with BUILD files for CLI
* Our new plugin subsystem (`ai.backend.plugin`) uses
  `importlib`-based entrypoints.
  - This removes dependency to setuptools at runtime.
* Update gitignore
* Add `./py`, `./backend.ai` shortcuts to run commands in exported venv
  - Finally implement truly unified CLI via `./backend.ai`!
* Rewrite GitHub Actions workflows
* Notable non-trivial Pants configs
  - Set `[GLOBAL].local_execution_root_dir` to a non-tmp directory
    because Snap-based Docker cannot access it!
  - `aiosqlite` is not explicitly imported but specified as a SQLAlchemy
    server URL scheme, so declare a manual dependency in manager tests.
  - Add `[pytest].execution_slot_var` config for test parallelization
* Mark storage-proxy tests as "integration" that require external dependency
  - Integration tests are skipped!
* Update test fixtures
  - Spawn a single-node etcd container with OS-assigned port numbers
  - Self-bootstrap db containers for isolated and parallel testing
* Reorganize packages
  - ai.backend.helpers -> backend.ai-kernel-helper
  - ai.backend.kernel -> backend.ai-kernel
  - ai.backend.runner -> backend.ai-kernel-binary
* Let it build packages for only Python 3.10
  - Without the upper bound, it tries to use Python 3.11-dev if available
    but aiohttp fails to build there. (aio-libs/aiohttp#6600)
* Rewrite scripts/install-dev and scripts/delete-dev
  - Change the container volume path for halfstack containers to "./volumes"
  - When Docker is installed via Snap, it must be 20.10.15 or later
    to have a working `docker compose` (v2) plugin with `sudo`.
  - Remove the auto-install routine but just show the guides
  - Now we support and use docker-compose v2 only
* Import `backend.ai-common` source (c864ccbe1)
* Import `backend.ai-agent` source (98aeeb98)
* Import `backend.ai-manager` source (85d16f0)
* Import `backend.ai-client-py` source (b6d03cc)
* Import `backend.ai-webserver` source (81506cc)
* Import `backend.ai-storage-proxy` source (8019533)
* Import `backend.ai-tester` source (ab85fab5c)

Co-authored-by: Andreas Stenius <git@astekk.se>
cooperlees pushed a commit to psf/black that referenced this issue Jul 13, 2022
- >= 3.10 add a warning that `get_event_loop` will not automatically create a loop
- Move to explicit API

Test:
- `python3.11 -m venv --upgrade-deps /tmp/tb`
  - `/tmp/tb/bin/pip install -e .`
  - Install deps and no blackd as aiohttp + yarl can't build still with 3.11
  - aio-libs/aiohttp#6600
- `export PYTHONWARNINGS=error`
```
cooper@l33t:~/repos/black$ /tmp/tb/bin/black .
All done! ✨ 🍰 ✨
44 files left unchanged.
```

Fixes #3110
@droserasprout
Copy link

droserasprout commented Jul 13, 2022

Python 3.11.0b4 is out. Should we revisit this one? Citing release notes here:

Our goal is have no ABI changes after beta 4 and as few code changes as possible after 3.11.0rc1, the first release candidate. To achieve that, it will be extremely important to get as much exposure for 3.11 as possible during the beta phase.

cooperlees added a commit to psf/black that referenced this issue Jul 14, 2022
* Move to explicitly creating a new loop

- >= 3.10 add a warning that `get_event_loop` will not automatically create a loop
- Move to explicit API

Test:
- `python3.11 -m venv --upgrade-deps /tmp/tb`
  - `/tmp/tb/bin/pip install -e .`
  - Install deps and no blackd as aiohttp + yarl can't build still with 3.11
  - aio-libs/aiohttp#6600
- `export PYTHONWARNINGS=error`
```
cooper@l33t:~/repos/black$ /tmp/tb/bin/black .
All done! ✨ 🍰 ✨
44 files left unchanged.
```

Fixes #3110

* Add to CHANGES.md

* Fix a cooper typo yet again

* Set default asyncio loop to our explicitly created one + unset on exit

* Update CHANGES.md

Fix my silly typo.

Co-authored-by: Thomas Grainger <tagrain@gmail.com>

Co-authored-by: Cooper Ry Lees <me@wcooperlees.com>
Co-authored-by: Thomas Grainger <tagrain@gmail.com>
cclauss added a commit to cclauss/aiohttp that referenced this issue Jul 17, 2022
@cclauss
Copy link
Contributor

cclauss commented Jul 17, 2022

@bagusk99
Copy link

Okay, I was able to fix the problem. It turned out that I depended on some package which depended on an old version of cython

thank you this solve my problem, i just install it with pip3 install cython

bnmajor added a commit to bnmajor/itkwidgets that referenced this issue Sep 28, 2023
The imjoy-elfinder dependency brings in aiohttp but <= 3.8.2 does not build for
Python 3.11. For 3.11 make sure we use aiohttp>=3.8.2.

aio-libs/aiohttp#6600
JanneKiiskila added a commit to PelionIoT/e2e-edge-test-suite that referenced this issue Oct 20, 2023
Seems aiohttp has issues with being able to build with newer
Python versions.

aio-libs/aiohttp#6600
JanneKiiskila added a commit to PelionIoT/e2e-edge-test-suite that referenced this issue Oct 20, 2023
There are still some issues with that it seems.

aio-libs/aiohttp#6600
JanneKiiskila added a commit to PelionIoT/e2e-edge-test-suite that referenced this issue Oct 23, 2023
Seems aiohttp has issues with being able to build with newer
Python versions.

aio-libs/aiohttp#6600
JanneKiiskila added a commit to PelionIoT/e2e-edge-test-suite that referenced this issue Oct 23, 2023
There are still some issues with that it seems.

aio-libs/aiohttp#6600
JanneKiiskila added a commit to PelionIoT/e2e-edge-test-suite that referenced this issue Oct 23, 2023
Seems aiohttp has issues with being able to build with newer
Python versions.

aio-libs/aiohttp#6600
JanneKiiskila added a commit to PelionIoT/e2e-edge-test-suite that referenced this issue Oct 23, 2023
There are still some issues with that it seems.

aio-libs/aiohttp#6600
@TamAIRC
Copy link

TamAIRC commented Apr 17, 2024

I errorsame
pip install fastbpe
Collecting fastbpe
Using cached fastBPE-0.1.0.tar.gz (35 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Installing backend dependencies ... done
Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: fastbpe
Building wheel for fastbpe (pyproject.toml) ... error
error: subprocess-exited-with-error

× Building wheel for fastbpe (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [18 lines of output]
running bdist_wheel
running build
running build_py
running build_ext
building 'fastBPE' extension
creating build
creating build\temp.win-amd64-cpython-311
creating build\temp.win-amd64-cpython-311\Release
creating build\temp.win-amd64-cpython-311\Release\fastBPE
"C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.39.33519\bin\HostX86\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -Id:\Product\text_summarizer_bert.venv\include "-Ic:\Program Files\Python311\include" "-Ic:\Program Files\Python311\Include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.39.33519\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\cppwinrt" /EHsc /TpfastBPE/fastBPE.cpp /Fobuild\temp.win-amd64-cpython-311\Release\fastBPE/fastBPE.obj -std=c++11 -Ofast -pthread
cl : Command line warning D9025 : overriding '/Os' with '/Ot'
cl : Command line warning D9002 : ignoring unknown option '-std=c++11'
cl : Command line warning D9002 : ignoring unknown option '-Of'
cl : Command line warning D9002 : ignoring unknown option '-Oa'
cl : Command line warning D9002 : ignoring unknown option '-pthread'
fastBPE.cpp
fastBPE/fastBPE.cpp(205): fatal error C1083: Cannot open include file: 'longintrepr.h': No such file or directory
error: command 'C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.39.33519\bin\HostX86\x64\cl.exe' failed with exit code 2
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for fastbpe
Failed to build fastbpe
ERROR: Could not build wheels for fastbpe, which is required to install pyproject.toml-based projects

@Dreamsorcerer
Copy link
Member

fastbpe is not aiohttp...

@TamAIRC
Copy link

TamAIRC commented Apr 20, 2024

I see it same bug

@Dreamsorcerer
Copy link
Member

fastbpe has not been updated in 5 years, that's your problem. Nothing to do with us...

@aio-libs aio-libs locked as off-topic and limited conversation to collaborators Apr 20, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests