Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Python 3.11 issues for uaqmp #334

Closed
kashifkhan opened this issue Aug 26, 2022 · 8 comments
Closed

Python 3.11 issues for uaqmp #334

kashifkhan opened this issue Aug 26, 2022 · 8 comments
Assignees
@kashifkhan
Labels
AMQP Python

Comments

@kashifkhan
Copy link
Member

No description provided.

@EpicWink
Copy link

EpicWink commented Sep 15, 2022 •
edited
Loading

Seems to fail to build due to missing longintrepr.h during build of 'build/temp.linux-x86_64-cpython-311/uamqp/c_uamqp.o' (from 'uamqp/c_uamqp.c')

Reproduction:

docker run -it python:3.11-rc-slim bash
apt update
apt install --no-install-recommends g++ libc-dev cmake make libssl-dev uuid-dev
pip install uamqp

Quick google search gives possibly relevant:

Solution seems to be:

- #include "longintrepr.h"
+ #if PY_MAJOR_VERSION < 3
+   #include "longintrepr.h"
+ #endif

(or drop Python 2 support entirely, and just remove the include)

@frenck
Copy link

frenck commented Sep 21, 2022

Build output:

  Building wheel for uamqp (setup.py): started
  Building wheel for uamqp (setup.py): finished with status 'error'
  error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [209 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-311
      creating build/lib.linux-x86_64-cpython-311/uamqp
      copying uamqp/receiver.py -> build/lib.linux-x86_64-cpython-311/uamqp
      copying uamqp/types.py -> build/lib.linux-x86_64-cpython-311/uamqp
      copying uamqp/compat.py -> build/lib.linux-x86_64-cpython-311/uamqp
  Running setup.py clean for uamqp
      copying uamqp/address.py -> build/lib.linux-x86_64-cpython-311/uamqp
      copying uamqp/message.py -> build/lib.linux-x86_64-cpython-311/uamqp
      copying uamqp/session.py -> build/lib.linux-x86_64-cpython-311/uamqp
      copying uamqp/errors.py -> build/lib.linux-x86_64-cpython-311/uamqp
      copying uamqp/client.py -> build/lib.linux-x86_64-cpython-311/uamqp
      copying uamqp/constants.py -> build/lib.linux-x86_64-cpython-311/uamqp
      copying uamqp/connection.py -> build/lib.linux-x86_64-cpython-311/uamqp
      copying uamqp/utils.py -> build/lib.linux-x86_64-cpython-311/uamqp
      copying uamqp/__init__.py -> build/lib.linux-x86_64-cpython-311/uamqp
      copying uamqp/sender.py -> build/lib.linux-x86_64-cpython-311/uamqp
      copying uamqp/mgmt_operation.py -> build/lib.linux-x86_64-cpython-311/uamqp
      creating build/lib.linux-x86_64-cpython-311/uamqp/async_ops
      copying uamqp/async_ops/connection_async.py -> build/lib.linux-x86_64-cpython-311/uamqp/async_ops
      copying uamqp/async_ops/receiver_async.py -> build/lib.linux-x86_64-cpython-311/uamqp/async_ops
      copying uamqp/async_ops/mgmt_operation_async.py -> build/lib.linux-x86_64-cpython-311/uamqp/async_ops
      copying uamqp/async_ops/sender_async.py -> build/lib.linux-x86_64-cpython-311/uamqp/async_ops
      copying uamqp/async_ops/session_async.py -> build/lib.linux-x86_64-cpython-311/uamqp/async_ops
      copying uamqp/async_ops/utils.py -> build/lib.linux-x86_64-cpython-311/uamqp/async_ops
      copying uamqp/async_ops/__init__.py -> build/lib.linux-x86_64-cpython-311/uamqp/async_ops
      copying uamqp/async_ops/client_async.py -> build/lib.linux-x86_64-cpython-311/uamqp/async_ops
      creating build/lib.linux-x86_64-cpython-311/uamqp/authentication
      copying uamqp/authentication/cbs_auth.py -> build/lib.linux-x86_64-cpython-311/uamqp/authentication
      copying uamqp/authentication/common.py -> build/lib.linux-x86_64-cpython-311/uamqp/authentication
      copying uamqp/authentication/cbs_auth_async.py -> build/lib.linux-x86_64-cpython-311/uamqp/authentication
      copying uamqp/authentication/__init__.py -> build/lib.linux-x86_64-cpython-311/uamqp/authentication
      running egg_info
      writing uamqp.egg-info/PKG-INFO
      writing dependency_links to uamqp.egg-info/dependency_links.txt
      writing requirements to uamqp.egg-info/requires.txt
      writing top-level names to uamqp.egg-info/top_level.txt
      reading manifest file 'uamqp.egg-info/SOURCES.txt'
      reading manifest template 'MANIFEST.in'
      writing manifest file 'uamqp.egg-info/SOURCES.txt'
      copying uamqp/c_uamqp.c -> build/lib.linux-x86_64-cpython-311/uamqp
      running build_ext
      will build uamqp in build/temp.linux-x86_64-cpython-311/cmake
      Building with generator flags: -G "Unix Makefiles"
      calling cmake /tmp/pip-install-lqo3w3o_/uamqp/src/vendor/azure-uamqp-c/ -G "Unix Makefiles" -Duse_openssl:bool=ON -Duse_default_uuid:bool=ON  -Duse_builtin_httpapi:bool=ON  -Dskip_samples:bool=ON -DCMAKE_POSITION_INDEPENDENT_CODE=TRUE -DCMAKE_BUILD_TYPE=Release
      -- The C compiler identification is GNU 9.4.0
      -- The CXX compiler identification is GNU 9.4.0
      -- Detecting C compiler ABI info
      -- Detecting C compiler ABI info - done
      -- Check for working C compiler: /usr/bin/cc - skipped
      -- Detecting C compile features
      -- Detecting C compile features - done
      -- Detecting CXX compiler ABI info
      -- Detecting CXX compiler ABI info - done
      -- Check for working CXX compiler: /usr/bin/c++ - skipped
      -- Detecting CXX compile features
      -- Detecting CXX compile features - done
      CMake Deprecation Warning at deps/azure-macro-utils-c/CMakeLists.txt:4 (cmake_minimum_required):
        Compatibility with CMake < 2.8.12 will be removed from a future version of
        CMake.
      
        Update the VERSION argument <min> value or use a ...<max> suffix to tell
        CMake that the project does not need compatibility with older versions.
      
      
      CMake Warning (dev) at deps/azure-macro-utils-c/CMakeLists.txt:14 (option):
        Policy CMP0077 is not set: option() honors normal variables.  Run "cmake
        --help-policy CMP0077" for policy details.  Use the cmake_policy command to
        set the policy and suppress this warning.
      
        For compatibility with older versions of CMake, option is clearing the
        normal variable 'run_int_tests'.
      This warning is for project developers.  Use -Wno-dev to suppress it.
      
      CMake Deprecation Warning at deps/umock-c/CMakeLists.txt:4 (cmake_minimum_required):
        Compatibility with CMake < 2.8.12 will be removed from a future version of
        CMake.
      
        Update the VERSION argument <min> value or use a ...<max> suffix to tell
        CMake that the project does not need compatibility with older versions.
      
      
      -- Looking for include file stdint.h
      -- Looking for include file stdint.h - found
      -- Looking for include file stdbool.h
      -- Looking for include file stdbool.h - found
      -- target architecture: x86_64
      -- Performing Test CXX_FLAG_CXX11
      -- Performing Test CXX_FLAG_CXX11 - Success
      -- Found OpenSSL: /usr/lib/x86_64-linux-gnu/libcrypto.so (found version "1.1.1f")
      -- target architecture: x86_64
      -- Configuring done
      -- Generating done
      -- Build files have been written to: /tmp/pip-install-lqo3w3o_/uamqp/build/temp.linux-x86_64-cpython-311/cmake
      calling cmake --build . --config Release
      [  0%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/azure_base32.c.o
      [  1%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/azure_base64.c.o
      [  2%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/buffer.c.o
      [  3%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/constbuffer_array.c.o
      [  4%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/constbuffer_array_batcher.c.o
      [  5%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/connection_string_parser.c.o
      [  6%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/constbuffer.c.o
      [  7%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/consolelogger.c.o
      [  8%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/crt_abstractions.c.o
      [  9%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/constmap.c.o
      [ 10%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/doublylinkedlist.c.o
      [ 11%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/gballoc.c.o
      [ 12%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/gbnetwork.c.o
      [ 13%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/gb_stdio.c.o
      [ 14%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/gb_time.c.o
      [ 15%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/gb_rand.c.o
      [ 16%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/hmac.c.o
      [ 17%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/hmacsha256.c.o
      [ 18%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/xio.c.o
      [ 19%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/singlylinkedlist.c.o
      [ 20%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/map.c.o
      [ 21%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/sastoken.c.o
      [ 22%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/sha1.c.o
      [ 23%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/sha224.c.o
      [ 24%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/sha384-512.c.o
      [ 25%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/strings.c.o
      [ 26%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/string_token.c.o
      [ 27%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/string_tokenizer.c.o
      [ 28%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/uuid.c.o
      [ 29%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/urlencode.c.o
      [ 30%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/usha.c.o
      [ 31%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/vector.c.o
      [ 32%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/xlogging.c.o
      [ 33%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/optionhandler.c.o
      [ 34%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/memory_data.c.o
      [ 35%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/adapters/agenttime.c.o
      [ 36%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/adapters/condition_pthreads.c.o
      [ 37%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/adapters/lock_pthreads.c.o
      [ 38%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/adapters/platform_linux.c.o
      [ 39%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/adapters/socketio_berkeley.c.o
      [ 40%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/adapters/tickcounter_linux.c.o
      [ 41%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/adapters/threadapi_pthreads.c.o
      [ 42%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/adapters/uniqueid_stub.c.o
      [ 43%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/adapters/envvariable.c.o
      [ 44%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/dns_resolver_sync.c.o
      [ 45%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/adapters/linux_time.c.o
      [ 46%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/httpapiex.c.o
      [ 47%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/httpapiexsas.c.o
      [ 48%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/httpheaders.c.o
      [ 49%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/adapters/httpapi_compact.c.o
      [ 50%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/http_proxy_io.c.o
      [ 51%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/adapters/tlsio_openssl.c.o
      [ 52%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/adapters/x509_openssl.c.o
      [ 53%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/wsio.c.o
      [ 54%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/uws_client.c.o
      [ 55%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/uws_frame_encoder.c.o
      [ 56%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/utf8_checker.c.o
      [ 57%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/src/ws_url.c.o
      [ 58%] Linking C static library libaziotsharedutil.a
      [ 58%] Built target aziotsharedutil
      [ 59%] Building C object CMakeFiles/uamqp.dir/src/amqp_definitions.c.o
      [ 60%] Building C object CMakeFiles/uamqp.dir/src/amqp_frame_codec.c.o
      [ 61%] Building C object CMakeFiles/uamqp.dir/src/amqp_management.c.o
      [ 62%] Building C object CMakeFiles/uamqp.dir/src/amqpvalue.c.o
      [ 63%] Building C object CMakeFiles/uamqp.dir/src/amqpvalue_to_string.c.o
      [ 64%] Building C object CMakeFiles/uamqp.dir/src/async_operation.c.o
      [ 65%] Building C object CMakeFiles/uamqp.dir/src/cbs.c.o
      [ 66%] Building C object CMakeFiles/uamqp.dir/src/connection.c.o
      [ 67%] Building C object CMakeFiles/uamqp.dir/src/frame_codec.c.o
      [ 68%] Building C object CMakeFiles/uamqp.dir/src/header_detect_io.c.o
      [ 69%] Building C object CMakeFiles/uamqp.dir/src/link.c.o
      [ 70%] Building C object CMakeFiles/uamqp.dir/src/message.c.o
      [ 71%] Building C object CMakeFiles/uamqp.dir/src/message_receiver.c.o
      [ 72%] Building C object CMakeFiles/uamqp.dir/src/message_sender.c.o
      [ 73%] Building C object CMakeFiles/uamqp.dir/src/messaging.c.o
      [ 74%] Building C object CMakeFiles/uamqp.dir/src/sasl_anonymous.c.o
      [ 75%] Building C object CMakeFiles/uamqp.dir/src/sasl_frame_codec.c.o
      [ 76%] Building C object CMakeFiles/uamqp.dir/src/sasl_mechanism.c.o
      [ 77%] Building C object CMakeFiles/uamqp.dir/src/sasl_server_mechanism.c.o
      [ 78%] Building C object CMakeFiles/uamqp.dir/src/sasl_mssbcbs.c.o
      [ 79%] Building C object CMakeFiles/uamqp.dir/src/sasl_plain.c.o
      [ 80%] Building C object CMakeFiles/uamqp.dir/src/saslclientio.c.o
      [ 81%] Building C object CMakeFiles/uamqp.dir/src/session.c.o
      [ 82%] Building C object CMakeFiles/uamqp.dir/src/socket_listener_berkeley.c.o
      [ 83%] Linking C static library libuamqp.a
      [ 83%] Built target uamqp
      [ 84%] Building C object deps/umock-c/CMakeFiles/umock_c.dir/src/umock_c.c.o
      [ 85%] Building C object deps/umock-c/CMakeFiles/umock_c.dir/src/umock_c_negative_tests.c.o
      [ 86%] Building C object deps/umock-c/CMakeFiles/umock_c.dir/src/umockalloc.c.o
      [ 87%] Building C object deps/umock-c/CMakeFiles/umock_c.dir/src/umockcall.c.o
      [ 88%] Building C object deps/umock-c/CMakeFiles/umock_c.dir/src/umockcallrecorder.c.o
      [ 89%] Building C object deps/umock-c/CMakeFiles/umock_c.dir/src/umocktypename.c.o
      [ 90%] Building C object deps/umock-c/CMakeFiles/umock_c.dir/src/umocktypes.c.o
      [ 91%] Building C object deps/umock-c/CMakeFiles/umock_c.dir/src/umocktypes_bool.c.o
      [ 92%] Building C object deps/umock-c/CMakeFiles/umock_c.dir/src/umocktypes_c.c.o
      [ 93%] Building C object deps/umock-c/CMakeFiles/umock_c.dir/src/umocktypes_stdint.c.o
      [ 94%] Building C object deps/umock-c/CMakeFiles/umock_c.dir/src/umocktypes_charptr.c.o
      [ 95%] Building C object deps/umock-c/CMakeFiles/umock_c.dir/src/umocktypes_wcharptr.c.o
      [ 96%] Building C object deps/umock-c/CMakeFiles/umock_c.dir/src/umockcallpairs.c.o
      [ 97%] Building C object deps/umock-c/CMakeFiles/umock_c.dir/src/umockstring.c.o
      [ 98%] Building C object deps/umock-c/CMakeFiles/umock_c.dir/src/umockautoignoreargs.c.o
      [ 99%] Building C object deps/umock-c/CMakeFiles/umock_c.dir/src/umock_log.c.o
      [100%] Linking C static library libumock_c.a
      [100%] Built target umock_c
      building 'uamqp.c_uamqp' extension
      creating build/temp.linux-x86_64-cpython-311/uamqp
      gcc -pthread -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I./src/vendor/inc -I./src/vendor/azure-uamqp-c/deps/azure-macro-utils-c/inc -I./src/vendor/azure-uamqp-c/deps/umock-c/inc -I./src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/pal/inc -I./src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/inc -I./src/vendor/azure-uamqp-c/inc -I./src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/pal/linux -I/home/runner/work/core/core/venv/include -I/opt/hostedtoolcache/Python/3.11.0-rc.2/x64/include/python3.11 -c uamqp/c_uamqp.c -o build/temp.linux-x86_64-cpython-311/uamqp/c_uamqp.o -g -O0 -std=gnu99 -fPIC
      uamqp/c_uamqp.c:268:12: fatal error: longintrepr.h: No such file or directory
        268 |   #include "longintrepr.h"
            |            ^~~~~~~~~~~~~~~
      compilation terminated.
      error: command '/usr/bin/gcc' failed with exit code 1
      [end of output]

@frenck frenck mentioned this issue Sep 21, 2022
Closed
23 tasks
@kashifkhan
Copy link
Member Author

Seems to fail to build due to missing longintrepr.h during build of 'build/temp.linux-x86_64-cpython-311/uamqp/c_uamqp.o' (from 'uamqp/c_uamqp.c')

Reproduction:

docker run -it python:3.11-rc-slim bash
apt update
apt install --no-install-recommends g++ libc-dev cmake make libssl-dev uuid-dev
pip install uamqp

Quick google search gives possibly relevant:

Solution seems to be:

- #include "longintrepr.h"
+ #if PY_MAJOR_VERSION < 3
+   #include "longintrepr.h"
+ #endif

(or drop Python 2 support entirely, and just remove the include)

Thanks for this @EpicWink and @frenck ...Looks like the cython requirement needs to be bumped up to fix this.

@kashifkhan
Copy link
Member Author

Wanted to give yall an update, I'm working on generating wheels for 3.11 in this feature branch. Bumping up the cython was indeed the fix, the version we had pinned to had a bug which was fixed subsequently. I am able to build wheels so a simple workaround is to just bump up the version to 0.29.32 to get yall moving :)

@potiuk
Copy link

potiuk commented Oct 26, 2022

Cool if there is a help you need on that - we are looking forward to it in Apache Airflow too

I know it is eaarly (Python 3.11 has just been released yesterday) but we are hoping in Apache Airflow to a much faster cycle of adding new Python releases - especially that Pyhon 3.11 introduces huge performance improvements (25% is the average number claimed) due to a very focused effort to increase single-threaded Python performance (Specialized interpreter being the core of it but also many other improvements) without actually changing any of the Python code.

The azure-uamqp-python is one of the dependencies of Airlfow that need to support newer Python version so this issue is here to make you aware that we are eaager (also to help if needed) to make it faster, possibly by talking to dependencies of Beam as well (who are oftten also direct dependencies of Airflow) and help them as well. I perfectly understand we need to bubble up support so that your dependencies support them first.

I opened PR in Apache Airflow yesterday and plan to keep it open until it gets green :)

apache/airflow#27264

I think it would be fantastic if we could as the open source community migrate to the new Python much faster.

Looking forward to cooperation on that one :)

@kashifkhan
Copy link
Member Author

Hi @potiuk, Im the in the process of running a few tests on uamqp and making sure everything is working the way we intend it to be, so that everyone has a smooth experience whilst using 3.11 :) We are also quite excited to see the effect of 3.11 on our code base.

As you know macOS runners on Github added 3.11 to their images just a few hours ago, so I need to go through those and some internal checks before pushing out new wheels.

appreciate the offer for help, Ill update this thread if I end up needing help etc.

@potiuk potiuk mentioned this issue Oct 26, 2022
Closed
@potiuk potiuk mentioned this issue Oct 26, 2022
Merged
11 tasks
potiuk added a commit to apache/airflow that referenced this issue Oct 26, 2022
@potiuk
Attempt to add Python 3.11 support
cf340d7 
Python 3.11 has been released as scheduled on October 25, 2022 and
this is the first attempt to see how far Airflow (mostly dependencies)
are from being ready to officially support 3.11.

So far we had to exclude the following dependencies:

- [ ] Pyarrow dependency: apache/arrow#14499
- [ ] Google Provider: #27292
  and googleapis/python-bigquery#1386
- [ ] Databricks Provider:
  databricks/databricks-sql-python#59
- [ ] Papermill Provider: nteract/papermill#700
- [ ] Azure Provider: Azure/azure-uamqp-python#334
  and Azure/azure-sdk-for-python#27066
- [ ] Apache Beam Provider: apache/beam#23848
- [ ] Snowflake Provider:
  snowflakedb/snowflake-connector-python#1294
- [ ] JDBC Provider: jpype-project/jpype#1087
- [ ] Hive Provider: cloudera/python-sasl#30

We might decide to release Airflow in 3.11 with those providers
disabled in case they are lagging behind eventually, but for the
moment we want to work with all the projects in concert to be
able to release all providers (Google Provider requires quite
a lot of work and likely Google Team stepping up and community helping
with migration to latest Goofle cloud libraries)
potiuk added a commit to apache/airflow that referenced this issue Oct 27, 2022
@potiuk
Attempt to add Python 3.11 support
bdd765d 
Python 3.11 has been released as scheduled on October 25, 2022 and
this is the first attempt to see how far Airflow (mostly dependencies)
are from being ready to officially support 3.11.

So far we had to exclude the following dependencies:

- [ ] Pyarrow dependency: apache/arrow#14499
- [ ] Google Provider: #27292
  and googleapis/python-bigquery#1386
- [ ] Databricks Provider:
  databricks/databricks-sql-python#59
- [ ] Papermill Provider: nteract/papermill#700
- [ ] Azure Provider: Azure/azure-uamqp-python#334
  and Azure/azure-sdk-for-python#27066
- [ ] Apache Beam Provider: apache/beam#23848
- [ ] Snowflake Provider:
  snowflakedb/snowflake-connector-python#1294
- [ ] JDBC Provider: jpype-project/jpype#1087
- [ ] Hive Provider: cloudera/python-sasl#30

We might decide to release Airflow in 3.11 with those providers
disabled in case they are lagging behind eventually, but for the
moment we want to work with all the projects in concert to be
able to release all providers (Google Provider requires quite
a lot of work and likely Google Team stepping up and community helping
with migration to latest Goofle cloud libraries)
potiuk added a commit to apache/airflow that referenced this issue Oct 27, 2022
@potiuk
Attempt to add Python 3.11 support
71cbff2 
Python 3.11 has been released as scheduled on October 25, 2022 and
this is the first attempt to see how far Airflow (mostly dependencies)
are from being ready to officially support 3.11.

So far we had to exclude the following dependencies:

- [ ] Pyarrow dependency: apache/arrow#14499
- [ ] Google Provider: #27292
  and googleapis/python-bigquery#1386
- [ ] Databricks Provider:
  databricks/databricks-sql-python#59
- [ ] Papermill Provider: nteract/papermill#700
- [ ] Azure Provider: Azure/azure-uamqp-python#334
  and Azure/azure-sdk-for-python#27066
- [ ] Apache Beam Provider: apache/beam#23848
- [ ] Snowflake Provider:
  snowflakedb/snowflake-connector-python#1294
- [ ] JDBC Provider: jpype-project/jpype#1087
- [ ] Hive Provider: cloudera/python-sasl#30

We might decide to release Airflow in 3.11 with those providers
disabled in case they are lagging behind eventually, but for the
moment we want to work with all the projects in concert to be
able to release all providers (Google Provider requires quite
a lot of work and likely Google Team stepping up and community helping
with migration to latest Goofle cloud libraries)
@kashifkhan
Copy link
Member Author

kashifkhan commented Oct 27, 2022 •
edited
Loading

hello everyone, we just published uamqp ver 1.6.3 that has support for py311

@potiuk
Copy link

potiuk commented Oct 27, 2022

WOOOHO!!!

potiuk added a commit to apache/airflow that referenced this issue Oct 31, 2022
@potiuk
Attempt to add Python 3.11 support
0463899 
Python 3.11 has been released as scheduled on October 25, 2022 and
this is the first attempt to see how far Airflow (mostly dependencies)
are from being ready to officially support 3.11.

So far we had to exclude the following dependencies:

- [ ] Pyarrow dependency: apache/arrow#14499
- [ ] Google Provider: #27292
  and googleapis/python-bigquery#1386
- [ ] Databricks Provider:
  databricks/databricks-sql-python#59
- [ ] Papermill Provider: nteract/papermill#700
- [ ] Azure Provider: Azure/azure-uamqp-python#334
  and Azure/azure-sdk-for-python#27066
- [ ] Apache Beam Provider: apache/beam#23848
- [ ] Snowflake Provider:
  snowflakedb/snowflake-connector-python#1294
- [ ] JDBC Provider: jpype-project/jpype#1087
- [ ] Hive Provider: cloudera/python-sasl#30

We might decide to release Airflow in 3.11 with those providers
disabled in case they are lagging behind eventually, but for the
moment we want to work with all the projects in concert to be
able to release all providers (Google Provider requires quite
a lot of work and likely Google Team stepping up and community helping
with migration to latest Goofle cloud libraries)
potiuk added a commit to apache/airflow that referenced this issue Nov 24, 2022
@potiuk
Attempt to add Python 3.11 support
af33700 
Python 3.11 has been released as scheduled on October 25, 2022 and
this is the first attempt to see how far Airflow (mostly dependencies)
are from being ready to officially support 3.11.

So far we had to exclude the following dependencies:

- [ ] Pyarrow dependency: apache/arrow#14499
- [ ] Google Provider: #27292
  and googleapis/python-bigquery#1386
- [ ] Databricks Provider:
  databricks/databricks-sql-python#59
- [ ] Papermill Provider: nteract/papermill#700
- [ ] Azure Provider: Azure/azure-uamqp-python#334
  and Azure/azure-sdk-for-python#27066
- [ ] Apache Beam Provider: apache/beam#23848
- [ ] Snowflake Provider:
  snowflakedb/snowflake-connector-python#1294
- [ ] JDBC Provider: jpype-project/jpype#1087
- [ ] Hive Provider: cloudera/python-sasl#30

We might decide to release Airflow in 3.11 with those providers
disabled in case they are lagging behind eventually, but for the
moment we want to work with all the projects in concert to be
able to release all providers (Google Provider requires quite
a lot of work and likely Google Team stepping up and community helping
with migration to latest Goofle cloud libraries)
@kashifkhan kashifkhan self-assigned this Jan 12, 2023
potiuk added a commit to potiuk/airflow that referenced this issue Jan 19, 2023
@potiuk
Attempt to add Python 3.11 support
ff8aebf 
Python 3.11 has been released as scheduled on October 25, 2022 and
this is the first attempt to see how far Airflow (mostly dependencies)
are from being ready to officially support 3.11.

So far we had to exclude the following dependencies:

- [ ] Pyarrow dependency: apache/arrow#14499
- [ ] Google Provider: apache#27292
  and googleapis/python-bigquery#1386
- [ ] Databricks Provider:
  databricks/databricks-sql-python#59
- [ ] Papermill Provider: nteract/papermill#700
- [ ] Azure Provider: Azure/azure-uamqp-python#334
  and Azure/azure-sdk-for-python#27066
- [ ] Apache Beam Provider: apache/beam#23848
- [ ] Snowflake Provider:
  snowflakedb/snowflake-connector-python#1294
- [ ] JDBC Provider: jpype-project/jpype#1087
- [ ] Hive Provider: cloudera/python-sasl#30

We might decide to release Airflow in 3.11 with those providers
disabled in case they are lagging behind eventually, but for the
moment we want to work with all the projects in concert to be
able to release all providers (Google Provider requires quite
a lot of work and likely Google Team stepping up and community helping
with migration to latest Goofle cloud libraries)
@bdraco bdraco mentioned this issue Feb 13, 2023
Merged
53 tasks
@frenck frenck mentioned this issue Feb 15, 2023
Merged
19 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kashifkhan kashifkhan

Labels
AMQP Python
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@frenck @kashifkhan @potiuk @EpicWink