Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GH-43845: [Docs] Update Python Development Build Instructions #43617

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
+8 −11
Open

GH-43845: [Docs] Update Python Development Build Instructions #43617

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 8 additions & 11 deletions docs/source/developers/python.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -398,7 +398,7 @@ Now, build pyarrow:

$ pushd arrow/python
$ export PYARROW_PARALLEL=4
$ python setup.py build_ext --inplace
$ python -m pip install -e .
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
$ python -m pip install -e .
$ python -m pip install -e . --no-build-isolation --no-deps -v

Those additional flags make it more similar to what we did before. It's probably debatable whether those should be added (disabling build isolation and installing deps), but since in this tutorial we have set up all build dependencies in advance, not doing that gives unnecessary pip overhead.
(this is also the command that I use locally, although for quick rebuilds I still use python setup.py build_ext --inplace)

$ popd

If you did build one of the optional components in C++, the equivalent components
Expand All @@ -414,8 +414,8 @@ to the ``arrow/python`` folder and run ``git clean -Xfd .``.

By default, PyArrow will be built in release mode even if Arrow C++ has been
built in debug mode. To create a debug build of PyArrow, run
``export PYARROW_BUILD_TYPE=debug`` prior to running ``python setup.py
build_ext --inplace`` above. A ``relwithdebinfo`` build can be created
``export PYARROW_BUILD_TYPE=debug`` prior to running ``python -m pip
install -e .`` above. A ``relwithdebinfo`` build can be created
similarly.

Now you are ready to install test dependencies and run `Unit Testing`_, as
Expand All @@ -427,12 +427,9 @@ libraries), one can set ``--bundle-arrow-cpp``:
.. code-block::

$ pip install wheel # if not installed
$ python setup.py build_ext --build-type=$ARROW_BUILD_TYPE \
--bundle-arrow-cpp bdist_wheel

.. note::
To install an editable PyArrow build run ``pip install -e . --no-build-isolation``
in the ``arrow/python`` directory.
$ python -m pip wheel . \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
$ python -m pip wheel . \
$ python -m build . \

Using https://build.pypa.io/en/stable/ might be the recommended way nowadays to build a wheel?

When using that, it might need --config-setting instead of --global-option (but haven't verified this works the same)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When using that, it might need --config-setting instead of --global-option (but haven't verified this works the same)

I believe the --global-option is a requirement as long as setuptools is the backend. It's extremely murky, but here's the best conversation I could find on that topic:

pypa/pip#11859 (comment)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we need some way to pass down the option, but what I meant is that when using python -m build instead of python -m pip wheel, the name of the keyword to pass options might be different.

Because in the docs of build, I only see --config-setting and not --global-option

--global-option="--build-type=$ARROW_BUILD_TYPE" \
--global-option="--bundle-arrow-cpp"

Docker examples
~~~~~~~~~~~~~~~
Expand Down Expand Up @@ -548,7 +545,7 @@ Now, we can build pyarrow:

$ pushd arrow\python
$ set CONDA_DLL_SEARCH_MODIFICATION_ENABLE=1
$ python setup.py build_ext --inplace
$ python pip install -e .
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
$ python pip install -e .
$ python pip install -e . --no-build-isolation --no-deps -v

(depending on what is done above)

$ popd

.. note::
Expand Down Expand Up @@ -588,7 +585,7 @@ Then run the unit tests with:
.. code-block::

$ set PYARROW_BUNDLE_ARROW_CPP=1
$ python setup.py build_ext --inplace
$ python -m pip install -e .

Note that bundled Arrow C++ libraries will not be automatically
updated when rebuilding Arrow C++.
Expand Down
Loading