pip-tools can not pip-compile itself

[klahnakoski]

We have pip-tools part of our development environment.

https://github.com/mozilla/treeherder/blob/9a158667b4c8dc4ec1bdac202ab57d93ca0b3318/requirements/dev.in

pip-tools requires pip, but after running

pip-compile --upgrade --generate-hashes --output-file requirements/dev.txt requirements/dev.in

pip is not included in the output

https://github.com/mozilla/treeherder/blob/9a158667b4c8dc4ec1bdac202ab57d93ca0b3318/requirements/dev.txt

of course this creates problems with pip install

In --require-hashes mode, all requirements must have their versions pinned with ==. These do not:

    pip>=20.0 from https://files.pythonhosted.org/packages/43/84/23ed6a1796480a6f1a2d38f2802901d078266bda38388954d01d3f2e821d/pip-20.1.1-py2.py3-none-any.whl#sha256=b27c4dedae8c41aa59108f2fa38bf78e0890e590545bc8ece7cdceb4ba60f6e4 (from pip-tools==5.2.1->-r requirements/dev.txt (line 156))

here is the full travis run

https://travis-ci.org/github/mozilla/treeherder/jobs/699143689

EDIT: pip-tools==4.5.1 works just fine

Environment Versions

pip-tools==5.2.1

$ python --version

Python 3.7.1

$ pip --version

pip 19.1.1 from /home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/pip (python 3.7)

[atugushev]

Hello @klahnakoski,

Thanks for the issue! Have you tried --allow-unsafe option as the warning suggests?

# WARNING: The following packages were not pinned, but pip requires them to be
# pinned when the requirements file includes hashes. Consider using the --allow-unsafe flag.
# pip
# setuptools

[atugushev]

FYI, we considering setting this option on by default, see #989.

[klahnakoski]

@atugushev yes, we considered --allow-unsafe, but that would defeat the purpose of preventing installation of unknown code onto our testing and production systems.

[atugushev]

Could you elaborate on what do you mean by "unknown code"?

[atugushev]

The --allow-unsafe option allows to pin the following packages:

• distribute
• pip
• setuptools

They called "unsafe", because pip-tools relies on them internally.

[klahnakoski]

@atugushev let me get back to you. I was unaware --allow-unsafe was part of pip-tools, I thought it was a pip flag

[atugushev]

@klahnakoski it's also documented here:

$ pip-compile --help | grep allow-unsafe -A 1
  --allow-unsafe                  Pin packages considered unsafe: distribute,
                                  pip, setuptools

Hope it helps! 👍

[klahnakoski]

@atugushev Thank you for your time. I apologize for my confusion; the --unallow-unsafe option works

[atugushev]

Not a problem! Feel free to reach out if you have any questions.