Pip install/upgrade fails on Windows if path too long

[tjprescott]

Steps to repro on Windows CMD:

• pip install azure-cli
• set AZURE_COMPONENT_PACKAGE_INDEX_URL=https://azureclinightly.blob.core.windows.net/packages
• az component update --private

Yields the following error:

[‎11/‎3/‎2016 10:20 AM] Ali Zaman: 
No Title
C:\Temp>az component update --private
Exception:
Traceback (most recent call last):
  File "C:\Users\alzam\AppData\Local\Programs\Python\Python35\lib\shutil.py", line 538, in move
    os.rename(src, real_dst)
FileNotFoundError: [WinError 3] The system cannot find the path specified: 'c:\\users\\alzam\\appdata\\local\\programs\\python\\python35\\lib\\site-packages\\azure\\cli\\command_modules\\network\\mgmt_express_route_circuit\\lib\\models\\__pycache__\\express_route_circuit_creation_client_enums.cpython-35.pyc' -> 'C:\\Users\\alzam\\AppData\\Local\\Temp\\pip-3bug3zw9-uninstall\\users\\alzam\\appdata\\local\\programs\\python\\python35\\lib\\site-packages\\azure\\cli\\command_modules\\network\\mgmt_express_route_circuit\\lib\\models\\__pycache__\\express_route_circuit_creation_client_enums.cpython-35.pyc'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\alzam\AppData\Local\Programs\Python\Python35\lib\site-packages\pip\basecommand.py", line 215, in main
    status = self.run(options, args)
  File "C:\Users\alzam\AppData\Local\Programs\Python\Python35\lib\site-packages\pip\commands\install.py", line 342, in run
    prefix=options.prefix_path,
  File "C:\Users\alzam\AppData\Local\Programs\Python\Python35\lib\site-packages\pip\req\req_set.py", line 778, in install
    requirement.uninstall(auto_confirm=True)
  File "C:\Users\alzam\AppData\Local\Programs\Python\Python35\lib\site-packages\pip\req\req_install.py", line 752, in uninstall
    paths_to_remove.remove(auto_confirm)
  File "C:\Users\alzam\AppData\Local\Programs\Python\Python35\lib\site-packages\pip\req\req_uninstall.py", line 115, in remove
    renames(path, new_path)
  File "C:\Users\alzam\AppData\Local\Programs\Python\Python35\lib\site-packages\pip\utils\__init__.py", line 267, in renames
    shutil.move(old, new)
  File "C:\Users\alzam\AppData\Local\Programs\Python\Python35\lib\shutil.py", line 552, in move
    copy_function(src, real_dst)
  File "C:\Users\alzam\AppData\Local\Programs\Python\Python35\lib\shutil.py", line 251, in copy2
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "C:\Users\alzam\AppData\Local\Programs\Python\Python35\lib\shutil.py", line 115, in copyfile
    with open(dst, 'wb') as fdst:
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\alzam\\AppData\\Local\\Temp\\pip-3bug3zw9-uninstall\\users\\alzam\\appdata\\local\\programs\\python\\python35\\lib\\site-packages\\azure\\cli\\command_modules\\network\\mgmt_express_route_circuit\\lib\\models\\__pycache__\\express_route_circuit_creation_client_enums.cpython-35.pyc' 

At this point, half of the packages have updated to the nightly version and the others have not. The only solution is to run pip install azure-cli-<component> -I for each package that was the nightly build in order to return to a stable state.

[tjprescott]

cc/ @derekbekoe @anzaman

[derekbekoe]

Thanks for reporting this @tjprescott and @anzaman!

Firstly, I've verified that it works on Linux (verified on Ubuntu) and OS X.

After investigation, it looks like you've run into the Maximum Path Length Limitation on Windows. It just happens that the file path of express_route_circuit_creation_client_enums.cpython-35.pyc exceeds the 260 character limit.

Looking at the filepath of the file not found, C:\Users\alzam\AppData\Local\Temp\pip-3bug3zw9-uninstall\users\alzam\appdata\local\programs\python\python35\lib\site-packages\azure\cli\command_modules\network\mgmt_express_route_circuit\lib\models\__pycache__\express_route_circuit_creation_client_enums.cpython-35.pyc it is 268 characters long.

In the Windows API (with some exceptions discussed in the following paragraphs), the maximum length for a path is MAX_PATH, which is defined as 260 characters.
https://msdn.microsoft.com/en-us/library/aa365247(VS.85).aspx#maxpath

• Related to Upgrading pip fails on Windows when install path is too long #3055
• Related to pip install fails to remove its temporary directory on Windows when it contains a deep folder hierarchy. #2892
• Python bug tracking this - Windows: support path longer than 260 bytes using "?" prefix

---

Another repro that excludes az component from this issue is:

$ pip install azure-cli
$ pip install --upgrade azure-cli --extra-index-url https://azureclinightly.blob.core.windows.net/packages

On Windows, if the filepath happens to be too long, you'll see a FileNotFoundError error from pip.

[derekbekoe]

Marking this as FAQ.
This can be resolved by installing the CLI in a higher directory to prevent reaching Windows MAX_PATH.

[lmiroslaw]

This instruction does not work for Azure CLI 2.0. I am getting 0.10.9 version (azure -v). How can I upgrade to CLI 2.0?

[derekbekoe]

It's az instead of azure for 2.0.

[derekbekoe]

Removing the MAX_PATH Limitation for Python 3.6 -https://docs.python.org/3/using/windows.html#removing-the-max-path-limitation

[lmiroslaw]

Installing Python 3.6.1 fixed the issue. I also had to remove the max-path-limitation during the installation proces.

[piercerob]

Long Path Tool can help you to remove blocked files or too long path files. You can easily fix file errors with this software.

[pradyunsg]

Linking to pypa/pip#3055, since that is the pip issue regarding this problem.