Projects won't upload with error: python3: No such file or directory

[technoblogy]

I have just upgraded the Arduino ESP8266 core to 2.7.1 and my projects no longer load, with the error:

python3: No such file or directory

I'm on MacOS 10.13.6. I've installed Python 3 but this hasn't fixed it. Downgrading to the 2.5.2 version of the core works OK. Any suggestions?

[devyte]

#6931

[technoblogy]

Thank you. I think it would be useful to document this problem and the solution somewhere more accessible.

[devyte]

The issue template, which you ignored here, has a checklist with an item directing to search for a similar issue before opening. It took me 15s to search "python" and find it.
If you think this still merits better documentation, please feel free to make a proposal in a PR.

[technoblogy]

The problem is that the solution is buried in the middle of that long thread.

I'm not familiar enough with GitHub to be comfortable making a pull request, but I suggest you add the following to the FAQ at:

https://arduino-esp8266.readthedocs.io/en/latest/faq/readme.html

On MacOS uploading from the Arduino IDE fails with the error: python3: No such file or directory

The latest versions of the ESP8266 core (later than 2.5.2) require Python 3 which is not installed by default on MacOS. You can install it as follows:

• Download the latest Python 3 installer from:

https://www.python.org/downloads/

• Run the installer.

• Also double-click the file Install Certificates.command in the Python 3.8 folder in Applications.

You may also need to update the symlink in the ESP8266 core files to point to the copy of Python you've installed by giving the following commands from the Terminal:

cd ~/Library/Arduino15/packages/esp8266/tools/python3/3.7.2-post1
unlink python3
ln -s /usr/local/bin/python3 python3

[earlephilhower]

Thanks, @technoblogy, that seems like a good writeup. Unfortunately, none of the maintainers use a Mac so we've been using the Travis-CI Mac instances.

FWIW, the CI actually does run just fine (compile a sketch to a BIN) MacOS builds in VMs so I'm curious what the difference is between the MacOS VM and whatever OS version you're running.

Could you take a look at the build steps in:
https://travis-ci.org/github/esp8266/Arduino/jobs/693112801
(or any other MacOS build) and see where they diverge from your own setup. Maybe there's something we can do in the packaging stage of the tools to make this work seamlessly.

And are you aware of any way to do a portable install of Python3 on the Mac? We do that on Windows so users don't have to install a thing other than the base IDE.

[earlephilhower]

@technoblogy, there's something fishy in your installation. The 2.7.1 install does not have a 3.7.2-post1 dir, that's from a much older version. Maybe the tools weren't updated properly or something (this is a tool tarball).

That said, I've had success on a clean Mac and Arduino just following the steps in:
#7347 (comment)

[technoblogy]

@earlephilhower thanks for your replies. I'll try and test what you're suggesting, but I don't understand some of your explanations. What is "CI" and what are "Travis-CI Mac instances"?

[technoblogy]

@earlephilhower I've just deleted the installation and reinstalled and I get this:

which definitely does have a 3.7.2-post1 dir.

[earlephilhower]

Ah, looks like it's added by your IDE.

If you go to the tarball itself, https://github.com/earlephilhower/esp-quick-toolchain/releases/download/2.5.0-4/python3-via-env.tar.gz
there is no 3.7.2-post1 in there.

In any case, there is now a script which uses /usr/bin/env to find the Mac system-wide installed Python3. If that doesn't work, then there's something wrong w/the Python installation (/usr/bin/env is the standard tool for finding locations of executable files on UNIX-like OSes). So no symlink magic should be required.

[technoblogy]

OK, I don't quite understand, but you're saying that people shouldn't get my problem in future? Perhaps the FAQ should still suggest they'll need to manually install Python 3 on Mac?

[earlephilhower]

The latest Mac OS actually prompts the user with a dialog to install it on first use (see the screenshot from the linked comment above). So, I don't think the FAQ needs anything...Apple has taken care of it and made it pretty painless.

After reinstall of the toolchain, does the build work w/o the symlink for you, then?

[technoblogy]

No, unfortunately not. I again get:

env: python3: No such file or directory

but doing this again fixed it:

cd ~/Library/Arduino15/packages/esp8266/tools/python3/3.7.2-post1
unlink python3
ln -s /usr/local/bin/python3 python3

[earlephilhower]

Oh, I just noticed that you're on a very old MacOS 10.3 (Panther? early 2000s?). I had no idea that was still viable today. That's the root of things here. That version only had Python 2 (many years obsolete, and completely unsupported as of 1/1/2020 or so)...Python 3 wasn't even a thing back then.

Apple added the automatic Python3 install goodness in modern versions. For your own case, sure make the link or install a proper version so that /usr/bin/env can find it. If you do the symlink, it will break again on every update. Alternatively, install the latest Xcode and it should come with the dev tools.

I'm against putting a symlink hack into the FAQ because in most cases it's just not needed and it's not really the right way to get things working for most people.

[technoblogy]

Oh, I just noticed that you're on a very old MacOS 10.3 (Panther? early 2000s?).

No, I'm on High Sierra (10.13.6). Sorry, that was a typo.

[earlephilhower]

Whew. I thought you were still rocking a PowerPC based Mac. :)

The proper way would be to say "Install Xcode as a prerequisite for OS X prior to 10.(14?15?)" 15 has the nice auto-installer, bet that was only just added) and give the URL in the FAQ.

The alternative would be to prepackage a portable Python (say that 5 times fast) just like we do with Windows. Let me see if I can throw together a tarball that can be used by OSX.

[earlephilhower]

OK, I've got something that passes CI so can you manually try the blob pr #7348 attempts to use for python3 in lieu of any OS or brew installed executables?

1. Rename the tools/python3/3.7.2-post1 directory to tools/python3/3.7.2-post1-XXX
2. Download the tarball here: https://github.com/earlephilhower/esp-quick-toolchain/releases/download/2.5.0-4/python3-macosx-portable.tar.gz
3. Expand the tarball into tools/python3/. You will now have a dir tree of tools/python3/python3/... (i.e. 2 dirs as "python3").
4. Rename tools/python3/python3 to tools/python3/3.7.2-post1
5. You should have the Python3 interpreter now in tools/python3/3.7.2-post1/python3
6. Run a new build and upload and verify if it works/fails/etc. Please get logs of any errors.

These steps would be automatically done by the Arduino IDE should a 2.7.2 release be done which includes this #7348 patch, so end users will just click and no, no need for these gymnastics or making symlinks manually.

[technoblogy]

Yes, that works like a dream - no errors to report! Thanks for getting to the bottom of this.