Error building on MacOS due to python3 not found #6931

harveydobson · 2019-12-21T12:36:24Z

Basic Infos

This issue complies with the issue POLICY doc.
I have read the documentation at readthedocs and the issue is not addressed there.
I have tested that the issue is present in current master branch (aka latest git).
I have searched the issue tracker for a similar issue.
If there is a stack dump, I have decoded it.
I have filled out all fields below.

Platform

Hardware: [ESP8266-12e]
Core Version: [21/12/2019 12:10pm GMT]
Development Env: [Arduino IDE]
Operating System: [MacOS]

Settings in IDE

Problem Description

Edit by @devyte:
The problem seems to be that the build process doesn't find python3 on MacOS. Duplicate WiFi libs warning is unrelated.

Original Problem Description follows.
--------------------------------------------------------------------------------------------------------------------

When including the library and selecting the board, any code i use whether it is the wifi search or the code I was looking to use in my project, I get the following error:

Arduino: 1.8.10 (Mac OS X), Board: "NodeMCU 0.9 (ESP-12 Module), 80 MHz, Flash, Legacy (new can return nullptr), All SSL ciphers (most compatible), 4MB (FS:2MB OTA:~1019KB), v2 Lower Memory, Disabled, None, Only Sketch, 115200"

Multiple libraries were found for "ESP8266WiFi.h"
 Used: /Users/xxxx/Library/Arduino15/packages/esp8266/hardware/esp8266/2.6.3/libraries/ESP8266WiFi
fork/exec /Users/xxxx/Library/Arduino15/packages/esp8266/tools/python3/3.7.2-post1/python3: no such file or directory
Error compiling for board NodeMCU 1.0 (ESP-12E Module).

This report would have more information with
"Show verbose output during compilation"
option enabled in File -> Preferences.

If i remove the library it finds nothing. This is a new installation on my mac so there wouldn't be any conflicts. Not sure if anyone has experienced the same..? Seems likely unless i'm doing something very wrong, but looking at the current issue list I cannot see anything like this.

Thanks

MCVE Sketch

/*
    This sketch demonstrates how to scan WiFi networks.
    The API is almost the same as with the WiFi Shield library,
    the most obvious difference bei›‹‹›ng the different file you need to include:
*/

#include "ESP8266WiFi.h"

void setup() {
  Serial.begin(115200);

  // Set WiFi to station mode and disconnect from an AP if it was previously connected
  WiFi.mode(WIFI_STA);
  WiFi.disconnect();
  delay(100);

  Serial.println("Setup done");
}

void loop() {
  Serial.println("scan start");

  // WiFi.scanNetworks will return the number of networks found
  int n = WiFi.scanNetworks();
  Serial.println("scan done");
  if (n == 0) {
    Serial.println("no networks found");
  } else {
    Serial.print(n);
    Serial.println(" networks found");
    for (int i = 0; i < n; ++i) {
      // Print SSID and RSSI for each network found
      Serial.print(i + 1);
      Serial.print(": ");
      Serial.print(WiFi.SSID(i));
      Serial.print(" (");
      Serial.print(WiFi.RSSI(i));
      Serial.print(")");
      Serial.println((WiFi.encryptionType(i) == ENC_TYPE_NONE) ? " " : "*");
      delay(10);
    }
  }
  Serial.println("");

  // Wait a bit before scanning again
  delay(5000);
}

Debug Messages

Arduino: 1.8.10 (Mac OS X), Board: "NodeMCU 0.9 (ESP-12 Module), 80 MHz, Flash, Legacy (new can return nullptr), All SSL ciphers (most compatible), 4MB (FS:2MB OTA:~1019KB), v2 Lower Memory, Disabled, None, Only Sketch, 115200"

Multiple libraries were found for "ESP8266WiFi.h"
 Used: /Users/xxxx/Library/Arduino15/packages/esp8266/hardware/esp8266/2.6.3/libraries/ESP8266WiFi
fork/exec /Users/xxxx/Library/Arduino15/packages/esp8266/tools/python3/3.7.2-post1/python3: no such file or directory
Error compiling for board NodeMCU 1.0 (ESP-12E Module).

This report would have more information with
"Show verbose output during compilation"
option enabled in File -> Preferences.

The text was updated successfully, but these errors were encountered:

JAndrassy · 2019-12-21T13:17:19Z

the "Multiple libraries were found for " is a bug in Arduino IDE (builder). it reports it at any error.
your true error is tools/python3/3.7.2-post1/python3: no such file or directory

harveydobson · 2019-12-23T09:15:51Z

the "Multiple libraries were found for " is a bug in Arduino IDE (builder). it reports it at any error.
your true error is tools/python3/3.7.2-post1/python3: no such file or directory

Thanks, I will investigate further!

JAndrassy · 2019-12-23T09:30:29Z

on arduino forum was discussed that the python3 link is wrong on Mac and should be set to correct path
https://forum.arduino.cc/index.php?topic=649739.msg4401959#msg4401959

@earlephilhower, do you know about the problem?

earlephilhower · 2019-12-23T15:09:15Z

@JAndrassy I think the problem is that there are now 2 spots where python3 may be on the Mac. We're actually building on MacOS as part of CI, now, and when homebrew installs python3, it's in the directory linked in the tarball.

Maybe there was an OS update/release recently where python3 is already installed? MacOS in CI (sorry, no idea what rev but a Mac guy might be able to look at any build log and tell from any PR) only had the obsolete Python2 installed under /usr/bin/python and no Python3 at all (hence the homebrew installation). Can anyone with a Mac confirm?

liebman · 2019-12-23T15:34:19Z

I have an up to date Mac 10.15.2 and there is no python3 in /usr/bin. Mine is installed via brew and its located in /usr/local/bin.

earlephilhower · 2019-12-23T16:15:17Z

Thanks, @liebman . So it looks like on an updated Mac the link in the tarball is correct (at least when brew is used).

@harveydobson, how did you install Python3 on your Mac? brew or something else?

tonywhitfort · 2019-12-28T09:07:37Z

I get the same error (no such file or directory)
MacOS 10.15.2
ESP V2.6.3
python3 is in /usr/bin
I have updated the alias to point to this folder and it now works

earlephilhower · 2019-12-28T18:27:52Z

@tonywhitfort , how did you get Python3 on your Mac? Is it now part of the standard distro, did you download a standalone installer, use brew, or something else?

tonywhitfort · 2019-12-28T23:10:18Z

I didn’t install Python3 myself it was possibly installed by one of these Eclipse IDE (which I installed earlier this year for some Arduino projects) Platform IO and also Xcode (which I installed earlier this year but haven’t used since) Python3 version is 3.7.3 dated 15 Nov 2019, so something is keeping it up to date. I installed MacOS Catalina as an upgrade, I haven’t done a clean install for years

…

On 29 Dec 2019, at 5:27 am, Earle F. Philhower, III ***@***.***> wrote: @tonywhitfort <https://github.com/tonywhitfort> , how did you get Python3 on your Mac? Is it now part of the standard distro, did you download a standalone installer, use brew, or something else? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#6931?email_source=notifications&email_token=AE3IZCW5XHWRME2SRWA3MWLQ26LCTA5CNFSM4J6G5DOKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHYPTLQ#issuecomment-569440686>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE3IZCVAUPJDXSVQE7HTIN3Q26LCTANCNFSM4J6G5DOA>.

farthinder · 2019-12-29T10:48:45Z

Just chiming in, I have this problem as well on my Mac OS.
I cant honestly say where I got my python installation from but likely it came with MacOs or I got it via brew.

Currently /Users/USER/Library/Arduino15/packages/esp8266/tools/python3/3.7.2-post1/python3 points at /usr/local/bin/python3

But it should point at /usr/bin/python3
However, creating such a symlink seems problematic: https://stackoverflow.com/questions/36730549/cannot-create-a-symlink-inside-of-usr-bin-even-as-sudo

farthinder · 2019-12-29T10:57:20Z

My workaround that seems to work:

cd ~/Library/Arduino15/packages/esp8266/tools/python3/3.7.2-post1
sudo unlink python3
sudo ln -s /usr/bin/python3 /usr/local/bin/python3
sudo ln -s /usr/local/bin/python3 python3

earlephilhower · 2019-12-29T16:08:37Z

<rant>What is Apple thinking, keeping only Python 2 in the latest OS</rant>

Anyway, does MacOS have /usr/bin/env and does it work with brew installed P3 as well as P3 direct from python.org? I'm thinking a wrapper script could be written using /usr/bin/env python3 and letting the OS handle disambiguation.

liebman · 2019-12-29T16:17:16Z

I have an up to date Mac 10.15.2 and there is no python3 in /usr/bin. Mine is installed via brew and its located in /usr/local/bin.

UPDATE: I've updated xcode and now I have a python3 in /usr/bin. (and one from brew in /usr/local/bin)

@earlephilhower - yes /usr/bin/env exists and is functional on MacOS.

It's odd, but because Windows requires a full Python3 install we must have an executable called "tools/python3/python3" to use Python3 in the toolchain. Before, we simply symlinked to /usr/bin/python3 (for Linux) or /usr/local/bin/python3 (Mac). Unfortunately, depending on the method of installation, on MacOS the Python3 executable can be in /usr/bin/python3 instead. To avoid the entire issue, unify the Mac and Linux python3 placeholders to use python3 itself to jump to the real executable. Fixes esp8266#6931

earlephilhower · 2019-12-29T17:52:30Z

@farthinder, @tonywhitfort , others with /usr/bin/python3 : Can you please try #6960 and report?

You'll need to re-run the get.py to retrive the new python3 redirect stuff?

You'll want to remove any symlinks you made in /usr/bin/local to make the old version work, of course. This new one uses the env command to find the proper python3 to call and should work on Mac and Linux as well.

farthinder · 2020-01-07T19:47:25Z

@earlephilhower I happened to be setting up a new mac so I gave this a try to the best of my abilities.

First of, on a clean mac when running python3 you get asked if you want to install "command line dev tools" which I declined at first. Then when cloning your pullrequest #6960 I got the same dialogbox when running git so I allowed the "command line dev tools" to get installed.
I cloned the repo, fetched the pull request and initilized the submodules then when running python3 get.py I get a cert warning:

esp8266 % git fetch origin pull/6960/head:pythonfix
esp8266 % git submodule update --init
esp8266 % git checkout pythonfix
esp8266 % cd tools
tools % python3 get.py
Platform: x86_64-apple-darwin
Downloading python3-macosx-placeholder.tar.gz
Traceback (most recent call last):
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 1317, in do_open
encode_chunked=req.has_header('Transfer-encoding'))
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/http/client.py", line 1229, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/http/client.py", line 1275, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/http/client.py", line 1224, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/http/client.py", line 1016, in _send_output
self.send(msg)
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/http/client.py", line 956, in send
self.connect()
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/http/client.py", line 1392, in connect
server_hostname=server_hostname)
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/ssl.py", line 412, in wrap_socket
session=session
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/ssl.py", line 853, in _create
self.do_handshake()
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/ssl.py", line 1117, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1056)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "get.py", line 131, in
main()
File "get.py", line 128, in main
get_tool(tool)
File "get.py", line 80, in get_tool
urlretrieve(url, local_path, report_progress)
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 247, in urlretrieve
with contextlib.closing(urlopen(url, data)) as fp:
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 525, in open
response = self._open(req, data)
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 543, in _open
'_open', req)
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 503, in _call_chain
result = func(*args)
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 1360, in https_open
context=self._context, check_hostname=self._check_hostname)
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 1319, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1056)>

earlephilhower · 2020-01-07T20:01:53Z

That's a MacOS python problem, unfortunately, and not much we can do. When Python3 is installed (any way) it doesn't register the SSL certificates needed to actually do any internet access and dumps the error you're seeing. You'll have the same problem running any other Python3 wget-like thing that needs HTTPS.

The solution is included in the Python3 readme, or should be if it's been repackaged.

( cd "/Applications/Python 3.7/" && sudo "./Install Certificates.command" )

You're going to have to adjust the CD to wherever you've got the Python3 directory installed. That script from the python team does the registration of those certs. After that, https connections in Python should work.

* Use a python3 script to call python3 It's odd, but because Windows requires a full Python3 install we must have an executable called "tools/python3/python3" to use Python3 in the toolchain. Before, we simply symlinked to /usr/bin/python3 (for Linux) or /usr/local/bin/python3 (Mac). Unfortunately, depending on the method of installation, on MacOS the Python3 executable can be in /usr/bin/python3 instead. To avoid the entire issue, unify the Mac and Linux python3 placeholders to use python3 itself to jump to the real executable. Fixes #6931 * Explicitly remove old symlink to python3 The tar extraction for the updated python3 tarball will fail on systems that already have a symlink in /tools/python3/python3 because the tar extractor attempts to open the *target of the symlink* (i.e. the actual interpreter in /usr/bin or /usr/local/bin). Add a commented hack to destroy this symlink before expanding the tarballs, if the file exists. This is safe to do since it will be overwritten by any extractions of the python3 tarball later in the process. Co-authored-by: david gauchard <gauchard@laas.fr>

fabiradi · 2020-04-24T13:17:14Z

My workaround that seems to work:

cd ~/Library/Arduino15/packages/esp8266/tools/python3/3.7.2-post1
sudo unlink python3
sudo ln -s /usr/bin/python3 /usr/local/bin/python3
sudo ln -s /usr/local/bin/python3 python3

Workaround without admin permissions (i.e. in a corporate environment):

cd ~/Library/Arduino15/packages/esp8266/tools/python3/3.7.2-post1
unlink python3
ln -s /usr/bin/python3 python3

The symlink can be created without the need for sudo.

tohqecom · 2020-05-01T09:07:53Z

the first you must run command to identify your current python:
python --version => Python 3.7.7
which python
and ressult like this:

/usr/local/opt/python/libexec/bin/python
cd ~/Library/Arduino15/packages/esp8266/tools/python3/3.7.2-post1
unlink python3
ln -s /usr/local/opt/python/libexec/bin/python ~/Library/Arduino15/packages/esp8266/tools/python3/3.7.2-post1/python3
Just that!

michelcmorel · 2020-05-04T08:42:12Z

cd ~/Library/Arduino15/packages/esp8266/tools/python3/3.7.2-post1
sudo unlink python3
sudo ln -s /usr/bin/python3 /usr/local/bin/python3
sudo ln -s /usr/local/bin/python3 python3

After 12 hrs of headaches trying to figure out how to bypass iMac SIP, so to get python3 path to work with my Wemos d1.. This worked like a charm! Thank you so much.

davecramer · 2020-05-06T21:06:10Z

I get the same error and when I run
/Users/davec/Library/Arduino15/packages/esp8266/tools/python3/3.7.2-post1/python3
it runs fine

danbates2 · 2020-05-18T11:53:38Z

This seems to be still an issue?

Latest versions of all IDE and ESP8266 core except I run High Sierra.

cd ~/Library/Arduino15/packages/esp8266/tools/python3/3.7.2-post1
sudo unlink python3
sudo ln -s /usr/bin/python3 /usr/local/bin/python3
sudo ln -s /usr/local/bin/python3 python3

this works

michelcmorel · 2020-05-18T12:17:01Z

This seems to be still an issue?

Latest versions of all IDE and ESP8266 core except I run High Sierra.

cd ~/Library/Arduino15/packages/esp8266/tools/python3/3.7.2-post1
sudo unlink python3
sudo ln -s /usr/bin/python3 /usr/local/bin/python3
sudo ln -s /usr/local/bin/python3 python3

this works

No, it's no longer an issue for me at least.
Farthinder's answer worked, I'm grateful for his post.

danbates2 · 2020-05-18T12:28:58Z

Yes but I updated the esp core from 2.7.0 to 2.7.1 and the problem resurfaced.
No big deal for most I'm sure.

davecramer · 2020-05-18T13:11:58Z

Mine was an update as well. I had to remove something else to get it to work. Sad that macos is still shipping python 2.

danbates2 · 2020-05-18T13:37:49Z

High Sierra was released September 25, 2017.
Catalina, October 7, 2019, I read now has the placeholder for the python3 bins, so it's easier to integrate using command line tools.
Yeah, it's unfortunate it's not been included by default in recent releases.

I don't know if it's worth accommodating the brew installed python3 package. They're pretty common. The symlink is at /usr/local/bin/python3 right.

blazoncek · 2020-06-13T22:18:10Z

I have python3 installed by HomeBrew and the issue resurfaces every time I update ESP8266 board.
To fix it I run:
cd ~/Library/Arduino15/packages/esp8266/tools/python3/3.7.2-post1
unlink python3
ln -s /usr/local/bin/python3 python3

hellvesper · 2020-06-29T08:19:27Z

I propose to change default python3 location to /usr/bin/python3 as this is system python3 location

superyarik · 2023-04-05T20:00:09Z

solve problem by adding this symlink here Library/Arduino15/packages/esp8266/tools/python3/3.7.2-post1

sudo ln -s /usr/local/bin/python3 python

devyte changed the title ~~Multiple libraries were found for "ESP8266WiFi.h"~~ Error building on MacOS due to python3 not found Dec 26, 2019

earlephilhower mentioned this issue Dec 29, 2019

Use a python3 script to call python3 #6960

Merged

earlephilhower added this to the 2.7.0 milestone Dec 29, 2019

earlephilhower self-assigned this Dec 29, 2019

earlephilhower closed this as completed in #6960 Jan 14, 2020

devyte mentioned this issue May 30, 2020

Projects won't upload with error: python3: No such file or directory #7346

Closed

Tech-TX mentioned this issue Jun 1, 2020

python3: error: unable to find utility "python3", not a developer tool or in PATH #7347

Closed

AlexHedley mentioned this issue Jun 12, 2020

README additions praeclarum/BuildLight#1

Closed

per1234 mentioned this issue Dec 7, 2020

esp32's python dependency won't be installed by the board manager Arduino-CI/arduino_ci#235

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Error building on MacOS due to python3 not found #6931

Error building on MacOS due to python3 not found #6931

harveydobson commented Dec 21, 2019 •

edited by devyte

Loading

JAndrassy commented Dec 21, 2019 •

edited

Loading

harveydobson commented Dec 23, 2019

JAndrassy commented Dec 23, 2019 •

edited

Loading

earlephilhower commented Dec 23, 2019

liebman commented Dec 23, 2019

earlephilhower commented Dec 23, 2019

tonywhitfort commented Dec 28, 2019

earlephilhower commented Dec 28, 2019

tonywhitfort commented Dec 28, 2019 via email

farthinder commented Dec 29, 2019 •

edited

Loading

farthinder commented Dec 29, 2019 •

edited

Loading

earlephilhower commented Dec 29, 2019

liebman commented Dec 29, 2019

earlephilhower commented Dec 29, 2019

farthinder commented Jan 7, 2020 •

edited

Loading

earlephilhower commented Jan 7, 2020

fabiradi commented Apr 24, 2020

tohqecom commented May 1, 2020

michelcmorel commented May 4, 2020

davecramer commented May 6, 2020

danbates2 commented May 18, 2020

michelcmorel commented May 18, 2020

danbates2 commented May 18, 2020

davecramer commented May 18, 2020

danbates2 commented May 18, 2020

blazoncek commented Jun 13, 2020

hellvesper commented Jun 29, 2020

superyarik commented Apr 5, 2023 •

edited

Loading

Error building on MacOS due to python3 not found #6931

Error building on MacOS due to python3 not found #6931

Comments

harveydobson commented Dec 21, 2019 • edited by devyte Loading

Basic Infos

Platform

Settings in IDE

Problem Description

MCVE Sketch

Debug Messages

JAndrassy commented Dec 21, 2019 • edited Loading

harveydobson commented Dec 23, 2019

JAndrassy commented Dec 23, 2019 • edited Loading

earlephilhower commented Dec 23, 2019

liebman commented Dec 23, 2019

earlephilhower commented Dec 23, 2019

tonywhitfort commented Dec 28, 2019

earlephilhower commented Dec 28, 2019

tonywhitfort commented Dec 28, 2019 via email

farthinder commented Dec 29, 2019 • edited Loading

farthinder commented Dec 29, 2019 • edited Loading

earlephilhower commented Dec 29, 2019

liebman commented Dec 29, 2019

earlephilhower commented Dec 29, 2019

farthinder commented Jan 7, 2020 • edited Loading

earlephilhower commented Jan 7, 2020

fabiradi commented Apr 24, 2020

tohqecom commented May 1, 2020

michelcmorel commented May 4, 2020

davecramer commented May 6, 2020

danbates2 commented May 18, 2020

michelcmorel commented May 18, 2020

danbates2 commented May 18, 2020

davecramer commented May 18, 2020

danbates2 commented May 18, 2020

blazoncek commented Jun 13, 2020

hellvesper commented Jun 29, 2020

superyarik commented Apr 5, 2023 • edited Loading

harveydobson commented Dec 21, 2019 •

edited by devyte

Loading

JAndrassy commented Dec 21, 2019 •

edited

Loading

JAndrassy commented Dec 23, 2019 •

edited

Loading

farthinder commented Dec 29, 2019 •

edited

Loading

farthinder commented Dec 29, 2019 •

edited

Loading

farthinder commented Jan 7, 2020 •

edited

Loading

superyarik commented Apr 5, 2023 •

edited

Loading