Upgrading node-gyp independently of npm and handling multiple versions

[rvagg]

(This could be made into a top-level doc, or integrated into the README, it comes up enough. Please edit this post or comment below if you have improvement suggestions. If someone wants to turn this into a PR then you're welcome to.)

Wot?

npm ships with its own version of node-gyp. Older versions of npm have older versions of node-gyp. npm has caught up recently to node-gyp's release cycle but it's expected we'll get out of sync and there will be cases where you need an old npm but a newer node-gyp, or npm doesn't ship yet with a node-gyp that we've released and it has fixes you need.

Generally, npm's library files are installed inside your global "node_modules", where npm is installed (run npm prefix and add lib/node_modules, or just node_modules for Windows [I think, Windows person - please confirm and edit this]). There are some exceptions to this. Inside this global node_modules/ there will be an npm/ directory and inside this you'll find a node_modules/node-gyp/ directory. So it may look something like /usr/local/lib/node_modules/npm/node_modules/node-gyp/. This is the version of node-gyp that ships with npm.

When you install a new version of node-gyp with outside of npm, it'll go into your global node_modules, but not under the npm/node_modules. So that may look like /usr/local/lib/node_modules/node-gyp/. It'll have the node-gyp executable linked into your PATH so running node-gyp will use this version.

The catch is that npm won't use this version unless you tell it to, it'll keep on using the one you have installed. You need to instruct it to by setting the node_gyp config variable (which goes into your ~/.npmrc). You do this by running the npm config set command as below. Then npm will use the command in the path you supply whenever it needs to build a native addon.

Important: You also need to remember to unset this when you upgrade npm with a newer version of node-gyp, or you have to manually keep your globally installed node-gyp to date. See "Undo" below.

Linux and macOS

npm install --global node-gyp@latest
npm config set node_gyp $(npm prefix -g)/lib/node_modules/node-gyp/bin/node-gyp.js

sudo may be required for the first command if you get a permission error.

Windows

@joaocgreis' Windows instructions from #1753 (comment)

Windows Command Prompt

npm install --global node-gyp@latest
for /f "delims=" %P in ('npm prefix -g') do npm config set node_gyp "%P\node_modules\node-gyp\bin\node-gyp.js"

Powershell

npm install --global node-gyp@latest
npm prefix -g | % {npm config set node_gyp "$_\node_modules\node-gyp\bin\node-gyp.js"}

Undo

Beware if you don't unset the node_gyp config option, npm will continue to use the globally installed version of node-gyp rather than the one it ships with, which may end up being newer.

npm uninstall --global node-gyp
npm config delete node_gyp

[rvagg]

We could also mention that you can achieve the same thing by setting the env var npm_config_node_gyp=/path/to/node-gyp.js, which is a less permanent solution.

[cclauss]

@liuliangsir Your issue is related to #661 #2058 #2110 #2112

[liuliangsir]

@cclauss okay, thanks!

[owl-from-hogvarts]

I think that should be added to node-gyp's wiki

[charles-ekube]

I need help!!!!!!

npm ERR! code 1
npm ERR! path C:\Users\Fragile\Desktop\omnipublish-for-testing\OPV5\OMNIPUBLISH\srcDispatch\node_modules\iltorb
npm ERR! command failed
npm ERR! command C:\WINDOWS\system32\cmd.exe /d /s /c node ./scripts/install.js || node-gyp rebuild
npm ERR! info install installing standalone, skipping download.
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using node-gyp@3.8.0
npm ERR! gyp info using node@16.4.2 | win32 | x64
npm ERR! gyp ERR! configure error
npm ERR! gyp ERR! stack Error: Command failed: C:\Users\Fragile\AppData\Local\Programs\Python\Python39\Python.EXE -c import sys; print "%s.%s.%s" % sys.version_info[:3];
npm ERR! gyp ERR! stack File "", line 1
npm ERR! gyp ERR! stack import sys; print "%s.%s.%s" % sys.version_info[:3];
npm ERR! gyp ERR! stack ^
npm ERR! gyp ERR! stack SyntaxError: invalid syntax
npm ERR! gyp ERR! stack
npm ERR! gyp ERR! stack at ChildProcess.exithandler (node:child_process:397:12)
npm ERR! gyp ERR! stack at ChildProcess.emit (node:events:394:28)
npm ERR! gyp ERR! stack at maybeClose (node:internal/child_process:1067:16)
npm ERR! gyp ERR! stack at Process.ChildProcess._handle.onexit (node:internal/child_process:301:5)
npm ERR! gyp ERR! System Windows_NT 10.0.18363
npm ERR! gyp ERR! command "C:\Program Files\nodejs\node.exe" "C:\Users\Fragile\Desktop\omnipublish-for-testing\OPV5\OMNIPUBLISH\srcDispatch\node_modules\node-gyp\bin\node-gyp.js" "rebuild"
npm ERR! gyp ERR! cwd C:\Users\Fragile\Desktop\omnipublish-for-testing\OPV5\OMNIPUBLISH\srcDispatch\node_modules\iltorb
npm ERR! gyp ERR! node -v v16.4.2
npm ERR! gyp ERR! node-gyp -v v3.8.0
npm ERR! gyp ERR! not ok

npm ERR! A complete log of this run can be found in:
npm ERR! C:\Users\Fragile\AppData\Local\npm-cache_logs\2021-09-01T02_58_14_892Z-debug.log

[cclauss]

npm ERR! gyp info using node-gyp@3.8.0

https://github.com/nodejs/node-gyp/blob/master/docs/Updating-npm-bundled-node-gyp.md

[technoken35]

Hi, the solution above did not work for me. I am running Linux.

[owl-from-hogvarts]

@technoken35 hi. Tell us your environment (versions):

1. Node
2. Npm
3. Node-gyp

Also send your npm config file

[technoken35]

I also have npm installed via homebrew. I'm not sure if you want to see the user config or so I placed contents of both.

Node - 16.9.2
Npm - 7.2.1
Node-gyp - 8.2.0

contents of user config
python=python2.7 node_gyp=/home/linuxbrew/.linuxbrew/lib/node_modules/node-gyp config=set node-gyp=/home/linuxbrew/.linuxbrew/home/kendall/node-gyp/

contents of built in
prefix = /home/linuxbrew/.linuxbrew

Thank you!

[cclauss]

Python 2 died 623 days ago on 1/1/2020 so node-gyp is no longer compatible with it. Current Python is 3.9.7. https://www.python.org/downloads/

[technoken35]

@cclauss I wasn't using python 2 when I ran the command

npm install --python=python3

[cclauss]

Edit user config to remove Py2.

npm ERR! gyp info using node-gyp@3.8.0

https://github.com/nodejs/node-gyp/blob/master/docs/Updating-npm-bundled-node-gyp.md

[technoken35]

Changed to python 3. Looked at the logs and noticed something weird. npm ERR! gyp info using node-gyp@3.8.0. Shouldn't that be version 8?

[technoken35]

@cclauss it was node-sass, I saw your answer ##here

[cclauss]

Yes. That is why you should do the process in the URL that I provided above.