Skip to content

Commit

Permalink
feat(gyp): update gyp to v0.18.1 (#3039)
Browse files Browse the repository at this point in the history
* feat(gyp): update gyp to v0.18.1

* ci: setup ninja

* ci: visual-studio job use python 3.12
  • Loading branch information
toyobayashi committed Jun 12, 2024
1 parent c495083 commit ea99fea
Show file tree
Hide file tree
Showing 34 changed files with 3,669 additions and 89 deletions.
1 change: 1 addition & 0 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,7 @@ jobs:
python-version: ${{ matrix.python }}
env:
PYTHON_VERSION: ${{ matrix.python }} # Why do this?
- uses: seanmiddleditch/gha-setup-ninja@v4
- name: Install Dependencies
run: |
npm install
Expand Down
8 changes: 5 additions & 3 deletions .github/workflows/visual-studio.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,12 @@ jobs:
steps:
- name: Checkout Repository
uses: actions/checkout@v4
- name: Use Python 3.12
uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install Dependencies
run: npm install
- name: Run Node tests
shell: pwsh
run: |
$pythonLocation = (Get-Command python).Source
npm run test --python="${pythonLocation}" --msvs-version="${{ matrix.msvs-version }}"
run: npm run test --python="${env:pythonLocation}\\python.exe" --msvs-version="${{ matrix.msvs-version }}"
20 changes: 20 additions & 0 deletions gyp/.github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Keep GitHub Actions up to date with Dependabot...
# https://docs.github.com/en/code-security/dependabot/working-with-dependabot/keeping-your-actions-up-to-date-with-dependabot
version: 2
updates:
- package-ecosystem: "github-actions"
directory: "/"
groups:
GitHub_Actions:
patterns:
- "*" # Group all Actions updates into a single larger pull request
schedule:
interval: weekly
- package-ecosystem: "pip"
directory: "/"
groups:
pip:
patterns:
- "*" # Group all pip updates into a single larger pull request
schedule:
interval: weekly
11 changes: 8 additions & 3 deletions gyp/.github/workflows/Python_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,15 +15,16 @@ jobs:
fail-fast: false
max-parallel: 5
matrix:
os: [macos-latest, ubuntu-latest] # , windows-latest]
python-version: ["3.8", "3.9", "3.10", "3.11", "3.12"]
os: [macos-13, macos-14, ubuntu-latest] # , windows-latest]
python-version: ["3.8", "3.9", "3.10", "3.11", "3.12", "3.13"]
steps:
- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
allow-prereleases: true
- uses: seanmiddleditch/gha-setup-ninja@v4
- name: Install dependencies
run: |
python -m pip install --upgrade pip setuptools
Expand All @@ -35,3 +36,7 @@ jobs:
run: pytest
# - name: Run doctests with pytest
# run: pytest --doctest-modules
- name: Test CLI commands on a pipx install
run: |
pipx run --no-cache --spec ./ gyp --help
pipx run --no-cache --spec ./ gyp --version
14 changes: 7 additions & 7 deletions gyp/.github/workflows/node-gyp.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,9 @@ jobs:
strategy:
fail-fast: false
matrix:
os: [macos-latest, ubuntu-latest, windows-latest]
python: ["3.8", "3.10", "3.12"]

node-version: ["22"]
os: [macos-13, macos-14, ubuntu-latest, windows-latest]
python-version: ["3.8", "3.10", "3.12", "3.13"]
runs-on: ${{ matrix.os }}
steps:
- name: Clone gyp-next
Expand All @@ -24,12 +24,12 @@ jobs:
with:
repository: nodejs/node-gyp
path: node-gyp
- uses: actions/setup-node@v3
- uses: actions/setup-node@v4
with:
node-version: 18.x
- uses: actions/setup-python@v4
node-version: ${{ matrix.node-version }}
- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python }}
python-version: ${{ matrix.python-version }}
allow-prereleases: true
- name: Install Python dependencies
run: |
Expand Down
93 changes: 87 additions & 6 deletions gyp/.github/workflows/release-please.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,91 @@ name: release-please
jobs:
release-please:
runs-on: ubuntu-latest
outputs:
release_created: ${{ steps.release.outputs.release_created }}
tag_name: ${{ steps.release.outputs.tag_name }}
permissions:
contents: write
pull-requests: write
steps:
- uses: google-github-actions/release-please-action@v3
with:
token: ${{ secrets.GITHUB_TOKEN }}
release-type: python
package-name: gyp-next
bump-minor-pre-major: true
- uses: google-github-actions/release-please-action@v4
id: release

build:
name: Build distribution
needs:
- release-please
if: ${{ needs.release-please.outputs.release_created }} # only publish on release
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install pypa/build
run: >-
python3 -m pip install build --user
- name: Build a binary wheel and a source tarball
run: python3 -m build
- name: Store the distribution packages
uses: actions/upload-artifact@v4
with:
name: python-package-distributions
path: dist/

publish-to-pypi:
name: >-
Publish Python distribution to PyPI
needs:
- release-please
- build
if: ${{ needs.release-please.outputs.release_created }} # only publish on release
runs-on: ubuntu-latest
environment:
name: pypi
url: https://pypi.org/p/gyp-next
permissions:
id-token: write # IMPORTANT: mandatory for trusted publishing
steps:
- name: Download all the dists
uses: actions/download-artifact@v4
with:
name: python-package-distributions
path: dist/
- name: Publish distribution to PyPI
uses: pypa/gh-action-pypi-publish@release/v1

github-release:
name: >-
Publish Python distribution to GitHub Release
needs:
- release-please
- build
if: ${{ needs.release-please.outputs.release_created }} # only publish on release
runs-on: ubuntu-latest
permissions:
contents: write # IMPORTANT: mandatory for making GitHub Releases
id-token: write # IMPORTANT: mandatory for sigstore
steps:
- name: Download all the dists
uses: actions/download-artifact@v4
with:
name: python-package-distributions
path: dist/
- name: Sign the dists with Sigstore
uses: sigstore/gh-action-sigstore-python@v2.1.1
with:
inputs: >-
./dist/*.tar.gz
./dist/*.whl
- name: Upload artifact signatures to GitHub Release
env:
GITHUB_TOKEN: ${{ github.token }}
# Upload to GitHub Release using the `gh` CLI.
# `dist/` contains the built packages, and the
# sigstore-produced signatures and certificates.
run: >-
gh release upload
${{ needs.release-please.outputs.tag_name }} dist/**
--repo '${{ github.repository }}'
3 changes: 3 additions & 0 deletions gyp/.release-please-manifest.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
".": "0.18.1"
}
41 changes: 41 additions & 0 deletions gyp/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,46 @@
# Changelog

## [0.18.1](https://github.com/nodejs/gyp-next/compare/v0.18.0...v0.18.1) (2024-05-26)


### Bug Fixes

* **ci:** add Python 3.13 pre-release to test matrix ([#257](https://github.com/nodejs/gyp-next/issues/257)) ([8597203](https://github.com/nodejs/gyp-next/commit/8597203b687325c7516367135e026586279d0583))


### Documentation

* vendor docs from gyp.gsrc.io ([#254](https://github.com/nodejs/gyp-next/issues/254)) ([8d7ba6e](https://github.com/nodejs/gyp-next/commit/8d7ba6e784dedf1122a0456150c739d2a09ecf57))

## [0.18.0](https://github.com/nodejs/gyp-next/compare/v0.17.0...v0.18.0) (2024-05-08)


### Features

* support language standard keys in msvs_settings ([#252](https://github.com/nodejs/gyp-next/issues/252)) ([322f6d5](https://github.com/nodejs/gyp-next/commit/322f6d5d5233967522f3e55c623a8e7d7281e024))

## [0.17.0](https://github.com/nodejs/gyp-next/compare/v0.16.2...v0.17.0) (2024-04-29)


### Features

* generate compile_commands.json with ninja ([#228](https://github.com/nodejs/gyp-next/issues/228)) ([7b20b46](https://github.com/nodejs/gyp-next/commit/7b20b4673d8cf46ff61898eb19569007d55c854a))


### Bug Fixes

* failed to detect flavor if compiler path include white spaces ([#240](https://github.com/nodejs/gyp-next/issues/240)) ([f3b9753](https://github.com/nodejs/gyp-next/commit/f3b9753e7526377020e7d40e66b624db771cf84a))
* support cross compiling for wasm with make generator ([#222](https://github.com/nodejs/gyp-next/issues/222)) ([de0e1c9](https://github.com/nodejs/gyp-next/commit/de0e1c9a5791d1bf4bc3103f878ab74814864ab4))
* support empty dictionary keys in input ([#245](https://github.com/nodejs/gyp-next/issues/245)) ([178459f](https://github.com/nodejs/gyp-next/commit/178459ff343a2771d5f30f04467d2f032d6b3565))
* update Ruff to 0.3.1 ([876ccaf](https://github.com/nodejs/gyp-next/commit/876ccaf5629e1b95e13aaa2b0eb6cbd08fa80593))

## [0.16.2](https://github.com/nodejs/gyp-next/compare/v0.16.1...v0.16.2) (2024-03-07)


### Bug Fixes

* avoid quoting cflag name and parameter with space separator ([#223](https://github.com/nodejs/gyp-next/issues/223)) ([2b9703d](https://github.com/nodejs/gyp-next/commit/2b9703dbd5b3b8a935faf257c6103033b47bf8bf))

## [0.16.1](https://github.com/nodejs/gyp-next/compare/v0.16.0...v0.16.1) (2023-10-25)


Expand Down
4 changes: 4 additions & 0 deletions gyp/CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# Contributing to gyp-next

## Start contributing

Read the docs at [`./docs/Hacking.md`](./docs/Hacking.md) to get started.

## Code of Conduct

This project is bound to the [Node.js Code of Conduct](https://github.com/nodejs/admin/blob/HEAD/CODE_OF_CONDUCT.md).
Expand Down
2 changes: 1 addition & 1 deletion gyp/README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
GYP can Generate Your Projects.
===================================

Documents are available at [gyp.gsrc.io](https://gyp.gsrc.io), or you can check out ```md-pages``` branch to read those documents offline.
Documents are available at [`./docs`](./docs).

__gyp-next__ is [released](https://github.com/nodejs/gyp-next/releases) to the [__Python Packaging Index__](https://pypi.org/project/gyp-next) (PyPI) and can be installed with the command:
* `python3 -m pip install gyp-next`
Expand Down
4 changes: 4 additions & 0 deletions gyp/data/ninja/build.ninja
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
rule cc
command = cc $in $out

build my.out: cc my.in
116 changes: 116 additions & 0 deletions gyp/docs/GypVsCMake.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
# vs. CMake

GYP was originally created to generate native IDE project files (Visual Studio, Xcode) for building [Chromium](http://www.chromim.org).

The functionality of GYP is very similar to the [CMake](http://www.cmake.org)
build tool. Bradley Nelson wrote up the following description of why the team
created GYP instead of using CMake. The text below is copied from
http://www.mail-archive.com/webkit-dev@lists.webkit.org/msg11029.html

```
Re: [webkit-dev] CMake as a build system?
Bradley Nelson
Mon, 19 Apr 2010 22:38:30 -0700
Here's the innards of an email with a laundry list of stuff I came up with a
while back on the gyp-developers list in response to Mike Craddick regarding
what motivated gyp's development, since we were aware of cmake at the time
(we'd even started a speculative port):
I did an exploratory port of portions of Chromium to cmake (I think I got as
far as net, base, sandbox, and part of webkit).
There were a number of motivations, not all of which would apply to other
projects. Also, some of the design of gyp was informed by experience at
Google with large projects built wholly from source, leading to features
absent from cmake, but not strictly required for Chromium.
1. Ability to incrementally transition on Windows. It took us about 6 months
to switch fully to gyp. Previous attempts to move to scons had taken a long
time and failed, due to the requirement to transition while in flight. For a
substantial period of time, we had a hybrid of checked in vcproj and gyp generated
vcproj. To this day we still have a good number of GUIDs pinned in the gyp files,
because different parts of our release pipeline have leftover assumptions
regarding manipulating the raw sln/vcprojs. This transition occurred from
the bottom up, largely because modules like base were easier to convert, and
had a lower churn rate. During early stages of the transition, the majority
of the team wasn't even aware they were using gyp, as it integrated into
their existing workflow, and only affected modules that had been converted.
2. Generation of a more 'normal' vcproj file. Gyp attempts, particularly on
Windows, to generate vcprojs which resemble hand generated projects. It
doesn't generate any Makefile type projects, but instead produces msvs
Custom Build Steps and Custom Build Rules. This makes the resulting projects
easier to understand from the IDE and avoids parts of the IDE that simply
don't function correctly if you use Makefile projects. Our early hope with
gyp was to support the least common denominator of features present in each
of the platform specific project file formats, rather than falling back on
generated Makefiles/shell scripts to emulate some common abstraction. CMake by
comparison makes a good faith attempt to use native project features, but
falls back on generated scripts in order to preserve the same semantics on
each platforms.
3. Abstraction on the level of project settings, rather than command line
flags. In gyp's syntax you can add nearly any option present in a hand
generated xcode/vcproj file. This allows you to use abstractions built into
the IDEs rather than reverse engineering them possibly incorrectly for
things like: manifest generation, precompiled headers, bundle generation.
When somebody wants to use a particular menu option from msvs, I'm able to
do a web search on the name of the setting from the IDE and provide them
with a gyp stanza that does the equivalent. In many cases, not all project
file constructs correspond to command line flags.
4. Strong notion of module public/private interface. Gyp allows targets to
publish a set of direct_dependent_settings, specifying things like
include_dirs, defines, platforms specific settings, etc. This means that
when module A depends on module B, it automatically acquires the right build
settings without module A being filled with assumptions/knowledge of exactly
how module B is built. Additionally, all of the transitive dependencies of
module B are pulled in. This avoids their being a single top level view of
the project, rather each gyp file expresses knowledge about its immediate
neighbors. This keep local knowledge local. CMake effectively has a large
shared global namespace.
5. Cross platform generation. CMake is not able to generate all project
files on all platforms. For example xcode projects cannot be generated from
windows (cmake uses mac specific libraries to do project generation). This
means that for instance generating a tarball containing pregenerated
projects for all platforms is hard with Cmake (requires distribution to
several machine types).
6. Gyp has rudimentary cross compile support. Currently we've added enough
functionality to gyp to support x86 -> arm cross compiles. Last I checked
this functionality wasn't present in cmake. (This occurred later).
That being said there are a number of drawbacks currently to gyp:
1. Because platform specific settings are expressed at the project file
level (rather than the command line level). Settings which might otherwise
be shared in common between platforms (flags to gcc on mac/linux), end up
being repeated twice. Though in fairness there is actually less sharing here
than you'd think. include_dirs and defines actually represent 90% of what
can be typically shared.
2. CMake may be more mature, having been applied to a broader range of
projects. There a number of 'tool modules' for cmake, which are shared in a
common community.
3. gyp currently makes some nasty assumptions about the availability of
chromium's hermetic copy of cygwin on windows. This causes you to either
have to special case a number of rules, or swallow this copy of cygwin as a
build time dependency.
4. CMake includes a fairly readable imperative language. Currently Gyp has a
somewhat poorly specified declarative language (variable expansion happens
in sometimes weird and counter-intuitive ways). In fairness though, gyp assumes
that external python scripts can be used as an escape hatch. Also gyp avoids
a lot of the things you'd need imperative code for, by having a nice target
settings publication mechanism.
5. (Feature/drawback depending on personal preference). Gyp's syntax is
DEEPLY nested. It suffers from all of Lisp's advantages and drawbacks.
-BradN
```
Loading

0 comments on commit ea99fea

Please sign in to comment.