Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

src: remove cached data tag from snapshot metadata #54122

Merged
merged 2 commits into from
Aug 23, 2024

Conversation

joyeecheung
Copy link
Member

@joyeecheung joyeecheung commented Jul 30, 2024

This only served as a preemptive check, but serializing this in the snapshot would make it unreproducible on different hardware. In the current cached data version tag, the V8 version can already be checked as part of the existing Node.js version check. The V8 flags aren't necessarily important for snapshot/code cache mismatches (only a small subset are), and the CPU features currently don't matter, so doing an exact match is stricter than necessary. Removing the check to help making the snapshot more reproducible on different hardware.

Refs: nodejs/build#3043

@nodejs-github-bot
Copy link
Collaborator

Review requested:

  • @nodejs/startup

@joyeecheung joyeecheung added the request-ci Add this label to start a Jenkins CI on a PR. label Jul 30, 2024
@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. needs-ci PRs that need a full CI run. labels Jul 30, 2024
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jul 30, 2024
@nodejs-github-bot

This comment was marked as outdated.

Copy link

codecov bot commented Jul 30, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 87.32%. Comparing base (2c14615) to head (d93f0a0).
Report is 381 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #54122      +/-   ##
==========================================
+ Coverage   87.08%   87.32%   +0.24%     
==========================================
  Files         648      648              
  Lines      182341   182306      -35     
  Branches    34982    34976       -6     
==========================================
+ Hits       158783   159199     +416     
+ Misses      16831    16377     -454     
- Partials     6727     6730       +3     
Files with missing lines Coverage Δ
src/env.cc 85.29% <ø> (-0.20%) ⬇️
src/env.h 97.87% <ø> (ø)
src/node_snapshotable.cc 78.21% <ø> (-0.55%) ⬇️

... and 50 files with indirect coverage changes

Copy link
Member

@legendecas legendecas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The V8 flags aren't necessarily important for snapshot/code cache mismatches (only a small subset are), and the CPU features currently don't matter

If there are still a subset of V8 flags that matter, can the CpuFeatures flag be removed from v8::ScriptCompiler::CachedDataVersionTag() at https://source.chromium.org/chromium/chromium/src/+/main:v8/src/api/api.cc;l=2854;drc=105770df485ace262780d95126bb60b1a16ec340;bpv=1;bpt=1 instead?

@joyeecheung
Copy link
Member Author

If there are still a subset of V8 flags that matter, can the CpuFeatures flag be removed

We had that discussion in https://chromium-review.googlesource.com/c/v8/v8/+/4905290 and the short answer is no,because the V8 team wants to reserve the ability to make the code cache CPU dependent. This API is used by Chromium as a key for cache storage to detect staleness, so it's not really something we can just break for Node.js either.

For the hardware-specific data in code cache, I think the better way forward would be something like what I mentioned in nodejs/build#3043 (comment) - we should ideally update V8 to persist "flags/CPU features required by this cache", instead of "flags/CPU features used to compile this cache", and let the cache get rejected if it doesn't work. But for the data accompanying snapshots, this is what we can do for now.

@joyeecheung joyeecheung added the request-ci Add this label to start a Jenkins CI on a PR. label Jul 30, 2024
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jul 30, 2024
@nodejs-github-bot

This comment was marked as outdated.

Copy link
Member

@legendecas legendecas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good to me. A minor fix to address complaining CI.

test/sequential/sequential.status Outdated Show resolved Hide resolved
@joyeecheung joyeecheung added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 5, 2024
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 5, 2024
@nodejs-github-bot

This comment was marked as outdated.

@joyeecheung joyeecheung added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 7, 2024
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 7, 2024
@nodejs-github-bot

This comment was marked as outdated.

@joyeecheung joyeecheung added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 8, 2024
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 8, 2024
@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@joyeecheung joyeecheung added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 14, 2024
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 14, 2024
@nodejs-github-bot
Copy link
Collaborator

@joyeecheung joyeecheung added request-ci Add this label to start a Jenkins CI on a PR. and removed request-ci Add this label to start a Jenkins CI on a PR. labels Aug 16, 2024
@nodejs-github-bot
Copy link
Collaborator

@joyeecheung joyeecheung added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 18, 2024
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 18, 2024
@nodejs-github-bot
Copy link
Collaborator

@joyeecheung
Copy link
Member Author

Fixed linter complaints..

@joyeecheung joyeecheung added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 19, 2024
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 19, 2024
@nodejs-github-bot
Copy link
Collaborator

@joyeecheung joyeecheung added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 19, 2024
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 19, 2024
@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@joyeecheung joyeecheung added commit-queue Add this label to land a pull request using GitHub Actions. commit-queue-squash Add this label to instruct the Commit Queue to squash all the PR commits into the first one. labels Aug 22, 2024
@nodejs-github-bot nodejs-github-bot added commit-queue-failed An error occurred while landing this pull request using GitHub Actions. and removed commit-queue Add this label to land a pull request using GitHub Actions. labels Aug 22, 2024
@nodejs-github-bot
Copy link
Collaborator

Commit Queue failed
- Loading data for nodejs/node/pull/54122
✔  Done loading data for nodejs/node/pull/54122
----------------------------------- PR info ------------------------------------
Title      src: remove cached data tag from snapshot metadata (#54122)
Author     Joyee Cheung <joyeec9h3@gmail.com> (@joyeecheung)
Branch     joyeecheung:remove-flag -> nodejs:main
Labels     c++, needs-ci, commit-queue-squash
Commits    2
 - src: remove cached data tag from snapshot metadata
 - fixup! src: remove cached data tag from snapshot metadata
Committers 1
 - Joyee Cheung <joyeec9h3@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/54122
Refs: https://github.com/nodejs/build/issues/3043
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
------------------------------ Generated metadata ------------------------------
PR-URL: https://github.com/nodejs/node/pull/54122
Refs: https://github.com/nodejs/build/issues/3043
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
--------------------------------------------------------------------------------
   ⚠  Commits were pushed since the last approving review:
   ⚠  - src: remove cached data tag from snapshot metadata
   ⚠  - fixup! src: remove cached data tag from snapshot metadata
   ℹ  This PR was created on Tue, 30 Jul 2024 11:11:10 GMT
   ✔  Approvals: 2
   ✔  - Yagiz Nizipli (@anonrig) (TSC): https://github.com/nodejs/node/pull/54122#pullrequestreview-2208197134
   ✔  - Chengzhong Wu (@legendecas) (TSC): https://github.com/nodejs/node/pull/54122#pullrequestreview-2219446395
   ✔  Last GitHub CI successful
   ℹ  Last Full PR CI on 2024-08-22T16:08:27Z: https://ci.nodejs.org/job/node-test-pull-request/61351/
- Querying data for job/node-test-pull-request/61351/
   ✔  Last Jenkins CI successful
--------------------------------------------------------------------------------
   ✔  Aborted `git node land` session in /home/runner/work/node/node/.ncu
https://github.com/nodejs/node/actions/runs/10514538417

@legendecas legendecas added commit-queue Add this label to land a pull request using GitHub Actions. and removed commit-queue-failed An error occurred while landing this pull request using GitHub Actions. labels Aug 23, 2024
@nodejs-github-bot nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Aug 23, 2024
@nodejs-github-bot nodejs-github-bot merged commit 9ee3a72 into nodejs:main Aug 23, 2024
67 checks passed
@nodejs-github-bot
Copy link
Collaborator

Landed in 9ee3a72

RafaelGSS pushed a commit that referenced this pull request Aug 25, 2024
This only served as a preemptive check, but serializing this in
the snapshot would make it unreproducible on different hardware.
In the current cached data version tag, the V8 version can already
be checked as part of the existing Node.js version check. The V8
flags aren't necessarily important for snapshot/code cache mismatches
(only a small subset are), and the CPU features currently don't
matter, so doing an exact match is stricter than necessary.
Removing the check to help making the snapshot more reproducible on
different hardware.

PR-URL: #54122
Refs: nodejs/build#3043
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
@RafaelGSS RafaelGSS mentioned this pull request Aug 25, 2024
RafaelGSS pushed a commit that referenced this pull request Aug 30, 2024
This only served as a preemptive check, but serializing this in
the snapshot would make it unreproducible on different hardware.
In the current cached data version tag, the V8 version can already
be checked as part of the existing Node.js version check. The V8
flags aren't necessarily important for snapshot/code cache mismatches
(only a small subset are), and the CPU features currently don't
matter, so doing an exact match is stricter than necessary.
Removing the check to help making the snapshot more reproducible on
different hardware.

PR-URL: #54122
Refs: nodejs/build#3043
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
@targos targos added the dont-land-on-v20.x PRs that should not land on the v20.x-staging branch and should not be released in v20.x. label Sep 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c++ Issues and PRs that require attention from people who are familiar with C++. commit-queue-squash Add this label to instruct the Commit Queue to squash all the PR commits into the first one. dont-land-on-v20.x PRs that should not land on the v20.x-staging branch and should not be released in v20.x. needs-ci PRs that need a full CI run.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants