feat(infra): migrate to Next.js infrastructure

[ovflowd]

This PR is a draft to migrate the underlying infrastructure from Handlebars and Metalsmith to JSX (React) and Next.js.

PR-Progress (Checklist)

• Migrating the Styles and Layouts
  • Migrate the Handlebars template engine scripts into JSX Hooks/JavaScript utilities
    • Around 100% of the scripts are now migrated to Next.js. The remaining scripts will be ported together with their respective Layouts.
  • Migrating the Handlebars layouts (templates) into JSX/React Components
    • Around 100% of the layouts are migrated and ready.
  • Re-linking the existing styles (Sass) to the new JSX Components
• Update the Nextra Theme with:
  • Understanding the current Frontmatter
  • Linking the correct layouts based on Frontmatter
  • Update the theme.tsx to use the correct layout
• Migration of Content
  • All content is buildable
  • Non-translated pages within translation folders were removed (As they serve no use)
  • Markdown build errors were fixed and non-conforming Markdown was fixed
• Final underlying mechanisms such as:
  • Migrating the pure JavaScript scripts (public/static/scripts) to be part of JSX
    • I've decided to do this outside this PR, as it would be changing the content, and this can be done as they will be discarded in the future anyways.
  • Remove old/legacy scripts/
  • Update other scripts, such as the release-post script
  • Add the meta/header/metadata tags with Nextra
  • Implement the RSS feed feature
  • Implement the Sitemap feature
  • i18n functionality
    • Port all translations from their JSON files to thei18n/locales/{localeCode}.json
  • Node.js version & release schedule metadata
• Preparing the PR to its readiness for review
  • Check the Markdown files are OK and working with the current MDX engine
  • Update the tests and final preparations
  • Check if everything is working, such as build, links, and other things
  • Re-enable build on CI
  • Optimise legacy scripts from public/static/js to be modern-proof and work well with Next.js
  • Moved the external/survey content to public/en/
• PR ready to be reviewed
• Performance Optimisations (Updated on 24/01/2023)
  • Compression and modification of MDX HTML output on Next.js-side
  • Reduction of build-time and bundled content
  • Compression of the bundled i18n JSON

Changes

Migration of Structure

A few directories have been changed to accommodate the new Next.js folder structure. Notably, these are the significant changes:

• Some Handlebar utilities became React Components under components
• Sass files (SCSS) were moved to styles
• Pages were moved from locale/ to pages/
• Static Files were moved from static/ to public/static (Links will be preserved)
• Layouts will be ported over React to layouts
• Data files (such as versions.json and schedule.json) were moved to data
• I18n Configuration files and Translations will be moved to i18n

The deletion of per-locale site.json and creation of a global site.json

i18n specific metatdata will be moved to i18n/config.json as the aggregate of all i18n metadata. Specific translation texts (such as "Next" button) or other translated text will be moved to i18n localization files such as i18n/locales/en.json. This is the same approach adopted currently on nodejs.dev repository to ease the migration.

Navigation structure and other non-related global metadata will be moved towards a global data/site.json and respectively for Navigation to pages/_meta.json files.

Migration of Handlebars Templates to JSX

Many of the current "Handlebars" scripts (for templating-engine) will be ported to either JavaScript utils or JSX Components, and the current Layouts will also be ported to JSX.

Migration from old tooling/scripts

Currently, we have scripts to generate the metadata of releases, versions, and the release schedule of Node.js; we should adopt the nodes from Nodejs.dev as they're going to be adopted anyways. This can be done either in this PR or in a follow-up, depending on the complexity of migrating the current scripts.

Items to be done after this PR

As mentioned, this is an initial implementation for this issue, following-up there will still be some improvements to be done, such as removing more legacy content and normalizing the JSX code and a few other things.

After that, we can start porting the layout and content from the Nodejs.dev repository.

Final Considerations (Updated on 24/01/2023)

As mentioned, this is an initial implementation (part of a transitionary phase). Much of the content added within this PR will be discarded in future PRs.

A few quirks from the original nodejs.org website are not available within this initial PR, as they do not bring value in the short-term (having the consideration that these same things will be discarded and considering the value x effort to have them done).

What was not ported from the original Nodejs.org codebase?

• Edit on GitHub links. It used a complex logic that would need to be replicated differently, which would be too complex and non-ideal for this current implementation.
  • Edit Links will be available in future PRs from the ported codebase of Nodejs.dev
• Non-translated pages will not be fallback to the English page + translated React Components
  • On the main branch, all non-translated pages have their respective i18n context injected and the original English content file (Markdown) copied and bundled together.
  • This would allow you to access nodejs.org/{otherLanguage}/some-page-not-translated-in-that-language without being redirected to https://nodejs.org/en/the-original-page-in-english
  • This was an exciting gimmick as at least the menubars, and other static translatable content would be translated.
  • Yet this is not ideal as we would have hundreds of new pages that, in the end, do not bring much context and value to that language for just the benefit of having a few buttons and header texts translated. With the enormous cost of performance and amount of generated files.
  • The routing of non-translated pages is handled by Nginx, meaning if the page does not exist on a translated version, it will be fallback to English.
  • This will be addressed on future PRs with SSR (Server-Side-Rendering) and the usage of Next.js's built-in i18n feature.
• Download Page => Previous Builds, instead of showing all versions since Node.js 0.10, we adopted the same approach done on https://nodejs.dev and show all the previous versions that are still "maintained" or in "maintenance" mode. All that page data was significant, and we wonder if people want to search for such old versions of Node.js on the website.
• The Handlebars Unit Tests were removed. As these layouts and components are soon to be removed, we figured that porting a few unit tests would not bring benefits.
• HTML tag with dir and htmlLang attributes cannot be added due of how Next.js document composition works. This will be solved with SSR on a following PR.

What is going to be done on very immediate PRs

• All the current content (except en/blog) will be replaced by the pages from nodejs.dev. Meaning translations will also be dropped. This was discussed here.
• We're planning on adopting either Vercel infrastructure (which would remove a lot of burden on our Build WG) or just a regular SSR installation on our servers. This would allow ISB/ISR (Incremental Static Builds/Rendering) and SSR (Server Side Rendering). It would also allow a better i18n implementation that doesn't require the two-letter folder approach and way less file generation at build time.
• Once the migration of the layouts from nodejs.dev starts, all the current layouts, scripts, and many other existing components will be dropped or adapted.
• OS detection, Dark Mode toggle, and Language Picker are done through legacy vanilla JS scripts that are not-minified/optimized. These will be removed in favor of JSX code that comes from nodejs.dev

Testing Locally / Caveats on the Static Build

TL;DR: npm ci && npm run serve. Browser: http://localhost:3000/en/

• A few things are handled directly by NGINX on our Infrastructure. Such as / to /en redirection, 404-page handling, and redirection from untranslated pages to /en/original-apge. So please do not worry if these don't work if you're trying out a complete exported build of this branch locally. These should work out of the box with the current NGINX configuration.
• To run the current branch on development mode, we recommend npm ci && npm run serve
• To run the current branch on production mode, we recommend npm ci && npm run build && npm run start

---

Live version: (Updated on 23/01/2023) https://nodejs-org-next-92b1511700.netlify.app/en/

Relates to #4958

cc @nodejs/website @nodejs/nodejs-dev @nodejs/build

[richardlau]

This PR is a draft to migrate the underlying infrastructure from Handlebars and Metalsmith to JSX (React) and Next.js.

Changes

Migration of Structure

A few directories are changed to accommodate the new Next.js folder structure. Notably, these are the significant changes:

* Pages were moved from `locale/` to `pages/`

...
Since this PR is in a draft state, there are still a few things to be done before we can say we're ready to get this PR reviewed:

...
* Move all blog posts to the new pages folder and new file format and update its Frontmatter

Can you add updating scripts/release-post.js to the list of things to be done. From a @nodejs/releasers POV, we want running that script (to generate the blog posts for releases) to create the post in the correct folder.

[ovflowd]

Oh @richardlau I'm going to inspect the whole scripts folder to remove any non-needed script and update all the needed ones. Don't worry about that 👀

But yea, going to update the PR description.

[tniessen]

Do we have any insights into how this will affect performance, file sizes, latency, etc.?

Do URLs to existing resources remain the same?

[ovflowd]

@tniessen this is only an infrastructure migration meaning:

• layouts and content should be the same
• all existing URLs should be the same
• deployment should also be the same without anything being changed on any config in the server side.

This means this PR is a foundational change for:

• A new framework behind the website
• Foundational milestone for migrating the layout, content and features from nodejs.dev over here
• Allowing all blazing-edge features from Next.js
• The way how the website is built, bundled and minified is different.

In terms of DX and development this means:

• Many things such as SEO, loading of components, linting, the set of features, and many other things are managed by Next.js
• Instead of pure HTML templates with "php" era of templating engine are now replaced by JSX/React
• Markdown is now standardised and supports MDX
• Easier to develop new features and styling as support of CSS modules, React, and et cetera

For content publishers:

• Creating Markdown pages should be the same experience
• Translating pages is the same experience
• Translating component translation strings now is done on a ICU-alike format

[ovflowd]

For performance and how long build takes time, this is something we can only benchmark once the whole thing is buildable. Right now it doesn't build yet, so there are a lot of things to be done :)

But of course I expect instead of taking less than one minute to at least 2-7 minutes for full cold-build time. (First time it gets built)

For CI we will be leveraging cache and other things but it's honestly too soon to talk about any of these.

[ovflowd]

Happy to announce this PR is finally ready to be reviewed 🎉

[AugustinMauroy]

Is it me or is the toggle dark mode not working?

[bnb]

@ovflowd is there a built demo version?

[ovflowd]

@ovflowd is there a built demo version?

Sadly there's not, I believe I could set-up a GitHub pages, but it would not "route" the .html files I guess? Any advise here?

[ovflowd]

Is it me or is the toggle dark mode not working?

@AugustinMauroy it seems to be working fine for me.

[mikeesto]

Props for the effort on this. FWIW, the theme and language pickers don't work for me on the news (/en/blog) page but work fine elsewhere. I also visually compared most of the other pages to the current live version and couldn't see any obvious issues. /de/blog/ and other locales show English for the blog/news page currently, but on this PR it's a 404 - not sure if that's intended

[targos]

Here's a build: https://nodejs-org-next-92b1511700.netlify.app/en/

[ovflowd]

Props for the effort on this. FWIW, the theme and language pickers don't work for me on the news (/en/blog) page but work fine elsewhere. I also visually compared most of the other pages to the current live version and couldn't see any obvious issues. /de/blog/ and other locales show English for the blog/news page currently, but on this PR it's a 404 - not sure if that's intended

Hey, @mikeesto I've just updated the main PR description explaining about these behaviors. But a TL;DR is: As this is a fully static generated website, the i18n redirection (pages that not exist within a certain language) is done via NGINX on our production infrastructure.

[ovflowd]

Hey y'all, I've just updated the main description of this PR containing more details regarding divergences within this implementation, immediate next steps, etc

[richardlau]

RSLGTM

[richardlau]

RSLGTM

[Trott]

Rubber-stamp LGTM