support node 18 and 24 officially

williamstein · williamstein · commit 57e3184bd86c · 2025-05-11T15:19:53.000Z
- this required switching to node-fetch for a lot of tests
- a very small number of tests are disabled with node 18
- add to github CI
- script to automate testing locally on all platforms (if you have nvm
  setup)
diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
@@ -20,9 +20,6 @@ on:
         required: false
         default: false
 
-env:
-  GITHUB_ACTIONS: yes
-
 jobs:
   build:
     runs-on: ubuntu-latest
@@ -31,8 +28,10 @@ jobs:
       matrix:
         # See supported Node.js release schedule at https://nodejs.org/en/about/releases/
         node-version:
+          - "18"
           - "20"
           - "22"
+          - "24"
 
     steps:
       - uses: actions/checkout@v4
diff --git a/README.md b/README.md
@@ -13,7 +13,7 @@ can be done using nginx or haproxy.
 
 **PR's welcome!**
 
-May 10, 2025 STATUS compared to [http-proxy](https://www.npmjs.com/package/http-proxy) and [httpxy](https://www.npmjs.com/package/httpxy):
+May 11, 2025 STATUS compared to [http-proxy](https://www.npmjs.com/package/http-proxy) and [httpxy](https://www.npmjs.com/package/httpxy):
 
 - Library entirely rewritten in Typescript in a modern style, with many typings added internally.
 - All dependent packages updated to latest versions, addressing all security vulnerabilities according to `pnpm audit`.
@@ -34,6 +34,12 @@ I've been adding load tests to the unit tests in various places. Generally speak
 
 - https://github.com/unjs/httpxy: it has the same motivation as this project -- it's a modern maintained rewrite of http-proxy. Unfortunately, it seems to have [very little unit testing](https://github.com/unjs/httpxy/tree/main/test). In http-proxy-3 (and the original http-proxy), there's an order of magnitude more unit test code than code in the actual library.
 
+**Officially supported platforms:**
+
+We run GitHUB CI on the following:
+
+- nodejs versions 18, 20, 22, and 24
+
 **Development:**
 
 ```sh
@@ -52,8 +58,6 @@ pnpm tsc
 
 and make changes to code under lib/.
 
-Tested with nodejs versions 20 and 22.
-
 [![Build package and run tests](https://github.com/sagemathinc/http-proxy-3/actions/workflows/test.yml/badge.svg)](https://github.com/sagemathinc/http-proxy-3/actions/workflows/test.yml)
 
 ## User's Guide
@@ -601,7 +605,7 @@ pnpm test
 
 ### Contributing and Issues
 
-- Submit a PR! I want this project to be active again! Port ideas from [https://github.com/http\-party/node\-http\-proxy/pulls](https://github.com/http-party/node-http-proxy/pulls) and [https://github.com/http\-party/node\-http\-proxy/issues](https://github.com/http-party/node-http-proxy/issues).  Email me at [wstein@sagemath.com](mailto:wstein@sagemath.com). 
+- Submit a PR! I want this project to be active again! Port ideas from [https://github.com/http\-party/node\-http\-proxy/pulls](https://github.com/http-party/node-http-proxy/pulls) and [https://github.com/http\-party/node\-http\-proxy/issues](https://github.com/http-party/node-http-proxy/issues). Email me at [wstein@sagemath.com](mailto:wstein@sagemath.com).
 
 **[Back to top](#table-of-contents)**
 
diff --git a/package.json b/package.json
@@ -1,6 +1,6 @@
 {
   "name": "http-proxy-3",
-  "version": "1.20.0",
+  "version": "1.20.1",
   "repository": {
     "type": "git",
     "url": "https://github.com/sagemathinc/http-proxy-3.git"
@@ -46,14 +46,15 @@
   "scripts": {
     "preinstall": "npx only-allow pnpm",
     "test": "NODE_TLS_REJECT_UNAUTHORIZED=0 pnpm exec jest",
-    "version": "auto-changelog -p && git add CHANGELOG.md",
+    "test-all": "pnpm audit && TEST_EXTERNAL_REVERSE_PROXY=yes pnpm test --runInBand",
+    "test-versions": ". \"$NVM_DIR/nvm.sh\" && nvm use 18 && pnpm test && nvm use 20 && pnpm test && nvm use 22 && pnpm test && nvm use 24 && pnpm test && nvm use 20",
     "clean": "rm -rf dist node_modules",
     "build": "pnpm exec tsc --build .tsconfig-bootstrap.json && pnpm exec tsc --build",
     "tsc": "pnpm exec tsc --watch  --pretty --preserveWatchOutput",
-    "prepublishOnly": "pnpm audit && pnpm test --runInBand && rm -f dist/*.tsbuildinfo dist/.tsconfig-bootstrap* && rm -rf dist/test"
+    "prepublishOnly": "pnpm test-versions && pnpm test-all && rm -f dist/*.tsbuildinfo dist/.tsconfig-bootstrap* && rm -rf dist/test"
   },
   "engines": {
-    "node": ">=20.0.0"
+    "node": ">=18"
   },
   "license": "MIT"
 }
diff --git a/test/README.md b/test/README.md
@@ -1,3 +1,7 @@
+# Testing Nodes
+
+## Ports
+
 A note about ports. These tests use a large number of ports and are run in
 parallel. There will sometimes be failures sometimes due to a port conflict
 between different tests being run at the same time.
@@ -6,4 +10,41 @@ Use `pnpm test --runInBand` to avoid this by running the tests in serial. This
 is good for CI and release testing. Just use `pnpm test` for dev, since it's
 much faster.
 
-Another issue is that sometimes tests that proxy google.com will get rejected by Google due to anti-abuse by them. If you set the environment variable GITHUB_ACTIONS then those tests are not run.
+## Reverse Proxy test
+
+There are a few unit tests that involve a reverse proxy of google.com. These will often fail due to Google rejecting these robotic requests, so they are not run unless you
+set this environment variable:
+
+```sh
+TEST_EXTERNAL_REVERSE_PROXY=yes
+```
+
+I run these manually before making a release, but not as part of CI. Use 
+
+```sh
+pnpm test-all
+```
+to include these tests and also pnpm audit.
+
+## fetch
+
+We use
+
+```
+import fetch from "node-fetch";
+```
+
+in the unit tests instead of the fetch builtin to nodejs, since in node 18 the builtin fetch leaves an open handling hanging the test suite.
+
+An exception is blacklist-headers.test.ts, where we can't even run the test using node-fetch, since it blocks it.
+
+In particular, it is not acceptable for the test suite to exit with: _"A worker process has failed to exit gracefully and has been force exited. This is likely caused by tests leaking due to improper teardown. Try running with --detectOpenHandles to find leaks. Active timers can also cause this, ensure that .unref() was called on them."_
+
+## WARNINGS
+
+These warnings in the test suite are expected, because we're testing ssl using self signed certs:
+
+```
+(node:52812) Warning: Setting the NODE_TLS_REJECT_UNAUTHORIZED environment variable to '0' makes TLS connections and HTTPS requests insecure by disabling certificate verification.
+(Use `node --trace-warnings ...` to show where the warning was created)
+```
diff --git a/test/balancer/simple-balancer-with-websockets.test.ts b/test/balancer/simple-balancer-with-websockets.test.ts
@@ -8,6 +8,7 @@ import * as http from "http";
 import * as httpProxy from "../..";
 import getPort from "../get-port";
 import { once } from "../wait";
+import fetch from "node-fetch";
 
 describe("A simple round-robin load balancer that supports websockets", () => {
   let addresses;
diff --git a/test/balancer/simple-balancer.test.ts b/test/balancer/simple-balancer.test.ts
@@ -7,6 +7,7 @@ pnpm test simple-balancer.test.ts
 import * as http from "http";
 import * as httpProxy from "../..";
 import getPort from "../get-port";
+import fetch from "node-fetch";
 
 describe("A simple round-robin load balancing strategy.", () => {
   let addresses;
diff --git a/test/http/basic-proxy.test.ts b/test/http/basic-proxy.test.ts
@@ -10,6 +10,7 @@ import * as http from "http";
 import * as httpProxy from "../..";
 import log from "../log";
 import getPort from "../get-port";
+import fetch from "node-fetch";
 
 export async function server() {
   const httpPort = await getPort();
diff --git a/test/http/custom-proxy-error.test.ts b/test/http/custom-proxy-error.test.ts
@@ -6,6 +6,7 @@ pnpm test ./custom-proxy-error.test.ts
 
 import * as httpProxy from "../..";
 import getPort from "../get-port";
+import fetch from "node-fetch";
 
 const CUSTOM_ERROR =
   "Something went wrong. And we are reporting a custom error message.";
diff --git a/test/http/error-handling.test.ts b/test/http/error-handling.test.ts
@@ -7,6 +7,7 @@ import * as http from "http";
 import getPort from "../get-port";
 import log from "../log";
 import { callback } from "awaiting";
+import fetch from "node-fetch";
 
 const CUSTOM_ERROR = "There was an error proxying your request";
 
diff --git a/test/http/forward-and-target-proxy.test.ts b/test/http/forward-and-target-proxy.test.ts
@@ -12,6 +12,7 @@ import * as httpProxy from "../..";
 import log from "../log";
 import getPort from "../get-port";
 import wait from "../wait";
+import fetch from "node-fetch";
 
 describe("Example of proxying over HTTP with additional forward proxy to a different server", () => {
   let ports;
diff --git a/test/http/forward-proxy.test.ts b/test/http/forward-proxy.test.ts
@@ -14,6 +14,7 @@ import * as httpProxy from "../..";
 import log from "../log";
 import getPort from "../get-port";
 import wait from "../wait";
+import fetch from "node-fetch";
 
 describe("Example of proxying over HTTP with additional forward proxy", () => {
   let forwardingServer,
diff --git a/test/http/proxy-http-to-https.test.ts b/test/http/proxy-http-to-https.test.ts
@@ -7,6 +7,7 @@ We proxy https://google.com via a local non-https http server.
 import * as https from "https";
 import * as httpProxy from "../..";
 import getPort from "../get-port";
+import fetch from "node-fetch";
 
 describe(" Basic example of proxying over HTTP to a target HTTPS server", () => {
   let port, server;
@@ -24,7 +25,7 @@ describe(" Basic example of proxying over HTTP to a target HTTPS server", () =>
   });
 
   it("queries the proxy server as a test", async () => {
-    if (process.env.GITHUB_ACTIONS) {
+    if (!process.env.TEST_EXTERNAL_REVERSE_PROXY) {
       // google tends to block CI
       return;
     }
diff --git a/test/http/proxy-https-to-http.test.ts b/test/http/proxy-https-to-http.test.ts
@@ -7,6 +7,7 @@ import * as httpProxy from "../..";
 import getPort from "../get-port";
 import { join } from "path";
 import { readFile } from "fs/promises";
+import fetch from "node-fetch";
 
 const fixturesDir = join(__dirname, "..", "fixtures");
 
diff --git a/test/http/proxy-https-to-https.test.ts b/test/http/proxy-https-to-https.test.ts
@@ -8,6 +8,7 @@ import * as httpProxy from "../..";
 import getPort from "../get-port";
 import { join } from "path";
 import { readFile } from "fs/promises";
+import fetch from "node-fetch";
 
 const fixturesDir = join(__dirname, "..", "fixtures");
 
diff --git a/test/http/reverse-proxy.test.ts b/test/http/reverse-proxy.test.ts
@@ -8,7 +8,6 @@ import * as http from "http";
 import * as httpProxy from "../..";
 import getPort from "../get-port";
 import * as net from "net";
-import * as url from "url";
 import log from "../log";
 import { HttpsProxyAgent } from "https-proxy-agent";
 import fetch from "node-fetch";
@@ -25,8 +24,8 @@ describe("Reverse proxying -- create a server that...", () => {
     server = http
       .createServer((req, res) => {
         log("Receiving reverse proxy request for:", req.url);
-        const parsedUrl = url.parse(req.url ?? "");
-        const target = parsedUrl.protocol + "//" + parsedUrl.hostname;
+        const urlObj = new URL(req.url ?? "", "http://dummy.org");
+        const target = urlObj.origin;
         proxy.web(req, res, { target, secure: false });
       })
       .listen(port);
@@ -35,10 +34,9 @@ describe("Reverse proxying -- create a server that...", () => {
     server.on("connect", (req, socket) => {
       log("Receiving reverse proxy request for:", req.url);
 
-      const serverUrl = url.parse("https://" + req.url);
-
+      const serverUrl = new URL(`https://${req.url}`);
       const srvSocket = net.connect(
-        parseInt(serverUrl.port ?? "0"),
+        parseInt(serverUrl.port ? serverUrl.port : "443"),
         serverUrl.hostname!,
         () => {
           socket.write(
@@ -54,7 +52,7 @@ describe("Reverse proxying -- create a server that...", () => {
   });
 
   it("Tests the reverse proxy out to access https://www.google.com using an http proxy running on localhost.", async () => {
-    if (process.env.GITHUB_ACTIONS) {
+    if (!process.env.TEST_EXTERNAL_REVERSE_PROXY) {
       // google tends to block CI
       return;
     }
@@ -69,7 +67,7 @@ describe("Reverse proxying -- create a server that...", () => {
   });
 
   it("Tests the reverse proxy out to access http://www.google.com and https://www.google.com using an http proxy running on localhost.", async () => {
-    if (process.env.GITHUB_ACTIONS) {
+    if (!process.env.TEST_EXTERNAL_REVERSE_PROXY) {
       // google tends to block CI
       return;
     }
diff --git a/test/http/server-sent-events.test.ts b/test/http/server-sent-events.test.ts
@@ -10,6 +10,7 @@ import getPort from "../get-port";
 import { createSession } from "better-sse";
 import { EventSource } from "eventsource";
 import { callback } from "awaiting";
+import fetch from "node-fetch";
 
 describe("proxying server sent events over HTTP", () => {
   let ports;
@@ -42,18 +43,6 @@ describe("proxying server sent events over HTTP", () => {
     servers.http.listen(ports.http);
   });
 
-  it("test receiving an SSE WITHOUT using the proxy", async () => {
-    const f = (cb) => {
-      const sse = new EventSource(`http://localhost:${ports.http}/sse`);
-      sse.addEventListener("message", ({ data }) => {
-        sse.close();
-        cb(undefined, JSON.parse(data));
-      });
-    };
-    const resp = await callback(f);
-    expect(resp).toEqual("Hello world! - 1");
-  });
-
   it("creates the proxy server", () => {
     servers.proxy = httpProxy
       .createProxyServer({
@@ -67,17 +56,33 @@ describe("proxying server sent events over HTTP", () => {
     expect(a).toContain("request successfully proxied");
   });
 
-  it("test receiving an SSE USING the proxy", async () => {
-    const f = (cb) => {
-      const sse = new EventSource(`http://localhost:${ports.proxy}/sse`);
-      sse.addEventListener("message", ({ data }) => {
-        sse.close();
-        cb(undefined, JSON.parse(data));
-      });
-    };
-    const resp = await callback(f);
-    expect(resp).toEqual("Hello world! - 2");
-  });
+  if (!process.version.startsWith("v18.")) {
+    // These two tests leave open handles on node v18, so we disable them ONLY
+    // with node v18. 
+    it("test receiving an SSE WITHOUT using the proxy", async () => {
+      const f = (cb) => {
+        const sse = new EventSource(`http://localhost:${ports.http}/sse`);
+        sse.addEventListener("message", ({ data }) => {
+          sse.close();
+          cb(undefined, JSON.parse(data));
+        });
+      };
+      const resp = await callback(f);
+      expect(resp).toEqual("Hello world! - 1");
+    });
+
+    it("test receiving an SSE USING the proxy", async () => {
+      const f = (cb) => {
+        const sse = new EventSource(`http://localhost:${ports.proxy}/sse`);
+        sse.addEventListener("message", ({ data }) => {
+          sse.close();
+          cb(undefined, JSON.parse(data));
+        });
+      };
+      const resp = await callback(f);
+      expect(resp).toEqual("Hello world! - 2");
+    });
+  }
 
   it("Clean up", () => {
     Object.values(servers).map((x: any) => x?.close());
diff --git a/test/lib/http-proxy-common.test.ts b/test/lib/http-proxy-common.test.ts
diff --git a/test/lib/https-proxy.test.ts b/test/lib/https-proxy.test.ts
diff --git a/test/middleware/body-decoder-middleware.test.ts b/test/middleware/body-decoder-middleware.test.ts
diff --git a/test/middleware/gzip-middleware.test.ts b/test/middleware/gzip-middleware.test.ts
diff --git a/test/middleware/modify-response-middleware.test.ts b/test/middleware/modify-response-middleware.test.ts
diff --git a/test/security/blacklist-headers.test.ts b/test/security/blacklist-headers.test.ts
diff --git a/test/setup.js b/test/setup.js
