Implement Vitest testing environment for Miniflare 3 #4795

Adds a `unsafeUseModuleFallbackService` worker option and a `unsafeModuleFallbackService` shared option for using `workerd`'s module fallback service. Note `workerd` only accepts an address for fallback service configuration. This means if we wanted per-worker fallback service configuration, we'd need to start a new HTTP server for each worker that enabled it. To avoid starting additional servers, we reuse Miniflare's existing loopback server, and allow each worker to enable/disable the fallback service instead.

Previously, Miniflare would throw an error if the `Upgrade: websocket` header was set, but a non-WebSocket response was returned. This change ensures the response is returned, just without `webSocket` set. This allows the caller to handle the error case/response themselves.

Implements a basic Vitest pool that runs tests inside `workerd`, using the module fallback service and a test runner Durable Object. Provides access to bindings via `env` from the `cloudflare:test` module. Closes DEVX-1052 and DEVX-1053

Adds a `singleWorker` option for running all a project's test files serially in the same worker. This can speed up execution if the project contains lots of small test files, as Vitest only needs to be initialised once.

Adds a `runInDurableObject()` test helper for running arbitrary code inside a Durable Object's I/O context, and a `runDurableObjectAlarm()` helper for running a Durable Object's scheduled alarms. These require the new `main` option to be set to the worker's entrypoint, so the Vitest pool can create instances of the user's Durable Objects. The `main` option also allows service bindings to be declared to the worker running the tests. Events dispatched to these service bindings will use the handler exported by the `main` worker. Closes DEVX-1056, DEVX-1057 and DEVX-1058

Adds `fetchMock` to the `cloudflare:test` module. This is an instance of `undici`'s `MockAgent`, and can be used to mock global `fetch()` requests in the test worker. This works by bundling just the `undici` `MockAgent` code, with a customisable `Dispatcher` that is set to the original workers `fetch()` function in the pool. Closes DEVX-1059

A custom test runner allows us to hook into test execution. Specifically, this allows us to run code before and after files/suites/tests are executed. We'll need this for isolated storage.

By default, Vitest uses the `node:fs` module for reading/writing snapshots. This isn't available in workers, so this change implements a custom snapshot client using service bindings. Closes DEVX-1054

We'd like to use Zod for validating the pool's options. Miniflare has a `formatZodError` function that makes pretty error messages from Zod validation errors.

For isolated per-test storage, we'll be pushing/popping databases to a "stack". After pushing a database stack frame, we may delete a record in a test. We'd like to avoid deleting the associated blob here so when we pop the frame, the record still links up with it. This change adds an `unsafeStickyBlobs` option that disables garbage blob cleanup if enabled.

Isolated storage will need access to SQLite database files. This change adds a `unsafeGetPersistPaths()` function for getting the persistence paths used by a `Miniflare` instance. This is unsafe as the layout of persistence directories is not a public interface.

For isolated storage, we'd like the runner object not to persist any state to disk, so we don't have to worry about excluding it when managing the storage stack. We currently have an `unsafeEphemeralDurableObjects` option that makes _all_ Durable Objects in a worker use in-memory state. This isn't suitable for our Vitest pool, as user objects will run in the same worker as the runner object. This change adds support for `workerd` "colo-local" actors. These are like Durable Objects, but don't provide any durable storage (i.e. no `state.storage` API), and have a slightly different namespace binding API.

We'd like to extend our Vitest pool's custom loopback service with endpoints for managing isolated storage stacks. To do this, we need to know the persistence paths of the `Miniflare` instance we're currently running in. A previous commit added an `unsafeGetPersistPaths()` function. If we wanted to access these paths in the pool's loopback service, we'd need to capture the paths/`Miniflare` instance in a closure used as the service binding function. Unfortunately, we use `util.deepStrictEqual()` on Miniflare options to determine if we should restart the `Miniflare` instance. This would always return `true` if we created a new bound/arrow function on each test run. This change binds `this` to the current `Miniflare` instance when calling custom service functions, meaning we can reuse the same function when building options.

This change adds an `isolatedStorage` pool option. If enabled, each test gets its own isolated storage environment. Storage writes in a test are automatically undone at the end of the test. The storage environment is copied from the parent suite, ensuring writes from `beforeAll()`s show up in tests. This is implemented with an on-disk stack of `.sqlite` files. Before a suite/test attempt begins, the current storage state is "pushed". When the suite/test attempt completes/fails, the state is "popped", undoing all changes. Sticky blobs are used to ensure deleting records in a test keeps the corresponding blobs around, so they can be used when the state is "popped". Closes DEVX-1055

Previously, we were using `z.infer()` on our option schemas to derive `WorkerOptions` and `SharedOptions` types. This gave the _output_ types for schemas, rather than the user _input_ types. Usually, these were the same. However, if a schema was `.transform()`ed, they could be different. Notably, `hyperdrives` values output objects, but accept strings, leading to type errors if these were used.

Tries to remove unnecessary polyfills/functions, so users are less likely to import APIs that would be unsupported in production in their tests.

Tunnels through the error thrown when running tests fails. This is useful for debugging, as it means the startup error thrown by Vitest includes a useful message and stack.

While we expect most users to write their tests in TypeScript, JavaScript tests are also supported. Since we can't guarantee correct types at runtime, this change adds type validation to test helpers, throwing errors in the same JSG-style as other `workerd` APIs.

Previously, Miniflare would always resolve path-valued options relative to the current working directory. For the Vitest pool, we'd like to be able to resolve these relative to the project config file. This change adds back Miniflare 2's `rootPath` option for controlling the resolution directory. `rootPath` can be set as a shared or per-worker option. Path-valued options resolve relative to the closest `rootPath`.

...rather than the current working directory. This ensures paths are always resolved to the same location when `vitest` is run in different directories.

Vitest global setup hooks provide a good place to start auxiliary servers required by tests (e.g. upstream). This change allows provided values to be injected into the Miniflare config (e.g. using port from global setup in hyperdrive local connection string).

This change adds a `setupEnvironment(env)` function called with a bindings proxy per-`Miniflare`-instance per-test-run. This should be used for seeding data from disk (e.g. applying D1 migrations).

Enabling Durable Object JS RPC changes the class name of `DurableObjectStub` from `DurableObject` to `WorkerRpc`. This was breaking `Fetcher#fetch()` detection and WebSocket-upgrading `DurableObjectStub#fetch()`es when using the magic proxy.

This change adds some new test helpers useful when importing Worker source code and calling handlers directly. Namely... - `createExecutionContext()` for creating an `ExecutionContext` - `getWaitUntil()` for waiting for all `ctx.waitUntil()` callbacks - `createScheduledController()` for creating a `scheduled` handler's `ScheduledController` - `getScheduledResult()` for getting the "no-retry" state of a `ScheduledController` - `createMessageBatch()` for creating a `queue` handler's `MessageBatch` - `getQueueResult()` for getting ack/retry state of a `MessageBatch` Note `createScheduledController()`/`getScheduledResult()` and `createMessageBatch()`/`getQueueResult()` take/return the same types as `Fetcher#scheduled()` and `Fetcher#queue()` respectively. Closes DEVX-1060

Our Vitest pool runs tests inside `workerd` itself. This means we don't need to use Vitest's custom environment feature to setup the correct global scope. Modifying the global scope would result in some franken-`workerd`-environment that didn't match production. With Miniflare 2's `vitest-environment-miniflare`, we _did_ require users to enable a custom environment. Considering this is a significant difference, this change adds a very helpful error message if we detect a custom `environment` is configured.

Our Vitest pool currently depends on internal Vitest APIs that are not protected by semantic-versioning guarantees. Therefore, this change adds a runtime sanity check for a peer-dependency version constraint on `vitest`.

`WorkspaceProject#path` was actually giving the path to the directory containing the Vitest config, not the config file itself, for non-workspace projects.

`vite-node` was transforming the pool files during development, but externalising them when the pool was depended on externally. When Vite transforms files, it automatically injects `__dirname`, even though these are ES modules, which shouldn't have this by default.

These can be referenced by adding `@cloudflare/vitest-pool-workers` to the `types` array in `tsconfig.json`.

Vitest will dispose of the pool and recreate it when the Vitest config changes. This was causing `Miniflare` instances to be disposed, then reused when the pool was recreated. This change moves the project cache from the module-level to a "pool-level" local variable.

Our mock agent implementation depends on `undici` internals. This change ensures we pin our version, so we don't accidentally upgrade to an incompatible version.

…ices Switch to passing `Miniflare` instance as parameter instead of `this`

Rename abort-all helper worker

Update not-implemented messages

Adds a `listDurableObjectIds()` test helper that behaves similarly to `getMiniflareDurableObjectIds()` from Miniflare 2's test environments.

Importantly, this change makes the Zod output type of `hyperdrives` assignable to the input type. This means we can pass the output of parsing options schemas back to the `new Miniflare()` constructor. Also adds a few more `hyperdrives` tests to verify this doesn't change behaviour.

Exposing `kCurrentWorker` to users for integration testing wasn't very nice. This is a common use case so this change ensures it's always bound and adds it directly to the `cloudflare:test` module.

This function aims to allow users to wait on side-effecting `waitUntil`s, then assert on their effects. `workerd` ignores the resolved value of `Promise`s, so exposing these may make it trickier to change the implementation of this function in the future.

`durableObjectBindingDesignators` includes the same and more information, so there's no need to pass this.

This makes it explicit which flags we're enabling, and helps avoid behaviour differences when running `wrangler dev` or deployed code.

Clarify `StackedStorageState` is per `Miniflare`-instance

Update docs in `config.ts`

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement Vitest testing environment for Miniflare 3 #4795

Implement Vitest testing environment for Miniflare 3 #4795

Commits on Feb 16, 2024