Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

worker: initial implementation #20876

Closed
wants to merge 26 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
f0ded20
src: cleanup per-isolate state on platform on isolate unregister
addaleax May 17, 2018
2bb055b
src: remove unused fields isolate_
danbev May 23, 2018
e7695cc
src: simplify handle closing
addaleax Sep 10, 2017
6d4931a
src: make handle onclose property a Symbol
addaleax Jun 2, 2018
315efb5
worker: implement `MessagePort` and `MessageChannel`
addaleax Sep 5, 2017
f2e297b
fixup! worker: implement `MessagePort` and `MessageChannel`
addaleax Jun 2, 2018
ef24c5c
worker: support MessagePort passing in messages
addaleax Oct 7, 2017
a57c54f
worker: add `SharedArrayBuffer` sharing
addaleax May 13, 2018
ca0edc1
src: remove unused fields msg_ and env_
danbev May 23, 2018
d2d9061
src: add Env::profiler_idle_notifier_started()
TimothyGu Sep 25, 2017
0c7012e
worker: initial implementation
addaleax Sep 1, 2017
396d785
fixup! worker: initial implementation
addaleax Jun 3, 2018
65d4542
fixup! worker: initial implementation
addaleax Jun 4, 2018
d0535eb
fixup! worker: initial implementation
addaleax Jun 5, 2018
17e01a2
test: add test against unsupported worker features
TimothyGu Sep 25, 2017
53b660b
worker: restrict supported extensions
TimothyGu Sep 20, 2017
004075b
worker: enable stdio
addaleax May 13, 2018
d44594d
benchmark: port cluster/echo to worker
TimothyGu Sep 24, 2017
fa53bd5
worker: improve error (de)serialization
addaleax Sep 25, 2017
79970f3
test,tools: enable running tests under workers
addaleax May 17, 2018
168d8a6
doc: explain Worker semantics in async_hooks.md
addaleax May 22, 2018
a12f76f
worker: rename to worker_threads
addaleax Jun 1, 2018
ab02dbc
fixup! worker: rename to worker_threads
addaleax Jun 3, 2018
2947dea
fixup! worker: rename to worker_threads
addaleax Jun 3, 2018
735227e
fixup! worker: rename to worker_threads
addaleax Jun 5, 2018
cbbddea
fixup! worker: rename to worker_threads
addaleax Jun 5, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions benchmark/fixtures/echo.worker.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
'use strict';

const { parentPort } = require('worker_threads');

parentPort.on('message', (msg) => {
parentPort.postMessage(msg);
});
73 changes: 73 additions & 0 deletions benchmark/worker/echo.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
'use strict';

const common = require('../common.js');
const path = require('path');
const bench = common.createBenchmark(main, {
workers: [1],
payload: ['string', 'object'],
sendsPerBroadcast: [1, 10],
n: [1e5]
}, { flags: ['--experimental-worker'] });

const workerPath = path.resolve(__dirname, '..', 'fixtures', 'echo.worker.js');

function main(conf) {
const { Worker } = require('worker_threads');

const n = +conf.n;
const workers = +conf.workers;
const sends = +conf.sendsPerBroadcast;
const expectedPerBroadcast = sends * workers;
var payload;
var readies = 0;
var broadcasts = 0;
var msgCount = 0;

switch (conf.payload) {
case 'string':
payload = 'hello world!';
break;
case 'object':
payload = { action: 'pewpewpew', powerLevel: 9001 };
break;
default:
throw new Error('Unsupported payload type');
}

const workerObjs = [];

for (var i = 0; i < workers; ++i) {
const worker = new Worker(workerPath);
workerObjs.push(worker);
worker.on('online', onOnline);
worker.on('message', onMessage);
}

function onOnline() {
if (++readies === workers) {
bench.start();
broadcast();
}
}

function broadcast() {
if (broadcasts++ === n) {
bench.end(n);
for (const worker of workerObjs) {
worker.unref();
}
return;
}
for (const worker of workerObjs) {
for (var i = 0; i < sends; ++i)
worker.postMessage(payload);
}
}

function onMessage() {
if (++msgCount === expectedPerBroadcast) {
msgCount = 0;
broadcast();
}
}
}
1 change: 1 addition & 0 deletions doc/api/_toc.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@
* [Utilities](util.html)
* [V8](v8.html)
* [VM](vm.html)
* [Worker Threads](worker_threads.html)
* [ZLIB](zlib.html)

<div class="line"></div>
Expand Down
1 change: 1 addition & 0 deletions doc/api/all.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,4 +46,5 @@
@include util
@include v8
@include vm
@include worker_threads
@include zlib
6 changes: 5 additions & 1 deletion doc/api/async_hooks.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,9 @@ A resource can also be closed before the callback is called. `AsyncHook` does
not explicitly distinguish between these different cases but will represent them
as the abstract concept that is a resource.

If [`Worker`][]s are used, each thread has an independent `async_hooks`

This comment was marked as resolved.

interface, and each thread will use a new set of async IDs.

## Public API

### Overview
Expand Down Expand Up @@ -224,7 +227,7 @@ clearTimeout(setTimeout(() => {}, 10));
```

Every new resource is assigned an ID that is unique within the scope of the
current process.
current Node.js instance.

###### `type`

Expand Down Expand Up @@ -733,3 +736,4 @@ never be called.
[Hook Callbacks]: #async_hooks_hook_callbacks
[PromiseHooks]: https://docs.google.com/document/d/1rda3yKGHimKIhg5YeoAmCOtyURgsbTH_qaYR79FELlk
[promise execution tracking]: #async_hooks_promise_execution_tracking
[`Worker`]: worker_threads.html#worker_threads_class_worker
51 changes: 51 additions & 0 deletions doc/api/errors.md
Original file line number Diff line number Diff line change
Expand Up @@ -629,6 +629,12 @@ An operation outside the bounds of a `Buffer` was attempted.
An attempt has been made to create a `Buffer` larger than the maximum allowed
size.

<a id="ERR_CANNOT_TRANSFER_OBJECT"></a>
### ERR_CANNOT_TRANSFER_OBJECT

The value passed to `postMessage()` contained an object that is not supported
for transferring.

<a id="ERR_CANNOT_WATCH_SIGINT"></a>
### ERR_CANNOT_WATCH_SIGINT

Expand All @@ -650,12 +656,23 @@ Used when a child process is being forked without specifying an IPC channel.
Used when the main process is trying to read data from the child process's
STDERR / STDOUT, and the data's length is longer than the `maxBuffer` option.

<a id="ERR_CLOSED_MESSAGE_PORT"></a>
### ERR_CLOSED_MESSAGE_PORT

There was an attempt to use a `MessagePort` instance in a closed
state, usually after `.close()` has been called.

<a id="ERR_CONSOLE_WRITABLE_STREAM"></a>
### ERR_CONSOLE_WRITABLE_STREAM

`Console` was instantiated without `stdout` stream, or `Console` has a
non-writable `stdout` or `stderr` stream.

<a id="ERR_CONSTRUCT_CALL_REQUIRED"></a>
### ERR_CONSTRUCT_CALL_REQUIRED

A constructor for a class was called without `new`.

<a id="ERR_CPU_USAGE"></a>
### ERR_CPU_USAGE

Expand Down Expand Up @@ -1203,6 +1220,11 @@ urlSearchParams.has.call(buf, 'foo');
// Throws a TypeError with code 'ERR_INVALID_THIS'
```

<a id="ERR_INVALID_TRANSFER_OBJECT"></a>
### ERR_INVALID_TRANSFER_OBJECT

An invalid transfer object was passed to `postMessage()`.

<a id="ERR_INVALID_TUPLE"></a>
### ERR_INVALID_TUPLE

Expand Down Expand Up @@ -1278,13 +1300,26 @@ strict compliance with the API specification (which in some cases may accept
`func(undefined)` and `func()` are treated identically, and the
[`ERR_INVALID_ARG_TYPE`][] error code may be used instead.

<a id="ERR_MISSING_MESSAGE_PORT_IN_TRANSFER_LIST"></a>
### ERR_MISSING_MESSAGE_PORT_IN_TRANSFER_LIST

A `MessagePort` was found in the object passed to a `postMessage()` call,
but not provided in the `transferList` for that call.

<a id="ERR_MISSING_MODULE"></a>
### ERR_MISSING_MODULE

> Stability: 1 - Experimental

An [ES6 module][] could not be resolved.

<a id="ERR_MISSING_PLATFORM_FOR_WORKER"></a>
### ERR_MISSING_PLATFORM_FOR_WORKER

The V8 platform used by this instance of Node.js does not support creating
Workers. This is caused by lack of embedder support for Workers. In particular,
this error will not occur with standard builds of Node.js.

<a id="ERR_MODULE_RESOLUTION_LEGACY"></a>
### ERR_MODULE_RESOLUTION_LEGACY

Expand Down Expand Up @@ -1694,6 +1729,22 @@ The fulfilled value of a linking promise is not a `vm.Module` object.
The current module's status does not allow for this operation. The specific
meaning of the error depends on the specific function.

<a id="ERR_WORKER_NEED_ABSOLUTE_PATH"></a>
### ERR_WORKER_NEED_ABSOLUTE_PATH

The path for the main script of a worker is not an absolute path.

<a id="ERR_WORKER_UNSERIALIZABLE_ERROR"></a>
### ERR_WORKER_UNSERIALIZABLE_ERROR

All attempts at serializing an uncaught exception from a worker thread failed.

<a id="ERR_WORKER_UNSUPPORTED_EXTENSION"></a>
### ERR_WORKER_UNSUPPORTED_EXTENSION

The pathname used for the main script of a worker has an
unknown file extension.

<a id="ERR_ZLIB_INITIALIZATION_FAILED"></a>
### ERR_ZLIB_INITIALIZATION_FAILED

Expand Down
27 changes: 27 additions & 0 deletions doc/api/process.md
Original file line number Diff line number Diff line change
Expand Up @@ -410,6 +410,8 @@ added: v0.7.0
The `process.abort()` method causes the Node.js process to exit immediately and
generate a core file.

This feature is not available in [`Worker`][] threads.

## process.arch
<!-- YAML
added: v0.5.0
Expand Down Expand Up @@ -517,6 +519,8 @@ try {
}
```

This feature is not available in [`Worker`][] threads.

## process.config
<!-- YAML
added: v0.7.7
Expand Down Expand Up @@ -918,6 +922,8 @@ console.log(process.env.test);
// => 1
```

`process.env` is read-only in [`Worker`][] threads.

## process.execArgv
<!-- YAML
added: v0.7.7
Expand Down Expand Up @@ -1030,6 +1036,9 @@ If it is necessary to terminate the Node.js process due to an error condition,
throwing an *uncaught* error and allowing the process to terminate accordingly
is safer than calling `process.exit()`.

In [`Worker`][] threads, this function stops the current thread rather
than the current process.

## process.exitCode
<!-- YAML
added: v0.11.8
Expand Down Expand Up @@ -1203,6 +1212,7 @@ console.log(process.getgroups()); // [ 27, 30, 46, 1000 ]

This function is only available on POSIX platforms (i.e. not Windows or
Android).
This feature is not available in [`Worker`][] threads.

## process.kill(pid[, signal])
<!-- YAML
Expand Down Expand Up @@ -1306,6 +1316,9 @@ The _heap_ is where objects, strings, and closures are stored. Variables are
stored in the _stack_ and the actual JavaScript code resides in the
_code segment_.

When using [`Worker`][] threads, `rss` will be a value that is valid for the
entire process, while the other fields will only refer to the current thread.

## process.nextTick(callback[, ...args])
<!-- YAML
added: v0.1.26
Expand Down Expand Up @@ -1569,6 +1582,7 @@ if (process.getegid && process.setegid) {

This function is only available on POSIX platforms (i.e. not Windows or
Android).
This feature is not available in [`Worker`][] threads.

## process.seteuid(id)
<!-- YAML
Expand Down Expand Up @@ -1596,6 +1610,7 @@ if (process.geteuid && process.seteuid) {

This function is only available on POSIX platforms (i.e. not Windows or
Android).
This feature is not available in [`Worker`][] threads.

## process.setgid(id)
<!-- YAML
Expand Down Expand Up @@ -1623,6 +1638,7 @@ if (process.getgid && process.setgid) {

This function is only available on POSIX platforms (i.e. not Windows or
Android).
This feature is not available in [`Worker`][] threads.

## process.setgroups(groups)
<!-- YAML
Expand All @@ -1639,6 +1655,7 @@ The `groups` array can contain numeric group IDs, group names or both.

This function is only available on POSIX platforms (i.e. not Windows or
Android).
This feature is not available in [`Worker`][] threads.

## process.setuid(id)
<!-- YAML
Expand All @@ -1664,6 +1681,7 @@ if (process.getuid && process.setuid) {

This function is only available on POSIX platforms (i.e. not Windows or
Android).
This feature is not available in [`Worker`][] threads.

This comment was marked as resolved.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this was intended by me, because the sentence before also explains when this feature is not available → it made sense to me to have them be a single paragraph

This comment was marked as resolved.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vsemozhetbyt Yup, done! (Feel free to push documentation change commits to this branch, btw.)


## process.setUncaughtExceptionCaptureCallback(fn)
<!-- YAML
Expand Down Expand Up @@ -1700,6 +1718,8 @@ a [Writable][] stream.
`process.stderr` differs from other Node.js streams in important ways, see
[note on process I/O][] for more information.

This feature is not available in [`Worker`][] threads.

## process.stdin

* {Stream}
Expand Down Expand Up @@ -1732,6 +1752,8 @@ In "old" streams mode the `stdin` stream is paused by default, so one
must call `process.stdin.resume()` to read from it. Note also that calling
`process.stdin.resume()` itself would switch stream to "old" mode.

This feature is not available in [`Worker`][] threads.

## process.stdout

* {Stream}
Expand All @@ -1750,6 +1772,8 @@ process.stdin.pipe(process.stdout);
`process.stdout` differs from other Node.js streams in important ways, see
[note on process I/O][] for more information.

This feature is not available in [`Worker`][] threads.

### A note on process I/O

`process.stdout` and `process.stderr` differ from other Node.js streams in
Expand Down Expand Up @@ -1865,6 +1889,8 @@ console.log(
);
```

This feature is not available in [`Worker`][] threads.

## process.uptime()
<!-- YAML
added: v0.5.0
Expand Down Expand Up @@ -1992,6 +2018,7 @@ cases:
[`ChildProcess`]: child_process.html#child_process_class_childprocess
[`Error`]: errors.html#errors_class_error
[`EventEmitter`]: events.html#events_class_eventemitter
[`Worker`]: worker_threads.html#worker_threads_class_worker
[`console.error()`]: console.html#console_console_error_data_args
[`console.log()`]: console.html#console_console_log_data_args
[`domain`]: domain.html
Expand Down
2 changes: 1 addition & 1 deletion doc/api/vm.md
Original file line number Diff line number Diff line change
Expand Up @@ -174,7 +174,7 @@ const contextifiedSandbox = vm.createContext({ secret: 42 });

Creates a new ES `Module` object.

*Note*: Properties assigned to the `import.meta` object that are objects may
Properties assigned to the `import.meta` object that are objects may
allow the `Module` to access information outside the specified `context`, if the
object is created in the top level context. Use `vm.runInContext()` to create
objects in a specific context.
Expand Down
Loading