stream: improve Readable.read() performance #7077

mscdex · 2016-05-31T17:17:40Z

Checklist

tests and code linting passes
a test and/or benchmark is included
the commit message follows commit guidelines

Affected core subsystem(s)

stream

Description of change

read() performance is improved most by switching from an array to a linked list for storing buffered data. However, other changes that also contribute include: making some hot functions inlinable, faster
read() argument checking, and misc code rearrangement to avoid unnecessary code execution.

These improvements only exist for when n is supplied to read(), otherwise performance is the same.

Note: I did change how negative n values are interpreted when in object mode. Before this PR, negative n values were interpreted as 1 in howMuchToRead(). I changed this to instead return 0 to match how negative n values are interpreted for non-object mode streams.

Results from the included benchmarks:

streams/readable-bigread.js n=1000: ./node: 306.98 ./node-master: 171.91 ....... 78.57%
streams/readable-bigunevenread.js n=1000: ./node: 213.85 ./node-master: 125.26 . 70.72%
streams/readable-boundaryread.js n=2000: ./node: 448.09 ./node-master: 298.88 .. 49.92%
streams/readable-unevenread.js n=1000: ./node: 49.565 ./node-master: 34.442 .... 43.91%
streams/readable-readall.js n=5000: ./node: 772.56 ./node-master: 766.4 ......... 0.80%

mscdex · 2016-05-31T17:18:56Z

/cc @nodejs/streams

mscdex · 2016-05-31T17:19:21Z

CI: https://ci.nodejs.org/job/node-test-pull-request/2877/

calvinmetcalf · 2016-05-31T17:25:58Z

should buffer list maybe live in the internals folder ?

mscdex · 2016-05-31T17:28:09Z

@calvinmetcalf Debatable but I left it there since it's only used by Readable and it's better if Readable isn't referencing anything from internal/ since streams are extracted from node?

calvinmetcalf · 2016-05-31T17:29:49Z

I can extract that other file easily, I'm more worried about exporting that from readable which is not on the api

mscdex · 2016-05-31T17:32:16Z

@calvinmetcalf I don't understand what you mean by "is not on the api."

calvinmetcalf · 2016-05-31T17:33:44Z

sorry, misspoke, I meant that I didn't like exporting that new thing from readable

mscdex · 2016-05-31T17:36:22Z

@calvinmetcalf I can remove it if necessary I suppose, but then we'd end up duplicating some of it in the tests I had to modify. I thought it would be better not to have to keep two separate "implementations" in sync.

calvinmetcalf · 2016-05-31T17:57:12Z

I was thinking that if that file got moved to something in internal, then you could just require that for the tests and require it for from stream_readable and don't worry about the readable-stream module, it won't be much of an inconvenience.

mscdex · 2016-05-31T23:52:53Z

@calvinmetcalf Ok, moved to internal.

ronkorving · 2016-06-01T08:10:39Z

lib/internal/streams/BufferList.js

+}
+
+BufferList.prototype.push = function(v) {
+  const entry = { data: v, next: null };


Do you think it could help to create an Entry class to increase property access performance?

I just tried it and it doesn't seem to make any real difference. If anything it seems to slow things down a little bit.

mscdex · 2016-06-07T03:00:02Z

/cc @nodejs/collaborators Anyone have any comments/questions/LGTMs on this?

ronkorving · 2016-06-07T03:12:51Z

lib/_stream_readable.js

-        else
-          buf.copy(ret, c, 0, cpy);
+    // result spans more than one buffer
+    ret = (hasStrings


I'm not sure on the style guide on this, but the outer braces seem useless.

You mean the parens? shrug I always add that when "assigning" a ternary statement to a variable as it seems more readable/clear to me.

I noticed :) I don't really mind, just curious if there's any consensus on this. No blocker, no worries.

And sorry, yeah, parens (I, non-native English speaker, always get these mixed up... sigh)

jasnell · 2016-06-07T03:36:03Z

benchmark/streams/readable-bigread.js

+
+var common = require('../common');
+var v8 = require('v8');
+var Readable = require('stream').Readable;


jasnell · 2016-06-07T03:49:52Z

First read through looks good but I'll have to go through in more detail tomorrow. In general tho, if @nodejs/streams is happy, then ++

mcollina · 2016-06-07T05:30:46Z

Do we have a CITGM run?

If that passes.. LGTM. just with one catch.. Let's give this a long time before we backport.

mscdex · 2016-06-07T06:38:09Z

@mcollina citgm: https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker/296/

mcollina · 2016-06-07T14:24:34Z

the job was aborted, should we kick it off again?

mscdex · 2016-06-07T14:42:33Z

@mcollina It was aborted most likely because the ppcle machine is/was still down. The lodash failure on OSX1010 looks like an npm or citgm failure since it failed on installation of the package.

Otherwise citgm is green.

mcollina · 2016-06-07T14:54:44Z

LGTM as semver-minor.

read() performance is improved most by switching from an array to a linked list for storing buffered data. However, other changes that also contribute include: making some hot functions inlinable, faster read() argument checking, and misc code rearrangement to avoid unnecessary code execution. PR-URL: #7077 Reviewed-By: Calvin Metcalf <calvin.metcalf@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

Fishrock123 · 2016-07-06T14:24:40Z

@mcollina Why is this semver-minor?

Notable changes: * buffer: Added `buffer.swap64()` to compliment `swap16()` & `swap32()`. (Zach Bjornson) #7157 * build: New `configure` options have been added for building Node.js as a shared library. (Stefan Budeanu) #6994 - The options are: `--shared`, `--without-v8-platform` & `--without-bundled-v8`. * crypto: Root certificates have been updated. (Ben Noordhuis) #7363 * debugger: The server address is now configurable via `--debug=<address>:<port>`. (Ben Noordhuis) #3316 * npm: Upgraded npm to v3.10.3 (Kat Marchán) #7515 & (Rebecca Turner) #7410 * readline: Added the `prompt` option to the readline constructor. (Evan Lucas) #7125 * repl / vm: `sigint`/`ctrl+c` will now break out of infinite loops without stopping the Node.js instance. (Anna Henningsen) #6635 * src: - Added a `node::FreeEnvironment` public C++ API. (Cheng Zhao) #3098 - Refactored `require('constants')`, constants are now available directly from their respective modules. (James M Snell) #6534 * stream: Improved `readable.read()` performance by up to 70%. (Brian White) #7077 * timers: `setImmediate()` is now up to 150% faster in some situations. (Andras) #6436 * util: Added a `breakLength` option to `util.inspect()` to control how objects are formatted across lines. (cjihrig) #7499 * v8-inspector: Experimental support has been added for debugging Node.js over the inspector protocol. (Ali Ijaz Sheikh) #6792 - *Note: This feature is experimental, and it could be altered or removed.* - You can try this feature by running Node.js with the `--inspect` flag. Refs: #7441 PR-URL: #7550

mcollina · 2016-07-06T16:41:29Z

@Fishrock123 avoiding possible breakages in a patch release. IMHO performance optimizations are semver-minor, as performance is a backward-compatible feature.

Fishrock123 · 2016-07-06T16:47:05Z

@mcollina So you are saying this is semver-major then? This is at obvious risk of breaking something? Do I need to back it out of 6.3.0?

How this works is, in this order:

Does it have a chance of breaking something? -> Major
Does it add something that won't work on previous versions (feature/api)? -> Minor
Everything else: -> Patch

Notable changes: * buffer: Added `buffer.swap64()` to compliment `swap16()` & `swap32()`. (Zach Bjornson) #7157 * build: New `configure` options have been added for building Node.js as a shared library. (Stefan Budeanu) #6994 - The options are: `--shared`, `--without-v8-platform` & `--without-bundled-v8`. * crypto: Root certificates have been updated. (Ben Noordhuis) #7363 * debugger: The server address is now configurable via `--debug=<address>:<port>`. (Ben Noordhuis) #3316 * npm: Upgraded npm to v3.10.3 (Kat Marchán) #7515 & (Rebecca Turner) #7410 * readline: Added the `prompt` option to the readline constructor. (Evan Lucas) #7125 * repl / vm: `sigint`/`ctrl+c` will now break out of infinite loops without stopping the Node.js instance. (Anna Henningsen) #6635 * src: - Added a `node::FreeEnvironment` public C++ API. (Cheng Zhao) #3098 - Refactored `require('constants')`, constants are now available directly from their respective modules. (James M Snell) #6534 * stream: Improved `readable.read()` performance by up to 70%. (Brian White) #7077 * timers: `setImmediate()` is now up to 150% faster in some situations. (Andras) #6436 * util: Added a `breakLength` option to `util.inspect()` to control how objects are formatted across lines. (cjihrig) #7499 * v8-inspector: Experimental support has been added for debugging Node.js over the inspector protocol. (Ali Ijaz Sheikh) #6792 - *Note: This feature is experimental, and it could be altered or removed.* - You can try this feature by running Node.js with the `--inspect` flag. Refs: #7441 PR-URL: #7550

mcollina · 2016-07-06T17:07:21Z

@Fishrock123

So you are saying this is semver-major then? This is at obvious risk of breaking something? Do I need to back it out of 6.3.0?

no and no.
We are touching streams, and based on latest history, there is a slight higher risk of breaking something compared to other part of core. Because of this, I would put them into semver-minor anything that is not strictly a bug fix.

This is a performance optimization, e.g. a new feature for me.

Fishrock123 · 2016-07-06T17:09:56Z

So... don't land on LTS is what I'm getting out of this. :)

mcollina · 2016-07-06T17:13:24Z

@Fishrock123 I think this one will be fine on LTS, but I would recommend porting it after some months, not weeks. Something like "land when the new LTS is out".

Notable changes: * buffer: Added `buffer.swap64()` to compliment `swap16()` & `swap32()`. (Zach Bjornson) #7157 * build: New `configure` options have been added for building Node.js as a shared library. (Stefan Budeanu) #6994 - The options are: `--shared`, `--without-v8-platform` & `--without-bundled-v8`. * crypto: Root certificates have been updated. (Ben Noordhuis) #7363 * debugger: The server address is now configurable via `--debug=<address>:<port>`. (Ben Noordhuis) #3316 * npm: Upgraded npm to v3.10.3 (Kat Marchán) #7515 & (Rebecca Turner) #7410 * readline: Added the `prompt` option to the readline constructor. (Evan Lucas) #7125 * repl / vm: `sigint`/`ctrl+c` will now break out of infinite loops without stopping the Node.js instance. (Anna Henningsen) #6635 * src: - Added a `node::FreeEnvironment` public C++ API. (Cheng Zhao) #3098 - Refactored `require('constants')`, constants are now available directly from their respective modules. (James M Snell) #6534 * stream: Improved `readable.read()` performance by up to 70%. (Brian White) #7077 * timers: `setImmediate()` is now up to 150% faster in some situations. (Andras) #6436 * util: Added a `breakLength` option to `util.inspect()` to control how objects are formatted across lines. (cjihrig) #7499 * v8-inspector: Experimental support has been added for debugging Node.js over the inspector protocol. (Ali Ijaz Sheikh) #6792 - *Note: This feature is experimental, and it could be altered or removed.* - You can try this feature by running Node.js with the `--inspect` flag. Refs: #7441 PR-URL: #7550

### Notable changes * **buffer**: Added `buffer.swap64()` to compliment `swap16()` & `swap32()`. (Zach Bjornson) [#7157](nodejs/node#7157) * **build**: New `configure` options have been added for building Node.js as a shared library. (Stefan Budeanu) [#6994](nodejs/node#6994) - The options are: `--shared`, `--without-v8-platform` & `--without-bundled-v8`. * **crypto**: Root certificates have been updated. (Ben Noordhuis) [#7363](nodejs/node#7363) * **debugger**: The server address is now configurable via `--debug=<address>:<port>`. (Ben Noordhuis) [#3316](nodejs/node#3316) * **npm**: Upgraded npm to v3.10.3 (Kat Marchán) [#7515](nodejs/node#7515) & (Rebecca Turner) [#7410](nodejs/node#7410) * **readline**: Added the `prompt` option to the readline constructor. (Evan Lucas) [#7125](nodejs/node#7125) * **repl / vm**: `sigint`/`ctrl+c` will now break out of infinite loops without stopping the Node.js instance. (Anna Henningsen) [#6635](nodejs/node#6635) * **src**: - Added a `node::FreeEnvironment` public C++ API. (Cheng Zhao) [#3098](nodejs/node#3098) - Refactored `require('constants')`, constants are now available directly from their respective modules. (James M Snell) [#6534](nodejs/node#6534) * **stream**: Improved `readable.read()` performance by up to 70%. (Brian White) [#7077](nodejs/node#7077) * **timers**: `setImmediate()` is now up to 150% faster in some situations. (Andras) [#6436](nodejs/node#6436) * **util**: Added a `breakLength` option to `util.inspect()` to control how objects are formatted across lines. (cjihrig) [#7499](nodejs/node#7499) * **v8-inspector**: Experimental support has been added for debugging Node.js over the inspector protocol. (Ali Ijaz Sheikh) [#6792](nodejs/node#6792) - **Note: This feature is _experimental_, and it could be altered or removed.** - You can try this feature by running Node.js with the `--inspect` flag.

mafintosh · 2016-07-08T16:14:20Z

@Fishrock123 @mcollina just re your above conversation, this ended up breaking duplexify (https://www.npmjs.com/package/duplexify) because it expected the _buffer to be an array

mcollina · 2016-07-08T16:21:43Z

I feared something like that would have happened.

I think we should make a final decision either to semver _readableState and _writableState API or not.
Formally, they are not part of the public API, and these are causing all sort of issues on streams modules.

I'm happy we got this early (before a backport to readable-stream).

ronkorving · 2016-07-08T23:26:14Z

They're clearly private. If people want to manipulate them, we can expose public API for that. The duplexify package authors could've pushed/asked for that, and still can imho.

mafintosh · 2016-07-08T23:37:57Z

@mcollina could you open a separate issue for the semver discussion so we don't take over this one? for the record i think reading some of the properties in _readableState is essential for a bunch of use-cases. like figuring out if a stream is in objectMode, figuring out if it ended but still has data in its read buffer etc.

mcollina · 2016-07-09T00:52:27Z

@mafintosh done ^

mscdex added stream Issues and PRs related to the stream subsystem. performance Issues and PRs related to the performance of Node.js. labels May 31, 2016

mscdex force-pushed the streams-readable-faster-read branch from e90c652 to 1ae67c9 Compare May 31, 2016 23:52

mscdex force-pushed the streams-readable-faster-read branch from 1ae67c9 to 47d6cc0 Compare May 31, 2016 23:54

ronkorving reviewed Jun 1, 2016
View reviewed changes

ronkorving reviewed Jun 7, 2016
View reviewed changes

mscdex force-pushed the streams-readable-faster-read branch 3 times, most recently from f1922f1 to 1f2fbc2 Compare June 7, 2016 03:33

jasnell reviewed Jun 7, 2016
View reviewed changes

mscdex force-pushed the streams-readable-faster-read branch from 1f2fbc2 to 729b645 Compare June 7, 2016 03:46

mscdex deleted the streams-readable-faster-read branch June 14, 2016 20:00

Fishrock123 mentioned this pull request Jun 27, 2016

Propose v6.3.0 #7441

Merged

Fishrock123 mentioned this pull request Jul 5, 2016

Propose v6.3.0 (v2) #7550

Merged

Fishrock123 removed the semver-minor PRs that contain new features and should be released in the next minor version. label Jul 6, 2016

Fishrock123 added the dont-land-on-v4.x label Jul 6, 2016

Fishrock123 added the lts-watch-v4.x label Jul 6, 2016

chrjean mentioned this pull request Jul 7, 2016

Update nodejs to 6.3.0 ScoopInstaller/Scoop#948

Merged

dignifiedquire mentioned this pull request Jul 8, 2016

Incompatible with ReadableStream from node@6.3.0 mafintosh/duplexify#6

Closed

mcollina mentioned this pull request Jul 9, 2016

[stream] Make _readableState and _writableState part of the public API #7629

Closed

MylesBorins removed the lts-watch-v4.x label Oct 10, 2016

stream: improve Readable.read() performance #7077

stream: improve Readable.read() performance #7077

Conversation

mscdex commented May 31, 2016

Checklist

Affected core subsystem(s)

Description of change

mscdex commented May 31, 2016

mscdex commented May 31, 2016

calvinmetcalf commented May 31, 2016

mscdex commented May 31, 2016

calvinmetcalf commented May 31, 2016

mscdex commented May 31, 2016

calvinmetcalf commented May 31, 2016

mscdex commented May 31, 2016

calvinmetcalf commented May 31, 2016

mscdex commented May 31, 2016

ronkorving Jun 1, 2016

Choose a reason for hiding this comment

mscdex Jun 1, 2016 • edited Loading

Choose a reason for hiding this comment

mscdex commented Jun 7, 2016

ronkorving Jun 7, 2016

Choose a reason for hiding this comment

mscdex Jun 7, 2016

Choose a reason for hiding this comment

ronkorving Jun 7, 2016

Choose a reason for hiding this comment

jasnell Jun 7, 2016

Choose a reason for hiding this comment

mscdex Jun 7, 2016

Choose a reason for hiding this comment

jasnell commented Jun 7, 2016

mcollina commented Jun 7, 2016

mscdex commented Jun 7, 2016

mcollina commented Jun 7, 2016

mscdex commented Jun 7, 2016 • edited Loading

mcollina commented Jun 7, 2016 • edited Loading

Fishrock123 commented Jul 6, 2016

mcollina commented Jul 6, 2016

Fishrock123 commented Jul 6, 2016

mcollina commented Jul 6, 2016

Fishrock123 commented Jul 6, 2016

mcollina commented Jul 6, 2016

mafintosh commented Jul 8, 2016 • edited Loading

mcollina commented Jul 8, 2016 • edited Loading

ronkorving commented Jul 8, 2016

mafintosh commented Jul 8, 2016

mcollina commented Jul 9, 2016

mscdex Jun 1, 2016 •

edited

Loading

mscdex commented Jun 7, 2016 •

edited

Loading

mcollina commented Jun 7, 2016 •

edited

Loading

mafintosh commented Jul 8, 2016 •

edited

Loading

mcollina commented Jul 8, 2016 •

edited

Loading