Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

buffer: hard-deprecate calling Buffer without new #8169

Closed
wants to merge 5 commits into from
Closed

buffer: hard-deprecate calling Buffer without new #8169

wants to merge 5 commits into from

Conversation

seishun
Copy link
Contributor

@seishun seishun commented Aug 18, 2016

Checklist
  • make -j4 test (UNIX), or vcbuild test nosign (Windows) passes
  • commit message follows commit guidelines
Affected core subsystem(s)

buffer

Description of change

Light version of #7152.

We want to make Buffer a class so that it can be subclassed. However, instantiating a class requires new. This hard-deprecates calling Buffer without new.

@nodejs-github-bot nodejs-github-bot added the buffer Issues and PRs related to the buffer subsystem. label Aug 18, 2016
@addaleax addaleax added the semver-major PRs that contain breaking changes and should be released in the next major version. label Aug 18, 2016
@addaleax
Copy link
Member

LGTM, and really, thanks for doing this.

@addaleax
Copy link
Member

@Qard
Copy link
Member

Qard commented Aug 18, 2016

LGTM too.

@@ -65,6 +71,9 @@ function alignPool() {
* would ever actually be removed.
**/
function Buffer(arg, encodingOrOffset, length) {
if (!(this instanceof Buffer)) {
Copy link
Member

@jasnell jasnell Aug 18, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps new.target instead? https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/new.target

if (!new.target) {
  // print deprecation warning
}

@ChALkeR
Copy link
Member

ChALkeR commented Aug 18, 2016

LGTM once the comments above are fixed. One question, though — are there no tests that are affected by this?

@jasnell
Copy link
Member

jasnell commented Aug 18, 2016

This should also include a test case that ensures that the deprecation warning is emitted when expected

@ChALkeR
Copy link
Member

ChALkeR commented Aug 18, 2016

Btw, should this also hard-deprecate SlowBuffer() without a new?

@jasnell
Copy link
Member

jasnell commented Aug 18, 2016

Good point @ChALkeR ... it probably should. /cc @trevnorris

@ChALkeR
Copy link
Member

ChALkeR commented Aug 18, 2016

We could start filing some issues in advance.

Edited.

@Qard
Copy link
Member

Qard commented Aug 18, 2016

With that many occurrences, I feel we might need to do a longer deprecation cycle on this. Even with PRs, it could take awhile for patches to trickle down from deps of deps.

@ChALkeR
Copy link
Member

ChALkeR commented Aug 18, 2016

@Quard, that's only 1193 modules (approx), and I expect this list to look much less scary once they are ordered by the actual downloads/month stats. I will begin filing issues to the most important ones, see sidorares/node-mysql2#380 for an example.

@seishun
Copy link
Contributor Author

seishun commented Aug 19, 2016

@ChALkeR

Btw, should this also hard-deprecate SlowBuffer() without a new?

I don't see the point. As far as I'm aware, we're not planning to make SlowBuffer subclassable.

@jasnell

This should also include a test case that ensures that the deprecation warning is emitted when expected

Could you or someone else link to an existing test that does a similar thing?

@ChALkeR
Copy link
Member

ChALkeR commented Aug 19, 2016

@seishun Yes, I totally missed that, thanks.

Perhaps we should hard-deprecate SlowBuffer whatsoever in another PR, then. Not sure yet about the target on that one, though, but I guess 7.0.0 could be fine, as there are not so many SlowBuffer users and those of them who wish to support 0.12 and older v4.x versions could just use the shim. That should be discussed, though.

@jasnell
Copy link
Member

jasnell commented Aug 19, 2016

@seishun ... you can look at test/parallel/test-crypto-deprecated.js for an example of validating that the deprecation warning is emitted. It's pretty straightforward.

process.on('warning', (warning) => {
  assert.strictEqual(warning.name, 'DeprecationWarning');
  // etc...
});

const bufferDeprecationWarning =
deprecate(() => {}, 'Using Buffer without `new` will soon stop working. ' +
'Use `new Buffer`, or preferably ' +
'Buffer.from, Buffer.allocUnsafe or Buffer.alloc instead.');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably do not mention allocUnsafe?

if people need that they will find it I think :)

newBufferWarned = true;
process.emitWarning(
'Using Buffer without `new` will soon stop working. ' +
'Use `new Buffer`, or preferably ' +
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually since this will be in v7... maybe we should only recommend Buffer.from() and Buffer.alloc() (pref with the parenthesis?)

Copy link
Member

@jasnell jasnell Aug 19, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That works for me misread the comment... see #8169 (comment) instead.

@ChALkeR
Copy link
Member

ChALkeR commented Aug 19, 2016

SlowBuffer deprecation would affect only 444 packages.

@addaleax
Copy link
Member

addaleax commented Nov 4, 2016

Yeah, I am in favor of reverting. The reactions we are hearing from outside of Node core make it pretty clear that a runtime deprecation for functionality like this is just too disruptive, and we’re not going to get away with it, possibly ever.

@addaleax
Copy link
Member

addaleax commented Nov 4, 2016

Oh and since @Trott put this on the agenda (thanks!), ping @nodejs/ctc

@seishun
Copy link
Contributor Author

seishun commented Nov 4, 2016

The reactions we are hearing from outside of Node core make it pretty clear that a runtime deprecation for functionality like this is just too disruptive

Could you link to some of these reactions, because I'm only seeing negative comments from other Node.js collaborators?

@MylesBorins
Copy link
Contributor

@mafintosh @yoshuawuyts @substack have contributed to various wg's but afaik are not part of core

@seishun
Copy link
Contributor Author

seishun commented Nov 4, 2016

To put that in perspective, there were 1193 packages that would affected by this in August (#8169 (comment)). Evidently most of their developers don't mind fixing their code, or their users don't mind the warning, or there are no users.

@MylesBorins
Copy link
Contributor

@seisun how many are still a problem?

also 7 has only been out for a couple weeks, and does not have mass adoption yet... we can expect way more problems with 8, especially when it goes Lts

@alex7kom
Copy link

alex7kom commented Nov 4, 2016

Maintained modules can be easily fixed. Unmaintained modules may have security issues or bugs, or they may simply be obsolete. So keeping old and insecure APIs just to keep unmaintained modules running will hurt the ecosystem long-term. Unmaintained things usually begin to decay, so building an ecosystem on them is not a great idea. Made by an unpaid volunteer or not, old code is old code.

@feross
Copy link
Contributor

feross commented Nov 4, 2016

If we want to hard-deprecate Buffer() and then eventually new Buffer(), then IMO we should do them both at the same time, and we should be clear about the reasoning: this is for security reasons. Buffer.alloc() is preferred over Buffer() since it zeroes out the memory.

It would be really unfortunate if folks started updating their code to use new just to have that get deprecated shortly thereafter.

@billiegoose
Copy link

So I've been following this thread, and what isn't clear to me is from the security point of view, why you can't you just zero-fill buffers by default? Specifically, make Buffer() and new Buffer() aliases for Buffer.from(). At worst that would cause old code that hasn't been updated to Buffer.allocUnsafe()to run slower.

@jasnell
Copy link
Member

jasnell commented Nov 7, 2016

I've been trying to think of a way forward on this that would allow us to maintain the existing uses of Buffer() with or without the use of new while still allowing us to progress with improved internals. There's one possible approach that I've come up with that's going to need a bit more investigation, but at first blush it's doable.

If we followed a pattern similar to:

class BufferArray extends Uint8Array {
  constructor(/*.. args ..*/) {
    /* ... */
  }
}

function Buffer(/* .. args ..*/) {
  if (!(this instanceof Buffer))
    return new Buffer(/* .. args ..*/);
  return Reflect.construct(BufferArray, [/** args **/], new.target);
}
util.inherits(Buffer, BufferArray);

Then calling Buffer() and new Buffer() as currently done would still continue to work and the resulting object would properly inherit from the new BufferArray ES6 class where the improved internal implementation would live. It would mean introducing a new object into the prototype chain, but it allows us a path forward that would enable existing code to keep functioning. Buffer itself essentially just becomes a backwards compatible shim for the new implementation.

There are obviously some details that would need to be worked out in this but, at the very least, it provides on possible option.

@addaleax
Copy link
Member

addaleax commented Nov 7, 2016

@jasnell have you seen addaleax/node@ec09d43? It goes pretty much into the direction you’re describing, with tests passing and everything, it would just take a bit of work to match the current performance (because it currently creates unnecessary intermediate objects).

(I’ve linked this before – in the other thread, I think – but I do see that these things can get a bit lost 😄)

@evanlucas
Copy link
Contributor

I think I am +1 on reverting as well

@jasnell
Copy link
Member

jasnell commented Nov 7, 2016

@addaleax ... I think I saw the link at some point but I don't believe I'd actually seen the code. Will dig in more! Thank you!

@seishun
Copy link
Contributor Author

seishun commented Nov 8, 2016

@jasnell I am -1 on introducing an extra class. Things are already complicated enough with Array, Buffer, ArrayBuffer and UintXArray. Please try to imagine yourself in the shoes of someone who is just getting familiar with JavaScript and Node.js. Backward compatibility is important, but it would be unfortunate to permanently increase complexity in order to avoid a temporary disruption.

@mafintosh
Copy link
Member

@jasnell I don't fully understand the Reflect thing but that seems like something that could work and not break things.

@addaleax
Copy link
Member

addaleax commented Nov 8, 2016

@seishun If we do this right, the additional complexity will be limited to something that can be explained in a single sentence; namely, the only difference between Buffer and BufferArray would be that BufferArray is that the latter is an ES6 class, meaning that it can’t be instantiated without new and that it can be inherited from.

And yeah, I’m not too keen on the name BufferArray (or my BufferClass) either, but I get that it’s hard to come up with a name for something that basically already exists under a different name. If nothing else works, I’d be for SubclassableBuffer, because that’s what it is.

addaleax added a commit to addaleax/node that referenced this pull request Nov 9, 2016
This reverts commit f2fe558
(nodejs#8169) as the original
justification for the runtime-deprecation does not appear
to justify the disruption to Node’s existing ecosystem.
Futhermore, the possibility of deprecating the Buffer constructor
entirely in v8.0 might lead to people having to change their code twice.
@addaleax
Copy link
Member

addaleax commented Nov 9, 2016

I would like to ask everyone who has commented here to move the discussion to #9531, if only so the public discussion can take in a central place and not over multiple GH threads at once.

@Trott Trott removed the ctc-agenda label Nov 16, 2016
evanlucas pushed a commit that referenced this pull request Nov 28, 2016
This reverts commit f2fe558
(#8169) as the original
justification for the runtime-deprecation does not appear
to justify the disruption to Node’s existing ecosystem.
Futhermore, the possibility of deprecating the Buffer constructor
entirely in v8.0 might lead to people having to change their code twice.

PR-URL: #9529
Reviewed-By: Roman Reiss <me@silverwind.io>
Reviewed-By: Myles Borins <myles.borins@gmail.com>
Reviewed-By: Sam Roberts <vieuxtech@gmail.com>
Reviewed-By: Nikolai Vavilov <vvnicholas@gmail.com>
Reviewed-By: Evan Lucas <evanlucas@me.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
addaleax added a commit that referenced this pull request Dec 5, 2016
This reverts commit f2fe558
(#8169) as the original
justification for the runtime-deprecation does not appear
to justify the disruption to Node’s existing ecosystem.
Futhermore, the possibility of deprecating the Buffer constructor
entirely in v8.0 might lead to people having to change their code twice.

PR-URL: #9529
Reviewed-By: Roman Reiss <me@silverwind.io>
Reviewed-By: Myles Borins <myles.borins@gmail.com>
Reviewed-By: Sam Roberts <vieuxtech@gmail.com>
Reviewed-By: Nikolai Vavilov <vvnicholas@gmail.com>
Reviewed-By: Evan Lucas <evanlucas@me.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
@gibfahn gibfahn mentioned this pull request Jun 15, 2017
3 tasks
@seishun seishun deleted the new branch August 12, 2017 11:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
buffer Issues and PRs related to the buffer subsystem. semver-major PRs that contain breaking changes and should be released in the next major version.
Projects
None yet
Development

Successfully merging this pull request may close these issues.