Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

doc: buffer.from will return internal pool buffer #24312

Closed
wants to merge 4 commits into from

Conversation

mritunjayz
Copy link
Contributor

Checklist

#22139
Buffer.from will also use internal Buffer pool if it's size is less then Buffer.poolSize.

@nodejs-github-bot nodejs-github-bot added buffer Issues and PRs related to the buffer subsystem. doc Issues and PRs related to the documentations. labels Nov 11, 2018
@mritunjayz mritunjayz changed the title doc: Buffer.from will return internal pool buffer doc: buffer.from will return internal pool buffer Nov 11, 2018
@mritunjayz
Copy link
Contributor Author

ping @ChALkeR

@mritunjayz
Copy link
Contributor Author

@ChALkeR can you please review this ? 😊

doc/api/buffer.md Outdated Show resolved Hide resolved
doc/api/buffer.md Outdated Show resolved Hide resolved
@ChALkeR ChALkeR self-requested a review November 16, 2018 19:58
@mritunjayz
Copy link
Contributor Author

@ChALkeR I removed length condition from documentation.

doc/api/buffer.md Outdated Show resolved Hide resolved
@mritunjayz
Copy link
Contributor Author

@Trott

@Trott
Copy link
Member

Trott commented Nov 28, 2018

@nodejs/buffer @nodejs/documentation

const nodeBuffers = Buffer.from([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]);

console.log(nodeBuffers.offset);
// prints: 344
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not always correct. It may return 0, and any subsequent call will probably increase it.
So it doesn't always print 344.

// prints: 8192

// But you can change it
Buffer.poolSize = 5;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could be misleading. Changing Buffer.poolSize does have immediate effect on which buffers would be pooled, but it doesn't immediately change the size of the current pool, it changes the size of the next pool.
It does, however, take immediate effect for determining whether should the buffer be allocated from the pool or not.

@ChALkeR
Copy link
Member

ChALkeR commented Mar 2, 2019

@mritunjaygoutam12 Sorry for delay on this, that was mostly due to my personal issues :-/.
I am available and will respond more actively on this from now on.


Overall, I don't think that the code example is very useful -- it's the confusing part atm, as it seems for me. Perhaps we can get this to land faster without the code example at all.

As for the remaining part -- it specifically mentions Buffer.from(array) for some reason, when in fact many other buffer operations are pooled.

E.g. these are not covered:

> Buffer.from('sdf').offset
280
> Buffer.concat([Buffer.from('xxx'), Buffer.from('yyy')]).offset
304
> Buffer.from(Buffer.from('sdf')).offset
320
> Buffer.from(new String('test')).offset
328

The problem of the current documentation in master is that this section:

node/doc/api/buffer.md

Lines 100 to 103 in 17c6b1d

`Buffer` instances returned by [`Buffer.allocUnsafe()`] *may* be allocated off
a shared internal memory pool if `size` is less than or equal to half
[`Buffer.poolSize`]. Instances returned by [`Buffer.allocUnsafeSlow()`] *never*
use the shared internal memory pool.

makes it seem that only .allocUnsafe is pooled, when in fact mostly everything is pooled.

Same for:

node/doc/api/buffer.md

Lines 597 to 609 in 17c6b1d

Note that the `Buffer` module pre-allocates an internal `Buffer` instance of
size [`Buffer.poolSize`] that is used as a pool for the fast allocation of new
`Buffer` instances created using [`Buffer.allocUnsafe()`] and the deprecated
`new Buffer(size)` constructor only when `size` is less than or equal to
`Buffer.poolSize >> 1` (floor of [`Buffer.poolSize`] divided by two).
Use of this pre-allocated internal memory pool is a key difference between
calling `Buffer.alloc(size, fill)` vs. `Buffer.allocUnsafe(size).fill(fill)`.
Specifically, `Buffer.alloc(size, fill)` will *never* use the internal `Buffer`
pool, while `Buffer.allocUnsafe(size).fill(fill)` *will* use the internal
`Buffer` pool if `size` is less than or equal to half [`Buffer.poolSize`]. The
difference is subtle but can be important when an application requires the
additional performance that [`Buffer.allocUnsafe()`] provides.

Perhaps some of the rewording should touch those?

Also, /cc @nodejs/tsc @nodejs/buffer @nodejs/security-wg -- what was the reason for making Buffer.alloc non-pooled, but Buffer.from pooled? Buffer.alloc could also benefit from pre-allocating zero-filled buffers, can't it? If providing pooled buffers is an issue, we shouldn't do that for everything. If it's not, perhaps we should just pool Buffer.alloc in a semver-major, so users won't be punished for using it instead of Buffer.allocUnsafe().fill()?

@ChALkeR ChALkeR self-requested a review March 2, 2019 02:43
Copy link
Contributor

@vsemozhetbyt vsemozhetbyt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some nits for consistency with other examples.

const nodeBuffers = Buffer.from([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]);

console.log(nodeBuffers.offset);
// prints: 344
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// prints: 344
// Prints: 344


// Default Buffer.poolsize is 8192
console.log(Buffer.poolSize);
// prints: 8192
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// prints: 8192
// Prints: 8192

const buf = Buffer.from([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]);

console.log(buf.offset);
// prints: 0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// prints: 0
// Prints: 0

@gireeshpunathil
Copy link
Member

@mritunjayz - can you address the review comments?

@HarshithaKP
Copy link
Member

@mritunjayz, are you still interested in working on the PR ? If not, let me know; I can pick this up and progress.

@mritunjayz
Copy link
Contributor Author

@HarshithaKP yes, you can. Thanks

@HarshithaKP
Copy link
Member

This can be closed as the intention is fulfilled in #32703.

@Trott Trott closed this Apr 14, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
buffer Issues and PRs related to the buffer subsystem. doc Issues and PRs related to the documentations.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants