1# safe-buffer [![travis][travis-image]][travis-url] [![npm][npm-image]][npm-url] [![downloads][downloads-image]][downloads-url] [![javascript style guide][standard-image]][standard-url] 2 3[travis-image]: https://img.shields.io/travis/feross/safe-buffer/master.svg 4[travis-url]: https://travis-ci.org/feross/safe-buffer 5[npm-image]: https://img.shields.io/npm/v/safe-buffer.svg 6[npm-url]: https://npmjs.org/package/safe-buffer 7[downloads-image]: https://img.shields.io/npm/dm/safe-buffer.svg 8[downloads-url]: https://npmjs.org/package/safe-buffer 9[standard-image]: https://img.shields.io/badge/code_style-standard-brightgreen.svg 10[standard-url]: https://standardjs.com 11 12#### Safer Node.js Buffer API 13 14**Use the new Node.js Buffer APIs (`Buffer.from`, `Buffer.alloc`, 15`Buffer.allocUnsafe`, `Buffer.allocUnsafeSlow`) in all versions of Node.js.** 16 17**Uses the built-in implementation when available.** 18 19## install 20 21``` 22npm install safe-buffer 23``` 24 25[Get supported safe-buffer with the Tidelift Subscription](https://tidelift.com/subscription/pkg/npm-safe-buffer?utm_source=npm-safe-buffer&utm_medium=referral&utm_campaign=readme) 26 27## usage 28 29The goal of this package is to provide a safe replacement for the node.js `Buffer`. 30 31It's a drop-in replacement for `Buffer`. You can use it by adding one `require` line to 32the top of your node.js modules: 33 34```js 35var Buffer = require('safe-buffer').Buffer 36 37// Existing buffer code will continue to work without issues: 38 39new Buffer('hey', 'utf8') 40new Buffer([1, 2, 3], 'utf8') 41new Buffer(obj) 42new Buffer(16) // create an uninitialized buffer (potentially unsafe) 43 44// But you can use these new explicit APIs to make clear what you want: 45 46Buffer.from('hey', 'utf8') // convert from many types to a Buffer 47Buffer.alloc(16) // create a zero-filled buffer (safe) 48Buffer.allocUnsafe(16) // create an uninitialized buffer (potentially unsafe) 49``` 50 51## api 52 53### Class Method: Buffer.from(array) 54<!-- YAML 55added: v3.0.0 56--> 57 58* `array` {Array} 59 60Allocates a new `Buffer` using an `array` of octets. 61 62```js 63const buf = Buffer.from([0x62,0x75,0x66,0x66,0x65,0x72]); 64 // creates a new Buffer containing ASCII bytes 65 // ['b','u','f','f','e','r'] 66``` 67 68A `TypeError` will be thrown if `array` is not an `Array`. 69 70### Class Method: Buffer.from(arrayBuffer[, byteOffset[, length]]) 71<!-- YAML 72added: v5.10.0 73--> 74 75* `arrayBuffer` {ArrayBuffer} The `.buffer` property of a `TypedArray` or 76 a `new ArrayBuffer()` 77* `byteOffset` {Number} Default: `0` 78* `length` {Number} Default: `arrayBuffer.length - byteOffset` 79 80When passed a reference to the `.buffer` property of a `TypedArray` instance, 81the newly created `Buffer` will share the same allocated memory as the 82TypedArray. 83 84```js 85const arr = new Uint16Array(2); 86arr[0] = 5000; 87arr[1] = 4000; 88 89const buf = Buffer.from(arr.buffer); // shares the memory with arr; 90 91console.log(buf); 92 // Prints: <Buffer 88 13 a0 0f> 93 94// changing the TypedArray changes the Buffer also 95arr[1] = 6000; 96 97console.log(buf); 98 // Prints: <Buffer 88 13 70 17> 99``` 100 101The optional `byteOffset` and `length` arguments specify a memory range within 102the `arrayBuffer` that will be shared by the `Buffer`. 103 104```js 105const ab = new ArrayBuffer(10); 106const buf = Buffer.from(ab, 0, 2); 107console.log(buf.length); 108 // Prints: 2 109``` 110 111A `TypeError` will be thrown if `arrayBuffer` is not an `ArrayBuffer`. 112 113### Class Method: Buffer.from(buffer) 114<!-- YAML 115added: v3.0.0 116--> 117 118* `buffer` {Buffer} 119 120Copies the passed `buffer` data onto a new `Buffer` instance. 121 122```js 123const buf1 = Buffer.from('buffer'); 124const buf2 = Buffer.from(buf1); 125 126buf1[0] = 0x61; 127console.log(buf1.toString()); 128 // 'auffer' 129console.log(buf2.toString()); 130 // 'buffer' (copy is not changed) 131``` 132 133A `TypeError` will be thrown if `buffer` is not a `Buffer`. 134 135### Class Method: Buffer.from(str[, encoding]) 136<!-- YAML 137added: v5.10.0 138--> 139 140* `str` {String} String to encode. 141* `encoding` {String} Encoding to use, Default: `'utf8'` 142 143Creates a new `Buffer` containing the given JavaScript string `str`. If 144provided, the `encoding` parameter identifies the character encoding. 145If not provided, `encoding` defaults to `'utf8'`. 146 147```js 148const buf1 = Buffer.from('this is a tést'); 149console.log(buf1.toString()); 150 // prints: this is a tést 151console.log(buf1.toString('ascii')); 152 // prints: this is a tC)st 153 154const buf2 = Buffer.from('7468697320697320612074c3a97374', 'hex'); 155console.log(buf2.toString()); 156 // prints: this is a tést 157``` 158 159A `TypeError` will be thrown if `str` is not a string. 160 161### Class Method: Buffer.alloc(size[, fill[, encoding]]) 162<!-- YAML 163added: v5.10.0 164--> 165 166* `size` {Number} 167* `fill` {Value} Default: `undefined` 168* `encoding` {String} Default: `utf8` 169 170Allocates a new `Buffer` of `size` bytes. If `fill` is `undefined`, the 171`Buffer` will be *zero-filled*. 172 173```js 174const buf = Buffer.alloc(5); 175console.log(buf); 176 // <Buffer 00 00 00 00 00> 177``` 178 179The `size` must be less than or equal to the value of 180`require('buffer').kMaxLength` (on 64-bit architectures, `kMaxLength` is 181`(2^31)-1`). Otherwise, a [`RangeError`][] is thrown. A zero-length Buffer will 182be created if a `size` less than or equal to 0 is specified. 183 184If `fill` is specified, the allocated `Buffer` will be initialized by calling 185`buf.fill(fill)`. See [`buf.fill()`][] for more information. 186 187```js 188const buf = Buffer.alloc(5, 'a'); 189console.log(buf); 190 // <Buffer 61 61 61 61 61> 191``` 192 193If both `fill` and `encoding` are specified, the allocated `Buffer` will be 194initialized by calling `buf.fill(fill, encoding)`. For example: 195 196```js 197const buf = Buffer.alloc(11, 'aGVsbG8gd29ybGQ=', 'base64'); 198console.log(buf); 199 // <Buffer 68 65 6c 6c 6f 20 77 6f 72 6c 64> 200``` 201 202Calling `Buffer.alloc(size)` can be significantly slower than the alternative 203`Buffer.allocUnsafe(size)` but ensures that the newly created `Buffer` instance 204contents will *never contain sensitive data*. 205 206A `TypeError` will be thrown if `size` is not a number. 207 208### Class Method: Buffer.allocUnsafe(size) 209<!-- YAML 210added: v5.10.0 211--> 212 213* `size` {Number} 214 215Allocates a new *non-zero-filled* `Buffer` of `size` bytes. The `size` must 216be less than or equal to the value of `require('buffer').kMaxLength` (on 64-bit 217architectures, `kMaxLength` is `(2^31)-1`). Otherwise, a [`RangeError`][] is 218thrown. A zero-length Buffer will be created if a `size` less than or equal to 2190 is specified. 220 221The underlying memory for `Buffer` instances created in this way is *not 222initialized*. The contents of the newly created `Buffer` are unknown and 223*may contain sensitive data*. Use [`buf.fill(0)`][] to initialize such 224`Buffer` instances to zeroes. 225 226```js 227const buf = Buffer.allocUnsafe(5); 228console.log(buf); 229 // <Buffer 78 e0 82 02 01> 230 // (octets will be different, every time) 231buf.fill(0); 232console.log(buf); 233 // <Buffer 00 00 00 00 00> 234``` 235 236A `TypeError` will be thrown if `size` is not a number. 237 238Note that the `Buffer` module pre-allocates an internal `Buffer` instance of 239size `Buffer.poolSize` that is used as a pool for the fast allocation of new 240`Buffer` instances created using `Buffer.allocUnsafe(size)` (and the deprecated 241`new Buffer(size)` constructor) only when `size` is less than or equal to 242`Buffer.poolSize >> 1` (floor of `Buffer.poolSize` divided by two). The default 243value of `Buffer.poolSize` is `8192` but can be modified. 244 245Use of this pre-allocated internal memory pool is a key difference between 246calling `Buffer.alloc(size, fill)` vs. `Buffer.allocUnsafe(size).fill(fill)`. 247Specifically, `Buffer.alloc(size, fill)` will *never* use the internal Buffer 248pool, while `Buffer.allocUnsafe(size).fill(fill)` *will* use the internal 249Buffer pool if `size` is less than or equal to half `Buffer.poolSize`. The 250difference is subtle but can be important when an application requires the 251additional performance that `Buffer.allocUnsafe(size)` provides. 252 253### Class Method: Buffer.allocUnsafeSlow(size) 254<!-- YAML 255added: v5.10.0 256--> 257 258* `size` {Number} 259 260Allocates a new *non-zero-filled* and non-pooled `Buffer` of `size` bytes. The 261`size` must be less than or equal to the value of 262`require('buffer').kMaxLength` (on 64-bit architectures, `kMaxLength` is 263`(2^31)-1`). Otherwise, a [`RangeError`][] is thrown. A zero-length Buffer will 264be created if a `size` less than or equal to 0 is specified. 265 266The underlying memory for `Buffer` instances created in this way is *not 267initialized*. The contents of the newly created `Buffer` are unknown and 268*may contain sensitive data*. Use [`buf.fill(0)`][] to initialize such 269`Buffer` instances to zeroes. 270 271When using `Buffer.allocUnsafe()` to allocate new `Buffer` instances, 272allocations under 4KB are, by default, sliced from a single pre-allocated 273`Buffer`. This allows applications to avoid the garbage collection overhead of 274creating many individually allocated Buffers. This approach improves both 275performance and memory usage by eliminating the need to track and cleanup as 276many `Persistent` objects. 277 278However, in the case where a developer may need to retain a small chunk of 279memory from a pool for an indeterminate amount of time, it may be appropriate 280to create an un-pooled Buffer instance using `Buffer.allocUnsafeSlow()` then 281copy out the relevant bits. 282 283```js 284// need to keep around a few small chunks of memory 285const store = []; 286 287socket.on('readable', () => { 288 const data = socket.read(); 289 // allocate for retained data 290 const sb = Buffer.allocUnsafeSlow(10); 291 // copy the data into the new allocation 292 data.copy(sb, 0, 0, 10); 293 store.push(sb); 294}); 295``` 296 297Use of `Buffer.allocUnsafeSlow()` should be used only as a last resort *after* 298a developer has observed undue memory retention in their applications. 299 300A `TypeError` will be thrown if `size` is not a number. 301 302### All the Rest 303 304The rest of the `Buffer` API is exactly the same as in node.js. 305[See the docs](https://nodejs.org/api/buffer.html). 306 307 308## Related links 309 310- [Node.js issue: Buffer(number) is unsafe](https://github.com/nodejs/node/issues/4660) 311- [Node.js Enhancement Proposal: Buffer.from/Buffer.alloc/Buffer.zalloc/Buffer() soft-deprecate](https://github.com/nodejs/node-eps/pull/4) 312 313## Why is `Buffer` unsafe? 314 315Today, the node.js `Buffer` constructor is overloaded to handle many different argument 316types like `String`, `Array`, `Object`, `TypedArrayView` (`Uint8Array`, etc.), 317`ArrayBuffer`, and also `Number`. 318 319The API is optimized for convenience: you can throw any type at it, and it will try to do 320what you want. 321 322Because the Buffer constructor is so powerful, you often see code like this: 323 324```js 325// Convert UTF-8 strings to hex 326function toHex (str) { 327 return new Buffer(str).toString('hex') 328} 329``` 330 331***But what happens if `toHex` is called with a `Number` argument?*** 332 333### Remote Memory Disclosure 334 335If an attacker can make your program call the `Buffer` constructor with a `Number` 336argument, then they can make it allocate uninitialized memory from the node.js process. 337This could potentially disclose TLS private keys, user data, or database passwords. 338 339When the `Buffer` constructor is passed a `Number` argument, it returns an 340**UNINITIALIZED** block of memory of the specified `size`. When you create a `Buffer` like 341this, you **MUST** overwrite the contents before returning it to the user. 342 343From the [node.js docs](https://nodejs.org/api/buffer.html#buffer_new_buffer_size): 344 345> `new Buffer(size)` 346> 347> - `size` Number 348> 349> The underlying memory for `Buffer` instances created in this way is not initialized. 350> **The contents of a newly created `Buffer` are unknown and could contain sensitive 351> data.** Use `buf.fill(0)` to initialize a Buffer to zeroes. 352 353(Emphasis our own.) 354 355Whenever the programmer intended to create an uninitialized `Buffer` you often see code 356like this: 357 358```js 359var buf = new Buffer(16) 360 361// Immediately overwrite the uninitialized buffer with data from another buffer 362for (var i = 0; i < buf.length; i++) { 363 buf[i] = otherBuf[i] 364} 365``` 366 367 368### Would this ever be a problem in real code? 369 370Yes. It's surprisingly common to forget to check the type of your variables in a 371dynamically-typed language like JavaScript. 372 373Usually the consequences of assuming the wrong type is that your program crashes with an 374uncaught exception. But the failure mode for forgetting to check the type of arguments to 375the `Buffer` constructor is more catastrophic. 376 377Here's an example of a vulnerable service that takes a JSON payload and converts it to 378hex: 379 380```js 381// Take a JSON payload {str: "some string"} and convert it to hex 382var server = http.createServer(function (req, res) { 383 var data = '' 384 req.setEncoding('utf8') 385 req.on('data', function (chunk) { 386 data += chunk 387 }) 388 req.on('end', function () { 389 var body = JSON.parse(data) 390 res.end(new Buffer(body.str).toString('hex')) 391 }) 392}) 393 394server.listen(8080) 395``` 396 397In this example, an http client just has to send: 398 399```json 400{ 401 "str": 1000 402} 403``` 404 405and it will get back 1,000 bytes of uninitialized memory from the server. 406 407This is a very serious bug. It's similar in severity to the 408[the Heartbleed bug](http://heartbleed.com/) that allowed disclosure of OpenSSL process 409memory by remote attackers. 410 411 412### Which real-world packages were vulnerable? 413 414#### [`bittorrent-dht`](https://www.npmjs.com/package/bittorrent-dht) 415 416[Mathias Buus](https://github.com/mafintosh) and I 417([Feross Aboukhadijeh](http://feross.org/)) found this issue in one of our own packages, 418[`bittorrent-dht`](https://www.npmjs.com/package/bittorrent-dht). The bug would allow 419anyone on the internet to send a series of messages to a user of `bittorrent-dht` and get 420them to reveal 20 bytes at a time of uninitialized memory from the node.js process. 421 422Here's 423[the commit](https://github.com/feross/bittorrent-dht/commit/6c7da04025d5633699800a99ec3fbadf70ad35b8) 424that fixed it. We released a new fixed version, created a 425[Node Security Project disclosure](https://nodesecurity.io/advisories/68), and deprecated all 426vulnerable versions on npm so users will get a warning to upgrade to a newer version. 427 428#### [`ws`](https://www.npmjs.com/package/ws) 429 430That got us wondering if there were other vulnerable packages. Sure enough, within a short 431period of time, we found the same issue in [`ws`](https://www.npmjs.com/package/ws), the 432most popular WebSocket implementation in node.js. 433 434If certain APIs were called with `Number` parameters instead of `String` or `Buffer` as 435expected, then uninitialized server memory would be disclosed to the remote peer. 436 437These were the vulnerable methods: 438 439```js 440socket.send(number) 441socket.ping(number) 442socket.pong(number) 443``` 444 445Here's a vulnerable socket server with some echo functionality: 446 447```js 448server.on('connection', function (socket) { 449 socket.on('message', function (message) { 450 message = JSON.parse(message) 451 if (message.type === 'echo') { 452 socket.send(message.data) // send back the user's message 453 } 454 }) 455}) 456``` 457 458`socket.send(number)` called on the server, will disclose server memory. 459 460Here's [the release](https://github.com/websockets/ws/releases/tag/1.0.1) where the issue 461was fixed, with a more detailed explanation. Props to 462[Arnout Kazemier](https://github.com/3rd-Eden) for the quick fix. Here's the 463[Node Security Project disclosure](https://nodesecurity.io/advisories/67). 464 465 466### What's the solution? 467 468It's important that node.js offers a fast way to get memory otherwise performance-critical 469applications would needlessly get a lot slower. 470 471But we need a better way to *signal our intent* as programmers. **When we want 472uninitialized memory, we should request it explicitly.** 473 474Sensitive functionality should not be packed into a developer-friendly API that loosely 475accepts many different types. This type of API encourages the lazy practice of passing 476variables in without checking the type very carefully. 477 478#### A new API: `Buffer.allocUnsafe(number)` 479 480The functionality of creating buffers with uninitialized memory should be part of another 481API. We propose `Buffer.allocUnsafe(number)`. This way, it's not part of an API that 482frequently gets user input of all sorts of different types passed into it. 483 484```js 485var buf = Buffer.allocUnsafe(16) // careful, uninitialized memory! 486 487// Immediately overwrite the uninitialized buffer with data from another buffer 488for (var i = 0; i < buf.length; i++) { 489 buf[i] = otherBuf[i] 490} 491``` 492 493 494### How do we fix node.js core? 495 496We sent [a PR to node.js core](https://github.com/nodejs/node/pull/4514) (merged as 497`semver-major`) which defends against one case: 498 499```js 500var str = 16 501new Buffer(str, 'utf8') 502``` 503 504In this situation, it's implied that the programmer intended the first argument to be a 505string, since they passed an encoding as a second argument. Today, node.js will allocate 506uninitialized memory in the case of `new Buffer(number, encoding)`, which is probably not 507what the programmer intended. 508 509But this is only a partial solution, since if the programmer does `new Buffer(variable)` 510(without an `encoding` parameter) there's no way to know what they intended. If `variable` 511is sometimes a number, then uninitialized memory will sometimes be returned. 512 513### What's the real long-term fix? 514 515We could deprecate and remove `new Buffer(number)` and use `Buffer.allocUnsafe(number)` when 516we need uninitialized memory. But that would break 1000s of packages. 517 518~~We believe the best solution is to:~~ 519 520~~1. Change `new Buffer(number)` to return safe, zeroed-out memory~~ 521 522~~2. Create a new API for creating uninitialized Buffers. We propose: `Buffer.allocUnsafe(number)`~~ 523 524#### Update 525 526We now support adding three new APIs: 527 528- `Buffer.from(value)` - convert from any type to a buffer 529- `Buffer.alloc(size)` - create a zero-filled buffer 530- `Buffer.allocUnsafe(size)` - create an uninitialized buffer with given size 531 532This solves the core problem that affected `ws` and `bittorrent-dht` which is 533`Buffer(variable)` getting tricked into taking a number argument. 534 535This way, existing code continues working and the impact on the npm ecosystem will be 536minimal. Over time, npm maintainers can migrate performance-critical code to use 537`Buffer.allocUnsafe(number)` instead of `new Buffer(number)`. 538 539 540### Conclusion 541 542We think there's a serious design issue with the `Buffer` API as it exists today. It 543promotes insecure software by putting high-risk functionality into a convenient API 544with friendly "developer ergonomics". 545 546This wasn't merely a theoretical exercise because we found the issue in some of the 547most popular npm packages. 548 549Fortunately, there's an easy fix that can be applied today. Use `safe-buffer` in place of 550`buffer`. 551 552```js 553var Buffer = require('safe-buffer').Buffer 554``` 555 556Eventually, we hope that node.js core can switch to this new, safer behavior. We believe 557the impact on the ecosystem would be minimal since it's not a breaking change. 558Well-maintained, popular packages would be updated to use `Buffer.alloc` quickly, while 559older, insecure packages would magically become safe from this attack vector. 560 561 562## links 563 564- [Node.js PR: buffer: throw if both length and enc are passed](https://github.com/nodejs/node/pull/4514) 565- [Node Security Project disclosure for `ws`](https://nodesecurity.io/advisories/67) 566- [Node Security Project disclosure for`bittorrent-dht`](https://nodesecurity.io/advisories/68) 567 568 569## credit 570 571The original issues in `bittorrent-dht` 572([disclosure](https://nodesecurity.io/advisories/68)) and 573`ws` ([disclosure](https://nodesecurity.io/advisories/67)) were discovered by 574[Mathias Buus](https://github.com/mafintosh) and 575[Feross Aboukhadijeh](http://feross.org/). 576 577Thanks to [Adam Baldwin](https://github.com/evilpacket) for helping disclose these issues 578and for his work running the [Node Security Project](https://nodesecurity.io/). 579 580Thanks to [John Hiesey](https://github.com/jhiesey) for proofreading this README and 581auditing the code. 582 583 584## license 585 586MIT. Copyright (C) [Feross Aboukhadijeh](http://feross.org) 587