Safe to drop a Buffer before GC?

[overlookmotel]

If you create an ArrayBuffer in Rust and pass it to JS, is it safe to drop its backing memory before the JS garbage collector collects the buffer if you know it's no longer in use?

My use case is this: I'm running a series of calls to a synchronous NAPI function which returns a large buffer. That buffer is then used (again, synchronously) in JS, and then discarded. The program does that over and over.

If I make all these calls without any ticks in between, JS garbage collector never gets a chance to run, and memory usage explodes. e.g.:

for (let i = 1; i <= 100; i++) {
  const buf = createOneGbBuffer();
  // Do stuff with buffer
}
// At this point 100 GB of memory is consumed!

I'm wondering if something like this is possible:

const { createBuffer, destroyBuffer } = require('./binding.js');

for (let i = 1; i <= 100; i++) {
  const buf = createBuffer(i);
  // Do stuff with buffer
  destroyBuffer();
}

Rust side: (assumes that there's only ever 1 buffer in play at a time)

use std::sync::Mutex;
static LAST_VEC: Mutex<Option<Vec<u8>>> = Mutex::new(None);

#[napi]
pub fn create_buffer(num: u8) -> Uint8Array {
  let mut vec = vec![num; 1024 * 1024 * 1024]; // 1 GiB
  let ptr = vec.as_mut_ptr();
  let len = vec.len();

  // Cache vec to be dropped later
  *LAST_VEC.lock().unwrap() = Some(vec);

  unsafe {
    Uint8Array::with_external_data( ptr, len, |_ptr, _len| destroy_last_vec() )
  }
}

#[napi]
pub fn destroy_buffer() {
  destroy_last_vec();
}

fn destroy_last_vec() {
  // Drop last vec if it's not already been dropped
  let mut guard = LAST_VEC.lock().unwrap();
  if guard.is_some() {
    *guard = None;
  }
}

In theory this makes some sense. We know that the buffer is no longer referenced before the garbage collector does, so why not free it early?

And the above example seems to work - memory consumption is massively reduced.

However, is it safe? The following chain of events is very likely:

1. create_buffer() creates a buffer, stored at memory address X.
2. NAPI sends buffer to V8.
3. Call to destroy_buffer() drops buffer's backing memory.
4. V8 believes buffer to still be "alive" - it's not GCed it yet.
5. create_buffer() called again. Allocator reuses previously freed memory, and 2nd buffer also has memory address X.
6. NAPI sends 2nd buffer to V8.
7. V8 now has two buffers referenced which both supposedly occupy the same memory.
8. Time passes...
9. V8 runs GC. Freaks out when it tries to collect twice for same memory address.

Is that likely to go horribly wrong?

[Brooooooklyn]

@overlookmotel sorry, I missed this discussion. Maybe this API will help https://nodejs.org/api/n-api.html#napi_open_handle_scope. I'd like to investigate how to integrate this API in this scenario

[overlookmotel]

Thanks for coming back. I'll look into napi_open_handle_scope.

One other question in meantime: Is the Mutex in above example necessary, or could it be replaced with Cell? Is the notify callback provided to Uint8Array::with_external_data() always called in main thread?

[Brooooooklyn]

there is Env::run_in_scope, but I have never tried this API.

[overlookmotel]

Sorry if this is stupid question, but was that reply relevant to use of napi_open_handle_scope? Or in answer to my question about whether the notify callback is run in main thread?

[Brooooooklyn]

Oh, my bad.

Is the notify callback provided to Uint8Array::with_external_data() always called in main thread?

Yes, it's always called in the main thread.

[overlookmotel]

Just a note, while I continue to investigate this. The reason for my observation of memory not getting freed in a tight loop is not that GC doesn't run, but that the finalize callback is scheduled to run with setImmediate(). So likely GC has identified this memory as ready to free, but just that NAPI-RS notify callback doesn't get called until next turn of the event loop.

Source: nodejs/node-addon-api#917 (comment)

[overlookmotel]

I've looked into napi_open_handle_scope.

As far as I can see, that API is designed to solve a different problem - telling V8 to keep a handle for a js_value "alive" for longer than it would by default, or to free it earlier than it would by default.

In my example, the default is correct. V8 knows when it can GC the buffer. The problem is that either:

1. it hasn't run GC yet or
2. the buffers have been GCed, but the notify callbacks aren't called until next turn of the event loop, so Rust-side code doesn't know this.

In my example above, (2) is the most likely explanation.

I wonder if napi_detach_arraybuffer might be the solution here? Presumably after detaching the buffer, it's legitimate to free the memory, and V8 won't have any problem with that.

@Brooooooklyn Does that sound correct to you? If so, do you think it'd be useful to include this in napi-rs's API somehow? Like adding a .detach() method to the ArrayBuffers it sends to JS?