How can I wait for multiple async operations in Writable stream in Node.js?

34 views Asked by At

Assume I have a writable stream implementation:

  • On each writable chunk, I'm firing an async operation. I don't care about the resolved value.
  • I'm firing the operations in parallel
  • I do want to wait for all async calls to be fulfilled before declaring that the stream has finished.

By Node.JS docs for ._final internal:

This optional function will be called before the stream closes, delaying the 'finish' event until callback is called

Thus, how should I wait for all promises to complete before calling the callback within ._final(...)?

I've came up with the following, but now sure that's the right way to do so using the Streams API. What would be the right approach here?

// Some async operation
const getStatus = async (chunk, ms) => {
  return new Promise((resolve, reject) => {
    setTimeout((resolve), ms);
  })
}

class MyWritable extends Writable {
  private asyncCalls = [];

  constructor() {
    super({objectMode: true});
  }

  _write(chunk, encoding, next): void {
    this.asyncCalls.push(getStatus(chunk, 2000))
    next(); // Keep firing parallel async operations when data is available
  }

  _final(callback): void {
    Promise.all(this.asyncCalls).then(() => {
      // Only here I'm actually finished - delay the finish event
      callback();
    })
  }
}
0

There are 0 answers