Running node child processes asynchronously in a loop and receiving stdout results after loop ends

Question

I have the following sample code...

const { exec } = require('child_process');

const Main = async () => {
    const ret = await GetAllItems();

    console.log('results', ret);
};

const GetAllItems = async () => {
    const promises = [];

    for (let i = 0; i < 15; i++) {
        promises.push(GetItem(i));
    }

        console.log('=== Loop ended ===')

    return await Promise.all(promises).then((values) => {
        return values;
    });
};

const GetItem = async (item = '') => {
    console.log('Getting data:', item);

    return new Promise((resolve, reject) => {
        exec('echo', [item], (error, stdout, stderr) => {
            if (error) {
                throw error;
            }
            // console.log(stdout);
            console.log('Receiving Data: ' + item);
            resolve('Receiving Data: ' + item);
        });
    });
};

Main();

And this is the result I'm getting...

Getting data: 0
Getting data: 1
Getting data: 2
Getting data: 3
Getting data: 4
Getting data: 5
Getting data: 6
Getting data: 7
Getting data: 8
Getting data: 9
Getting data: 10
Getting data: 11
Getting data: 12
Getting data: 13
Getting data: 14
=== Loop ended ===
Receiving Data: 12
Receiving Data: 11
Receiving Data: 10
Receiving Data: 9
Receiving Data: 8
Receiving Data: 7
Receiving Data: 6
Receiving Data: 5
Receiving Data: 4
Receiving Data: 3
Receiving Data: 2
Receiving Data: 1
Receiving Data: 0
Receiving Data: 14
Receiving Data: 13

So my question is, how come I'm receiving data only after the loop ends? any idea why it's in that specific order?

Ideally I'd like to receive the data as soon as the command is executed (before the loop ends), is this even possible?

Thanks for the help!

=====================

Answer 1

const EventEmitter = require('events')
const { exec } = require('child_process')

class ItemGetter extends EventEmitter {
  constructor () {
    super();
    this.on('item', item => this.handleItem(item));
  }

  handleItem (item) {
    console.log('Receiving Data: ' + item);
  }

  getAllItems () {
    for (let i = 0; i < 15; i++) {
      this
        .getItem(i)
        .then(item => this.emit('item', item))
        .catch(console.error);
    }
    console.log('=== Loop ended ===')
  }

  async getItem (item = '') {
    console.log('Getting data:', item);
    return new Promise((resolve, reject) => {
      exec('echo', [item], (error, stdout, stderr) => {
        if (error) {
          throw error;
        }
        resolve(item);
      });
    });
  }
}

(new ItemGetter()).getAllItems()

You logic, for first, run loop with calling all GetItem, then output '=== Loop ended ===', and only after that run all promices resolution, so, if you want get result of each getItem execution independently of eachother, just don't abuse asynchronous logic, frequently right solution much simpliest, than it seems ;)

Note: in this solution, you will get the same output, because loop with getItem calling, runnung faster, then promises with exec, but in this case, each item will be handled exactly after appropriate promise will be resolved, except of awaiting of all promises resolution. But in case, when you whant to make method getAllItems, asyncronous too, and await, when all results will be handled, but still remaining each promise running independently, you should implement you own logic, eg, using counter of started getItem methods, which increments in a loop and decrements on each promise resolution, so, this can be not so trivial. From other side, when you starts getItems in a loop sincronuosly, its have a sence to leave using promise.all (or even promise.allSettled, depeding on you needs), and then handle all results also sincronously.

UPD: By the way, you should understand, how event loop works, and how its supply async behavior: code in the node.js run in one sequence, step by step syncronously, from start to end, in continuos loop. Any async operations, by fact just a sheduled for execution on next loop, if all conditions for start that code mutched (eg promise is resolved). Read about event loop for detailes. You can start parallel sequences in a child processes, for examle, using fork, or cluster, but each thread also will run syncronuosly in its own event loop.