Batch Subscribes on Angular | Managing multiples subscriptions at the same time

Question

Imagine this situation...

I have a list of about +1k itens to display in a grid.

The user can change the grid values individually or he has some options to change the same property in all the itens.

Because I don't know how much itens the request can return, when the user choose to make a batch change, like change the datatime for today. I have to storage those changes in some properties, so when he clicks a Confirm button I have to make some "GET" requests to know how many requests I will have to handle, do the property change in my Angular project and send the update request to the back-end.

I already made all those changes and it was working, the only problem was that the browser would freeze cuz it had to handle many subscriptions at the same time.

To fix that I coded the function below. So I made an List full os requests ready to be subscribed and I subscribe X requests ( controlled by QUANTITY_OPEN_REQUESTS ) so the browser won't freeze.

When I'm at max parallel requests I just sleep and try again after Y seconds ( controlled by LOOP_INTERVAL_MS ).

The code is working but I feel like I'm making a huge mess, but I tried to find some articles or posts in Stackoverflow, Medium, Angular Docs, RXJS documentation and don't know whatelse can I do.

    async loopRequests(){
        let lsWaitingReqs: Array<ISubControl> = this.lsRequests.filter((x)=> x.status === StatusSubscription.WAITING)
        let lsRunningReqs: Array<ISubControl> = this.lsRequests.filter((x)=> x.status === StatusSubscription.STARTED)
        let lsDoneReqs: Array<ISubControl> = this.getFinishedRequests()
        let count: number = 0;
        this.loopExecuting = true

        // While my Done List has less itens then my requestList we must check if I can subscribe

        while(lsDoneReqs.length !== this.lsRequests.length){
            
            // While I have less running requests then max quantity, create another subscribe
            if (lsRunningReqs.length <= QUANTITY_OPEN_REQUESTS){
                if (lsWaitingReqs.length > 0){
                    const nextReq: IAltLoteTipoSubControl = lsWaitingReqs[0]
                    this.callProgressRefresh();

                    this.updateItemReqStack(nextReq, StatusSubscription.STARTED);
                    // The execRequest subscribes the request and when done it moves the StatusSubscription to Done
                    this.lsSubscriptions.push(this.execRequest(nextReq));
                } else {
                    await sleep(LOOP_INTERVAL_MS);  
                }
            } else {
                // If its at the maximum parallel requests, sleep to try again
                await sleep(LOOP_INTERVAL_MS);
            }
            
            // update the progressbar
            this.callProgressRefresh();

            // Update control lists
            lsRunningReqs = this.lsRequests.filter((x)=> x.status === StatusSubscription.STARTED);
            lsWaitingReqs = this.lsRequests.filter((x)=> x.status === StatusSubscription.WAITING);
            lsDoneReqs    = this.getFinishedRequests()
        }
        this.loopExecuting = false
    }

Can you guys leave some tips or anything that could help me handle all this mess in my project? My main concearn is Performance, refresh rate and those kinda things.

Even if you guys can recommend tools for performance evaluation I would try cuz most of the time my Motto is "Lets delivery something and then we think about performance" and I want to anticipate perfomance issues.

Edit¹: One thing I forgot to mention is that each item can send an update request.

Obs: I know it would be a lot easier to process all those requests on the Backend, but I just can't.... It's more like a Corporate problem and I have to give an solution in my Angular project.

Answer 1

Rxjs can handle concurrency quite easily. I will show a generic example since I don't know your exact code that makes the request.

makeReqeusts(updatedItems: Item[]) {
   from(updatedItems).pipe(
      mergeMap(item => this.http.post('/url', item), 5) // <-- concurrency = 5
   ).subscribe();
}

With the above code, we use from to create an observable that emits each array element individually. We when use mergeMap to subscribe to the http call for each item. You can pass the concurrency as the second parameter, which will limit the number of active requests.

---

I see you want to indicate progress of this operation. We can use the use the map operator to calculate a percentage as updates are completed:

makeReqeusts(updatedItems: Item[]) {
   from(updatedItems).pipe(
      mergeMap(item => this.http.post('/url', item), 5),
      map((_, i) => (i+1) / updatedItems.length),
      startWith(0)
   ).subscribe(
      percent => this.updateProgress(percent)
   );
}

Here the code inside map will be hit whenever an http call is completed. map takes two arguments, the first is the emitted value (which we don't need in this case), the other is the "emission index", which is just an increasing integer starting at 0.

We use startWith(0) to immediately emit 0.