DeployR Server & Javascript API, Large Data returns "Project in Use"

Question

So let me preface this with I'm a JS dev, who's started working on a project in R/DeployR, so my knowledge is limited to what I've learned in the past few months.

I'm using the DeployR Javascript API in node, which is connecting to DeployR Open Edition 7.4.1.

I've been dealing with an error that has both myself and the data engineer stumped. This ONLY occurs when a large amount of data is passed in (28k+ row database).

I integrated Pubnub to get logging for the process, here's the gist of what's happening, and the issue I'm trying to resolve.

Process: Node is making a request via the Javascript API to an R Script that processes the data against algorithms. The R script runs the data against each algorithm by using a for loop.

Issue: On the third loop/algorithm, deployR returns an error to the javascript API, which then returns the error as JSON to the app. However, the script keeps running, and shows a completed status in PubNub. It strangely, then returns the data object expected, invoking the .end function in the Javascript API. This returns an empty response. So the error is invoking, however the R Script continues to run.

Error Returned:

{ get: [Function],
  deployr: 
   { response: 
      { success: false,
        call: '/r/project/execute/script',
        errorCode: 403,
        error: '403: Project currently in-use by node_api, call rejected.\n' } } }

Answer 1

After some complex problem solving, we figured out that DeployR server was returning prior to the R script (a fairly complex one) completing the data, and the subsequent call couldn't load the project due to this.

This is mostly due to creating a larger scale app than DeployR was really meant to handle, but we implemented PubNub - any similar watcher feature to let a node (or any front end) environment know when the actual script itself completes does the trick!