I have a tensorflowjs model that I plan to use in nodejs. As the model inference is CPU intensive, I leveraged workerthreads.
I do not want to load the model to workerthread every single time before I call model.predict() as it would slow down the server.
I could load the model in app.js once and reuse it ideally. However, as far as I understand, the workerthread can only communicate with the app.js via messaging. What is the best practice of reusing the model in such a scenario?
Below is how I launch the thread and I tried to pass the model as an argument but looks like you cannot pass complex objects as arguments to threads.
workerPool.runTask({ imagePixelData: data }, (err, result) => {
console.log("* Thread completed:", result);
res.send(result);
});
I would be grateful for any pointer you might have.
Thanks,
Doug