Node - getting ENOMEM error when spawning child process

Question

I'm running Node 14 on an AWS ECS machine with has 5120 CPU and 8192 Memory. At times when running on heavy load (either compressing large number of images using imagemin or copying files from S3 to the local machine using S3 sync) I get the ENOMEM error.

This causes the ECS instance to crash.

The S3 sync command is executed via child_process spawn, while imagemin package also spawns a child process in order to compress images. Both errors are displayed below

(node:23) UnhandledPromiseRejectionWarning: Error: spawn ENOMEM
    at ChildProcess.spawn (internal/child_process.js:403:11)
    at spawn (child_process.js:553:9)
    at new SpawnTimeout (/app/src/utils/SpawnTimeout.ts:33:25)
    at s3Cp (/app/src/utils/s3.utils.ts:62:24)
    at copyFile (/app/src/utils/s3.utils.ts:52:10)
    at executeImageCompression (/app/src/processors/Processor.ts:63:9)

I tried increasing the ECS resources and adding custom -max-old-space-size but it did not help.

I saw this post - Node.js catch ENOMEM error thrown after spawn

But I cannot configure --memory-swap param in the docker as the deployment is controlled by someone else.

Please advise on how I can resolve this?

Answer 1

Memory is allocated to every child_process.spawn() that you call. From what I see, imagemin could be calling child_process.spawn() more times than the computer can handle, leading to nodejs allocating more memory than the computer has to the newly spawned child_processes, leading to the ENOMEM error. Either that, or the processes may be taking up too much memory. This is just my speculation, as I can't see the code.

You could try decreasing or putting a limit on the number of new child_processes being spawned.

Again, this solution is purely speculative as I have no access to your code, so it would be helpful if you could add it to your question.

This link may further help you: How can I limit the number of child proccesses in imagemin?