Sidekiq terminated when high memory usage during generating watermark in server

Question

I'm using have a server which running on 4gb ram...

Each uploaded picture will be watermark, so i decided to put in background process. However, when there are a lot of requests of uploading pictures..the server will facing high memory issue, and the memory don't deallocate themselves.

My Questions: - why sidekiq worker terminate? - is rmagick memory leak? - how to handle this situation?

Answer 1

You've provided nowhere near the amount of details needed for us to give you informed advice, but I'll try something general: How many Sidekiq workers are you running? Consider reducing the #, then queue up tons of request to simulate a heavy load; keep doing that until you have few enough workers that Sidekiq can comfortably handle the worst load. (Or until you confirm that the issue appears the same even when there's only 1 Sidekiq worker!)

Once you've done that, then you'll have a better feel for the contours of the problem: something that looks like an Rmagick memory leak issue when your server is overloaded, may look different (or give you more ideas on how to address it) when you reduce the workload.

Also take a look at this similar SO question about Rmagick and memory leaks; it might be worth forcing garbage collection to limit how much damage any given leak can cause.