Process mail bursts one at a time

Question

we receive bursts of mails from our IDS that we are postprocessing to create incident reports, forward them to request tracker, etc. Right now, we have one script that does all the postprocessing that is called by procmail according to rules in /etc/aliases. Something like:

ids-report: "|/opt/ids/process.pl"

Now, the problem is that running many instances of this script because of mail bursts can lead to memory depletion and some synchronization nastyness.

In my script, I tried using Sys::RunAlone, but found it suboptimal.

I would like procmail to feed my script one mail from queue at a time, but I have not found a way to do it.

Do you guys have any idea, what could I do? Out-of-the-box solutions are welcomed too...

Thanks.

Answer 1

I second Georgi Rangelov's answer but if you are seriously asking how to run only one message at a time in Procmail, the answer is to use a lock file.

:0:only1.lock
| /opt/ids/process.pl

This means, if the file only1.lock exists, wait. Once it's gone, create it, run the pipe, and then remove the lock file when done.

See also http://porkmail.org/era/procmail/mini-faq.html#locking

Answer 2

Why not save every mail to a file or database as soon as you receive them and process the saved records one at a time from another script ?

If you post some code you may receive more (and possibly better) answers :)

Answer 3

"IMAP with idle" mailbox as buffer/pipeline of email messages

You may deliver messages to IMAP mailbox. You can use fetchmail with --idle (to get real time processing) and --mda (to execute your script).

You can make procmail deliver messages to maildir as use dovecot even without daemon running to allow fetchmail access maildir using IMAP protocol.

P.S. I may provide a few more details if you are interested. AFAIR I had used it to handle spams catched by my spamtrap server.